You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-20 - 16:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Core i7-M620 @2667 MHz, Linux 4.19.106-rt45+ (Profile)

Latency plot of shadow in rack #3, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack3slot6s.osadl.org (updated Mon Mar 20, 2023 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
102172116105,6sleep30-21swapper/307:05:183
10910211594,16sleep20-21swapper/207:08:512
9378211189,17sleep00-21swapper/007:05:080
10815210589,11sleep10-21swapper/107:07:381
11031993616,18cyclictest0-21swapper/209:30:172
198102260,1sleep00-21swapper/007:30:270
11031992418,5cyclictest0-21swapper/209:35:162
1103199222,18cyclictest0-21swapper/208:05:112
1103199220,2cyclictest0-21swapper/208:30:112
1103199210,19cyclictest0-21swapper/208:20:142
1102999211,17cyclictest0-21swapper/010:43:320
1103199200,2cyclictest0-21swapper/210:35:142
1103199200,2cyclictest0-21swapper/210:25:122
1102999190,17cyclictest0-21swapper/010:55:080
199532180,1sleep10-21swapper/111:30:191
1102999188,8cyclictest0-21swapper/007:57:080
1102999188,8cyclictest0-21swapper/007:57:070
1102999187,9cyclictest9633-21ssh12:19:040
1102999180,17cyclictest0-21swapper/009:00:200
1102999180,16cyclictest0-21swapper/008:05:210
210932170,1sleep221095-21latency07:35:172
1102999174,11cyclictest12826-21cpuspeed_turbos10:00:160
1102999171,14cyclictest10333-21cut12:20:170
1102999170,16cyclictest0-21swapper/008:25:190
1102999170,15cyclictest0-21swapper/012:35:060
1102999170,15cyclictest0-21swapper/009:40:480
1102999169,6cyclictest0-21swapper/011:55:540
1102999169,6cyclictest0-21swapper/011:55:530
1102999169,6cyclictest0-21swapper/011:44:430
1102999169,6cyclictest0-21swapper/011:15:210
1102999169,6cyclictest0-21swapper/009:45:140
1102999169,6cyclictest0-21swapper/007:20:140
1102999165,9cyclictest9261-21wc09:50:290
1102999165,9cyclictest7149-21ls11:00:260
1102999165,9cyclictest31207-21grep09:25:270
1102999165,9cyclictest10056-21head11:05:290
1102999162,11cyclictest0-21swapper/009:59:020
1102999161,13cyclictest0-21swapper/008:30:220
1102999160,15cyclictest0-21swapper/009:10:540
1103299151,3cyclictest17180-21grep07:25:143
1103099150,13cyclictest26951-21pzem_voltage07:50:251
1103099150,13cyclictest0-21swapper/110:30:211
1102999157,7cyclictest0-21swapper/010:36:260
1102999154,9cyclictest3701-21runrttasks09:39:560
1102999154,9cyclictest28568-21ssh11:49:260
1102999154,9cyclictest21473-1kworker/u9:112:00:340
1102999154,9cyclictest21473-1kworker/u9:112:00:330
1102999151,3cyclictest18281-21grep08:55:140
1102999150,14cyclictest0-21swapper/011:31:190
1102999150,14cyclictest0-21swapper/011:10:580
1102999150,14cyclictest0-21swapper/010:11:320
1102999150,13cyclictest21521-21cut10:20:190
1102999150,13cyclictest0-21swapper/011:35:240
11031991410,2cyclictest0-21swapper/211:55:172
11031991410,2cyclictest0-21swapper/211:55:162
11031991410,2cyclictest0-21swapper/207:58:032
11031991410,2cyclictest0-21swapper/207:58:032
1103199140,12cyclictest0-21swapper/210:14:592
1103099140,2cyclictest0-21swapper/107:55:211
1103099140,2cyclictest0-21swapper/107:55:211
1102999144,8cyclictest26315-21df_abs07:50:160
1102999144,8cyclictest0-21swapper/011:22:380
1102999143,9cyclictest3701-21runrttasks12:31:200
1102999143,9cyclictest30919-21cron11:55:010
1102999143,9cyclictest27920-21munin-node0
1102999143,9cyclictest26963-21unixbench_multi09:15:310
1102999143,9cyclictest24820-21latency07:45:170
1102999143,9cyclictest22687-21switchtime09:05:300
1102999143,9cyclictest15243-21tr08:45:280
1102999143,9cyclictest12573-21fschecks_count12:25:210
1102999143,9cyclictest0-21swapper/011:25:160
1102999142,7cyclictest0-21swapper/008:40:010
1102999142,7cyclictest0-21swapper/007:20:010
1102999142,10cyclictest23328-21diskmemload10:28:280
1102999142,10cyclictest15862-21timerandwakeup10:05:280
1102999141,11cyclictest29971-21taskset08:00:150
1102999141,11cyclictest2934-21sort08:15:010
11029991410,2cyclictest0-21swapper/010:45:190
1102999140,6cyclictest0-21swapper/010:51:440
1102999140,13cyclictest0-21swapper/012:06:190
1102999140,11cyclictest16865-21ntp_kernel_pll_08:50:160
35922130,2sleep30-21swapper/308:15:163
1103299139,2cyclictest0-21swapper/310:30:163
1103299135,6cyclictest281-21jbd2/sda1-807:15:003
1103299131,3cyclictest0-21swapper/308:20:303
11032991310,2cyclictest0-21swapper/311:05:163
11032991310,2cyclictest0-21swapper/309:50:233
1103299130,2cyclictest0-21swapper/312:35:343
1103299130,2cyclictest0-21swapper/312:05:283
1103299130,12cyclictest0-21swapper/310:40:253
1103299130,12cyclictest0-21swapper/307:35:143
1103299130,11cyclictest26689-21mailstats09:15:293
1103199139,2cyclictest0-21swapper/212:30:192
11031991310,2cyclictest0-21swapper/211:40:172
11031991310,2cyclictest0-21swapper/211:30:222
11031991310,2cyclictest0-21swapper/210:15:302
11031991310,2cyclictest0-21swapper/207:50:012
1103199130,2cyclictest30711-21ntp_states08:00:292
1103199130,2cyclictest0-21swapper/211:35:282
1103199130,12cyclictest0-21swapper/211:45:212
1103199130,12cyclictest0-21swapper/211:05:172
1103199130,11cyclictest0-21swapper/212:20:202
1103199130,11cyclictest0-21swapper/210:55:242
11030991310,2cyclictest0-21swapper/111:50:231
11030991310,2cyclictest0-21swapper/110:35:181
11030991310,2cyclictest0-21swapper/109:20:131
11030991310,2cyclictest0-21swapper/109:14:011
1103099130,2cyclictest0-21swapper/112:30:321
1103099130,2cyclictest0-21swapper/108:45:201
1103099130,12cyclictest0-21swapper/110:40:221
1102999133,8cyclictest12611-21expr08:40:130
1102999132,9cyclictest32199-21cpu09:30:180
1102999131,10cyclictest5755-21df_abs12:10:170
1102999131,10cyclictest5228-21sh08:20:140
1102999131,10cyclictest3327-21snmpd07:40:110
1102999131,10cyclictest3183-21cut08:15:130
1102999131,10cyclictest25669-21df_abs10:30:170
1102999131,10cyclictest20540-21runrttasks07:35:110
1102999131,10cyclictest19076-21apt-get10:15:120
1102999131,10cyclictest16999-21df_abs07:25:130
1102999131,10cyclictest11320-21cpuspeed_turbos07:10:120
1103299129,2cyclictest0-21swapper/309:35:243
1103299129,2cyclictest0-21swapper/309:12:443
1103299129,2cyclictest0-21swapper/308:55:113
1103299127,3cyclictest21951-21ntpq10:20:233
1103299123,3cyclictest281-21jbd2/sda1-811:45:003
1103299121,9cyclictest2145-21python08:10:243
1103299120,2cyclictest0-21swapper/311:20:183
1103299120,2cyclictest0-21swapper/311:16:433
1103299120,2cyclictest0-21swapper/307:45:143
1103299120,11cyclictest0-21swapper/312:10:243
1103299120,11cyclictest0-21swapper/311:30:203
1103299120,11cyclictest0-21swapper/310:56:343
1103299120,11cyclictest0-21swapper/310:00:303
1103299120,11cyclictest0-21swapper/308:45:223
1103199129,2cyclictest0-21swapper/212:25:212
1103199129,2cyclictest0-21swapper/211:20:082
1103199129,2cyclictest0-21swapper/210:45:282
1103199129,2cyclictest0-21swapper/210:20:282
1103199129,2cyclictest0-21swapper/210:00:152
1103199129,2cyclictest0-21swapper/209:00:252
1103199122,7cyclictest0-21swapper/212:16:592
1103199122,7cyclictest0-21swapper/211:11:022
1103199122,7cyclictest0-21swapper/210:40:022
1103199122,7cyclictest0-21swapper/209:06:592
1103199122,7cyclictest0-21swapper/208:36:592
1103199122,6cyclictest0-21swapper/207:45:252
1103199122,5cyclictest0-21swapper/209:50:022
1103199120,2cyclictest0-21swapper/212:00:252
1103199120,2cyclictest0-21swapper/212:00:242
1103199120,2cyclictest0-21swapper/210:05:192
1103199120,2cyclictest0-21swapper/209:10:262
1103199120,2cyclictest0-21swapper/207:15:012
1103199120,11cyclictest0-21swapper/212:35:202
1103199120,11cyclictest0-21swapper/212:10:292
1103199120,11cyclictest0-21swapper/211:25:162
1103199120,11cyclictest0-21swapper/209:40:182
1103199120,11cyclictest0-21swapper/209:15:262
1103199120,11cyclictest0-21swapper/208:25:272
1103199120,11cyclictest0-21swapper/208:10:182
1103199120,11cyclictest0-21swapper/207:40:012
1103199120,10cyclictest0-21swapper/210:30:242
1103199120,10cyclictest0-21swapper/208:50:202
1103099129,2cyclictest26782-21switchtime09:15:311
1103099129,2cyclictest0-21swapper/111:45:341
1103099129,2cyclictest0-21swapper/111:18:311
1103099129,2cyclictest0-21swapper/110:15:141
1103099129,2cyclictest0-21swapper/108:10:181
1103099120,2cyclictest0-21swapper/111:28:131
1103099120,2cyclictest0-21swapper/110:50:271
1103099120,2cyclictest0-21swapper/109:30:211
1103099120,2cyclictest0-21swapper/108:15:131
1103099120,11cyclictest0-21swapper/112:00:531
1103099120,11cyclictest0-21swapper/112:00:521
1103099120,11cyclictest0-21swapper/110:49:311
1103099120,11cyclictest0-21swapper/107:35:261
171882110,2sleep10-21swapper/111:23:121
1103299118,2cyclictest0-21swapper/310:50:183
1103299118,2cyclictest0-21swapper/310:15:223
1103299118,2cyclictest0-21swapper/309:56:143
1103299118,2cyclictest0-21swapper/309:05:163
1103299118,2cyclictest0-21swapper/309:00:133
1103299115,4cyclictest3065-21nscd12:03:013
1103299115,4cyclictest3065-21nscd12:03:003
1103299111,8cyclictest0-21swapper/309:40:263
1103299110,9cyclictest281-21jbd2/sda1-811:50:203
1103299110,3cyclictest0-21swapper/312:25:263
1103299110,10cyclictest0-21swapper/310:10:173
1103299110,10cyclictest0-21swapper/307:57:243
1103299110,10cyclictest0-21swapper/307:57:233
1103199118,2cyclictest0-21swapper/211:18:072
1103199118,2cyclictest0-21swapper/209:24:042
1103199112,6cyclictest0-21swapper/209:45:012
1103199112,6cyclictest0-21swapper/207:31:592
1103199112,6cyclictest0-21swapper/207:11:592
1103199110,2cyclictest0-21swapper/211:50:572
1103199110,2cyclictest0-21swapper/210:50:452
1103199110,2cyclictest0-21swapper/209:25:152
1103199110,10cyclictest0-21swapper/207:20:122
1103099118,2cyclictest0-21swapper/111:55:191
1103099118,2cyclictest0-21swapper/111:55:181
1103099118,2cyclictest0-21swapper/110:04:021
1103099118,2cyclictest0-21swapper/108:50:131
1103099118,2cyclictest0-21swapper/108:40:141
1103099113,6cyclictest0-21swapper/108:05:221
1103099111,2cyclictest0-21swapper/111:05:181
1103099110,9cyclictest6236-21munin-run11:00:001
1103099110,9cyclictest4143-21munin-node1
1103099110,9cyclictest21222-21munin-node1
1103099110,9cyclictest0-21swapper/107:25:141
1103099110,9cyclictest0-21swapper/107:15:161
1103099110,10cyclictest0-21swapper/112:13:041
1103099110,10cyclictest0-21swapper/107:10:261
8152100,2sleep30-21swapper/310:46:403
80232100,1sleep10-21swapper/109:48:411
129972100,2sleep30-21swapper/311:14:333
1103299107,2cyclictest0-21swapper/311:37:243
1103299107,2cyclictest0-21swapper/309:45:273
1103299107,2cyclictest0-21swapper/309:25:173
1103299105,3cyclictest7482-21sensors_temp11:00:333
1103299105,3cyclictest32338-21sed09:30:183
1103299105,3cyclictest31586-21cron08:05:013
1103299105,3cyclictest10586-21cat12:20:183
1103299103,3cyclictest6702-21latency_hist08:25:013
1103299101,7cyclictest20363-21latency_hist07:35:003
1103299100,9cyclictest0-21swapper/311:55:193
1103299100,9cyclictest0-21swapper/311:55:193
1103199107,2cyclictest0-21swapper/212:09:002
1103199107,2cyclictest0-21swapper/211:01:582
1103199107,2cyclictest0-21swapper/209:55:232
1103199107,2cyclictest0-21swapper/208:45:202
1103199105,3cyclictest3173-21grep08:15:132
1103199102,6cyclictest0-21swapper/207:25:592
1103099107,2cyclictest0-21swapper/111:41:511
1103099107,2cyclictest0-21swapper/111:36:441
1103099107,2cyclictest0-21swapper/110:25:141
1103099107,2cyclictest0-21swapper/109:50:371
1103099101,2cyclictest3198-21munin-node1
1103099100,9cyclictest19246-21kworker/u8:109:00:041
1103099100,9cyclictest0-21swapper/111:00:361
1103099100,8cyclictest0-21swapper/110:10:201
1103099100,8cyclictest0-21swapper/108:35:181
1103099100,8cyclictest0-21swapper/108:00:291
1103099100,2cyclictest0-21swapper/108:25:171
12246290,1sleep10-21swapper/112:25:141
110329994,3cyclictest9141-21fschecks_time08:30:143
110329994,3cyclictest8490-21grep12:15:193
110329994,3cyclictest32066-21cut08:05:133
110329994,3cyclictest3128-21lldpd11:25:253
110329994,3cyclictest3065-21nscd08:40:213
110329994,3cyclictest29773-21chrt09:24:503
110329994,3cyclictest27797-21df_abs10:35:153
110329994,3cyclictest26103-21cpu07:50:153
110329994,3cyclictest24450-21timerandwakeup10:25:273
110329994,3cyclictest22400-21munin-node3
110329994,3cyclictest16351-21sed08:50:123
110329994,3cyclictest15455-21switchtime12:30:323
110329994,3cyclictest15183-21df_inode07:20:143
110329994,3cyclictest14970-21cpuspeed_turbos10:05:153
110329994,3cyclictest13873-21processes07:15:223
110329994,3cyclictest10611-21cpu08:35:123
110329990,1cyclictest0-21swapper/311:45:173
110319994,3cyclictest18411-21gltestperf08:55:142
110319991,6cyclictest0-21swapper/208:40:242
110309996,2cyclictest0-21swapper/111:10:331
110309996,2cyclictest0-21swapper/109:35:181
110309996,2cyclictest0-21swapper/107:40:171
110309996,2cyclictest0-21swapper/107:20:171
110309993,2cyclictest6806-21date08:25:011
110309991,6cyclictest9799-21munin-run12:20:011
110309990,8cyclictest0-21swapper/110:05:211
110309990,8cyclictest0-21swapper/109:58:361
110309990,8cyclictest0-21swapper/108:30:121
110309990,3cyclictest17998-21munin-node1
110309990,2cyclictest16600-21munin-node1
110309990,2cyclictest0-21swapper/112:22:321
110309983,3cyclictest30580-21latency09:25:171
110309983,3cyclictest19261-21cat07:30:171
110309980,7cyclictest23890-21kworker/u8:007:48:031
110309980,7cyclictest23004-21kworker/u8:009:07:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional