You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-18 - 16:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Wed Jun 18, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
159899944,14cyclictest0-21swapper/012:14:140
160299935,28cyclictest0-21swapper/107:39:481
159899934,13cyclictest0-21swapper/009:39:240
159899934,12cyclictest0-21swapper/011:04:280
160299926,27cyclictest0-21swapper/107:29:251
159899924,13cyclictest0-21swapper/010:09:380
159899924,13cyclictest0-21swapper/010:09:380
159899924,13cyclictest0-21swapper/007:04:300
159899923,69cyclictest0-21swapper/007:09:370
159899923,69cyclictest0-21swapper/007:09:370
159899913,62cyclictest25494-21munin-run12:09:000
159899913,11cyclictest19113-21unixbench_multi08:29:420
159899904,64cyclictest0-21swapper/011:24:160
159899904,14cyclictest0-21swapper/011:44:230
159899904,13cyclictest0-21swapper/009:29:310
159899904,12cyclictest0-21swapper/009:44:450
159899894,78cyclictest0-21swapper/010:14:240
159899894,14cyclictest0-21swapper/007:18:320
159899894,13cyclictest0-21swapper/007:44:380
159899893,12cyclictest0-21swapper/010:49:100
159899884,64cyclictest0-21swapper/011:54:160
159899884,13cyclictest0-21swapper/010:29:040
159899864,13cyclictest0-21swapper/009:19:050
159899863,21cyclictest0-21swapper/011:59:220
159899863,13cyclictest0-21swapper/008:49:090
159899863,13cyclictest0-21swapper/007:19:270
159899854,13cyclictest0-21swapper/011:22:370
159899854,13cyclictest0-21swapper/009:04:420
159899854,13cyclictest0-21swapper/006:59:300
159899854,12cyclictest0-21swapper/009:39:070
159899853,67cyclictest0-21swapper/009:59:340
159899853,60cyclictest0-21swapper/007:59:390
160399849,68cyclictest26823-21latency_hist11:04:042
160399844,12cyclictest0-21swapper/207:04:302
160399844,11cyclictest0-21swapper/209:55:382
159899844,64cyclictest0-21swapper/011:14:250
159899844,13cyclictest0-21swapper/006:54:290
159899843,74cyclictest0-21swapper/007:55:050
159899843,74cyclictest0-21swapper/007:55:050
82302830,6sleep18234-21cpuspeed_turbos08:09:241
160399834,71cyclictest0-21swapper/208:54:422
160399833,57cyclictest0-21swapper/207:24:462
159899834,14cyclictest0-21swapper/007:34:260
159899834,13cyclictest0-21swapper/007:49:350
159899834,12cyclictest0-21swapper/011:34:160
159899834,12cyclictest0-21swapper/011:34:160
1598998320,48cyclictest0-21swapper/007:24:430
160399824,9cyclictest8817-21systemd-cgroups09:19:072
160399824,9cyclictest1265-21runrttasks10:44:562
160399824,62cyclictest0-21swapper/210:49:292
160399823,11cyclictest0-21swapper/207:39:432
160399822,61cyclictest1160-21snmpd11:49:202
159899824,64cyclictest0-21swapper/008:09:290
159899824,14cyclictest0-21swapper/009:24:280
159899824,14cyclictest0-21swapper/009:24:280
159899824,13cyclictest0-21swapper/009:54:230
159899824,12cyclictest0-21swapper/008:39:410
159899824,12cyclictest0-21swapper/008:39:410
159899823,64cyclictest0-21swapper/011:51:340
159899823,16cyclictest0-21swapper/008:04:230
160399814,12cyclictest0-21swapper/212:04:272
160399814,12cyclictest0-21swapper/208:39:362
160399814,12cyclictest0-21swapper/208:39:362
160399813,10cyclictest0-21swapper/207:32:022
1603998116,44cyclictest0-21swapper/209:09:332
159899814,60cyclictest0-21swapper/010:34:360
159899814,13cyclictest0-21swapper/008:59:080
159899813,71cyclictest0-21swapper/006:49:480
159899813,63cyclictest0-21swapper/009:01:350
160399804,61cyclictest0-21swapper/210:09:392
160399804,61cyclictest0-21swapper/210:09:392
160399804,54cyclictest0-21swapper/211:44:352
160399804,53cyclictest0-21swapper/206:54:312
160399804,12cyclictest0-21swapper/211:24:162
160399804,12cyclictest0-21swapper/207:04:132
160399803,9cyclictest27868-21crond12:14:002
160399803,9cyclictest1265-21runrttasks07:58:552
160399803,9cyclictest1265-21runrttasks07:58:552
160399803,12cyclictest0-21swapper/209:59:382
160399803,10cyclictest20620-21gltestperf08:34:282
159899805,65cyclictest0-21swapper/006:49:130
159899804,13cyclictest0-21swapper/008:34:250
159899804,13cyclictest0-21swapper/008:24:400
159899804,13cyclictest0-21swapper/007:29:350
159899803,70cyclictest1160-21snmpd07:39:190
159899803,13cyclictest0-21swapper/008:14:360
1598998015,10cyclictest10393-21telnet10:29:180
160399794,67cyclictest0-21swapper/208:19:312
160399794,12cyclictest0-21swapper/211:04:172
160399794,12cyclictest0-21swapper/208:29:212
160399794,12cyclictest0-21swapper/208:09:392
1603997911,45cyclictest106250irq/46-eth009:49:352
159899794,13cyclictest0-21swapper/011:09:320
159899794,13cyclictest0-21swapper/010:04:190
160399785,59cyclictest0-21swapper/207:59:222
160399784,12cyclictest0-21swapper/209:44:222
160399784,11cyclictest0-21swapper/208:04:322
160399784,11cyclictest0-21swapper/206:49:332
160399783,8cyclictest11956-21df_inode11:39:192
160399783,11cyclictest0-21swapper/207:14:262
1603997818,41cyclictest0-21swapper/210:54:312
1603997818,41cyclictest0-21swapper/210:54:312
159899784,12cyclictest0-21swapper/009:14:080
159899783,60cyclictest0-21swapper/012:09:200
159899783,13cyclictest0-21swapper/008:49:400
160399774,59cyclictest0-21swapper/211:54:162
160399773,12cyclictest0-21swapper/211:34:172
160399773,12cyclictest0-21swapper/211:34:172
1603997722,47cyclictest6928-21cpuspeed_turbos11:29:172
159899773,58cyclictest0-21swapper/010:59:170
159899773,14cyclictest0-21swapper/009:49:220
160399769,45cyclictest0-21swapper/207:34:252
160399764,65cyclictest0-21swapper/209:29:222
160399764,11cyclictest0-21swapper/211:59:272
160399763,9cyclictest1808-21aten2.4-expect11:19:152
160399763,9cyclictest11327-21ls10:29:292
159899769,8cyclictest106250irq/46-eth011:29:350
159899764,65cyclictest0-21swapper/011:42:540
159899764,65cyclictest0-21swapper/010:44:310
159899763,60cyclictest0-21swapper/010:39:340
159899763,12cyclictest0-21swapper/010:54:210
159899763,12cyclictest0-21swapper/010:54:210
284722750,3sleep1201ktimersoftd/111:04:361
160399754,14cyclictest0-21swapper/208:49:382
160399754,12cyclictest0-21swapper/208:24:322
160399754,11cyclictest0-21swapper/207:19:472
160399753,11cyclictest0-21swapper/209:19:252
160399752,50cyclictest17176-21latency_hist10:44:052
1603997510,44cyclictest30616-21aten_r1power_cu07:49:242
159899753,20cyclictest0-21swapper/010:19:190
159899753,13cyclictest0-21swapper/008:19:260
160399747,9cyclictest106250irq/46-eth008:14:362
160399744,13cyclictest0-21swapper/207:09:422
160399744,13cyclictest0-21swapper/207:09:422
160399744,11cyclictest0-21swapper/210:14:272
160399743,11cyclictest0-21swapper/209:24:232
160399743,11cyclictest0-21swapper/209:24:232
159899743,13cyclictest0-21swapper/009:19:210
160399734,62cyclictest0-21swapper/211:09:282
160399734,57cyclictest0-21swapper/209:43:582
160399726,8cyclictest0-21swapper/209:04:072
160399724,55cyclictest0-21swapper/210:34:192
160399724,13cyclictest0-21swapper/211:19:022
160399723,8cyclictest15065-21kworker/2:108:44:222
160399723,11cyclictest0-21swapper/209:37:082
160399716,56cyclictest0-21swapper/210:24:272
160399715,57cyclictest0-21swapper/210:05:012
160399714,60cyclictest0-21swapper/212:18:262
160399714,17cyclictest0-21swapper/209:08:362
160399713,8cyclictest7604-21crond10:24:052
160399694,14cyclictest0-21swapper/207:44:262
160399664,12cyclictest0-21swapper/206:49:142
160299654,27cyclictest20716-21sadc11:59:001
128722624,14sleep30-21swapper/309:24:403
128722624,14sleep30-21swapper/309:24:403
1604996143,11cyclictest24579-21needreboot09:49:353
160299615,28cyclictest0-21swapper/110:29:381
160299615,28cyclictest0-21swapper/109:24:251
160299615,28cyclictest0-21swapper/109:24:251
160299615,27cyclictest0-21swapper/112:14:131
160299614,27cyclictest0-21swapper/110:19:251
1602996112,26cyclictest0-21swapper/108:23:431
160299605,28cyclictest0-21swapper/111:59:221
160299605,28cyclictest0-21swapper/110:34:421
160299604,28cyclictest0-21swapper/109:09:381
160299604,28cyclictest0-21swapper/108:54:401
160299604,28cyclictest0-21swapper/108:14:441
160299604,28cyclictest0-21swapper/107:14:461
160299604,28cyclictest0-21swapper/107:09:351
160299604,28cyclictest0-21swapper/107:09:351
160299604,27cyclictest0-21swapper/112:09:291
160299595,28cyclictest0-21swapper/111:24:361
160299595,28cyclictest0-21swapper/109:29:391
160299594,27cyclictest0-21swapper/110:44:201
1602995922,13cyclictest0-21swapper/107:59:231
1602995922,12cyclictest0-21swapper/109:54:251
279222589,41sleep327988-21latency_hist09:59:063
160499583,37cyclictest0-21swapper/310:14:273
160299585,31cyclictest0-21swapper/107:24:261
160299585,28cyclictest0-21swapper/110:14:351
160299585,28cyclictest0-21swapper/110:04:271
160299585,27cyclictest0-21swapper/110:24:411
160299585,27cyclictest0-21swapper/109:34:421
160299584,28cyclictest0-21swapper/109:19:391
160299584,27cyclictest0-21swapper/110:01:271
156582583,13sleep10-21swapper/111:45:231
324102574,46sleep10-21swapper/111:14:241
184342573,46sleep30-21swapper/307:24:133
160299574,28cyclictest0-21swapper/108:34:201
160299574,28cyclictest0-21swapper/107:04:271
160299564,42cyclictest0-21swapper/107:49:241
160299564,28cyclictest0-21swapper/107:34:261
1602995615,21cyclictest0-21swapper/110:14:051
1602995615,21cyclictest0-21swapper/110:14:051
313702554,44sleep10-21swapper/111:12:351
183092553,13sleep10-21swapper/107:22:541
1604995537,11cyclictest13502-21gltestperf10:34:263
160299554,28cyclictest0-21swapper/110:39:321
160299554,28cyclictest0-21swapper/109:49:061
160299554,27cyclictest0-21swapper/109:39:391
1602995534,13cyclictest7042-21tr06:59:291
73162542,11sleep30-21swapper/308:05:083
267042532,12sleep30-21swapper/311:03:583
172042533,9sleep116180-21munin-node11:49:261
160499534,39cyclictest0-21swapper/307:04:313
160499534,29cyclictest0-21swapper/308:14:103
193402523,41sleep10-21swapper/108:31:141
1604995221,8cyclictest361ktimersoftd/309:34:393
1604995219,26cyclictest6749-21tail06:59:273
1602995218,27cyclictest7544-21ls11:29:231
39092513,42sleep10-21swapper/106:51:551
303182514,40sleep10-21swapper/107:48:421
276992512,14sleep30-21swapper/312:11:533
236642513,42sleep10-21swapper/108:39:441
236642513,42sleep10-21swapper/108:39:441
160499514,29cyclictest0-21swapper/311:34:203
160499514,29cyclictest0-21swapper/311:34:203
1604995133,11cyclictest18986-21cpuspeed_turbos07:24:293
160299514,28cyclictest0-21swapper/110:54:481
160299514,28cyclictest0-21swapper/110:54:481
300382503,40sleep30-21swapper/312:16:163
29702504,38sleep30-21swapper/306:49:393
281402502,9sleep30-21swapper/308:49:343
277262502,41sleep30-21swapper/307:41:023
243262502,10sleep30-21swapper/308:43:453
243262502,10sleep30-21swapper/308:43:453
160499504,28cyclictest0-21swapper/307:49:413
160499504,28cyclictest0-21swapper/307:39:113
1604995032,10cyclictest32076-21cpuspeed_turbos11:14:193
1604995019,12cyclictest0-21swapper/309:19:263
1604995017,24cyclictest31924-21tail08:59:243
160299504,27cyclictest0-21swapper/109:05:421
1602995021,12cyclictest0-21swapper/108:04:311
86272492,13sleep10-21swapper/109:16:561
63082493,39sleep10-21swapper/106:57:031
160499495,27cyclictest0-21swapper/307:44:453
160499494,28cyclictest0-21swapper/311:04:353
160499494,27cyclictest0-21swapper/311:44:263
160499494,27cyclictest0-21swapper/310:54:403
160499494,27cyclictest0-21swapper/310:54:403
160499494,27cyclictest0-21swapper/310:44:183
160499494,27cyclictest0-21swapper/309:44:233
160499494,27cyclictest0-21swapper/309:14:423
160499494,27cyclictest0-21swapper/308:44:243
160499493,13cyclictest0-21swapper/310:09:413
160499493,13cyclictest0-21swapper/310:09:413
160499492,40cyclictest0-21swapper/309:29:193
160299495,27cyclictest0-21swapper/111:39:391
160299494,28cyclictest0-21swapper/110:49:291
160299494,27cyclictest0-21swapper/109:49:251
160299494,27cyclictest0-21swapper/108:24:331
60662483,9sleep30-21swapper/311:25:063
305692483,10sleep330648-21munin-node10:04:183
160499485,26cyclictest0-21swapper/310:24:203
160499484,27cyclictest0-21swapper/307:09:453
160499484,27cyclictest0-21swapper/307:09:453
1604994831,10cyclictest6634-21aten_r1power_cu11:29:143
1604994819,12cyclictest0-21swapper/308:24:313
1604994819,11cyclictest1160-21snmpd11:39:193
160299485,27cyclictest0-21swapper/112:04:241
160299484,27cyclictest0-21swapper/108:49:411
108642483,39sleep10-21swapper/111:35:241
108642483,39sleep10-21swapper/111:35:241
265332473,11sleep10-21swapper/111:01:431
234602472,38sleep323560-21cpuspeed12:04:153
1604994732,9cyclictest16550-21cpuspeed_turbos11:49:173
1604994718,14cyclictest0-21swapper/306:58:293
160299474,34cyclictest0-21swapper/111:19:151
160299474,24cyclictest0-21swapper/108:59:241
1602994725,13cyclictest25556-21if_err_eth008:44:311
38242462,10sleep30-21swapper/311:22:043
288482463,10sleep30-21swapper/309:59:263
231432462,11sleep30-21swapper/307:33:443
160499464,25cyclictest0-21swapper/311:54:213
160499464,25cyclictest0-21swapper/308:23:433
1604994624,15cyclictest0-21swapper/309:04:413
157742463,10sleep30-21swapper/307:15:593
160499458,31cyclictest16141-21missed_timers10:39:293
160499454,34cyclictest0-21swapper/308:34:423
1604994530,9cyclictest30145-21gltestperf11:09:243
160499453,35cyclictest3966-21gltestperf07:59:313
160499453,26cyclictest0-21swapper/309:42:543
1604994528,10cyclictest5612-21aten_r1power_vo10:19:193
1604994527,11cyclictest4532-21cpuspeed_turbos09:09:233
160299454,25cyclictest0-21swapper/107:56:391
160299454,25cyclictest0-21swapper/107:56:391
312762442,10sleep30-21swapper/308:55:593
160499444,14cyclictest0-21swapper/307:54:273
160499444,14cyclictest0-21swapper/307:54:273
1604994427,10cyclictest20133-21cpuspeed_turbos10:49:203
160499434,32cyclictest0-21swapper/308:29:363
1604994327,9cyclictest11052-21grep10:29:253
1604994326,11cyclictest10384-21aten_r1power_po08:14:243
228782424,32sleep30-21swapper/312:01:133
14352404,30sleep10-21swapper/106:47:401
223099380,4rtkit-daemon2229-21rtkit-daemon06:46:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional