You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-18 - 06:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Thu Dec 18, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2485991460,5rtkit-daemon27065-21diskstats21:20:381
1588421260,3chrt15886-21aten_r1power_po19:50:361
2485991120,7rtkit-daemon0-21swapper/120:17:561
1684321060,6sleep216842-21grep00:20:412
1684321060,6sleep216842-21grep00:20:412
2634599975,85cyclictest0-21swapper/219:10:512
2634299954,67cyclictest0-21swapper/022:05:310
2634599924,81cyclictest0-21swapper/219:30:562
2634299904,77cyclictest0-21swapper/020:10:420
2634299903,69cyclictest9508-21crond20:45:230
2634599894,26cyclictest0-21swapper/222:20:372
2634599894,11cyclictest9411-21gltestperf19:35:462
2634599885,76cyclictest0-21swapper/222:50:392
2634599874,24cyclictest0-21swapper/223:30:302
2634299874,76cyclictest0-21swapper/023:50:410
2634299873,76cyclictest0-21swapper/021:05:400
299002860,5sleep2281ktimersoftd/223:40:392
2634599865,25cyclictest0-21swapper/219:56:572
2634299864,74cyclictest0-21swapper/000:25:430
2634299864,64cyclictest0-21swapper/022:32:220
2634299864,46cyclictest4743-21awk20:35:230
2634299863,76cyclictest0-21swapper/000:00:350
2634299863,63cyclictest0-21swapper/023:10:300
2634699854,30cyclictest0-21swapper/320:36:063
26346998523,12cyclictest0-21swapper/321:40:233
2634299855,73cyclictest0-21swapper/021:00:440
2634299854,74cyclictest4634-21grep19:25:470
2634299854,74cyclictest4634-21grep19:25:470
2634299854,74cyclictest0-21swapper/023:20:330
2634299854,73cyclictest0-21swapper/021:15:330
2634299854,65cyclictest0-21swapper/023:38:230
2634299854,63cyclictest0-21swapper/019:50:360
2634299854,62cyclictest0-21swapper/019:20:450
2634299854,62cyclictest0-21swapper/019:20:450
2634299853,75cyclictest0-21swapper/023:55:300
26346998436,42cyclictest9475-21gltestperf19:35:473
2634299844,73cyclictest0-21swapper/020:55:560
2634299844,72cyclictest5900-21crond21:45:210
2634299844,61cyclictest0-21swapper/023:15:310
2634699833,10cyclictest0-21swapper/321:50:403
26346998320,9cyclictest22863-21munin-node22:20:313
2634599834,29cyclictest0-21swapper/219:45:262
2634299834,72cyclictest0-21swapper/021:45:460
2634299834,66cyclictest0-21swapper/019:30:380
2634299834,62cyclictest0-21swapper/021:50:310
2634299834,62cyclictest0-21swapper/019:05:460
2634299833,73cyclictest27618-21chrt21:20:460
2634299833,66cyclictest0-21swapper/020:10:260
2634299833,61cyclictest0-21swapper/020:20:400
2634299833,60cyclictest32413-21munin-node22:40:340
26346998211,63cyclictest0-21swapper/321:05:243
2634299824,71cyclictest0-21swapper/022:00:440
2634299824,71cyclictest0-21swapper/020:50:240
2634299823,70cyclictest0-21swapper/021:31:230
2634299823,70cyclictest0-21swapper/021:31:230
2634299823,62cyclictest0-21swapper/023:00:180
2634299823,62cyclictest0-21swapper/023:00:180
2634299823,61cyclictest0-21swapper/021:10:530
2634299823,44cyclictest11628-21sort19:40:440
2634299815,68cyclictest0-21swapper/020:25:400
2634299814,70cyclictest0-21swapper/023:45:400
2634299814,70cyclictest0-21swapper/023:40:450
2634299813,71cyclictest13718-21sed00:15:300
2634299813,60cyclictest0-21swapper/019:35:470
2634699803,29cyclictest7440-21munin-node21:45:473
2634599804,36cyclictest0-21swapper/222:10:202
2634599804,26cyclictest0-21swapper/219:15:542
26345998010,31cyclictest0-21swapper/221:05:572
2634299804,68cyclictest0-21swapper/023:00:470
2634299804,68cyclictest0-21swapper/022:10:460
2634299804,68cyclictest0-21swapper/022:10:460
2634299804,64cyclictest0-21swapper/019:55:380
2634299804,57cyclictest0-21swapper/020:05:260
2634299804,55cyclictest0-21swapper/020:15:360
2634299803,71cyclictest0-21swapper/022:27:220
2634299803,66cyclictest0-21swapper/000:06:470
2634699795,27cyclictest17795-21munin-run00:25:153
2634699795,27cyclictest17795-21munin-run00:25:153
2634699793,7cyclictest16682-21latency_hist23:15:193
2634699793,32cyclictest0-21swapper/320:35:253
2634699793,30cyclictest0-21swapper/322:35:193
26346997923,9cyclictest2109-21hwlatdetect23:50:373
26345997920,8cyclictest3515-21latency_hist21:40:222
2634299794,68cyclictest0-21swapper/022:50:350
2634299794,68cyclictest0-21swapper/020:37:440
2634299793,69cyclictest0-21swapper/022:35:350
2634699783,31cyclictest0-21swapper/321:31:223
2634699783,31cyclictest0-21swapper/321:31:223
2634699783,31cyclictest0-21swapper/321:30:233
2634699783,30cyclictest27039-21tail20:10:593
2634699783,30cyclictest25310-21aten_r1power_en22:25:303
26346997820,9cyclictest4389-21munin-node21:40:393
26346997820,9cyclictest2256-21gltestperf19:20:463
26346997820,9cyclictest2256-21gltestperf19:20:463
26346997819,10cyclictest12449-21cpuspeed_turbos20:50:373
26346997819,10cyclictest12449-21cpuspeed_turbos20:50:373
2634599785,27cyclictest0-21swapper/222:11:142
2634599785,27cyclictest0-21swapper/222:11:142
2634299783,69cyclictest0-21swapper/021:55:330
2634299783,68cyclictest1287-21snmpd23:07:280
2634299783,61cyclictest0-21swapper/019:46:550
2634299783,59cyclictest0-21swapper/023:30:500
2634299783,58cyclictest0-21swapper/022:20:370
2634299783,56cyclictest0-21swapper/021:35:470
2634699773,8cyclictest20376-21kernelversion21:05:463
2634699773,67cyclictest0-21swapper/322:55:193
2634699773,26cyclictest0-21swapper/322:05:343
26346997727,11cyclictest24971-21gltestperf21:15:423
26346997724,10cyclictest6974-21gltestperf00:00:383
26346997720,9cyclictest30371-21cpuspeed_turbos22:35:333
26346997720,9cyclictest13994-21fschecks_time00:15:333
2634299774,55cyclictest0-21swapper/019:15:280
2634299773,68cyclictest0-21swapper/022:19:360
2634299773,67cyclictest14341-21chrt20:55:240
2634299773,67cyclictest14341-21chrt20:55:240
2634299773,59cyclictest0-21swapper/021:27:120
2634299773,45cyclictest3956-21sendmail_mailtr22:45:490
2634699764,31cyclictest0-21swapper/323:45:073
2634699763,34cyclictest32185-21interrupts23:45:373
2634699763,33cyclictest0-21swapper/323:30:443
2634699763,31cyclictest0-21swapper/319:40:373
2634699763,30cyclictest0-21swapper/323:15:373
26346997626,11cyclictest11595-21grep21:55:393
26346997616,16cyclictest14244-21gltestperf19:45:463
2634299764,58cyclictest0-21swapper/019:15:550
2634299763,44cyclictest22489-21sshd00:34:250
2634699754,31cyclictest0-21swapper/300:15:163
2634699754,30cyclictest0-21swapper/323:40:173
26346997531,37cyclictest8634-21head00:05:283
2634699753,30cyclictest0-21swapper/322:15:213
2634699753,30cyclictest0-21swapper/322:15:213
26346997525,9cyclictest904-21sed20:25:433
26346997524,10cyclictest21647-21sort20:00:513
26346997522,9cyclictest14001-21fschecks_time22:00:393
26346997521,8cyclictest4702-21gltestperf19:25:473
26346997521,8cyclictest4702-21gltestperf19:25:473
26346997521,8cyclictest29811-21gltestperf19:10:483
2634299754,63cyclictest0-21swapper/000:25:160
2634299754,63cyclictest0-21swapper/000:25:160
2634299754,61cyclictest0-21swapper/023:25:510
2634299753,57cyclictest11409-21df_abs00:10:320
26346997421,8cyclictest9698-21munin-node23:00:423
26346997325,9cyclictest1419-21runrttasks20:59:143
26346997313,14cyclictest0-21swapper/322:40:383
2634599734,25cyclictest0-21swapper/222:57:272
2634599734,25cyclictest0-21swapper/222:57:272
26345997315,18cyclictest22629-21latency_hist22:20:212
2634699728,28cyclictest20332-21awk20:00:253
26346997225,8cyclictest22679-21kernelversion23:25:413
26346997222,9cyclictest7766-21df_abs22:55:343
26346997222,9cyclictest7766-21df_abs22:55:343
26346997222,9cyclictest1419-21runrttasks20:40:523
26346997221,10cyclictest23176-21switchtime21:10:533
26346997218,11cyclictest0-21swapper/323:20:283
26346997121,9cyclictest22089-21unixbench-2d00:30:483
2634299713,46cyclictest26318-21latency_hist19:05:280
2634699704,11cyclictest0-21swapper/319:15:383
2634699703,31cyclictest1287-21snmpd22:17:203
2634699703,28cyclictest23832-21smartctl20:05:433
26346997025,8cyclictest18829-21gltestperf00:25:343
26346997024,9cyclictest27317-21gltestperf21:20:413
26346997021,8cyclictest13301-21netstat23:05:423
26346997019,9cyclictest17525-21unixbench_singl19:50:583
26346996923,9cyclictest30311-21cpu20:20:383
26346996922,9cyclictest6261-21aten_r1power_en19:30:383
26346996922,10cyclictest0-21swapper/319:07:383
26346996820,9cyclictest10353-21fschecks_count20:45:413
26345996712,15cyclictest0-21swapper/220:45:222
26345996619,11cyclictest1419-21runrttasks22:38:322
211922603,48sleep10-21swapper/123:22:421
2634599594,48cyclictest0-21swapper/223:25:402
26345995824,13cyclictest0-21swapper/220:50:502
26345995824,13cyclictest0-21swapper/220:50:502
26344995839,12cyclictest12965-21cut23:05:361
26345995619,31cyclictest1419-21runrttasks22:45:492
26344995626,12cyclictest17817-21gltestperf21:00:441
2634599554,44cyclictest0-21swapper/220:36:012
26345995417,10cyclictest26618-21aten_r1power_en19:05:352
2634499544,30cyclictest0-21swapper/123:20:171
26344995418,22cyclictest1419-21runrttasks22:03:551
7132533,42sleep10-21swapper/122:40:391
2634599533,43cyclictest30631-21unixbench_singl21:25:552
2634499524,25cyclictest0-21swapper/121:12:231
26344995218,26cyclictest28823-21tail23:40:271
246962522,44sleep10-21swapper/122:21:171
226232523,12sleep10-21swapper/122:20:201
128962523,42sleep10-21swapper/100:13:521
32902512,42sleep10-21swapper/121:37:291
2634499514,28cyclictest0-21swapper/123:55:491
216942512,12sleep10-21swapper/100:30:441
111442514,9sleep111190-21switchtime23:00:501
96882502,42sleep10-21swapper/100:05:421
80092503,41sleep11287-21snmpd21:46:441
26345995023,11cyclictest0-21swapper/221:20:462
2634499504,29cyclictest0-21swapper/119:30:531
2634499504,28cyclictest0-21swapper/120:40:441
2634499504,28cyclictest0-21swapper/120:35:401
2634499504,28cyclictest0-21swapper/119:45:251
2634499504,28cyclictest0-21swapper/100:20:321
2634499504,28cyclictest0-21swapper/100:20:321
2634499504,27cyclictest0-21swapper/123:10:301
26344995020,13cyclictest0-21swapper/123:25:351
26344995020,13cyclictest0-21swapper/100:00:301
248599500,5rtkit-daemon2484-21rtkit-daemon19:00:441
57662492,39sleep20-21swapper/223:59:102
32652492,41sleep20-21swapper/219:21:172
32652492,41sleep20-21swapper/219:21:172
26345994922,9cyclictest4746-21awk20:35:242
2634499495,11cyclictest0-21swapper/123:46:431
2634499494,28cyclictest0-21swapper/122:10:421
2634499494,28cyclictest0-21swapper/122:10:421
2634499494,28cyclictest0-21swapper/121:50:311
2634499494,27cyclictest0-21swapper/122:30:451
2634499494,27cyclictest0-21swapper/121:40:511
2634499494,27cyclictest0-21swapper/120:45:561
2634499494,27cyclictest0-21swapper/120:30:541
2634499494,27cyclictest0-21swapper/119:20:281
2634499494,23cyclictest0-21swapper/119:20:391
2634499494,23cyclictest0-21swapper/119:20:391
2634499494,12cyclictest0-21swapper/121:25:421
2634499493,40cyclictest2152-21yum-updatesd-he22:45:221
26344994923,11cyclictest26938-21df_abs23:35:311
26344994919,13cyclictest0-21swapper/121:30:411
26344994919,13cyclictest0-21swapper/121:30:411
26344994919,13cyclictest0-21swapper/120:05:551
225272492,41sleep10-21swapper/120:03:151
211402492,40sleep10-21swapper/121:06:021
164082492,39sleep10-21swapper/120:56:391
151592492,41sleep20-21swapper/222:01:232
2634699482,41cyclictest0-21swapper/323:55:433
2634599484,26cyclictest0-21swapper/220:00:382
26345994833,9cyclictest27019-21grep20:10:592
2634599483,39cyclictest0-21swapper/221:00:362
26345994821,7cyclictest25084-21latency_hist20:10:262
26345994819,11cyclictest0-21swapper/219:50:432
2634499484,27cyclictest0-21swapper/122:55:371
2634499484,27cyclictest0-21swapper/122:55:371
2634499484,27cyclictest0-21swapper/122:50:431
2634499484,27cyclictest0-21swapper/120:25:401
2634499484,26cyclictest0-21swapper/122:05:521
2634499484,26cyclictest0-21swapper/120:22:011
2634499484,26cyclictest0-21swapper/119:50:261
2634499483,28cyclictest0-21swapper/122:27:171
258422482,40sleep10-21swapper/123:31:081
181272482,11sleep10-21swapper/100:25:281
128822482,40sleep10-21swapper/121:57:481
104352482,39sleep10-21swapper/119:36:271
284202472,38sleep10-21swapper/119:06:411
2634599475,23cyclictest9944-21find21:50:492
2634599474,25cyclictest0-21swapper/221:57:162
2634599473,9cyclictest15011-21df_abs23:10:342
26345994719,11cyclictest0-21swapper/223:20:452
26345994718,12cyclictest0-21swapper/200:10:372
2634499474,26cyclictest0-21swapper/123:50:271
2634499474,25cyclictest0-21swapper/120:10:451
26344994718,12cyclictest0-21swapper/119:10:501
152972473,38sleep20-21swapper/219:47:322
140622473,38sleep20-21swapper/223:08:082
2634599466,34cyclictest0-21swapper/222:30:312
2634599464,37cyclictest0-21swapper/223:50:492
2634599464,36cyclictest0-21swapper/220:25:462
2634599464,34cyclictest0-21swapper/219:25:392
2634599464,34cyclictest0-21swapper/219:25:392
2634599464,25cyclictest0-21swapper/221:10:372
2634599464,25cyclictest0-21swapper/220:50:232
2634599463,37cyclictest0-21swapper/221:30:352
2634599463,37cyclictest0-21swapper/221:30:352
26345994629,11cyclictest0-21swapper/200:30:492
26344994627,13cyclictest14994-21unixbench_singl00:15:481
26344994618,12cyclictest0-21swapper/122:40:131
2634599454,25cyclictest0-21swapper/223:45:342
2634599454,24cyclictest0-21swapper/200:00:352
2634599453,36cyclictest26504-21tail23:35:282
2634599453,24cyclictest0-21swapper/222:40:332
2634599453,23cyclictest13249-21munin-node00:15:422
26345994517,12cyclictest0-21swapper/223:00:342
2634499454,25cyclictest0-21swapper/121:17:101
26344994529,11cyclictest19073-21hwlatdetect19:55:471
131582453,36sleep10-21swapper/120:50:461
131582453,36sleep10-21swapper/120:50:461
57602442,36sleep20-21swapper/221:43:382
2634599445,32cyclictest0-21swapper/200:25:182
2634599444,34cyclictest0-21swapper/221:45:412
2634599444,25cyclictest0-21swapper/223:15:432
2634599444,13cyclictest0-21swapper/222:25:362
2634599443,36cyclictest0-21swapper/220:55:392
2634499443,25cyclictest0-21swapper/119:26:511
2634499443,25cyclictest0-21swapper/119:26:511
2634599435,31cyclictest0-21swapper/200:05:322
2634599432,35cyclictest0-21swapper/221:15:412
2634699422,23cyclictest4759-21sshd22:50:193
2634599424,32cyclictest0-21swapper/220:20:262
2634599423,33cyclictest0-21swapper/220:20:362
2634699413,31cyclictest0-21swapper/320:20:263
26346993218,8cyclictest26318-21latency_hist19:05:283
2634599323,23cyclictest26368-21latency_hist19:05:272
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional