You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 11:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Tue Apr 14, 2026 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3200999993,87cyclictest30275-21ssh20:10:503
3200999974,86cyclictest0-21swapper/320:40:203
3200999964,86cyclictest0-21swapper/319:18:283
3200999953,84cyclictest8775-21unixbench-2d20:26:013
3200999953,84cyclictest8775-21unixbench-2d20:26:013
3200999945,81cyclictest0-21swapper/321:35:503
32007999419,22cyclictest4524-21gltestperf20:20:491
32007999419,22cyclictest4524-21gltestperf20:20:491
3200999924,81cyclictest0-21swapper/319:56:083
3200999915,79cyclictest19021-21sed19:55:313
3200999914,81cyclictest0-21swapper/321:57:163
3200999913,80cyclictest0-21swapper/319:38:213
3200899913,80cyclictest0-21swapper/220:49:592
3200899913,80cyclictest0-21swapper/220:40:582
3200899913,79cyclictest0-21swapper/220:06:302
3200999905,78cyclictest0-21swapper/321:40:453
3200999904,80cyclictest0-21swapper/321:50:523
3200999904,79cyclictest9358-21tail21:15:463
3200999904,79cyclictest0-21swapper/322:04:503
3200999904,79cyclictest0-21swapper/321:32:583
3200999903,79cyclictest36351ssh20:20:313
3200999903,79cyclictest36351ssh20:20:313
3200999903,79cyclictest20521-21munin-node20:45:553
3200899904,78cyclictest0-21swapper/220:56:032
3200899903,82cyclictest0-21swapper/218:36:042
3200899903,63cyclictest28325-21switchtime22:30:542
3200999895,52cyclictest32055-21forks19:25:483
3200999894,78cyclictest29640-21runrttasks22:34:323
3200999894,78cyclictest0-21swapper/320:19:233
3200999894,78cyclictest0-21swapper/318:40:533
3200999893,79cyclictest15220-21munin-node21:25:533
3200899894,79cyclictest0-21swapper/220:23:242
3200899894,79cyclictest0-21swapper/220:23:242
3200899894,78cyclictest0-21swapper/222:27:202
3200899893,73cyclictest31070-21df_abs21:00:412
3200899893,54cyclictest6404-21netstat21:10:532
3200999884,77cyclictest0-21swapper/321:10:453
3200999883,77cyclictest1287-21snmpd17:16:403
32009998818,64cyclictest28837-21switchtime20:55:583
3200899883,76cyclictest26211-21grep21:40:452
3200999874,77cyclictest0-21swapper/322:45:233
3200999874,76cyclictest25226-21unixbench_singl22:25:573
3200999874,53cyclictest28014-21runrttasks20:07:053
3200999873,77cyclictest9341-21expr19:40:443
3200999873,77cyclictest9341-21expr19:40:443
3200999873,76cyclictest31697-21taskset21:50:013
3200899873,78cyclictest0-21swapper/219:47:382
3200899873,77cyclictest0-21swapper/219:27:572
3200999864,75cyclictest19090-21ssh22:17:583
3200999863,76cyclictest4306-21ssh19:32:333
3200999863,76cyclictest223511ssh22:22:353
3200999863,75cyclictest25742-21munin-run18:10:333
3200999863,75cyclictest104041munin-node20:30:583
3200899864,75cyclictest22176-21kworker/2:121:50:522
3200899863,77cyclictest0-21swapper/220:35:002
3200899863,77cyclictest0-21swapper/220:00:222
3200899863,76cyclictest24504-21diskmemload22:01:292
3200899863,76cyclictest16137-21head21:25:452
3200899863,76cyclictest14199-21ssh20:35:432
3200899863,76cyclictest0-21swapper/219:25:302
3200899862,77cyclictest188011ssh19:55:262
32007998634,13cyclictest0-21swapper/120:05:281
3200999854,75cyclictest4779-21runrttasks21:10:153
3200999854,75cyclictest4779-21runrttasks21:10:153
3200999853,75cyclictest4504-21ntp_offset18:31:003
3200899853,76cyclictest0-21swapper/222:41:132
3200899853,76cyclictest0-21swapper/221:15:552
3200899853,76cyclictest0-21swapper/219:42:132
3200899853,76cyclictest0-21swapper/219:42:132
3200899853,75cyclictest3833-21fschecks_count21:55:432
3200899853,75cyclictest307261ssh20:10:582
3200899853,75cyclictest0-21swapper/221:31:222
3200899853,74cyclictest30717-21ssh22:35:392
3200899853,74cyclictest30717-21ssh22:35:392
3200999844,74cyclictest0-21swapper/322:35:523
3200999844,74cyclictest0-21swapper/322:35:523
3200999844,73cyclictest14193-21wc18:51:023
3200999843,75cyclictest8642-21timerandwakeup17:31:063
3200999843,75cyclictest8642-21timerandwakeup17:31:063
3200999843,74cyclictest26646-21ssh20:54:103
3200999843,74cyclictest24928-21ssh20:03:103
3200999843,65cyclictest12129-21cpuspeed_turbos21:20:403
3200899843,75cyclictest15233-21ssh22:11:142
3200899843,73cyclictest0-21swapper/222:24:592
3200899843,66cyclictest12469-21fschecks_count21:20:442
3200899843,66cyclictest1056-21gltestperf20:15:482
3200899843,64cyclictest10164-21awk18:45:322
3200899843,59cyclictest10542-21tune2fs22:05:422
3200899843,50cyclictest0-21swapper/220:25:472
3200899843,50cyclictest0-21swapper/220:25:472
3200899842,63cyclictest2389-21grep21:05:452
3200899842,63cyclictest2389-21grep21:05:452
3200999835,72cyclictest0-21swapper/321:04:443
3200999834,72cyclictest15352-21nscd18:05:333
3200999834,71cyclictest14080-21perl22:10:433
3200999833,73cyclictest17490-21tr20:40:443
3200899833,74cyclictest0-21swapper/222:15:412
3200899833,73cyclictest8185-21taskset17:30:572
3200899833,73cyclictest8185-21taskset17:30:572
3200899833,73cyclictest0-21swapper/218:10:332
3200899833,70cyclictest0-21swapper/218:45:432
3200899832,49cyclictest25387-21grep19:15:482
3200999824,72cyclictest0-21swapper/319:00:503
3200999824,72cyclictest0-21swapper/319:00:503
3200999823,72cyclictest30819-21ssh19:24:273
3200999823,59cyclictest13711-21open_inodes19:45:583
3200899823,73cyclictest0-21swapper/219:33:342
3200899823,72cyclictest0-21swapper/220:51:392
3200899823,71cyclictest0-21swapper/219:37:362
3200899823,62cyclictest0-21swapper/218:01:022
3200999814,71cyclictest0-21swapper/318:00:353
3200999814,70cyclictest1686-21nscd17:25:353
3200999814,70cyclictest0-21swapper/317:40:553
3200999814,68cyclictest6472-21gltestperf18:35:533
3200899813,72cyclictest0-21swapper/217:18:322
3200899813,71cyclictest30667-21ssh21:46:172
3200899812,71cyclictest0-21swapper/217:52:262
3200999803,69cyclictest10173-21awk22:05:383
3200899803,70cyclictest0-21swapper/220:02:312
3200899803,70cyclictest0-21swapper/218:25:442
3200999795,66cyclictest11990-21timerwakeupswit18:46:043
3200999794,70cyclictest0-21swapper/318:20:463
3200999794,69cyclictest0-21swapper/317:30:363
3200899793,70cyclictest0-21swapper/221:35:452
3200999783,68cyclictest27227-21ntp_kernel_pll_18:10:593
3200899784,60cyclictest0-21swapper/218:15:472
3200899784,60cyclictest0-21swapper/218:15:472
3200899783,69cyclictest0-21swapper/217:20:552
3200999774,67cyclictest16080-21awk18:55:523
3200999774,67cyclictest16080-21awk18:55:523
3200999773,69cyclictest0-21swapper/318:25:463
3200899773,69cyclictest0-21swapper/217:25:512
3200999754,65cyclictest0-21swapper/318:18:193
3200999754,65cyclictest0-21swapper/318:18:193
3200999754,64cyclictest23785-21munin-node19:11:003
3200899753,66cyclictest0-21swapper/218:55:532
3200899753,66cyclictest0-21swapper/218:55:532
3200899753,65cyclictest1287-21snmpd19:03:502
3200899753,65cyclictest1287-21snmpd19:03:502
3200899753,43cyclictest0-21swapper/219:05:472
3200999744,63cyclictest0-21swapper/317:40:363
3200899743,65cyclictest0-21swapper/217:35:522
3200899743,63cyclictest22649-21tr19:10:442
3200999733,64cyclictest17224-21diskstats17:50:513
3200899733,64cyclictest0-21swapper/217:46:052
3200899733,53cyclictest21179-21sshd18:00:302
3200899733,39cyclictest0-21swapper/218:50:452
3200899723,47cyclictest0-21swapper/218:31:002
3200899723,42cyclictest13999-21chrt17:45:242
248599720,5rtkit-daemon2484-21rtkit-daemon18:45:530
3200999713,61cyclictest1419-21runrttasks19:06:383
3200899713,62cyclictest27728-21taskset18:11:062
32008997121,45cyclictest1419-21runrttasks18:21:252
248599700,5rtkit-daemon2484-21rtkit-daemon21:42:460
3200999683,57cyclictest14897-21fschecks_count17:45:523
3200699664,55cyclictest877-21needreboot21:50:520
32006996628,21cyclictest20840-21grep19:56:020
3200999634,53cyclictest0-21swapper/317:15:363
248599630,4rtkit-daemon2484-21rtkit-daemon20:15:570
248599630,4rtkit-daemon2484-21rtkit-daemon19:03:370
248599630,4rtkit-daemon2484-21rtkit-daemon19:03:370
3200899624,52cyclictest0-21swapper/217:15:352
32007996145,10cyclictest10686-21cpuspeed_turbos18:45:461
32006996120,13cyclictest0-21swapper/018:40:470
3200699595,40cyclictest0-21swapper/022:40:180
3200699595,40cyclictest0-21swapper/022:40:180
3200699594,48cyclictest0-21swapper/020:20:480
3200699594,48cyclictest0-21swapper/020:20:480
3200699594,40cyclictest0-21swapper/021:21:090
3200699594,39cyclictest0-21swapper/019:22:500
32006995921,20cyclictest0-21swapper/020:46:440
32006995916,27cyclictest0-21swapper/022:27:290
248599590,4rtkit-daemon2484-21rtkit-daemon21:58:150
83602583,10sleep00-21swapper/020:25:560
83602583,10sleep00-21swapper/020:25:560
321732582,49sleep10-21swapper/122:36:271
321732582,49sleep10-21swapper/122:36:271
32006995821,19cyclictest2778-21ssh21:05:500
32006995821,19cyclictest2778-21ssh21:05:500
75072573,12sleep00-21swapper/019:36:390
32007995725,18cyclictest32130-21munin-node21:00:561
305662574,9sleep00-21swapper/021:46:000
248599570,7rtkit-daemon2484-21rtkit-daemon17:26:040
248599570,6rtkit-daemon2484-21rtkit-daemon20:40:560
173742573,47sleep00-21swapper/021:26:580
57802562,47sleep10-21swapper/121:58:391
3200799563,26cyclictest27166-21ssh19:18:061
3200699565,13cyclictest0-21swapper/021:35:480
3200699565,12cyclictest0-21swapper/022:05:360
3200699564,13cyclictest0-21swapper/022:30:560
3200699564,13cyclictest0-21swapper/019:30:430
32006995625,13cyclictest0-21swapper/018:20:330
32006995625,13cyclictest0-21swapper/018:20:330
248599560,6rtkit-daemon2484-21rtkit-daemon18:51:080
173132563,10sleep00-21swapper/019:00:270
173132563,10sleep00-21swapper/019:00:270
165452566,42sleep10-21swapper/120:39:151
32007995522,19cyclictest14561-21ssh19:47:181
3200699554,28cyclictest0-21swapper/022:40:440
3200699554,12cyclictest0-21swapper/020:50:510
3200699553,47cyclictest0-21swapper/022:20:050
32006995511,26cyclictest0-21swapper/020:11:420
287272552,47sleep10-21swapper/122:31:151
248599550,6rtkit-daemon2484-21rtkit-daemon17:17:260
241452553,8sleep124250-21munin-node18:05:511
230632553,44sleep00-21swapper/018:02:120
220142553,10sleep00-21swapper/019:09:180
216932557,41sleep10-21swapper/121:35:251
3200799543,25cyclictest0-21swapper/120:11:051
3200699545,32cyclictest0-21swapper/021:17:120
3200699545,14cyclictest0-21swapper/021:10:440
3200699544,15cyclictest0-21swapper/022:14:110
3200699543,21cyclictest0-21swapper/020:00:420
32006995424,10cyclictest30281-21ssh21:00:110
248599540,6rtkit-daemon2484-21rtkit-daemon21:00:480
248599540,5rtkit-daemon2484-21rtkit-daemon21:49:231
248599540,5rtkit-daemon2484-21rtkit-daemon19:10:520
248599540,5rtkit-daemon2484-21rtkit-daemon17:55:470
3200799539,27cyclictest0-21swapper/121:15:431
32007995321,20cyclictest0-21swapper/122:09:381
32007995318,30cyclictest0-21swapper/121:08:431
32007995318,30cyclictest0-21swapper/121:08:431
3200699535,29cyclictest0-21swapper/020:05:510
3200699534,44cyclictest0-21swapper/019:48:550
3200699534,30cyclictest0-21swapper/019:50:420
3200699534,29cyclictest0-21swapper/022:20:360
3200699534,13cyclictest0-21swapper/019:25:350
32006995323,12cyclictest0-21swapper/018:25:330
32006995322,12cyclictest0-21swapper/018:36:030
32006995321,14cyclictest0-21swapper/019:43:160
32006995321,14cyclictest0-21swapper/019:43:160
279792533,44sleep00-21swapper/018:13:480
248599530,5rtkit-daemon2484-21rtkit-daemon20:00:131
42432522,9sleep10-21swapper/122:45:101
32007995235,11cyclictest29114-21gltestperf19:20:511
32007995230,7cyclictest13384-21rm21:20:581
32007995221,9cyclictest8612-21taskset22:01:411
3200699525,29cyclictest0-21swapper/017:20:520
3200699525,28cyclictest0-21swapper/017:50:480
32006995220,13cyclictest0-21swapper/022:00:440
248599520,5rtkit-daemon2484-21rtkit-daemon19:19:590
248599520,5rtkit-daemon2484-21rtkit-daemon17:50:180
236322523,43sleep10-21swapper/120:50:271
210532523,10sleep00-21swapper/021:33:060
17672522,43sleep10-21swapper/117:18:041
100162523,7sleep110020-21ssh20:30:041
100162523,7sleep110020-21ssh20:30:041
3200799514,23cyclictest0-21swapper/120:30:521
32007995129,10cyclictest2002-21unixbench_multi20:16:031
32007995118,20cyclictest19031-21vmstat20:41:051
3200699519,23cyclictest25747-21awk18:10:330
3200699515,27cyclictest0-21swapper/020:30:490
3200699514,28cyclictest0-21swapper/020:35:440
3200699514,28cyclictest0-21swapper/018:31:060
32006995122,12cyclictest0-21swapper/017:41:040
138762512,42sleep10-21swapper/117:43:471
115412513,10sleep00-21swapper/017:39:270
3200799504,39cyclictest0-21swapper/120:55:331
32007995033,11cyclictest0-21swapper/117:47:061
3200799503,39cyclictest0-21swapper/122:10:521
32007995027,9cyclictest10119-21sort19:40:541
32007995027,9cyclictest10119-21sort19:40:541
32007995023,11cyclictest26588-21needreboot21:40:501
32007995022,13cyclictest0-21swapper/119:30:481
3200699504,27cyclictest0-21swapper/017:30:570
3200699504,27cyclictest0-21swapper/017:30:570
99212495,10sleep10-21swapper/118:42:351
66362493,40sleep10-21swapper/117:27:431
32007994926,10cyclictest32127-21telnet21:50:381
32007994922,12cyclictest0-21swapper/122:25:241
3200699494,27cyclictest0-21swapper/018:25:460
186652494,10sleep10-21swapper/117:53:551
326462482,39sleep10-21swapper/118:22:241
3200799483,11cyclictest0-21swapper/120:50:451
32007994823,8cyclictest5466-21cat18:35:321
304842482,40sleep10-21swapper/118:20:221
304842482,40sleep10-21swapper/118:20:221
279182482,40sleep10-21swapper/118:13:011
248599480,4rtkit-daemon2484-21rtkit-daemon17:10:490
3200799475,27cyclictest0-21swapper/119:30:191
32007994727,8cyclictest1287-21snmpd19:37:031
32007994723,11cyclictest0-21swapper/121:10:511
32007994715,26cyclictest15220-21munin-node21:25:461
258092473,37sleep10-21swapper/120:05:331
210062472,39sleep10-21swapper/117:58:181
3200799464,26cyclictest0-21swapper/122:15:451
3200799464,11cyclictest0-21swapper/121:35:381
32007994631,10cyclictest10346-21gltestperf17:35:551
3200799463,30cyclictest0-21swapper/119:52:071
240742462,37sleep10-21swapper/119:11:041
230072462,38sleep10-21swapper/118:01:281
196532462,38sleep10-21swapper/119:04:521
196532462,38sleep10-21swapper/119:04:521
3200799453,37cyclictest24504-21diskmemload22:25:571
75742444,10sleep10-21swapper/118:38:031
32007994417,13cyclictest0-21swapper/119:00:321
32007994417,13cyclictest0-21swapper/119:00:321
3200799434,33cyclictest0-21swapper/117:21:061
32007994324,8cyclictest7357-21ls17:30:471
32007994324,8cyclictest7357-21ls17:30:471
32007994322,10cyclictest0-21swapper/118:52:291
32007994220,10cyclictest683-21aten_r1power_cu18:25:451
32007994117,11cyclictest0-21swapper/119:05:571
3200799372,30cyclictest0-21swapper/117:15:361
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional