You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 11:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun May 03, 2026 00:43:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1166953995747,7cyclictest1339241-21ssh00:00:011
1166953995329,15cyclictest1199945-21idleruntime-cro21:00:011
1166953995040,6cyclictest1166963-21systemd-run19:10:011
1166953994919,8cyclictest1240239-21ntpq21:55:461
1166954994738,5cyclictest1243010-21idleruntime-cro22:00:012
1166953994739,5cyclictest421-21dbus-daemon22:10:021
1166953994718,11cyclictest1286180-21ssh22:53:261
1166954994638,5cyclictest1234997-21idleruntime-cro21:50:012
1166954994635,4cyclictest0-21swapper/200:25:512
1166952994633,3cyclictest0-21swapper/021:25:370
1166952994633,3cyclictest0-21swapper/021:25:370
1166952994629,14cyclictest1210990-21idleruntime-cro21:20:010
1166954994538,4cyclictest0-21swapper/223:05:482
1166954994532,9cyclictest420-21avahi-daemon21:44:552
1166953994531,9cyclictest3572428-21cups-browsed22:00:421
1166953994517,11cyclictest1333925-21ssh23:52:351
1166952994522,16cyclictest151ktimers/019:25:470
1166953994436,4cyclictest1231003-21kworker/u8:4-xprtiod22:15:551
1166953994432,8cyclictest261-21systemd-journal22:30:011
1166953994420,8cyclictest1221183-21ssh21:31:331
1166953994420,8cyclictest1221183-21ssh21:31:331
1166953994418,8cyclictest1253626-21sh22:12:181
1166953994418,7cyclictest1316529-21proc_pri23:30:531
1166953994417,8cyclictest1327317-21latency_hist23:45:021
1166953994417,8cyclictest1272242-21ntpq22:35:461
1166953994416,8cyclictest1190343-21ntpq20:25:441
1166953994414,9cyclictest1205053-21ssh21:11:201
1166952994436,5cyclictest1365272-21grep00:30:590
1166952994434,6cyclictest151ktimers/020:45:520
1166952994423,14cyclictest151ktimers/020:00:470
116695499436,20cyclictest1055-21mta-sts-daemon19:40:472
1166954994336,3cyclictest0-21swapper/200:10:492
1166954994335,5cyclictest0-21swapper/223:15:392
1166954994335,4cyclictest0-21swapper/223:35:492
1166954994334,6cyclictest1339201-21ssh00:00:002
1166954994331,8cyclictest1258991-21systemctl22:20:002
1166953994335,5cyclictest1275818-21fschecks_count22:40:381
1166953994321,14cyclictest1280311-21ssh22:45:481
1166953994321,14cyclictest1280311-21ssh22:45:481
1166953994319,7cyclictest1225802-21sh21:37:461
1166952994326,5cyclictest1291714-21aten_r2power_vo23:00:350
1166953994233,6cyclictest1348770-21proc_pri00:10:531
1166953994228,8cyclictest1178330-21ntpq19:45:471
1166953994216,8cyclictest1352596-21ld00:15:471
1166953994216,10cyclictest1330118-21sh23:47:591
1166953994215,9cyclictest1218117-21ssh21:28:201
1166953994215,9cyclictest1218117-21ssh21:28:201
1166952994233,6cyclictest421-21dbus-daemon21:30:470
1166952994233,6cyclictest421-21dbus-daemon21:30:470
116695299422,35cyclictest1252291-21ssh22:10:460
1166954994131,6cyclictest0-21swapper/223:30:472
1166953994125,12cyclictest1304138-21missed_timers23:15:411
1166953994118,7cyclictest1362797-21sh00:28:551
1166953994118,7cyclictest1289058-21sh22:56:121
1166953994117,8cyclictest1357734-21sh00:21:511
1166953994117,8cyclictest1229162-21sh21:41:271
1166953994116,8cyclictest1208238-21ld21:15:471
1166952994118,16cyclictest12350irq/32-eth022:30:450
1166954994033,4cyclictest1335191-21cron23:55:012
1166954994031,6cyclictest3572428-21cups-browsed23:49:192
1166954994031,4cyclictest0-21swapper/219:35:502
1166954994031,4cyclictest0-21swapper/219:35:502
116695499401,32cyclictest1260376-21sh22:20:522
1166953994027,8cyclictest1344529-21ntpq00:05:461
1166953994026,9cyclictest1170799-21ntpq19:20:471
1166953994026,8cyclictest1312464-21ntpq23:25:501
1166953994026,8cyclictest1312464-21ntpq23:25:501
1166953994024,9cyclictest1308460-21ntpq23:20:511
1166953994024,9cyclictest1308460-21ntpq23:20:511
1166953994017,7cyclictest1367317-21sh00:34:531
1166953994016,9cyclictest1322145-21diskmemload23:38:081
1166953994014,6cyclictest1184343-21ntpq20:05:471
1166953994012,8cyclictest1194896-21ntpq20:40:481
1166952994032,5cyclictest1307139-21latency_hist23:20:020
1166952994029,7cyclictest1243735-21cpu22:00:360
1166952994029,6cyclictest12350irq/32-eth023:30:520
1166953993926,8cyclictest1368789-21ntpq00:35:511
1166953993926,8cyclictest1197799-21ntpq20:50:451
1166953993926,8cyclictest1176801-21ntpq19:40:441
1166953993925,8cyclictest1187307-21ntpq20:15:441
1166953993916,7cyclictest1295441-21sh23:05:201
1166953993915,8cyclictest1233064-21ssh21:46:161
1166952993925,10cyclictest151ktimers/020:55:460
1166954993830,5cyclictest1-21systemd21:25:022
1166954993830,4cyclictest1051-21mta-sts-daemon23:41:382
1166953993826,7cyclictest1167819-21ntpq19:10:451
1166953993825,8cyclictest1175409-21ntpq19:35:501
1166953993825,8cyclictest1175409-21ntpq19:35:501
1166953993824,11cyclictest1302326-21sshd23:13:371
1166953993817,7cyclictest1193092-21kernelversion20:35:411
1166953993814,7cyclictest0-21swapper/119:55:431
1166953993814,10cyclictest1341410-21rm00:01:231
1166952993832,4cyclictest1051-21mta-sts-daemon20:30:480
1166954993729,5cyclictest1283020-21diskmemload22:50:022
1166954993729,5cyclictest1283020-21diskmemload22:50:022
1166954993729,5cyclictest1180450-21systemd-run19:55:022
1166954993729,4cyclictest0-21swapper/221:25:542
1166954993729,4cyclictest0-21swapper/221:25:542
116695499372,30cyclictest1284261-21ssh22:50:462
1166953993728,6cyclictest1191861-21ntpq20:30:481
1166953993726,7cyclictest1202295-21ntpq21:05:451
1166953993726,7cyclictest1188809-21ntpq20:20:431
1166953993725,8cyclictest3572428-21cups-browsed21:22:091
1166953993725,8cyclictest1268286-21ntpq22:30:481
1166953993724,8cyclictest1200806-21ntpq21:00:461
1166953993723,9cyclictest1196003-21/usr/sbin/munin20:45:411
1166953993722,11cyclictest1235396-21ssh21:50:221
1166953993715,8cyclictest1261498-21sh22:22:051
1166952993730,4cyclictest1300889-21unixbench-2d23:10:590
1166952993728,6cyclictest3572428-21cups-browsed23:59:570
1166952993728,6cyclictest1311577-21/usr/sbin/munin23:25:570
1166952993728,6cyclictest1311577-21/usr/sbin/munin23:25:570
1166954993628,5cyclictest1267193-21latency_hist22:30:032
1166954993627,6cyclictest421-21dbus-daemon21:40:012
1166953993625,7cyclictest1182856-21ntpq20:00:481
1166953993625,7cyclictest1173802-21ntpq19:30:451
1166953993625,7cyclictest1173802-21ntpq19:30:451
1166953993616,8cyclictest1293243-21sh23:01:301
1166953993612,8cyclictest1172359-21ntpq19:25:471
1166952993631,2cyclictest0-21swapper/023:40:010
1166952993630,4cyclictest1351356-21ssh00:15:010
1166952993630,3cyclictest1243010-21awk22:00:010
1166952993629,4cyclictest1175947-21sendmail19:40:000
1166952993629,4cyclictest1175947-21sendmail19:40:000
116695499355,21cyclictest1368606-21sh00:35:442
116695499353,25cyclictest1204227-21ssh21:10:482
1166954993528,3cyclictest0-21swapper/221:10:012
1166954993528,3cyclictest0-21swapper/200:00:522
1166954993527,3cyclictest0-21swapper/223:10:472
1166954993523,9cyclictest1174443-21cut19:35:012
1166954993523,9cyclictest1174443-21cut19:35:012
1166953993525,6cyclictest1185814-21ntpq20:10:451
1166953993524,8cyclictest1168961-21/usr/sbin/munin19:15:401
1166953993524,7cyclictest1179843-21ntpq19:50:471
1166952993529,4cyclictest1327176-21systemd-run23:45:010
1166952993528,5cyclictest1202931-21diskmemload22:25:560
1166952993528,5cyclictest1054-21mta-sts-daemon00:08:270
1166952993527,5cyclictest1271487-21/usr/sbin/munin22:36:000
1166952993526,6cyclictest1308320-21ssh23:20:460
1166952993526,6cyclictest1308320-21ssh23:20:460
1166952993526,6cyclictest1284674-21timerandwakeup22:50:560
1166952993523,7cyclictest1-21systemd00:40:010
1166952993521,12cyclictest12350irq/32-eth000:25:510
116695499348,19cyclictest1276382-21ssh22:40:502
1166954993429,3cyclictest0-21swapper/219:20:012
1166954993426,5cyclictest1247016-21cron22:05:012
1166954993425,4cyclictest0-21swapper/221:50:542
1166954993423,8cyclictest1195442-21/usr/sbin/munin20:45:022
116695499342,26cyclictest1292428-21ssh23:00:502
1166952993428,4cyclictest1355355-21cron00:20:010
1166952993428,3cyclictest464458-21kworker/u8:8-nfsiod19:20:010
1166952993427,4cyclictest0-21swapper/022:20:000
116695299342,25cyclictest1332529-21ssh23:50:500
116695499337,20cyclictest1344628-21ssh00:05:492
116695499333,25cyclictest1364678-21ssh00:30:492
1166954993324,5cyclictest1201353-21unixbench_singl21:00:582
116695499332,25cyclictest1272272-21ssh22:35:472
116695499331,26cyclictest1288409-21ssh22:55:522
1166952993329,2cyclictest0-21swapper/022:55:590
1166952993326,4cyclictest1181964-21systemd-run20:00:020
1166952993320,7cyclictest1054-21mta-sts-daemon21:35:550
116695499326,20cyclictest1308488-21sh23:20:532
116695499326,20cyclictest1308488-21sh23:20:532
1166954993223,4cyclictest0-21swapper/221:30:452
1166954993223,4cyclictest0-21swapper/221:30:452
1166954993222,5cyclictest0-21swapper/222:10:522
116695299323,25cyclictest1260247-21ssh22:20:470
116695299323,21cyclictest1236307-21ssh21:50:490
1166952993226,3cyclictest1183738-21aten2.4-expect20:05:360
1166952993225,4cyclictest1201961-21fschecks_time21:05:390
1166952993224,6cyclictest1168431-21cron19:15:010
1166954993126,2cyclictest0-21swapper/221:00:012
1166954993123,5cyclictest3572428-21cups-browsed20:00:382
1166954993123,5cyclictest1247569-21aten_r2power_cu22:05:352
1166954993123,4cyclictest0-21swapper/219:10:462
116695299314,20cyclictest1280219-21ssh22:45:450
116695299314,20cyclictest1280219-21ssh22:45:450
1166952993125,2cyclictest0-21swapper/000:25:010
116695299312,22cyclictest1212360-21ssh21:20:490
1166952993121,7cyclictest3572428-21cups-browsed22:44:220
1166952993114,13cyclictest1-21systemd23:10:010
116695499304,20cyclictest1202931-21diskmemload00:20:482
116695499303,19cyclictest428-21mta-sts-daemon20:30:452
1166954993022,5cyclictest1270975-21ssh22:35:012
116695299302,24cyclictest1232327-21ssh21:45:490
116695299302,22cyclictest1248290-21sh22:05:470
1166952993015,11cyclictest12350irq/32-eth000:00:490
116695299301,26cyclictest1207056-21cut21:15:020
1166954992924,2cyclictest0-21swapper/220:40:012
1166954992922,3cyclictest0-21swapper/200:15:362
116695499292,21cyclictest1312295-21sh23:25:442
116695499292,21cyclictest1312295-21sh23:25:442
1166954992920,6cyclictest1190931-21sh20:30:012
1166952992921,5cyclictest1197226-21aten_r2power_en20:50:350
1166952992919,8cyclictest261-21systemd-journal19:30:380
1166952992919,8cyclictest261-21systemd-journal19:30:380
1166952992918,8cyclictest0-21swapper/020:20:010
1166954992822,3cyclictest0-21swapper/221:15:462
1166954992820,5cyclictest1180821-21aten2.4-expect19:55:372
1166954992820,5cyclictest1170284-21aten2.4-expect19:20:372
1166952992820,5cyclictest421-21dbus-daemon23:46:230
1166952992820,4cyclictest0-21swapper/021:40:530
116695499272,3cyclictest171rcu_preempt20:05:522
1166954992722,2cyclictest0-21swapper/220:55:012
1166952992720,5cyclictest516-21in:imuxsock19:45:000
1166952992718,6cyclictest1189368-21unixbench_singl20:20:560
1166954992618,5cyclictest3572428-21cups-browsed20:17:322
1166954992518,3cyclictest0-21swapper/220:10:392
1166954992517,5cyclictest1177899-21cpuspeed19:45:382
1166954992516,6cyclictest3572428-21cups-browsed19:28:192
1166954992516,6cyclictest1189410-21vmstat20:20:572
116695299252,18cyclictest3572428-21cups-browsed19:45:460
1166952992519,4cyclictest1051-21mta-sts-daemon20:39:220
1166954992417,4cyclictest1196921-21cron20:50:012
1166952992413,8cyclictest420-21avahi-daemon21:00:410
1166952992411,10cyclictest0-21swapper/019:20:440
1166952992315,6cyclictest1190717-21timerwakeupswit20:25:530
116695299230,8cyclictest1194744-21needreboot20:40:410
116695299221,18cyclictest1185469-21/usr/sbin/munin20:10:380
1166954992116,3cyclictest0-21swapper/219:10:022
116695299212,14cyclictest1179795-21ntpq19:50:450
116695599201,9cyclictest0-21swapper/321:30:463
116695599201,9cyclictest0-21swapper/321:30:463
116695299202,14cyclictest1166907-21munin-run19:10:010
116695599183,9cyclictest0-21swapper/322:10:483
116695599182,9cyclictest0-21swapper/323:00:433
116695599182,9cyclictest0-21swapper/322:30:443
116695599173,8cyclictest0-21swapper/321:25:493
116695599173,8cyclictest0-21swapper/321:25:493
116695599172,8cyclictest0-21swapper/322:35:473
116695599172,8cyclictest0-21swapper/321:50:463
116695599163,7cyclictest0-21swapper/322:50:503
116695599162,8cyclictest0-21swapper/321:45:453
116695599162,8cyclictest0-21swapper/321:10:463
116695599162,8cyclictest0-21swapper/300:15:483
116695599161,8cyclictest0-21swapper/322:25:453
116695599154,7cyclictest0-21swapper/322:05:443
116695599153,6cyclictest0-21swapper/323:45:493
116695599152,8cyclictest0-21swapper/323:20:493
116695599152,8cyclictest0-21swapper/323:20:493
116695599152,8cyclictest0-21swapper/323:15:443
116695599152,8cyclictest0-21swapper/322:00:403
116695599152,8cyclictest0-21swapper/321:15:473
116695599152,8cyclictest0-21swapper/300:10:463
116695599152,7cyclictest0-21swapper/323:05:453
116695599152,7cyclictest0-21swapper/322:20:523
116695599152,7cyclictest0-21swapper/320:40:513
116695599152,7cyclictest0-21swapper/300:25:503
116695599152,6cyclictest0-21swapper/320:05:453
116695599152,6cyclictest0-21swapper/319:50:453
116695599152,5cyclictest0-21swapper/320:30:473
116695599152,5cyclictest0-21swapper/320:15:453
116695599151,8cyclictest0-21swapper/323:40:483
116695599151,8cyclictest0-21swapper/323:30:453
116695599151,8cyclictest0-21swapper/322:55:483
116695599151,8cyclictest0-21swapper/300:00:453
116695599151,7cyclictest0-21swapper/321:40:493
116695599151,7cyclictest0-21swapper/321:05:503
116695599144,7cyclictest0-21swapper/323:55:503
116695599143,8cyclictest0-21swapper/321:55:373
116695599143,6cyclictest0-21swapper/319:35:503
116695599143,6cyclictest0-21swapper/319:35:503
116695599142,7cyclictest0-21swapper/323:35:393
116695599142,7cyclictest0-21swapper/323:25:433
116695599142,7cyclictest0-21swapper/323:25:433
116695599142,6cyclictest0-21swapper/319:45:463
116695599142,5cyclictest0-21swapper/321:00:483
116695599142,4cyclictest0-21swapper/320:45:453
116695599141,8cyclictest0-21swapper/322:15:563
116695599141,8cyclictest0-21swapper/300:35:443
116695599141,7cyclictest0-21swapper/319:20:483
116695599141,6cyclictest0-21swapper/319:30:463
116695599141,6cyclictest0-21swapper/319:30:463
116695599141,5cyclictest0-21swapper/319:10:443
116695599135,6cyclictest0-21swapper/300:05:503
116695599133,7cyclictest0-21swapper/321:35:363
116695599133,7cyclictest0-21swapper/300:20:483
116695599133,4cyclictest0-21swapper/319:55:473
116695599132,6cyclictest0-21swapper/322:45:363
116695599132,6cyclictest0-21swapper/322:45:363
116695599132,5cyclictest0-21swapper/320:20:433
116695599131,7cyclictest0-21swapper/322:40:443
116695599131,7cyclictest0-21swapper/321:20:373
116695599131,7cyclictest0-21swapper/300:30:473
116695599131,6cyclictest0-21swapper/320:55:433
116695599131,6cyclictest0-21swapper/320:50:493
116695599131,6cyclictest0-21swapper/320:25:423
116695599131,6cyclictest0-21swapper/319:25:483
116695599131,6cyclictest0-21swapper/319:15:483
116695599122,6cyclictest0-21swapper/323:10:463
116695599121,7cyclictest0-21swapper/323:50:423
116695599121,6cyclictest0-21swapper/320:35:463
116695599121,6cyclictest0-21swapper/320:00:473
116695599121,5cyclictest0-21swapper/319:40:443
116695599101,5cyclictest0-21swapper/320:10:393
116695599101,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional