You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 07:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Tue Aug 11, 2026 00:43:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1294124996049,8cyclictest1340210-21if_eth021:25:482
129412399532,36cyclictest1435412-21cron00:01:011
129412399532,36cyclictest1435412-21cron00:01:011
1294123994825,13cyclictest271ktimers/120:10:581
1294124994738,6cyclictest1296366-21/usr/sbin/munin19:15:562
1294122994736,7cyclictest0-21swapper/000:40:010
1294124994632,11cyclictest265-21systemd-journal19:55:012
1294122994636,5cyclictest0-21swapper/022:36:010
1294122994636,5cyclictest0-21swapper/022:36:010
1294124994537,5cyclictest1298644-21systemd-run19:25:012
1294123994537,5cyclictest1328640-21systemd-run21:05:011
1294124994434,6cyclictest1348605-21systemd-run21:40:022
1294123994435,5cyclictest1397455-21munin-plugin-st23:00:011
1294123994433,8cyclictest1330114-21munin-plugin-st21:10:001
1294123994432,9cyclictest1458587-21/usr/sbin/munin00:40:021
1294123994423,17cyclictest1002-21mta-sts-daemon23:05:551
129412499438,6cyclictest171rcu_preempt23:25:552
1294124994329,11cyclictest1364954-21needreboot22:05:522
129412399433,31cyclictest1343670-21ssh21:31:021
1294122994333,5cyclictest151ktimers/019:41:010
1294122994330,9cyclictest0-21swapper/021:50:530
1294122994330,9cyclictest0-21swapper/021:50:530
1294124994235,4cyclictest1455724-21latency_hist00:35:032
1294124994234,5cyclictest1333274-21latency_hist21:15:022
1294122994235,4cyclictest1424938-21sh23:45:010
1294122994235,4cyclictest1424938-21sh23:45:010
1294123994133,4cyclictest1003-21mta-sts-daemon22:52:141
1294123994122,14cyclictest271ktimers/123:25:551
1294124994028,9cyclictest1334635-21mailstats21:16:062
129412499402,30cyclictest1374188-21ssh22:20:532
1294123994033,4cyclictest1357740-21ssh21:55:001
1294123994033,4cyclictest1357740-21ssh21:55:001
1294123994028,9cyclictest1388316-21cut22:45:011
129412299403,31cyclictest1429299-21ssh23:51:010
129412299402,26cyclictest1368055-21ssh22:10:520
1294124993931,5cyclictest1434196-40systemctl00:00:022
1294124993924,8cyclictest1410918-21sh23:20:572
1294122993933,4cyclictest1387023-21unixbench_multi22:41:120
1294122993931,5cyclictest1331887-21unixbench_singl21:11:060
1294122993930,5cyclictest151ktimers/023:45:580
1294122993930,5cyclictest151ktimers/023:45:580
129412499382,32cyclictest1362075-21ssh22:01:002
1294124993823,11cyclictest3791945-21cups-browsed19:40:012
1294124993823,11cyclictest3791945-21cups-browsed19:40:012
1294123993831,3cyclictest0-21swapper/100:30:491
1294123993830,5cyclictest3791945-21cups-browsed22:35:471
1294123993830,5cyclictest3791945-21cups-browsed22:35:471
1294123993830,4cyclictest0-21swapper/122:05:551
1294123993829,5cyclictest1351652-21munin-plugin-st21:45:011
1294122993829,6cyclictest0-21swapper/023:20:010
1294122993828,6cyclictest151ktimers/021:45:550
1294124993729,5cyclictest1344133-21unixbench-2d21:31:122
1294124993729,4cyclictest1416150-21kworker/u8:2-xprtiod23:31:072
129412399374,4cyclictest1410882-21ssh23:20:561
1294123993730,3cyclictest0-21swapper/121:50:021
1294123993730,3cyclictest0-21swapper/121:15:591
1294123993728,5cyclictest1431812-21fschecks_time23:55:461
1294122993730,3cyclictest0-21swapper/019:11:170
1294122993729,4cyclictest1338155-21sshd21:21:170
1294124993628,5cyclictest1370923-21ls22:15:502
1294124993626,7cyclictest421-21dbus-daemon23:05:012
1294124993626,7cyclictest421-21dbus-daemon23:05:012
1294124993626,6cyclictest1383968-21head22:36:092
1294124993626,6cyclictest1383968-21head22:36:092
1294123993629,4cyclictest1397962-21aten_r2power_cu23:00:431
1294123993629,4cyclictest1397962-21aten_r2power_cu23:00:431
1294123993629,4cyclictest1370000-21munin-plugin-st22:15:011
1294123993628,5cyclictest1374812-21grep22:21:071
1294123993628,5cyclictest1339530-21latency_hist21:25:031
1294123993627,6cyclictest1450749-21proc_pri00:26:001
1294124993528,4cyclictest1366958-21kworker/u8:14-xprtiod22:30:592
1294124993528,3cyclictest0-21swapper/222:46:042
1294124993526,5cyclictest0-21swapper/221:20:582
1294124993526,4cyclictest0-21swapper/223:40:472
1294124993526,4cyclictest0-21swapper/223:40:472
1294124993525,6cyclictest1-21systemd21:05:012
1294123993529,4cyclictest1294121-21cyclictest23:11:081
1294123993527,5cyclictest1306624-21fschecks_count19:50:481
1294123993526,6cyclictest421-21dbus-daemon23:18:271
1294123993524,8cyclictest1324945-21ntp_kernel_err20:50:531
1294123993524,8cyclictest1298025-21ntp_kernel_pll_19:20:571
129412299353,26cyclictest1358862-21ssh21:55:510
1294122993528,4cyclictest0-21swapper/023:10:530
1294122993527,6cyclictest1413227-21aten_r2power_cu23:25:440
1294122993527,5cyclictest1328634-21systemd-run21:05:010
1294122993522,8cyclictest1443226-21sh00:14:470
1294124993427,4cyclictest1366958-21kworker/u8:14-rpciod00:05:512
1294124993427,4cyclictest1315113-21idleruntime-cro20:20:012
1294124993426,5cyclictest1397458-21munin-run23:00:012
1294124993424,7cyclictest1356185-21switchtime21:51:052
1294124993424,7cyclictest1356185-21switchtime21:51:052
1294123993427,4cyclictest1419052-21latency_hist23:35:031
1294123993426,5cyclictest420-21avahi-daemon23:41:041
1294123993426,5cyclictest420-21avahi-daemon23:41:041
1294123993426,5cyclictest1420396-21threads23:36:051
1294122993428,4cyclictest1348569-21munin-run21:40:010
1294122993427,5cyclictest1322606-21cron20:45:010
1294122993426,5cyclictest1000-21mta-sts-daemon21:15:520
1294122993424,6cyclictest151ktimers/022:51:020
129412299342,24cyclictest1401732-21ssh23:05:560
1294122993422,10cyclictest1327040-21unixbench_singl20:56:040
1294122993414,13cyclictest12650irq/32-eth022:00:580
129412499332,25cyclictest1429192-21ssh23:50:562
129412399337,4cyclictest1447599-21ssh00:21:011
1294123993327,3cyclictest0-21swapper/100:15:011
1294123993325,5cyclictest1-21systemd20:35:011
1294123993323,7cyclictest1326466-21ntp_kernel_pll_20:55:531
1294123993323,6cyclictest420-21avahi-daemon22:47:121
129412299334,23cyclictest1377460-21ssh22:26:010
1294122993327,4cyclictest1435549-21/usr/sbin/munin00:01:050
1294122993327,4cyclictest1435549-21/usr/sbin/munin00:01:050
1294122993325,5cyclictest3791945-21cups-browsed19:30:470
1294122993325,5cyclictest3791945-21cups-browsed19:30:470
1294122993325,4cyclictest0-21swapper/022:56:010
1294122993323,8cyclictest0-21swapper/000:21:120
1294122993313,13cyclictest420-21avahi-daemon21:31:010
129412299331,29cyclictest0-21swapper/022:15:570
129412499323,19cyclictest1404797-21ssh23:10:582
1294124993226,4cyclictest1294121-21cyclictest23:06:072
1294124993226,3cyclictest0-21swapper/222:45:002
1294124993224,5cyclictest1437207-21sh00:05:012
1294124993224,5cyclictest1437207-21sh00:05:012
1294123993226,3cyclictest0-21swapper/122:00:011
1294123993225,4cyclictest0-21swapper/100:10:011
1294123993225,3cyclictest0-21swapper/122:05:011
1294123993224,5cyclictest1318124-21systemd-run20:30:011
1294123993224,5cyclictest1318124-21systemd-run20:30:011
1294122993227,3cyclictest1377995-21kworker/u8:9-rpciod23:25:010
1294122993224,3cyclictest0-21swapper/022:05:550
1294122993223,5cyclictest1419455-21tail23:35:440
1294122993222,7cyclictest1451154-21grep00:26:050
129412299322,23cyclictest1398782-21ssh23:01:010
129412299322,23cyclictest1398782-21ssh23:01:010
1294122993215,10cyclictest1438113-21/usr/sbin/munin00:05:530
129412499317,14cyclictest1001-21mta-sts-daemon20:55:542
1294124993124,4cyclictest1354894-21latency_hist21:50:032
1294124993124,4cyclictest1002-21mta-sts-daemon21:44:242
1294124993123,5cyclictest1309009-21unixbench_singl19:56:052
1294124993122,6cyclictest1325468-21sed20:51:062
129412399313,12cyclictest1346711-21ssh21:35:571
1294123993126,2cyclictest0-21swapper/120:05:011
1294123993125,2cyclictest0-21swapper/100:20:011
1294123993124,4cyclictest1003-21mta-sts-daemon22:19:331
1294123993120,8cyclictest1305424-21needreboot19:45:511
1294122993125,3cyclictest1000-21mta-sts-daemon20:05:570
1294122993124,3cyclictest0-21swapper/020:15:570
1294122993123,4cyclictest0-21swapper/022:46:040
129412499303,21cyclictest1419991-21ssh23:35:532
1294124993023,5cyclictest1358482-21ssh21:55:452
1294124993023,4cyclictest1447840-21mailstats00:21:082
1294124993023,4cyclictest1445025-21unixbench_multi00:16:082
1294124993023,4cyclictest1428149-21latency_hist23:50:022
1294124993023,4cyclictest1428149-21latency_hist23:50:022
1294124993023,4cyclictest1406656-21kworker/u8:15-rpciod23:16:342
1294124993023,4cyclictest1392434-21needreboot22:50:542
1294124993023,3cyclictest0-21swapper/200:15:022
1294124993022,5cyclictest1313592-21cron20:15:002
1294124993022,5cyclictest1310549-21unixbench_singl20:01:102
1294124993022,4cyclictest1000-21mta-sts-daemon00:25:522
1294124993022,4cyclictest0-21swapper/222:11:002
1294123993024,3cyclictest0-21swapper/120:10:011
1294123993024,3cyclictest0-21swapper/119:20:011
1294123993023,4cyclictest0-21swapper/119:55:551
1294123993022,5cyclictest1304533-21grep19:41:061
129412399302,22cyclictest1426210-21ssh23:46:011
129412399302,22cyclictest1426210-21ssh23:46:011
1294123993021,6cyclictest1429762-21unixbench_singl23:51:061
1294123993020,8cyclictest1320504-21ntp_kernel_pll_20:35:561
1294123993018,5cyclictest1323002-21aten_r2power_vo20:45:461
1294122993022,5cyclictest3791945-21cups-browsed21:40:130
1294122993018,9cyclictest1430842-21kworker/u8:0+xprtiod00:00:010
1294122993013,13cyclictest1380508-21processes22:31:000
129412499293,15cyclictest428-21mta-sts-daemon19:25:552
1294124992923,3cyclictest0-21swapper/200:35:582
1294123992923,2cyclictest0-21swapper/121:26:071
129412399292,24cyclictest1377364-21ssh22:25:581
1294123992919,8cyclictest1302484-21ntp_kernel_pll_19:35:521
1294123992919,8cyclictest1302484-21ntp_kernel_pll_19:35:521
1294122992924,2cyclictest0-21swapper/019:30:010
1294122992922,3cyclictest0-21swapper/023:30:560
1294122992922,3cyclictest0-21swapper/021:25:590
129412399283,21cyclictest1331310-21ssh21:10:561
1294123992820,5cyclictest1366958-21kworker/u8:14+xprtiod22:31:051
1294122992822,4cyclictest1319604-21munin-plugin-st20:35:000
1294122992822,4cyclictest1001-21mta-sts-daemon22:25:020
1294122992819,6cyclictest1330265-21latency_hist21:10:020
1294122992816,10cyclictest1324423-21aten_r2power_fr20:50:440
1294124992721,3cyclictest0-21swapper/219:45:482
1294124992720,3cyclictest0-21swapper/222:25:542
1294124992714,9cyclictest351ktimers/220:25:572
1294124992714,9cyclictest351ktimers/220:25:572
1294123992719,6cyclictest271ktimers/120:40:561
1294123992717,5cyclictest1315453-21aten_r2power_fr20:20:441
1294123992713,9cyclictest421-21dbus-daemon19:11:201
1294122992720,4cyclictest1316634-21systemd-run20:25:010
1294122992712,12cyclictest151ktimers/019:20:570
1294124992621,2cyclictest0-21swapper/219:15:012
1294124992619,4cyclictest1003-21mta-sts-daemon20:45:452
1294122992620,4cyclictest1453241-21fschecks_time00:30:470
1294122992619,4cyclictest1324032-21head20:46:070
1294122992617,7cyclictest1305456-21ntp_kernel_err19:45:530
1294122992616,7cyclictest1443982-21cpu00:15:460
1294122992613,10cyclictest0-21swapper/020:05:010
1294124992520,2cyclictest0-21swapper/220:21:052
1294124992517,5cyclictest1319600-21sh20:35:002
1294123992516,6cyclictest1315093-21cron20:20:011
1294123992516,6cyclictest1299267-21grep19:25:491
1294122992517,3cyclictest0-21swapper/019:35:510
1294122992517,3cyclictest0-21swapper/019:35:510
1294122992516,3cyclictest1307868-21aten_r2power_cu19:55:430
129412499243,16cyclictest1330135-21munin-run21:10:012
1294124992417,4cyclictest1300953-21ntp_kernel_err19:30:542
1294124992417,4cyclictest1300953-21ntp_kernel_err19:30:542
1294124992417,4cyclictest1002-21mta-sts-daemon20:36:002
1294124992416,4cyclictest1003-21mta-sts-daemon19:45:012
1294122992414,8cyclictest0-21swapper/019:20:010
1294122992316,5cyclictest3791945-21cups-browsed20:12:260
1294124992214,5cyclictest1311427-21needreboot20:05:522
129412299222,15cyclictest1320558-21ntp_states20:36:000
129412299222,14cyclictest1317542-21ntpq20:25:540
129412299222,14cyclictest1317542-21ntpq20:25:540
129412499212,15cyclictest1322036-21ntpq20:40:572
1294123992114,3cyclictest0-21swapper/119:30:491
1294123992114,3cyclictest0-21swapper/119:30:491
129412299210,18cyclictest1306956-21ntp_kernel_err19:50:530
129412599202,10cyclictest0-21swapper/323:10:563
129412599184,9cyclictest0-21swapper/321:55:513
129412599183,9cyclictest0-21swapper/323:55:523
129412599182,9cyclictest0-21swapper/323:50:563
129412599182,8cyclictest0-21swapper/300:05:513
129412599182,10cyclictest0-21swapper/322:20:013
129412599181,8cyclictest0-21swapper/319:25:583
129412599174,9cyclictest0-21swapper/300:21:033
129412599173,9cyclictest0-21swapper/322:46:043
129412599173,8cyclictest0-21swapper/322:41:103
129412599172,9cyclictest0-21swapper/323:40:573
129412599172,9cyclictest0-21swapper/323:40:573
129412599172,9cyclictest0-21swapper/300:26:003
129412599172,8cyclictest0-21swapper/323:35:543
129412599172,8cyclictest0-21swapper/300:15:583
129412599172,5cyclictest0-21swapper/321:00:543
1294124991713,2cyclictest0-21swapper/219:10:022
1294123991710,5cyclictest1288786-21kworker/u8:13+xprtiod19:10:021
129412599163,8cyclictest0-21swapper/323:15:593
129412599163,8cyclictest0-21swapper/321:45:543
129412599163,7cyclictest0-21swapper/321:15:573
129412599162,9cyclictest0-21swapper/323:00:593
129412599162,9cyclictest0-21swapper/323:00:593
129412599162,9cyclictest0-21swapper/321:20:583
129412599162,8cyclictest0-21swapper/323:25:573
129412599162,8cyclictest0-21swapper/321:10:533
129412599162,7cyclictest0-21swapper/322:25:513
129412599162,6cyclictest0-21swapper/321:50:503
129412599162,6cyclictest0-21swapper/321:50:503
129412599162,11cyclictest0-21swapper/321:40:023
129412599161,9cyclictest0-21swapper/321:31:023
129412599161,9cyclictest0-21swapper/319:35:523
129412599161,9cyclictest0-21swapper/319:35:523
129412599161,8cyclictest0-21swapper/320:10:593
129412299164,7cyclictest0-21swapper/019:10:010
129412599154,7cyclictest0-21swapper/323:30:553
129412599153,5cyclictest0-21swapper/319:20:543
129412599152,8cyclictest0-21swapper/322:51:013
129412599152,8cyclictest0-21swapper/322:35:573
129412599152,8cyclictest0-21swapper/322:35:573
129412599152,8cyclictest0-21swapper/322:20:473
129412599152,8cyclictest0-21swapper/322:10:443
129412599152,6cyclictest0-21swapper/320:35:563
129412599151,8cyclictest0-21swapper/323:05:563
129412599151,8cyclictest0-21swapper/322:30:543
129412599151,8cyclictest0-21swapper/321:40:563
129412599151,8cyclictest0-21swapper/300:30:493
129412599151,8cyclictest0-21swapper/300:00:523
129412599151,8cyclictest0-21swapper/300:00:523
129412599151,7cyclictest0-21swapper/320:40:593
129412599151,10cyclictest0-21swapper/322:55:493
129412599144,7cyclictest0-21swapper/300:36:013
129412599143,6cyclictest0-21swapper/320:51:003
129412599142,8cyclictest0-21swapper/322:00:543
129412599142,7cyclictest0-21swapper/322:05:553
129412599142,6cyclictest0-21swapper/323:45:463
129412599142,6cyclictest0-21swapper/323:45:463
129412599142,6cyclictest0-21swapper/323:20:173
129412599142,6cyclictest0-21swapper/321:10:003
129412599142,5cyclictest0-21swapper/320:30:553
129412599141,7cyclictest0-21swapper/319:45:523
129412599141,7cyclictest0-21swapper/300:10:543
129412599141,6cyclictest0-21swapper/320:25:533
129412599141,6cyclictest0-21swapper/320:25:533
129412599141,6cyclictest0-21swapper/320:15:563
129412599132,5cyclictest0-21swapper/320:47:183
129412599131,7cyclictest0-21swapper/321:25:463
129412599131,7cyclictest0-21swapper/319:40:583
129412599131,7cyclictest0-21swapper/319:16:013
129412599121,6cyclictest0-21swapper/319:55:553
129412599112,5cyclictest0-21swapper/320:05:543
129412599111,6cyclictest0-21swapper/320:20:553
129412599111,5cyclictest0-21swapper/320:55:433
129412599101,6cyclictest0-21swapper/319:50:513
129412599101,6cyclictest0-21swapper/319:10:433
129412599101,5cyclictest0-21swapper/319:30:483
129412599101,5cyclictest0-21swapper/319:30:483
12941259993,4cyclictest0-21swapper/320:00:553
12941259981,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional