You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 19:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sat May 09, 2026 12:43:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1318656995749,5cyclictest1416627-21sshd10:25:540
1318656995645,8cyclictest1447516-21cron11:05:010
1318658995243,6cyclictest1451576-21systemd-run11:10:012
1318656995135,13cyclictest496353-21cups-browsed09:45:500
1318658995040,7cyclictest1455589-21systemd-run11:15:012
1318657995041,6cyclictest1449424-21unixbench_multi11:06:011
1318657994738,5cyclictest1382948-21ssh09:45:021
1318657994738,5cyclictest1382948-21ssh09:45:021
1318658994536,6cyclictest1483451-21systemd-run11:50:012
131865799443,34cyclictest1500073-21ssh12:10:551
1318656994436,5cyclictest421-21dbus-daemon09:55:010
1318656994431,10cyclictest1484845-21ssh11:50:570
1318656994429,7cyclictest1491167-21sendmail12:00:010
1318658994335,5cyclictest1332156-21systemd-run07:55:012
131865799433,5cyclictest171rcu_preempt09:20:521
1318657994333,6cyclictest1459773-21latency_hist11:20:031
1318656994336,4cyclictest0-21swapper/011:10:490
1318656994328,12cyclictest1356535-21unixbench_multi09:11:020
1318656994328,12cyclictest1356535-21unixbench_multi09:11:020
1318658994234,5cyclictest1350124-21cron08:55:012
1318658994232,6cyclictest1423342-21latency_hist10:35:022
1318658994231,7cyclictest1367425-21aten_r2power_en09:25:392
1318656994235,4cyclictest1-21systemd10:50:010
1318656994235,4cyclictest1-21systemd10:50:010
1318658994132,5cyclictest1354641-21diskmemload12:10:482
1318658994131,6cyclictest1495844-21fschecks_time12:05:432
1318658994130,7cyclictest0-21swapper/212:35:532
1318658994130,7cyclictest0-21swapper/212:35:532
1318658994130,6cyclictest1468564-21if_eth011:30:442
1318657994134,4cyclictest1435398-21ssh10:50:001
1318657994134,4cyclictest1435398-21ssh10:50:001
1318656994135,3cyclictest1051-21mta-sts-daemon12:08:390
1318656994129,9cyclictest0-21swapper/009:40:540
1318656994129,9cyclictest0-21swapper/009:40:540
1318656994114,21cyclictest1503550-21sh12:15:510
1318658994032,5cyclictest1354632-21cron09:10:012
1318658994032,5cyclictest1354632-21cron09:10:012
131865899402,29cyclictest1456910-21ssh11:15:522
1318657994032,5cyclictest1374883-21cut09:35:011
1318657994030,7cyclictest1354641-21diskmemload10:55:531
1318657994028,9cyclictest496353-21cups-browsed10:06:001
1318656994034,4cyclictest1509238-21idleruntime-cro12:25:010
1318656994034,4cyclictest0-21swapper/010:30:420
1318657993933,2cyclictest0-21swapper/109:30:011
1318657993931,5cyclictest1467633-21systemd-run11:30:011
1318657993931,5cyclictest1467633-21systemd-run11:30:011
131865799391,26cyclictest1056-21mta-sts-daemon11:30:501
131865699393,28cyclictest659-21snmpd07:10:480
131865699393,27cyclictest1360038-21ssh09:15:530
1318656993932,4cyclictest1431376-21ssh10:45:010
1318656993925,12cyclictest1395006-21latency_hist10:00:010
1318656993925,11cyclictest1469489-21unixbench_multi11:31:020
1318656993911,24cyclictest1408429-21ssh10:15:490
1318658993830,5cyclictest1487301-21ssh11:55:012
1318658993829,6cyclictest421-21dbus-daemon10:35:572
1318658993829,6cyclictest1351624-21grep09:00:012
1318658993828,6cyclictest496353-21cups-browsed10:40:432
1318658993827,8cyclictest1432192-21ssh10:45:402
1318658993827,8cyclictest1432192-21ssh10:45:402
1318657993830,5cyclictest1359277-21aten_r2power_cu09:15:401
1318657993829,6cyclictest261-21systemd-journal11:40:001
1318656993832,3cyclictest0-21swapper/012:25:520
131865899375,28cyclictest1444779-21sh11:00:492
1318658993730,4cyclictest1380167-21ssh09:40:532
1318658993730,4cyclictest1380167-21ssh09:40:532
1318658993729,5cyclictest1403134-21systemd-run10:10:012
1318658993729,5cyclictest1336662-21systemd-run08:10:012
1318658993729,5cyclictest0-21swapper/209:20:012
1318658993728,6cyclictest548-21gdbus07:24:552
1318658993728,6cyclictest548-21gdbus07:24:552
1318658993727,7cyclictest1320903-21missed_timers07:15:462
131865899372,27cyclictest1507139-21ssh12:20:542
1318657993730,4cyclictest1051-21mta-sts-daemon10:40:521
1318657993729,5cyclictest1326162-21systemd-run07:35:011
1318657993729,4cyclictest0-21swapper/111:40:541
1318657993728,6cyclictest1351619-21munin-plugin-st09:00:011
1318657993726,8cyclictest1480610-21latency11:45:501
131865799372,27cyclictest1408540-21ssh10:15:531
131865699373,26cyclictest1348122-21sshd08:45:530
1318656993729,5cyclictest1427326-21awk10:40:010
1318656993728,4cyclictest0-21swapper/010:20:560
131865699372,4cyclictest0-21swapper/008:20:510
131865699372,31cyclictest1499934-21ssh12:10:490
131865699371,30cyclictest1376190-21ssh09:35:560
1318658993629,4cyclictest1329677-21fschecks_time07:45:422
1318658993628,5cyclictest1333623-21cron08:00:012
1318658993628,5cyclictest1326659-21fschecks_count07:35:422
1318658993628,4cyclictest0-21swapper/211:35:482
1318657993630,3cyclictest0-21swapper/112:05:511
1318657993629,3cyclictest0-21swapper/109:55:541
1318657993628,5cyclictest1516353-21munin-plugin-st12:35:011
1318657993628,5cyclictest1427320-21sh10:40:011
1318657993628,5cyclictest1378841-21cut09:40:011
1318657993628,5cyclictest1321652-21basename07:20:001
1318657993627,6cyclictest496353-21cups-browsed10:20:421
1318657993627,6cyclictest1445355-21unixbench_multi11:01:011
1318657993625,8cyclictest1520088-21idleruntime-cro12:40:011
1318657993625,8cyclictest1520088-21idleruntime-cro12:40:011
1318656993629,4cyclictest1347216-21latency_hist08:45:010
1318656993629,4cyclictest1347216-21latency_hist08:45:010
1318656993626,7cyclictest1-21systemd11:45:010
1318656993624,7cyclictest1440713-21sh10:55:500
1318658993526,6cyclictest1387555-21aten_r2power_en09:50:392
1318657993528,4cyclictest1460467-21fschecks_count11:20:431
1318657993528,4cyclictest1460467-21fschecks_count11:20:431
1318657993528,4cyclictest0-21swapper/112:20:011
1318657993527,5cyclictest1439428-21cron10:55:001
1318657993527,4cyclictest0-21swapper/112:25:011
1318657993526,6cyclictest439-21systemd-logind12:05:011
1318657993526,6cyclictest439-21systemd-logind12:05:011
1318657993526,6cyclictest420-21avahi-daemon08:25:011
1318657993526,5cyclictest1452425-21gltestperf11:10:411
1318657993524,8cyclictest1396709-21switchtime10:00:591
131865699359,15cyclictest428-21mta-sts-daemon07:20:470
131865699359,15cyclictest428-21mta-sts-daemon07:20:470
1318656993526,6cyclictest496353-21cups-browsed09:31:210
131865699352,24cyclictest1460982-21sshd11:20:530
131865699352,24cyclictest1460982-21sshd11:20:530
1318656993522,3cyclictest171rcu_preempt08:25:490
1318656993518,13cyclictest12350irq/32-eth011:46:030
1318658993428,3cyclictest0-21swapper/209:40:012
1318658993426,5cyclictest1396198-21needreboot10:00:452
1318658993426,5cyclictest1341680-21fschecks_time08:25:442
1318658993425,6cyclictest1392862-21unixbench_multi09:56:032
131865799342,9cyclictest1510556-21ntpq12:25:531
1318657993426,5cyclictest1333798-21latency_hist08:00:021
1318657993426,5cyclictest1329150-21grep07:45:011
1318657993425,6cyclictest421-21dbus-daemon10:10:481
1318657993425,6cyclictest421-21dbus-daemon10:10:481
1318656993426,4cyclictest0-21swapper/009:22:090
131865699342,27cyclictest1400421-21ssh10:05:520
1318658993326,4cyclictest1497195-21kworker/u8:7-xprtiod12:35:012
1318658993325,5cyclictest1356138-21proc_pri09:10:562
1318658993325,5cyclictest1356138-21proc_pri09:10:562
1318658993320,3cyclictest171rcu_preempt11:25:522
1318658993320,3cyclictest171rcu_preempt11:25:522
1318657993324,5cyclictest1488803-21sendmail_mailqu11:55:571
1318657993324,5cyclictest1056-21mta-sts-daemon10:27:301
1318657993321,9cyclictest1391167-21latency_hist09:55:021
1318656993313,8cyclictest171rcu_preempt12:37:100
1318656993313,8cyclictest171rcu_preempt12:37:100
131865899327,5cyclictest171rcu_preempt11:55:532
1318658993225,4cyclictest1339530-21unixbench_multi08:16:042
1318658993225,3cyclictest0-21swapper/210:26:042
1318658993223,6cyclictest421-21dbus-daemon11:20:222
1318658993223,6cyclictest421-21dbus-daemon11:20:222
1318658993223,5cyclictest1510725-21mailstats12:25:572
1318658993223,5cyclictest1372685-21unixbench-2d09:31:012
1318657993224,5cyclictest1-21systemd07:50:021
1318656993228,2cyclictest0-21swapper/008:30:520
1318656993224,5cyclictest1335094-21head08:01:000
1318656993224,5cyclictest1055-21mta-sts-daemon09:28:040
1318656993224,5cyclictest0-21swapper/011:05:520
1318656993222,7cyclictest1439438-21/usr/sbin/munin10:55:010
1318656993217,10cyclictest12350irq/32-eth012:00:510
1318656993217,10cyclictest12350irq/32-eth012:00:510
1318658993124,4cyclictest0-21swapper/212:20:012
1318658993124,3cyclictest0-21swapper/209:21:022
1318658993123,5cyclictest1493044-21unixbench_multi12:01:022
1318658993123,5cyclictest1493044-21unixbench_multi12:01:022
1318658993122,6cyclictest0-21swapper/211:00:002
1318658993121,7cyclictest1345571-21head08:36:022
1318658993121,7cyclictest0-21swapper/210:50:402
1318657993125,4cyclictest1318655-21cyclictest08:11:531
1318657993125,3cyclictest0-21swapper/111:50:571
1318657993122,6cyclictest420-21avahi-daemon08:00:511
131865699318,16cyclictest1464842-21ntpq11:25:480
131865699318,16cyclictest1464842-21ntpq11:25:480
1318656993124,4cyclictest1475792-21latency_hist11:40:020
1318656993122,7cyclictest1335882-21missed_timers08:05:470
1318656993122,6cyclictest600-21gmain10:04:110
1318656993120,8cyclictest0-21swapper/009:00:010
1318656993119,3cyclictest171rcu_preempt08:35:520
1318656993118,7cyclictest12450irq/33-eth011:15:490
131865699311,25cyclictest659-21snmpd07:50:550
131865699311,14cyclictest1403661-21munin-node10:10:400
131865699311,14cyclictest1403661-21munin-node10:10:400
1318658993023,3cyclictest0-21swapper/210:10:572
1318658993023,3cyclictest0-21swapper/210:10:572
1318658993022,5cyclictest1323670-21fschecks_count07:25:432
1318658993021,6cyclictest1319458-21needreboot07:10:462
1318658993020,7cyclictest496353-21cups-browsed09:45:502
131865899301,24cyclictest1408492-21ssh10:15:512
131865899301,22cyclictest1328615-21sshd07:40:532
1318657993021,6cyclictest496353-21cups-browsed09:46:011
131865899291,22cyclictest1412555-21awk10:20:512
1318657992922,4cyclictest1349164-21fschecks_time08:50:421
1318657992920,6cyclictest496353-21cups-browsed07:15:011
1318657992916,7cyclictest0-21swapper/109:10:491
1318657992916,7cyclictest0-21swapper/109:10:491
131865799291,22cyclictest1420581-21ssh10:30:531
1318656992922,4cyclictest1338152-21systemd-run08:15:010
1318656992922,3cyclictest0-21swapper/008:15:540
1318658992819,6cyclictest1346169-21fschecks_count08:40:422
1318658992819,6cyclictest1346169-21fschecks_count08:40:422
1318658992812,4cyclictest0-21swapper/208:20:532
131865799283,20cyclictest1327633-21munin-run07:40:011
1318657992822,3cyclictest0-21swapper/108:45:531
1318657992821,4cyclictest1054-21mta-sts-daemon07:54:381
1318657992821,4cyclictest1051-21mta-sts-daemon07:25:511
1318657992820,5cyclictest1344015-21unixbench_multi08:31:011
1318657992820,5cyclictest1339675-21systemd-run08:20:011
1318657992818,7cyclictest1354639-21munin-plugin-st09:10:021
1318657992818,7cyclictest1354639-21munin-plugin-st09:10:021
131865699282,7cyclictest1350318-21wc08:55:030
1318656992812,3cyclictest0-21swapper/007:35:480
1318658992719,5cyclictest1338064-21head08:11:012
1318658992719,5cyclictest1335086-21head08:01:002
1318658992717,7cyclictest1343976-21unixbench-2d08:31:012
131865699272,22cyclictest0-21swapper/007:55:540
1318656992715,6cyclictest1056-21mta-sts-daemon12:30:450
131865699271,19cyclictest421-21dbus-daemon07:45:020
1318656992711,9cyclictest0-21swapper/007:25:530
131865699270,11cyclictest171rcu_preempt07:46:010
131865899262,18cyclictest1476926-21ssh11:40:552
1318658992618,5cyclictest1347790-21kernelversion08:45:442
1318658992617,6cyclictest439-21systemd-logind07:35:012
1318657992620,3cyclictest1317439-21kworker/u8:15-xprtiod09:01:011
1318657992620,3cyclictest0-21swapper/108:40:001
1318656992620,4cyclictest1353081-21sed09:01:010
131865699261,11cyclictest1-21systemd07:10:010
1318658992518,4cyclictest1352168-21fschecks_time09:00:432
1318657992520,4cyclictest1318655-21cyclictest08:25:571
131865699257,8cyclictest171rcu_preempt07:15:560
131865699255,18cyclictest1265014-21kworker/u8:1+events_unbound07:33:320
131865699251,20cyclictest1354668-21unin-run09:10:020
131865699251,20cyclictest1354668-21unin-run09:10:020
1318657992417,4cyclictest1335958-21needreboot08:05:481
131865799232,15cyclictest1346610-21ntpq08:40:531
131865799232,15cyclictest1346610-21ntpq08:40:531
131865999224,11cyclictest0-21swapper/309:30:543
131865999223,14cyclictest0-21swapper/309:40:543
131865999223,14cyclictest0-21swapper/309:40:543
131865999222,11cyclictest0-21swapper/312:30:543
1318657992215,5cyclictest261-21systemd-journal07:10:011
131865799221,16cyclictest1322685-21ps07:20:541
131865799221,16cyclictest1322685-21ps07:20:541
131865999213,8cyclictest0-21swapper/311:10:473
131865999213,11cyclictest0-21swapper/309:35:583
131865999211,12cyclictest0-21swapper/310:50:533
131865999211,11cyclictest0-21swapper/310:45:473
131865999211,11cyclictest0-21swapper/310:45:473
131865999203,9cyclictest0-21swapper/311:05:433
131865999202,11cyclictest0-21swapper/312:35:553
131865999202,11cyclictest0-21swapper/312:35:553
131865999202,10cyclictest0-21swapper/309:25:563
131865999201,11cyclictest0-21swapper/312:05:503
131865999201,11cyclictest0-21swapper/309:45:443
131865999201,10cyclictest0-21swapper/311:20:563
131865999201,10cyclictest0-21swapper/311:20:563
131865999201,10cyclictest0-21swapper/308:55:493
131865999193,9cyclictest0-21swapper/312:15:513
131865999193,8cyclictest0-21swapper/311:50:543
131865999193,8cyclictest0-21swapper/310:30:543
131865999192,10cyclictest0-21swapper/312:20:473
131865999184,8cyclictest0-21swapper/311:25:483
131865999184,8cyclictest0-21swapper/311:25:483
131865999184,7cyclictest0-21swapper/309:20:493
131865999183,9cyclictest0-21swapper/311:40:563
131865999183,9cyclictest0-21swapper/310:40:483
131865999183,10cyclictest0-21swapper/310:25:503
131865999182,9cyclictest0-21swapper/312:00:553
131865999182,9cyclictest0-21swapper/312:00:553
131865999182,9cyclictest0-21swapper/311:35:473
131865999182,8cyclictest0-21swapper/309:55:503
131865999182,11cyclictest0-21swapper/311:15:423
131865999182,10cyclictest0-21swapper/310:55:563
131865999182,10cyclictest0-21swapper/310:15:493
131865999181,10cyclictest0-21swapper/310:05:513
131865999174,4cyclictest0-21swapper/307:15:483
131865999173,9cyclictest0-21swapper/311:55:483
131865999173,9cyclictest0-21swapper/310:20:453
131865999173,8cyclictest0-21swapper/309:50:433
131865999172,9cyclictest0-21swapper/311:00:473
131865999172,8cyclictest0-21swapper/308:40:543
131865999172,8cyclictest0-21swapper/308:40:543
131865999172,10cyclictest0-21swapper/309:15:563
131865999171,9cyclictest0-21swapper/311:30:503
131865999171,9cyclictest0-21swapper/309:10:483
131865999171,9cyclictest0-21swapper/309:10:483
131865999171,8cyclictest0-21swapper/310:00:483
131865999171,8cyclictest0-21swapper/307:20:533
131865999171,8cyclictest0-21swapper/307:20:533
131865999171,10cyclictest0-21swapper/312:10:483
131865999171,10cyclictest0-21swapper/310:10:523
131865999171,10cyclictest0-21swapper/310:10:523
131865999163,7cyclictest0-21swapper/308:20:483
131865999163,7cyclictest0-21swapper/307:55:553
131865999162,7cyclictest0-21swapper/307:35:003
131865999161,9cyclictest0-21swapper/312:25:533
131865999161,8cyclictest0-21swapper/308:10:483
131865999161,7cyclictest0-21swapper/308:35:533
131865999152,9cyclictest0-21swapper/311:45:553
131865999152,8cyclictest0-21swapper/310:35:403
131865999152,8cyclictest0-21swapper/309:10:023
131865999152,8cyclictest0-21swapper/309:10:023
131865999152,8cyclictest0-21swapper/308:25:523
131865999152,8cyclictest0-21swapper/307:40:533
131865899154,2cyclictest171rcu_preempt07:10:012
131865999143,6cyclictest0-21swapper/307:25:493
131865999142,8cyclictest0-21swapper/307:50:503
131865999142,7cyclictest0-21swapper/308:45:533
131865999142,7cyclictest0-21swapper/307:45:483
131865999141,7cyclictest0-21swapper/308:05:503
131865999135,5cyclictest0-21swapper/308:30:493
131865999133,4cyclictest0-21swapper/308:15:513
131865999132,7cyclictest0-21swapper/309:00:483
131865999132,7cyclictest0-21swapper/307:10:533
131865999131,8cyclictest0-21swapper/308:50:493
131865999131,6cyclictest0-21swapper/307:35:483
131865999121,6cyclictest0-21swapper/307:10:013
131865999120,7cyclictest0-21swapper/308:00:503
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional