You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-01 - 01:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sat Aug 01, 2026 00:43:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3932530994738,6cyclictest4072541-21latency_hist00:00:031
3932529994538,5cyclictest4011470-21systemd-run22:20:020
3932530994436,5cyclictest4032706-21cron22:55:001
3932529994337,4cyclictest3986962-21awk21:40:010
3932531994233,6cyclictest3953533-21tr20:20:012
3932529994236,4cyclictest4055665-21find23:31:000
3932529994235,4cyclictest4026638-21systemd-run22:45:010
3932530994135,3cyclictest0-21swapper/122:05:011
3932530994133,5cyclictest4080429-21sh00:15:011
3932530994133,5cyclictest4060868-21aten_r2power_vo23:40:401
3932530994133,5cyclictest3968510-21diskmemload00:15:381
3932529994134,4cyclictest4095720-21cron00:40:010
3932529994134,4cyclictest4095720-21cron00:40:010
3932531994033,5cyclictest4023671-21latency_hist22:40:022
3932531994033,5cyclictest4023671-21latency_hist22:40:022
3932531994032,5cyclictest3977725-21ssh21:25:002
3932531994032,3cyclictest0-21swapper/221:15:502
3932529994031,7cyclictest3946024-21sh19:55:020
393252999401,32cyclictest151ktimers/020:15:490
3932531993930,5cyclictest4066408-21systemd-run23:50:022
3932530993932,4cyclictest3967019-21munin-plugin-st21:05:011
3932530993931,5cyclictest4066538-21latency_hist23:50:031
3932530993931,5cyclictest4037018-21ssh23:00:541
3932530993931,5cyclictest4018898-21mailstats22:30:591
3932530993929,7cyclictest4095728-21idleruntime-cro00:40:011
3932530993929,7cyclictest4095728-21idleruntime-cro00:40:011
3932530993929,7cyclictest4042817-21latency23:10:471
3932529993931,5cyclictest421-21dbus-daemon23:00:010
3932531993831,4cyclictest4057206-21cron23:35:012
3932530993828,7cyclictest4035796-21idleruntime-cro23:00:011
3932531993730,3cyclictest0-21swapper/222:10:552
3932531993728,6cyclictest4075586-21munin-run00:05:012
3932531993727,6cyclictest4043326-21mailstats23:11:022
3932530993729,5cyclictest3956536-21systemd-run20:30:011
3932530993729,4cyclictest1001-21mta-sts-daemon22:18:191
3932530993728,7cyclictest3994080-21missed_timers21:50:471
3932530993728,6cyclictest1003-21mta-sts-daemon23:35:021
3932529993731,4cyclictest4090672-21needreboot00:30:470
3932529993731,4cyclictest3968143-21kworker/u8:4+rpciod23:20:020
3932529993729,5cyclictest3974652-21ssh21:20:010
3932531993629,4cyclictest4048148-21latency_hist23:20:022
3932531993629,4cyclictest3963991-21cron20:55:002
3932531993628,5cyclictest4035784-21awk23:00:012
3932531993627,5cyclictest4080429-21idleruntime-cro00:15:012
3932531993626,6cyclictest420-21avahi-daemon23:07:002
3932530993631,2cyclictest0-21swapper/121:56:011
3932530993630,4cyclictest3932528-21cyclictest19:26:011
3932530993630,3cyclictest0-21swapper/121:25:421
3932530993628,5cyclictest4016230-21unixbench_singl22:26:001
3932529993628,6cyclictest3968815-21sh21:10:170
3932529993628,5cyclictest4085252-21unixbench_singl00:21:050
3932531993528,3cyclictest0-21swapper/221:45:542
3932531993527,5cyclictest3990148-21latency_hist21:45:022
3932531993527,5cyclictest3935517-21munin-plugin-st19:20:012
3932531993527,5cyclictest1-21systemd21:00:012
3932530993530,3cyclictest0-21swapper/121:20:001
3932530993529,3cyclictest0-21swapper/121:40:011
3932530993529,3cyclictest0-21swapper/100:20:511
3932530993528,5cyclictest0-21swapper/100:10:021
3932530993528,4cyclictest4051085-21latency_hist23:25:011
3932530993527,5cyclictest485-21in:imuxsock00:30:021
3932530993527,5cyclictest4058142-21latency23:35:451
3932529993529,4cyclictest2342389-21munin-node00:16:000
3932529993528,4cyclictest4042819-21latency23:10:470
3932529993528,4cyclictest4015993-21threads22:25:590
3932529993526,6cyclictest0-21swapper/023:25:010
3932531993428,3cyclictest0-21swapper/221:40:012
3932531993426,5cyclictest4052857-21head23:26:012
3932531993426,5cyclictest3994778-21unixbench_multi21:51:022
3932531993426,5cyclictest3947545-21systemd-run20:00:022
3932531993425,6cyclictest0-21swapper/222:05:022
3932531993423,5cyclictest0-21swapper/222:25:492
3932531993422,9cyclictest4024646-21missed_timers22:40:492
3932530993429,2cyclictest0-21swapper/100:30:461
3932530993428,3cyclictest0-21swapper/123:20:011
3932530993427,4cyclictest3957888-21unixbench_multi20:31:001
3932530993426,5cyclictest439-21systemd-logind20:10:011
3932530993426,5cyclictest4028250-21sed22:46:001
3932530993425,6cyclictest485-21in:imuxsock19:20:011
3932530993425,6cyclictest2243543-21cups-browsed23:08:281
3932530993425,6cyclictest2243543-21cups-browsed21:43:191
3932530993425,6cyclictest1000-21mta-sts-daemon22:24:101
3932530993425,5cyclictest1001-21mta-sts-daemon22:39:211
3932530993425,5cyclictest1001-21mta-sts-daemon22:39:211
3932529993422,9cyclictest3935456-21unixbench_singl19:16:010
3932529993422,2cyclictest0-21swapper/021:30:000
3932531993326,4cyclictest3951998-21cron20:15:012
3932531993324,6cyclictest485-21in:imuxsock23:40:032
3932531993324,6cyclictest420-21avahi-daemon20:40:582
3932531993324,6cyclictest3966481-21sshd21:01:012
3932531993324,6cyclictest0-21swapper/220:05:012
3932531993322,7cyclictest1003-21mta-sts-daemon23:01:352
3932530993326,5cyclictest0-21swapper/121:00:021
3932530993324,6cyclictest2243538-21cupsd23:50:231
3932529993328,3cyclictest0-21swapper/021:41:040
3932529993326,5cyclictest3952078-21latency_hist20:15:020
3932529993326,4cyclictest3997902-21ssh21:56:020
3932529993322,8cyclictest4040530-21unixbench_multi23:06:010
3932529993315,11cyclictest12650irq/32-eth021:30:590
3932531993224,5cyclictest3950510-21idleruntime-cro20:10:012
3932531993224,3cyclictest0-21swapper/200:21:002
3932531993223,6cyclictest4020594-21munin-plugin-st22:35:012
3932531993222,6cyclictest4088310-21unixbench_singl00:26:022
3932530993225,3cyclictest0-21swapper/122:05:531
3932530993225,3cyclictest0-21swapper/120:15:531
3932530993223,6cyclictest2243543-21cups-browsed22:12:071
393252999329,17cyclictest665-21snmpd19:30:500
3932529993225,4cyclictest3959544-21systemd-run20:40:010
3932529993223,6cyclictest0-21swapper/022:40:020
3932529993223,6cyclictest0-21swapper/022:40:020
3932531993124,3cyclictest0-21swapper/200:35:532
3932531993124,3cyclictest0-21swapper/200:35:532
3932531993123,5cyclictest3945053-21gltestperf19:50:482
3932531993121,6cyclictest0-21swapper/223:55:102
393253199311,23cyclictest3961912-21sshd20:45:542
3932530993125,3cyclictest0-21swapper/120:00:011
3932530993124,4cyclictest3962526-21systemd-run20:50:011
3932530993124,3cyclictest0-21swapper/121:10:521
3932530993123,5cyclictest694-21cron19:40:001
3932530993123,5cyclictest4074146-21unixbench-2d00:01:011
3932529993126,2cyclictest0-21swapper/019:15:010
3932529993122,6cyclictest2243543-21cups-browsed22:01:080
3932529993117,4cyclictest4089646-21systemd-run00:30:020
3932531993025,2cyclictest0-21swapper/222:50:592
3932531993023,4cyclictest0-21swapper/223:22:482
3932531993023,3cyclictest0-21swapper/223:40:012
3932531993022,4cyclictest0-21swapper/221:55:592
393253199302,23cyclictest3982050-21ssh21:30:572
3932531993022,3cyclictest0-21swapper/221:10:592
3932531993021,6cyclictest2243543-21cups-browsed20:33:322
3932531993020,7cyclictest4067366-21missed_timers23:50:492
3932530993024,3cyclictest0-21swapper/121:20:481
3932530993024,3cyclictest0-21swapper/119:10:011
3932530993023,3cyclictest0-21swapper/123:25:431
3932529993022,6cyclictest0-21swapper/023:40:030
3932529993022,5cyclictest4036214-21/usr/sbin/munin23:01:040
3932529993020,8cyclictest3957999-21cron20:35:010
3932531992923,3cyclictest0-21swapper/222:20:512
3932531992919,3cyclictest0-21swapper/200:16:022
3932531992916,9cyclictest421-21dbus-daemon21:29:442
3932530992923,3cyclictest0-21swapper/120:40:011
3932530992922,3cyclictest0-21swapper/121:30:591
3932530992921,6cyclictest3960215-21latency20:40:471
3932530992921,5cyclictest0-21swapper/121:48:161
3932530992919,8cyclictest3944234-21timerandwakeup19:46:021
393252999293,17cyclictest12750irq/33-eth022:45:510
3932529992924,2cyclictest0-21swapper/000:00:010
3932529992921,5cyclictest3975392-21fschecks_count21:20:430
3932529992920,6cyclictest4063458-21latency_hist23:45:020
3932529992920,6cyclictest0-21swapper/021:50:020
3932531992821,4cyclictest3968528-21kworker/u8:6+rpciod21:10:012
3932531992821,4cyclictest3959502-21sh20:40:012
3932531992821,3cyclictest0-21swapper/222:15:562
3932531992820,5cyclictest2243543-21cups-browsed22:45:432
3932531992820,5cyclictest2243543-21cups-browsed20:28:342
3932530992819,6cyclictest3948031-21fschecks_count20:00:431
393252999289,15cyclictest151ktimers/021:05:540
393252999288,11cyclictest4003345-21nfs4_client22:05:490
3932529992823,3cyclictest0-21swapper/022:11:020
3932529992823,2cyclictest0-21swapper/019:40:010
3932529992822,4cyclictest1002-21mta-sts-daemon21:50:430
393252999282,21cyclictest4018698-21ssh22:30:530
3932529992820,5cyclictest151ktimers/021:00:530
3932529992818,8cyclictest0-21swapper/020:45:010
3932529992818,7cyclictest4064241-21latency23:45:450
3932529992813,11cyclictest12650irq/32-eth022:50:490
393253199275,16cyclictest3944040-21ps19:45:562
3932531992719,5cyclictest3942910-21unixbench_multi19:40:592
3932531992719,5cyclictest3942910-21unixbench_multi19:40:592
3932530992720,4cyclictest4026789-21ssh22:45:021
3932530992718,7cyclictest3939693-21switchtime19:30:571
3932529992723,2cyclictest0-21swapper/023:50:570
3932529992716,8cyclictest439-21systemd-logind19:45:010
3932529992716,8cyclictest439-21systemd-logind19:45:010
3932529992712,11cyclictest420-21avahi-daemon22:22:350
3932531992619,4cyclictest1003-21mta-sts-daemon22:05:142
3932530992618,4cyclictest0-21swapper/120:10:581
393252999262,18cyclictest665-21snmpd19:25:490
3932529992619,5cyclictest567-21gdbus19:49:500
3932529992619,5cyclictest2243543-21cups-browsed20:24:100
393252999261,21cyclictest4073688-21ntpq00:00:520
3932531992518,4cyclictest1001-21mta-sts-daemon00:05:062
3932531992517,5cyclictest2243543-21cups-browsed19:24:292
3932531992516,6cyclictest2243543-21cups-browsed19:30:222
393253199251,19cyclictest4090777-21ntpq00:30:502
3932530992518,5cyclictest271ktimers/120:20:461
3932530992517,5cyclictest3963991-21cron20:55:001
3932530992517,5cyclictest3941924-21aten_r2power_vo19:40:391
3932530992517,5cyclictest3941924-21aten_r2power_vo19:40:391
3932530992517,5cyclictest0-21swapper/119:25:031
3932529992521,2cyclictest0-21swapper/020:30:010
3932529992519,3cyclictest0-21swapper/000:11:510
3932529992518,5cyclictest2243543-21cups-browsed20:48:260
393253299244,13cyclictest0-21swapper/323:25:023
393253199243,16cyclictest3937998-21ntp_states19:25:552
3932531992417,4cyclictest1003-21mta-sts-daemon20:22:342
3932530992418,3cyclictest0-21swapper/119:10:451
3932530992415,6cyclictest3968512-21munin-run21:10:011
3932530992411,9cyclictest171rcu_preempt19:50:541
3932529992419,3cyclictest3944648-21kworker/u8:13-xprtiod20:00:010
3932529992418,4cyclictest3948183-21kernelversion20:00:450
3932529992417,5cyclictest0-21swapper/023:30:020
3932531992314,6cyclictest3933497-21ntpq19:10:552
3932529992317,4cyclictest3964025-21systemd-run20:55:010
3932529992317,3cyclictest0-21swapper/000:09:050
3932529992316,5cyclictest265-21systemd-journal20:08:440
393253299223,12cyclictest0-21swapper/323:55:023
393253299223,12cyclictest0-21swapper/322:15:543
393253299223,12cyclictest0-21swapper/321:00:553
393253199222,15cyclictest3940865-21ntpq19:35:492
393253299214,9cyclictest0-21swapper/322:00:523
393253299214,9cyclictest0-21swapper/321:50:503
393253299214,12cyclictest0-21swapper/323:45:513
393253299214,11cyclictest0-21swapper/321:35:493
393253299214,11cyclictest0-21swapper/300:25:483
393253299213,11cyclictest0-21swapper/323:10:503
393253299213,10cyclictest0-21swapper/300:10:463
3932529992114,5cyclictest0-21swapper/019:21:040
393252999211,12cyclictest151ktimers/020:55:520
393253299206,11cyclictest0-21swapper/322:40:523
393253299205,10cyclictest0-21swapper/323:15:543
393253299205,10cyclictest0-21swapper/322:10:563
393253299203,9cyclictest0-21swapper/323:25:473
393253299203,11cyclictest0-21swapper/323:55:513
393253299203,11cyclictest0-21swapper/323:40:503
393253299203,11cyclictest0-21swapper/322:35:543
393253299203,11cyclictest0-21swapper/322:35:543
393253299203,10cyclictest0-21swapper/323:00:493
393253299202,11cyclictest0-21swapper/322:45:553
393253299202,11cyclictest0-21swapper/322:25:543
393253299202,11cyclictest0-21swapper/322:20:553
3932529992012,4cyclictest12650irq/32-eth019:10:010
393253299194,9cyclictest0-21swapper/322:30:543
393253299194,9cyclictest0-21swapper/321:10:543
393253299194,11cyclictest0-21swapper/323:35:513
393253299193,9cyclictest0-21swapper/320:10:583
393253299193,7cyclictest0-21swapper/321:55:483
393253299193,12cyclictest0-21swapper/321:15:563
393253299193,10cyclictest0-21swapper/321:30:013
393253299192,9cyclictest0-21swapper/322:05:513
393253299192,9cyclictest0-21swapper/320:25:503
393253299192,9cyclictest0-21swapper/300:35:403
393253299192,9cyclictest0-21swapper/300:35:403
393253299192,9cyclictest0-21swapper/300:30:503
393253299191,8cyclictest0-21swapper/321:05:553
3932531991915,2cyclictest0-21swapper/219:10:012
393253299184,6cyclictest0-21swapper/320:20:453
393253299183,9cyclictest0-21swapper/321:30:553
393253299183,7cyclictest0-21swapper/320:05:503
393253299183,6cyclictest0-21swapper/319:35:493
393253299182,7cyclictest0-21swapper/319:50:543
393253299182,12cyclictest0-21swapper/300:20:543
393253299182,10cyclictest0-21swapper/321:45:483
393253299182,10cyclictest0-21swapper/319:55:503
393253299181,8cyclictest0-21swapper/319:45:493
393253299174,6cyclictest0-21swapper/320:50:493
393253299173,9cyclictest0-21swapper/300:00:493
393253299173,6cyclictest0-21swapper/320:55:473
393253299172,9cyclictest0-21swapper/322:50:493
393253299172,9cyclictest0-21swapper/300:15:433
393253299172,8cyclictest0-21swapper/320:00:483
393253299172,10cyclictest0-21swapper/323:30:483
393253299172,10cyclictest0-21swapper/323:05:473
393253299172,10cyclictest0-21swapper/322:55:483
393253299172,10cyclictest0-21swapper/319:30:473
393253299163,9cyclictest0-21swapper/321:40:533
393253299163,9cyclictest0-21swapper/320:15:513
393253299162,9cyclictest0-21swapper/319:10:513
393253299162,8cyclictest0-21swapper/319:20:573
393253299162,7cyclictest0-21swapper/320:30:473
393253299162,10cyclictest0-21swapper/321:20:493
393253299161,7cyclictest0-21swapper/319:25:483
393253299153,6cyclictest0-21swapper/320:40:523
393253299152,9cyclictest0-21swapper/320:35:553
393253299152,8cyclictest0-21swapper/319:20:013
393253299151,8cyclictest0-21swapper/319:40:483
393253299151,8cyclictest0-21swapper/319:40:483
393253299143,9cyclictest0-21swapper/319:10:013
393253299143,7cyclictest0-21swapper/320:45:513
393253299131,6cyclictest0-21swapper/300:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional