You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-10 - 12:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Mon Aug 10, 2026 00:43:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3652921996053,4cyclictest1002-21mta-sts-daemon20:26:000
3652922995232,6cyclictest3777630-21latency23:35:501
3652923994736,8cyclictest3761684-21sh23:10:022
3652922994738,5cyclictest3791870-21ssh00:00:021
3652922994738,5cyclictest3791870-21ssh00:00:021
3652921994739,5cyclictest421-21dbus-daemon00:25:010
3652921994739,5cyclictest421-21dbus-daemon00:25:010
3652921994735,7cyclictest12650irq/32-eth023:35:550
3652921994537,5cyclictest3816040-21systemctl00:40:010
3652921994535,7cyclictest421-21dbus-daemon20:15:020
3652921994531,9cyclictest12650irq/32-eth000:06:050
3652921994527,13cyclictest0-21swapper/023:20:520
3652922994436,5cyclictest3770774-21latency_hist23:25:021
3652923994335,5cyclictest3746592-21systemd22:45:012
3652923994335,5cyclictest3746592-21systemd22:45:012
3652923994331,9cyclictest3777501-21idleruntime23:35:482
3652922994338,2cyclictest0-21swapper/119:20:011
3652921994333,7cyclictest3685892-21munin-plugin-st21:00:010
3652922994235,4cyclictest3803961-21sendmail00:20:011
3652922994235,4cyclictest3803961-21sendmail00:20:011
3652923994133,3cyclictest0-21swapper/200:20:452
3652923994133,3cyclictest0-21swapper/200:20:452
3652922994132,6cyclictest420-21avahi-daemon23:46:041
3652921994129,8cyclictest3687640-21kworker/u8:4-xprtiod21:11:120
3652921994128,10cyclictest3760002-21sendmail_mailst23:06:010
365292199412,32cyclictest3808210-21sh00:26:000
365292199412,32cyclictest3808210-21sh00:26:000
365292399402,30cyclictest3783945-21ssh23:45:542
365292299403,30cyclictest3787081-21ssh23:51:001
3652921994032,4cyclictest0-21swapper/023:55:530
3652921994032,4cyclictest0-21swapper/023:55:530
3652921994028,8cyclictest420-21avahi-daemon21:29:580
3652921994026,11cyclictest421-21dbus-daemon23:00:010
365292399393,29cyclictest3774858-21ssh23:30:582
3652923993930,6cyclictest3695096-21ssh21:20:022
3652922993931,5cyclictest3759611-21memory23:05:501
3652921993932,4cyclictest1003-21mta-sts-daemon21:05:540
365292199392,29cyclictest3711427-21ssh21:45:540
3652923993830,5cyclictest3660430-21systemd-run19:35:012
3652923993829,6cyclictest3679820-21head20:36:062
3652923993828,6cyclictest3754101-21timerandwakeup22:56:062
3652922993832,3cyclictest0-21swapper/121:55:011
3652922993831,3cyclictest0-21swapper/122:40:591
3652922993831,3cyclictest0-21swapper/122:40:591
3652922993830,5cyclictest3741265-21fschecks_time22:35:461
3652922993830,5cyclictest3725420-21systemd-run22:10:011
3652922993830,5cyclictest3663936-21fschecks_count19:45:471
3652922993830,4cyclictest3676916-21basename20:30:011
3652922993828,7cyclictest1002-21mta-sts-daemon22:02:121
3652922993822,3cyclictest171rcu_preempt22:55:011
365292199383,28cyclictest443-21gpm23:30:560
3652921993831,4cyclictest3739211-21grep22:31:040
3652921993829,6cyclictest439-21systemd-logind20:05:010
3652923993730,3cyclictest0-21swapper/222:05:542
3652923993729,5cyclictest3682930-21systemd-run20:50:012
3652923993729,5cyclictest3667894-21latency_hist20:00:012
3652923993729,5cyclictest3657439-21sh19:25:012
3652923993729,5cyclictest3657439-21sh19:25:012
3652922993731,3cyclictest0-21swapper/100:10:011
3652922993729,5cyclictest3707374-21ssh21:40:021
365292199372,28cyclictest3741792-21ssh22:36:000
3652923993627,5cyclictest0-21swapper/219:40:012
365292399361,28cyclictest3717574-21ssh21:55:572
3652922993629,4cyclictest3732819-21sendmail_mailqu22:21:041
3652922993628,5cyclictest3810726-21fschecks_count00:30:471
3652922993628,5cyclictest3773783-21latency_hist23:30:021
3652922993628,5cyclictest3755581-21grep23:00:011
3652921993626,7cyclictest3675435-21sh20:25:010
3652921993622,11cyclictest265-21systemd-journal23:55:010
3652921993619,15cyclictest1000-21mta-sts-daemon19:50:560
3652923993528,4cyclictest3767695-21cron23:20:012
3652923993528,3cyclictest0-21swapper/223:45:012
3652923993527,5cyclictest3690684-21unixbench_singl21:11:122
3652923993526,5cyclictest3795026-21latency_hist00:05:032
3652922993527,5cyclictest3667935-21systemd-run20:00:011
3652921993528,3cyclictest0-21swapper/023:00:520
3652921993526,6cyclictest3658983-21latency_hist19:30:020
3652921993525,7cyclictest3696012-21latency21:20:490
3652921993525,6cyclictest3735070-21aten2.4-expect22:25:450
3652923993428,3cyclictest0-21swapper/200:26:092
3652923993428,3cyclictest0-21swapper/200:26:092
3652923993426,5cyclictest3687300-21tail21:01:082
3652922993427,4cyclictest3764678-21munin-plugin-st23:15:011
3652922993427,4cyclictest3688919-21ssh21:10:011
3652922993427,4cyclictest3676936-21kworker/u8:8-rpciod21:56:491
3652922993427,3cyclictest0-21swapper/122:25:581
3652922993426,5cyclictest3814795-21grep00:36:081
3652922993426,5cyclictest3809976-21latency_hist00:30:001
3652922993426,5cyclictest3809976-21latency_hist00:30:001
3652922993426,5cyclictest3729083-21cpuspeed22:15:441
3652921993427,3cyclictest0-21swapper/021:55:560
3652921993426,5cyclictest1000-21mta-sts-daemon22:14:590
3652921993424,6cyclictest421-21dbus-daemon22:03:470
3652921993424,5cyclictest0-21swapper/022:09:520
365292399335,21cyclictest443-21gpm21:05:562
365292399333,23cyclictest3735621-21ssh22:25:562
3652923993326,4cyclictest0-21swapper/222:20:472
3652923993325,5cyclictest3714004-21cpuspeed21:50:452
3652923993325,5cyclictest3702201-21missed_timers21:30:542
3652923993323,7cyclictest3791844-21dpkg-db-backup00:00:012
3652923993323,7cyclictest3791844-21dpkg-db-backup00:00:012
3652923993323,7cyclictest3739073-21timerwakeupswit22:31:032
3652922993327,3cyclictest0-21swapper/120:25:011
3652922993325,5cyclictest3794907-21latency_hist00:05:021
3652922993325,5cyclictest3661776-21unixbench_multi19:36:061
3652922993325,4cyclictest1000-21mta-sts-daemon00:10:431
3652922993324,5cyclictest3670913-21cut20:10:011
3652922993322,4cyclictest1003-21mta-sts-daemon22:33:481
365292299331,26cyclictest3726653-21rm22:10:591
3652921993327,3cyclictest0-21swapper/023:16:060
3652921993325,4cyclictest0-21swapper/000:00:480
3652921993323,3cyclictest171rcu_preempt19:15:550
3652921993322,7cyclictest151ktimers/019:35:560
3652921993318,10cyclictest12650irq/32-eth000:15:570
3652921993318,10cyclictest12650irq/32-eth000:15:570
3652921993317,11cyclictest12650irq/32-eth022:40:580
3652921993317,11cyclictest12650irq/32-eth022:40:580
3652923993226,3cyclictest0-21swapper/221:45:012
3652923993225,4cyclictest3673905-21idleruntime-cro20:20:012
3652923993225,4cyclictest0-21swapper/223:20:572
3652923993224,5cyclictest3752590-21sh22:55:022
365292299325,15cyclictest3805114-21ssh00:20:561
365292299325,15cyclictest3805114-21ssh00:20:561
3652922993225,5cyclictest0-21swapper/121:30:021
3652922993224,5cyclictest3688907-21diskmemload21:45:501
3652922993224,3cyclictest0-21swapper/121:14:431
3652921993226,3cyclictest3683096-21latency_hist20:50:020
3652921993226,3cyclictest0-21swapper/021:31:050
3652921993225,4cyclictest1000-21mta-sts-daemon22:23:590
3652921993225,3cyclictest0-21swapper/021:35:450
3652921993221,7cyclictest420-21avahi-daemon21:52:580
3652921993217,8cyclictest3663408-21cron19:45:010
3652921993217,12cyclictest3678921-21fschecks_count20:35:470
365292399316,13cyclictest3741772-21ssh22:35:592
365292399314,23cyclictest3747742-21ssh22:45:552
3652923993124,3cyclictest0-21swapper/221:00:012
3652923993123,5cyclictest3816028-21munin-run00:40:012
3652923993123,5cyclictest3813208-21latency_hist00:35:022
3652923993123,5cyclictest3688907-21diskmemload00:10:422
3652923993123,4cyclictest0-21swapper/222:00:552
3652923993121,4cyclictest0-21swapper/221:25:572
365292299313,20cyclictest3747691-21ssh22:45:531
3652922993125,3cyclictest3766385-21unixbench_multi23:16:061
3652922993125,3cyclictest0-21swapper/120:40:001
3652922993124,4cyclictest3776702-21sh23:35:001
3652922993123,5cyclictest3782787-21systemd-run23:45:011
3652922993123,5cyclictest3695056-21latency_hist21:20:011
3652921993123,4cyclictest0-21swapper/022:15:550
3652921993122,6cyclictest1944774-21cups-browsed21:15:490
365292199312,22cyclictest3811122-21sh00:30:540
3652921993115,10cyclictest151ktimers/022:45:540
3652923993025,2cyclictest0-21swapper/222:10:482
3652923993023,4cyclictest3730102-21unixbench_multi22:16:052
3652923993021,6cyclictest3681422-21munin-plugin-st20:45:002
3652923993021,6cyclictest3681422-21munin-plugin-st20:45:002
3652923993021,5cyclictest3804113-21sh00:20:032
3652923993021,5cyclictest3804113-21sh00:20:032
3652923993021,4cyclictest0-21swapper/219:10:012
365292399301,24cyclictest3771893-21ssh23:25:562
3652922993024,2cyclictest0-21swapper/119:55:011
3652922993023,4cyclictest3656956-21processes19:21:011
3652922993023,4cyclictest3656956-21processes19:21:011
3652922993022,5cyclictest3686426-21fschecks_count21:00:471
3652921993026,2cyclictest0-21swapper/020:00:010
3652921993023,3cyclictest0-21swapper/021:41:050
3652921993023,3cyclictest0-21swapper/000:14:210
3652921993022,5cyclictest12650irq/32-eth023:10:520
3652921993010,14cyclictest665-21snmpd23:46:330
3652923992924,2cyclictest0-21swapper/219:10:522
3652923992923,4cyclictest3652920-21cyclictest23:00:462
3652923992922,3cyclictest0-21swapper/221:46:002
365292399292,22cyclictest3786908-21ssh23:50:532
3652923992921,5cyclictest3669351-21unixbench_singl20:01:132
3652923992920,7cyclictest3676915-21/usr/sbin/munin20:30:022
3652923992920,5cyclictest1003-21mta-sts-daemon21:24:102
365292299294,5cyclictest171rcu_preempt21:31:001
3652922992923,3cyclictest0-21swapper/119:11:041
3652922992922,4cyclictest0-21swapper/120:20:011
3652922992921,5cyclictest3698145-21idleruntime-cro21:25:011
3652922992921,5cyclictest3669441-21systemd-run20:05:011
365292199297,16cyclictest1001-21mta-sts-daemon20:50:540
3652921992922,4cyclictest3752563-21awk22:55:010
365292199291,23cyclictest3771973-21rm23:26:000
3652923992822,3cyclictest0-21swapper/223:15:022
3652923992821,4cyclictest0-21swapper/219:20:012
3652923992818,8cyclictest3665545-21kernelversion19:50:502
365292299282,19cyclictest665-21snmpd21:40:561
3652922992821,4cyclictest1001-21mta-sts-daemon20:45:491
3652922992820,5cyclictest3672420-21sh20:15:011
3652922992820,5cyclictest3660048-21sshd19:31:041
365292199285,4cyclictest171rcu_preempt20:40:540
365292199285,4cyclictest171rcu_preempt20:40:540
3652921992820,5cyclictest427-21polkitd21:00:440
3652921992816,10cyclictest57-21kcompactd023:43:270
3652923992720,3cyclictest0-21swapper/200:05:592
3652923992718,6cyclictest1944774-21cups-browsed20:23:522
3652922992710,11cyclictest1944774-21cups-browsed19:40:541
3652921992720,4cyclictest3678293-21unixbench_multi20:31:060
365292199271,6cyclictest0-21swapper/019:45:550
365292399263,13cyclictest3705370-21ssh21:35:512
3652923992621,2cyclictest0-21swapper/219:50:012
3652923992618,5cyclictest3670931-21systemd-run20:10:012
3652923992617,7cyclictest3684398-21sh20:55:002
3652922992620,3cyclictest0-21swapper/120:45:001
3652922992620,3cyclictest0-21swapper/120:45:001
3652922992619,5cyclictest3652920-21cyclictest23:00:451
3652922992617,5cyclictest1000-21mta-sts-daemon20:59:171
3652921992619,5cyclictest3660336-21unixbench_singl19:31:050
365292499251,6cyclictest0-21swapper/319:45:523
365292499251,5cyclictest0-21swapper/300:25:593
365292499251,5cyclictest0-21swapper/300:25:593
365292499251,19cyclictest0-21swapper/323:35:553
365292499251,19cyclictest0-21swapper/322:45:563
3652922992518,4cyclictest3658910-21cron19:30:001
365292199255,15cyclictest420-21avahi-daemon20:15:590
3652921992513,9cyclictest0-21swapper/019:15:000
365292499241,7cyclictest0-21swapper/323:15:573
365292499241,5cyclictest0-21swapper/323:10:593
365292499241,5cyclictest0-21swapper/321:25:443
365292499241,19cyclictest0-21swapper/322:30:553
3652923992416,5cyclictest420-21avahi-daemon19:41:022
3652923992416,5cyclictest3678402-21cron20:35:012
3652923992416,5cyclictest3672282-21tail20:11:092
3652921992412,9cyclictest3669904-21df20:05:460
365292499232,6cyclictest0-21swapper/319:50:563
365292499232,3cyclictest0-21swapper/323:31:033
365292499231,6cyclictest0-21swapper/322:55:553
365292499231,6cyclictest0-21swapper/320:05:593
365292499231,5cyclictest0-21swapper/300:06:023
365292499231,4cyclictest0-21swapper/323:00:533
365292499231,4cyclictest0-21swapper/322:40:583
365292499231,4cyclictest0-21swapper/322:40:583
365292499231,4cyclictest0-21swapper/322:35:593
365292499231,4cyclictest0-21swapper/321:40:553
365292499231,4cyclictest0-21swapper/320:50:513
365292499231,4cyclictest0-21swapper/319:10:563
365292499231,4cyclictest0-21swapper/300:15:543
365292499231,4cyclictest0-21swapper/300:15:543
365292499231,4cyclictest0-21swapper/300:10:553
365292499231,3cyclictest0-21swapper/320:40:533
365292499231,3cyclictest0-21swapper/320:40:533
365292499231,18cyclictest0-21swapper/322:05:573
365292299237,3cyclictest171rcu_preempt20:55:021
3652922992317,4cyclictest1003-21mta-sts-daemon20:30:391
365292199234,2cyclictest171rcu_preempt19:20:550
365292199234,2cyclictest171rcu_preempt19:20:550
365292499222,4cyclictest0-21swapper/323:45:523
365292499222,16cyclictest0-21swapper/323:10:013
365292499222,14cyclictest0-21swapper/321:30:593
365292499221,6cyclictest0-21swapper/320:45:533
365292499221,6cyclictest0-21swapper/300:01:023
365292499221,5cyclictest0-21swapper/323:40:563
365292499221,5cyclictest0-21swapper/321:46:073
365292499221,5cyclictest0-21swapper/320:30:563
365292499221,5cyclictest0-21swapper/300:35:553
365292499221,4cyclictest0-21swapper/323:25:533
365292499221,4cyclictest0-21swapper/322:20:593
365292499221,4cyclictest0-21swapper/322:15:543
365292499221,4cyclictest0-21swapper/321:05:523
365292499221,4cyclictest0-21swapper/320:25:523
365292499221,4cyclictest0-21swapper/319:40:543
365292499221,4cyclictest0-21swapper/319:35:533
365292499221,4cyclictest0-21swapper/319:26:023
365292499221,3cyclictest0-21swapper/323:51:003
365292499221,3cyclictest0-21swapper/322:50:523
365292499221,3cyclictest0-21swapper/321:20:523
365292499220,4cyclictest0-21swapper/320:15:583
365292499220,4cyclictest0-21swapper/300:20:553
365292499220,4cyclictest0-21swapper/300:20:553
365292499212,6cyclictest0-21swapper/323:55:593
365292499212,6cyclictest0-21swapper/323:55:593
365292499211,5cyclictest0-21swapper/321:51:083
365292499211,5cyclictest0-21swapper/319:10:013
365292499211,5cyclictest0-21swapper/300:30:543
365292499211,4cyclictest0-21swapper/323:21:003
365292499211,4cyclictest0-21swapper/322:25:553
365292499211,4cyclictest0-21swapper/322:00:543
365292499211,4cyclictest0-21swapper/321:55:523
365292499211,4cyclictest0-21swapper/321:35:553
365292499211,4cyclictest0-21swapper/321:16:003
365292499211,4cyclictest0-21swapper/320:55:573
365292499211,4cyclictest0-21swapper/320:35:543
365292499211,4cyclictest0-21swapper/320:20:573
365292499211,4cyclictest0-21swapper/319:55:583
365292499211,4cyclictest0-21swapper/319:30:563
365292499211,4cyclictest0-21swapper/319:20:553
365292499211,4cyclictest0-21swapper/319:20:553
365292499211,4cyclictest0-21swapper/319:20:013
365292499211,3cyclictest0-21swapper/322:10:443
365292499211,3cyclictest0-21swapper/320:11:033
365292499210,4cyclictest0-21swapper/321:10:483
365292499201,4cyclictest0-21swapper/320:00:543
365292499201,3cyclictest0-21swapper/321:00:533
365292399201,15cyclictest3658450-21ps19:26:052
365292199203,13cyclictest1-21systemd19:10:010
365292299151,11cyclictest3652933-21/usr/sbin/munin19:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional