You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 02:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Aug 26, 2026 00:42:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
294858996338,18cyclictest12650irq/32-eth021:21:090
294860995746,7cyclictest336966-21munin-run21:20:012
294860995444,6cyclictest349272-21munin-run21:40:012
29485999515,35cyclictest408418-21ssh23:16:001
294860995041,5cyclictest450095-21ssh00:25:012
29486099503,37cyclictest332145-21ssh21:11:082
294858994926,15cyclictest12650irq/32-eth022:06:070
294858994733,5cyclictest425640-21ssh23:45:010
294860994637,4cyclictest0-21swapper/221:26:062
294860994634,8cyclictest1002-21mta-sts-daemon20:51:152
294859994634,8cyclictest2702384-21cups-browsed20:56:111
294858994633,10cyclictest447510-40mandb00:20:500
294860994537,5cyclictest401187-21systemd-run23:05:022
294860994537,5cyclictest401187-21systemd-run23:05:022
294860994531,10cyclictest362959-21mailstats22:01:112
29486099444,30cyclictest454334-21ssh00:31:032
294860994332,7cyclictest421-21dbus-daemon23:13:282
294859994334,5cyclictest0-21swapper/121:15:591
294860994231,7cyclictest0-21swapper/223:36:002
294859994233,6cyclictest373711-21cron22:20:011
294859994233,6cyclictest373711-21cron22:20:011
294859994232,7cyclictest347018-21fschecks_time21:35:561
294859994232,4cyclictest0-21swapper/100:31:041
29486099414,8cyclictest171rcu_preempt21:06:122
294860994134,4cyclictest392042-21kworker/u8:3-xprtiod23:45:012
294860994133,5cyclictest318950-21latency_hist20:30:022
294860994132,6cyclictest439-21systemd-logind20:50:012
294860994127,9cyclictest2702384-21cups-browsed23:16:022
294860994029,8cyclictest459273-21sendmail00:40:012
294859994029,8cyclictest430120-21mailstats23:51:091
294858994035,2cyclictest0-21swapper/023:30:020
29486099398,22cyclictest1002-21mta-sts-daemon23:33:492
294860993931,5cyclictest398116-21ssh23:00:012
294860993929,6cyclictest485-21in:imuxsock00:00:022
294860993929,6cyclictest303848-21idleruntime-cro19:40:012
29485899395,27cyclictest665-21snmpd21:51:030
294858993932,4cyclictest151ktimers/023:31:040
29486099385,7cyclictest171rcu_preempt22:11:022
294860993830,5cyclictest416424-21/usr/sbin/munin23:30:012
294860993830,5cyclictest346198-21latency_hist21:35:012
294860993829,6cyclictest376783-21munin-plugin-st22:25:002
294860993826,5cyclictest0-21swapper/200:26:062
294859993830,5cyclictest436234-21proc_pri00:01:101
294859993830,5cyclictest296337-21munin-plugin-st19:15:001
294859993824,11cyclictest356502-21needreboot21:50:591
294858993829,6cyclictest434877-21cupsd00:05:350
294860993729,5cyclictest313372-21fschecks_time20:10:562
294860993729,5cyclictest313372-21fschecks_time20:10:562
294860993726,8cyclictest444000-21/usr/sbin/munin00:15:022
294860993726,7cyclictest420-21avahi-daemon00:05:252
294860993725,9cyclictest296340-21/usr/sbin/munin19:15:012
294860993724,10cyclictest322655-21needreboot20:41:022
294859993729,5cyclictest330838-21diskmemload23:20:591
294859993728,6cyclictest308328-21latency_hist19:55:011
294858993731,4cyclictest396869-21unixbench_singl22:56:190
294858993731,4cyclictest388962-21cron22:45:010
294860993628,5cyclictest416443-21kworker/u8:1+xprtiod00:20:012
294860993628,4cyclictest0-21swapper/222:50:572
294860993627,4cyclictest0-21swapper/223:51:282
294860993627,4cyclictest0-21swapper/222:30:022
294860993627,4cyclictest0-21swapper/222:30:022
294860993626,8cyclictest387209-21collect222:41:052
294860993626,5cyclictest0-21swapper/223:09:482
294859993629,4cyclictest368722-21cut22:11:001
294859993628,5cyclictest445004-21latency00:16:021
294859993628,5cyclictest439636-21unixbench-2d00:06:151
294859993628,5cyclictest320325-21cron20:35:011
294859993627,6cyclictest567-21gdbus20:51:191
294859993627,6cyclictest378537-21unixbench_multi22:26:161
294859993627,6cyclictest378537-21unixbench_multi22:26:161
294859993626,6cyclictest317352-21munin-plugin-st20:25:001
294859993626,6cyclictest317352-21munin-plugin-st20:25:001
29485899367,21cyclictest454347-21ssh00:31:040
294858993630,3cyclictest434879-21modprobe00:00:020
294858993630,3cyclictest416450-21kworker/u8:8-nfsiod00:15:010
294858993629,4cyclictest151ktimers/022:51:080
294860993528,4cyclictest327816-21cron21:00:012
294860993526,6cyclictest350191-21if_wlan021:40:562
294860993526,6cyclictest350191-21if_wlan021:40:562
294860993526,6cyclictest296870-21fschecks_time19:15:552
294860993526,5cyclictest436621-21unixbench_multi00:01:152
29486099352,25cyclictest411609-21ssh23:21:052
294859993530,2cyclictest0-21swapper/100:00:021
294859993528,4cyclictest0-21swapper/121:46:051
294859993528,4cyclictest0-21swapper/121:46:051
294859993527,5cyclictest439-21systemd-logind00:30:011
294859993527,4cyclictest1000-21mta-sts-daemon21:21:251
294859993526,4cyclictest0-21swapper/120:00:011
294859993524,8cyclictest399126-21latency23:00:581
294859993524,8cyclictest399126-21latency23:00:581
294859993524,8cyclictest363170-21timerwakeupswit22:01:141
29486099346,17cyclictest428-21mta-sts-daemon21:21:042
29486099343,20cyclictest359647-21ssh21:56:022
294860993426,5cyclictest427409-21tail23:46:142
294860993426,4cyclictest1003-21mta-sts-daemon22:15:392
294860993426,4cyclictest1003-21mta-sts-daemon22:15:392
294860993425,5cyclictest1000-21mta-sts-daemon22:45:532
294860993417,12cyclictest351ktimers/219:56:022
294859993428,3cyclictest0-21swapper/122:40:011
294859993427,4cyclictest304079-21/usr/sbin/munin19:41:011
294859993426,5cyclictest417392-21kernelversion23:30:591
294859993426,5cyclictest315830-21cron20:20:001
294859993426,4cyclictest0-21swapper/123:26:041
29485899343,24cyclictest371850-21ssh22:16:030
29485899343,24cyclictest371850-21ssh22:16:030
294858993428,3cyclictest292194-21kworker/u8:10+xprtiod19:40:010
294858993427,4cyclictest151ktimers/022:11:050
294858993426,5cyclictest1002-21mta-sts-daemon23:19:500
294858993425,6cyclictest385959-21ssh22:40:020
294858993424,7cyclictest299357-21/usr/sbin/munin19:25:010
29485899342,30cyclictest0-21swapper/000:01:040
294860993326,3cyclictest0-21swapper/222:40:012
294860993325,5cyclictest366371-21unixbench_singl22:06:152
294860993317,12cyclictest351ktimers/220:21:002
294860993317,12cyclictest351ktimers/220:21:002
294859993328,2cyclictest0-21swapper/121:45:011
294859993328,2cyclictest0-21swapper/121:45:011
294859993327,3cyclictest0-21swapper/121:30:011
294859993325,5cyclictest421-21dbus-daemon19:35:021
294859993324,4cyclictest0-21swapper/123:11:011
294859993323,7cyclictest393654-21timerandwakeup22:51:131
294859993321,9cyclictest427289-21timerandwakeup23:46:131
294858993325,5cyclictest2702384-21cups-browsed21:13:590
294860993226,3cyclictest308352-21kworker/u8:5+rpciod20:20:012
294860993224,5cyclictest1001-21mta-sts-daemon20:06:132
294860993224,3cyclictest0-21swapper/221:45:522
294860993224,3cyclictest0-21swapper/221:45:522
294860993221,8cyclictest355409-21kworker/u8:5-xprtiod21:54:482
294859993226,3cyclictest0-21swapper/100:11:021
294859993224,5cyclictest459264-21cut00:40:011
294859993224,4cyclictest0-21swapper/123:05:521
294859993223,6cyclictest387668-21tail22:41:131
294858993226,3cyclictest0-21swapper/020:55:010
294858993225,4cyclictest316386-21fschecks_time20:20:550
294858993225,4cyclictest316386-21fschecks_time20:20:550
294858993224,5cyclictest429188-21/usr/sbin/munin23:51:080
294858993222,7cyclictest457183-21latency00:35:580
29485899322,23cyclictest390127-21ssh22:46:000
294860993124,3cyclictest0-21swapper/222:31:022
294859993124,5cyclictest300376-21grep19:26:081
294859993123,5cyclictest398112-21sendmail23:00:011
294859993123,5cyclictest359255-21gltestperf21:55:541
294859993123,5cyclictest310367-21fschecks_count20:00:541
294859993123,5cyclictest303835-21munin-plugin-st19:40:011
294859993122,5cyclictest422585-21idleruntime-cro23:40:011
294859993121,7cyclictest382856-21cron22:35:011
294858993124,3cyclictest0-21swapper/021:42:480
294858993124,3cyclictest0-21swapper/021:42:480
294858993123,5cyclictest2702384-21cups-browsed22:33:390
294859993024,3cyclictest0-21swapper/122:21:091
294859993023,3cyclictest0-21swapper/122:06:031
29485999302,24cyclictest448264-21ssh00:21:041
294859993019,8cyclictest2702384-21cups-browsed23:45:031
294858993025,3cyclictest0-21swapper/021:46:050
294858993025,3cyclictest0-21swapper/021:46:050
294858993025,2cyclictest0-21swapper/021:40:010
294858993024,4cyclictest426455-21fschecks_count23:45:550
294858993023,5cyclictest2702384-21cups-browsed21:30:520
294858993023,4cyclictest401161-21cron23:05:010
294858993023,4cyclictest401161-21cron23:05:010
294858993023,4cyclictest359375-21kernelversion21:55:560
294858993023,4cyclictest315832-21idleruntime-cro20:20:000
294858993023,3cyclictest0-21swapper/000:25:530
29485899302,23cyclictest362756-21sh22:01:040
294858993021,7cyclictest444567-21aten_r2power_en00:15:540
294858993016,9cyclictest12650irq/32-eth021:16:090
294858993012,6cyclictest567-21gdbus23:40:010
29486099292,19cyclictest665-21snmpd19:21:012
294859992921,5cyclictest297841-21munin-run19:20:001
294859992920,6cyclictest330838-21diskmemload21:33:481
294858992925,2cyclictest0-21swapper/020:26:220
294858992921,6cyclictest312158-21needreboot20:06:000
294858992917,9cyclictest379813-21cut22:30:010
294858992917,9cyclictest379813-21cut22:30:010
294858992912,15cyclictest0-21swapper/022:25:000
294860992821,5cyclictest321539-21xargs20:36:122
294860992821,4cyclictest329268-21unixbench_singl21:01:122
294860992819,6cyclictest2702384-21cups-browsed20:31:012
294860992819,6cyclictest1-21systemd19:50:012
294860992818,7cyclictest300865-21sh19:30:022
294860992818,5cyclictest1001-21mta-sts-daemon20:01:042
294859992824,2cyclictest0-21swapper/120:40:011
294859992822,4cyclictest0-21swapper/122:45:531
29485899283,11cyclictest330838-21diskmemload23:14:410
294859992720,4cyclictest427-21polkitd21:06:491
294859992720,4cyclictest0-21swapper/121:11:051
294859992719,5cyclictest299199-21unixbench-2d19:21:121
294859992718,6cyclictest329336-21cron21:05:011
294858992720,4cyclictest401890-21cpuspeed23:05:510
294858992718,6cyclictest329870-21grep21:06:040
294858992718,6cyclictest329364-21systemd-run21:05:010
294858992717,7cyclictest341733-21timerwakeupswit21:26:130
29485899271,22cyclictest367624-21kworker/u8:7+rpciod23:21:020
294860992619,4cyclictest307478-21kernelversion19:50:572
294859992617,6cyclictest1000-21mta-sts-daemon19:45:561
294858992619,5cyclictest323869-21fschecks_time20:45:570
294858992618,5cyclictest2702384-21cups-browsed19:14:430
294858992616,7cyclictest0-21swapper/020:40:020
294860992519,3cyclictest0-21swapper/219:41:132
294860992510,9cyclictest421-21dbus-daemon19:10:012
294859992517,5cyclictest2702384-21cups-browsed20:43:301
294858992520,3cyclictest315878-21kworker/u8:4-xprtiod20:56:000
294858992519,4cyclictest2702384-21cups-browsed19:58:560
294859992417,3cyclictest0-21swapper/120:11:091
294859992417,3cyclictest0-21swapper/120:11:091
294859992412,8cyclictest317733-21/usr/sbin/munin20:25:521
29485899242,16cyclictest306254-21ntpq19:46:070
294858992417,5cyclictest313852-21proc_pri20:11:090
294858992417,5cyclictest313852-21proc_pri20:11:090
294858992417,5cyclictest265-21systemd-journal20:03:480
294858992417,5cyclictest1003-21mta-sts-daemon19:40:570
294858992417,4cyclictest420-21avahi-daemon20:44:580
29486099234,15cyclictest428-21mta-sts-daemon19:31:082
294859992315,5cyclictest312835-21cron20:10:011
294858992316,5cyclictest307899-21sendmail_mailst19:51:100
294858992315,5cyclictest567-21gdbus19:19:510
294858992314,3cyclictest320219-21unixbench_multi20:31:130
294859992214,5cyclictest2800149-21munin-node20:45:561
294858992217,3cyclictest301656-21needreboot19:31:010
29485899211,17cyclictest300270-21ntpq19:26:030
294858992015,2cyclictest0-21swapper/019:10:010
294859991914,2cyclictest0-21swapper/119:10:011
29486199161,7cyclictest0-21swapper/300:26:013
29486199151,7cyclictest0-21swapper/320:46:023
29486199143,4cyclictest0-21swapper/320:16:033
29486199142,7cyclictest0-21swapper/323:56:063
29486199142,6cyclictest0-21swapper/323:01:063
29486199142,6cyclictest0-21swapper/323:01:063
29486199142,5cyclictest0-21swapper/319:36:013
29486199142,5cyclictest0-21swapper/300:11:003
29486199141,8cyclictest0-21swapper/322:30:533
29486199141,7cyclictest0-21swapper/322:00:523
29486199141,6cyclictest0-21swapper/323:46:003
29486199141,6cyclictest0-21swapper/323:21:013
29486199141,6cyclictest0-21swapper/300:20:473
29486199134,6cyclictest0-21swapper/323:51:013
29486199133,6cyclictest0-21swapper/321:26:013
29486199132,6cyclictest0-21swapper/323:41:043
29486199132,6cyclictest0-21swapper/323:33:493
29486199132,6cyclictest0-21swapper/322:46:083
29486199132,6cyclictest0-21swapper/321:40:533
29486199132,6cyclictest0-21swapper/321:40:533
29486199132,6cyclictest0-21swapper/300:36:073
29486199132,5cyclictest0-21swapper/322:15:573
29486199132,5cyclictest0-21swapper/322:15:573
29486199132,5cyclictest0-21swapper/321:45:513
29486199132,5cyclictest0-21swapper/321:45:513
29486199132,5cyclictest0-21swapper/321:16:083
29486199132,5cyclictest0-21swapper/319:56:033
29486199132,4cyclictest0-21swapper/320:26:153
29486199131,7cyclictest0-21swapper/322:35:493
29486199131,7cyclictest0-21swapper/321:55:593
29486199131,6cyclictest0-21swapper/322:05:523
29486199131,5cyclictest0-21swapper/319:16:033
29486199131,4cyclictest0-21swapper/321:51:003
29486199124,5cyclictest0-21swapper/323:05:593
29486199122,6cyclictest0-21swapper/321:31:033
29486199122,6cyclictest0-21swapper/300:20:023
29486199122,5cyclictest0-21swapper/319:46:043
29486199121,6cyclictest0-21swapper/323:25:553
29486199121,6cyclictest0-21swapper/322:56:043
29486199121,6cyclictest0-21swapper/321:35:483
29486199121,6cyclictest0-21swapper/320:36:043
29486199121,6cyclictest0-21swapper/320:15:013
29486199121,6cyclictest0-21swapper/320:15:013
29486199121,6cyclictest0-21swapper/300:31:023
29486199121,5cyclictest0-21swapper/321:06:123
29486199121,5cyclictest0-21swapper/320:51:143
29486199121,5cyclictest0-21swapper/319:21:003
29486199121,3cyclictest0-21swapper/300:01:023
29486199115,4cyclictest0-21swapper/319:11:063
29486199114,5cyclictest0-21swapper/322:41:053
29486199114,4cyclictest0-21swapper/323:16:003
29486199112,5cyclictest0-21swapper/323:10:503
29486199112,5cyclictest0-21swapper/319:51:043
29486199112,5cyclictest0-21swapper/319:45:013
29486199111,6cyclictest0-21swapper/323:36:013
29486199111,6cyclictest0-21swapper/322:50:533
29486199111,6cyclictest0-21swapper/321:10:593
29486199111,6cyclictest0-21swapper/319:26:063
29486199111,6cyclictest0-21swapper/300:05:463
29486199111,5cyclictest0-21swapper/322:20:503
29486199111,5cyclictest0-21swapper/320:21:033
29486199111,5cyclictest0-21swapper/320:21:033
29486199111,5cyclictest0-21swapper/319:31:053
29486199102,6cyclictest0-21swapper/320:55:523
29486199102,5cyclictest0-21swapper/321:20:533
29486199102,5cyclictest0-21swapper/320:31:023
29486199101,6cyclictest0-21swapper/321:01:013
29486199101,5cyclictest0-21swapper/322:25:543
29486199101,5cyclictest0-21swapper/322:25:543
29486199101,5cyclictest0-21swapper/322:10:523
29486199101,5cyclictest0-21swapper/320:40:513
29486199101,5cyclictest0-21swapper/320:06:033
29486199101,5cyclictest0-21swapper/320:01:033
2948619981,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional