You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-31 - 09:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri Jul 31, 2026 00:43:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2103991995226,9cyclictest2208843-21timerwakeupswit23:01:052
2103991995126,7cyclictest420-21avahi-daemon00:24:162
2103991994840,5cyclictest2186502-21cut22:25:382
2103991994726,4cyclictest1003-21mta-sts-daemon23:12:332
2103991994537,5cyclictest2243473-21sendmail00:00:012
2103989994524,5cyclictest2243543-21cups-browsed00:26:500
2103990994435,6cyclictest420-21avahi-daemon00:20:021
2103990994434,7cyclictest2243501-21idleruntime-cro00:00:011
2103989994423,5cyclictest2155784-21/usr/sbin/munin21:35:410
2103991994334,6cyclictest421-21dbus-daemon20:45:022
2103990994234,5cyclictest2124982-21systemctl20:20:011
2103989994235,4cyclictest2170680-21munin-run22:00:010
2103989994224,6cyclictest389646-21cups-browsed21:41:230
2103989994224,6cyclictest389646-21cups-browsed21:41:230
2103991994133,5cyclictest2252635-21latency_hist00:15:012
2103991994133,5cyclictest2252635-21latency_hist00:15:012
2103991994132,5cyclictest2164608-21systemd-run21:50:012
2103990994133,5cyclictest2220035-21fschecks_count23:20:421
2103990994133,5cyclictest2182933-21latency_hist22:20:011
2103990994132,6cyclictest2231438-21systemd-run23:40:011
2103991994032,5cyclictest2136965-21awk21:00:012
2103991994029,8cyclictest2158488-21systemd-run21:40:022
2103990994032,5cyclictest421-21dbus-daemon21:15:011
2103990994031,5cyclictest2179890-21systemd-run22:15:011
2103991993931,5cyclictest2147003-21kernelversion21:20:442
2103991993930,6cyclictest265-21systemd-journal00:35:012
2103990993931,5cyclictest389646-21cups-browsed20:57:471
2103990993931,5cyclictest2170677-21sendmail22:00:011
2103990993929,6cyclictest420-21avahi-daemon22:20:211
2103990993929,6cyclictest1-21systemd23:34:341
2103989993930,6cyclictest2192092-21idleruntime-cro22:35:010
2103989993929,7cyclictest389646-21cups-browsed20:00:520
2103991993832,3cyclictest0-21swapper/223:40:012
2103991993831,4cyclictest2182942-21munin-plugin-st22:20:012
2103991993830,5cyclictest2120998-21fschecks_count20:05:412
2103991993828,7cyclictest2211781-21switchtime23:06:062
2103991993827,7cyclictest421-21dbus-daemon21:52:242
2103990993830,5cyclictest2198147-21cron22:45:001
2103990993828,7cyclictest2127295-21needreboot20:25:461
2103990993828,7cyclictest2127295-21needreboot20:25:461
2103991993729,5cyclictest2193833-21head22:36:002
2103991993729,5cyclictest2125524-21fschecks_time20:20:422
2103991993729,5cyclictest2112985-21idleruntime-cro19:40:012
2103990993729,5cyclictest2234433-21systemd-run23:45:011
2103990993729,5cyclictest2234433-21systemd-run23:45:011
2103990993729,5cyclictest2207209-21ssh23:00:011
2103990993729,5cyclictest2207209-21ssh23:00:011
2103990993728,7cyclictest2118167-21latency19:55:451
2103990993728,6cyclictest420-21avahi-daemon22:02:011
2103990993728,6cyclictest389646-21cups-browsed21:49:161
2103990993727,8cyclictest2189110-21sh22:30:011
2103989993731,4cyclictest2240474-21ssh23:55:010
2103991993628,5cyclictest2139963-21cron21:10:012
2103991993628,5cyclictest2128002-21systemd-run20:30:012
2103991993628,5cyclictest2128002-21systemd-run20:30:012
2103991993626,6cyclictest389646-21cups-browsed21:58:492
2103990993629,4cyclictest2267735-21munin-run00:40:011
2103990993628,5cyclictest2157117-21unixbench_singl21:36:021
2103989993630,4cyclictest2149204-21munin-plugin-st21:25:010
2103989993628,6cyclictest389646-21cups-browsed23:18:450
2103991993528,3cyclictest2103988-21cyclictest22:14:352
2103991993527,5cyclictest389646-21cups-browsed21:11:392
2103991993527,5cyclictest2202212-21needreboot22:50:462
2103991993527,4cyclictest2104327-21sshd19:10:382
2103991993526,5cyclictest421-21dbus-daemon22:34:412
2103990993527,5cyclictest2175455-21unixbench_multi22:06:011
2103989993529,4cyclictest2180071-21latency_hist22:15:030
2103989993523,2cyclictest0-21swapper/000:15:010
2103989993523,2cyclictest0-21swapper/000:15:010
2103991993426,5cyclictest421-21dbus-daemon19:25:012
2103991993426,5cyclictest421-21dbus-daemon19:25:012
2103991993426,5cyclictest2222327-21sh23:25:012
2103991993426,5cyclictest2204649-21aten_r2power_cu22:55:392
2103991993426,5cyclictest2204649-21aten_r2power_cu22:55:392
2103990993423,3cyclictest171rcu_preempt22:35:411
2103989993426,6cyclictest2243550-21ssh00:00:020
2103989993425,6cyclictest2214975-21unixbench_multi23:11:100
2103991993328,2cyclictest0-21swapper/223:20:012
2103991993328,2cyclictest0-21swapper/222:20:552
2103991993325,5cyclictest428-21mta-sts-daemon23:30:442
2103991993325,5cyclictest2143547-21aten_r2power_en21:15:422
2103991993325,5cyclictest2117477-21idleruntime-cro19:55:012
2103991993325,5cyclictest2110497-21fschecks_count19:30:442
2103991993320,5cyclictest1001-21mta-sts-daemon23:46:002
2103990993327,3cyclictest0-21swapper/123:20:011
2103989993326,4cyclictest1003-21mta-sts-daemon00:34:510
2103989993325,5cyclictest567-21gdbus23:20:450
2103989993325,5cyclictest2149624-21/usr/sbin/munin21:26:040
2103989993324,4cyclictest2266431-21unixbench_singl00:36:030
2103989993323,7cyclictest2186571-21basename22:25:390
2103991993226,3cyclictest0-21swapper/222:05:022
2103991993225,3cyclictest0-21swapper/219:15:562
2103991993224,5cyclictest2175112-21mailstats22:05:572
2103991993223,6cyclictest601-21gmain00:27:292
2103990993227,2cyclictest0-21swapper/121:55:011
2103990993226,4cyclictest0-21swapper/100:15:011
2103990993226,4cyclictest0-21swapper/100:15:011
2103990993224,5cyclictest265-21systemd-journal19:55:011
2103990993224,5cyclictest2263397-21unixbench_singl00:31:021
2103990993224,5cyclictest2225495-21ssh23:30:021
2103990993223,6cyclictest1-21systemd00:05:021
2103989993225,4cyclictest2153332-21needreboot21:30:470
2103989993225,3cyclictest0-21swapper/022:40:020
2103989993223,6cyclictest2198159-21cut22:45:010
2103991993125,3cyclictest1001-21mta-sts-daemon23:50:382
2103991993125,3cyclictest0-21swapper/221:01:012
2103991993124,4cyclictest2115690-21threads19:45:582
2103991993122,7cyclictest694-21cron23:30:012
2103991993122,7cyclictest694-21cron19:30:002
2103990993125,3cyclictest0-21swapper/122:50:391
2103990993122,6cyclictest1-21systemd22:35:021
2103990993122,5cyclictest2257362-21unixbench_singl00:21:031
2103990993120,8cyclictest2143998-21latency21:15:481
2103989993124,4cyclictest2207985-21fschecks_count23:00:460
2103989993124,4cyclictest2173771-21ssh22:05:010
2103989993124,4cyclictest1-21systemd20:50:020
2103989993121,7cyclictest2207224-21sendmail23:00:010
2103989993121,7cyclictest2207224-21sendmail23:00:010
2103991993024,3cyclictest0-21swapper/200:00:522
2103991993023,4cyclictest2233072-21unixbench-2d23:41:092
2103991993023,4cyclictest2233072-21unixbench-2d23:41:092
2103991993023,4cyclictest2103939-21latency_hist19:10:022
2103991993022,5cyclictest420-21avahi-daemon20:00:522
2103991993021,7cyclictest2153828-21timerandwakeup21:31:012
2103991993018,5cyclictest488920-21munin-node22:42:362
2103990993024,3cyclictest0-21swapper/121:20:391
2103990993024,3cyclictest0-21swapper/100:30:011
2103990993022,5cyclictest439-21systemd-logind19:50:011
2103990993021,6cyclictest421-21dbus-daemon23:50:571
2103990993021,6cyclictest420-21avahi-daemon20:05:381
2103990993021,6cyclictest389643-21cupsd23:00:121
2103989993025,3cyclictest2213285-21idleruntime-cro23:10:010
2103989993025,2cyclictest0-21swapper/000:20:010
2103989993024,4cyclictest2163257-21grep21:46:010
2103989993021,4cyclictest2204302-21ssh22:55:030
2103989993020,8cyclictest2257094-21switchtime00:21:010
2103989993020,7cyclictest2165793-21ntp_offset21:50:530
2103991992923,3cyclictest70-21kswapd022:45:532
2103991992923,3cyclictest0-21swapper/221:25:392
2103991992921,5cyclictest389646-21cups-browsed20:10:482
2103990992921,5cyclictest389646-21cups-browsed23:12:311
2103990992921,5cyclictest265-21systemd-journal20:40:011
2103990992920,7cyclictest2132479-21/usr/sbin/munin20:45:021
2103990992919,7cyclictest2155367-21cut21:35:021
2103990992919,7cyclictest2126491-21munin-plugin-st20:25:011
2103990992919,7cyclictest2106371-21ntp_kernel_pll_19:15:511
2103989992924,2cyclictest0-21swapper/022:15:550
2103989992924,2cyclictest0-21swapper/021:00:010
2103989992923,4cyclictest2118860-21tail19:56:010
2103989992922,5cyclictest421-21dbus-daemon23:50:010
2103989992922,5cyclictest420-21avahi-daemon00:00:310
2103989992922,5cyclictest0-21swapper/022:20:560
2103989992922,4cyclictest2129996-21fschecks_time20:35:410
2103989992922,4cyclictest1001-21mta-sts-daemon19:10:350
2103989992922,4cyclictest0-21swapper/020:45:010
210398999292,21cyclictest665-21snmpd20:30:520
2103989992921,6cyclictest2126823-21aten_r2power_fr20:25:390
2103989992921,6cyclictest2126823-21aten_r2power_fr20:25:390
2103989992921,6cyclictest2112975-21sendmail19:40:010
2103989992919,4cyclictest1003-21mta-sts-daemon20:15:370
2103990992823,2cyclictest0-21swapper/100:10:001
2103990992822,4cyclictest2103988-21cyclictest21:05:031
2103990992822,3cyclictest0-21swapper/123:45:391
2103990992822,3cyclictest0-21swapper/120:50:591
2103990992821,4cyclictest2161523-21debian-sa121:45:001
2103990992821,4cyclictest2161523-21debian-sa121:45:001
2103990992820,5cyclictest421-21dbus-daemon21:30:021
2103990992820,5cyclictest389646-21cups-browsed19:10:571
2103990992820,5cyclictest2201263-21latency_hist22:50:021
2103990992820,5cyclictest2133004-21fschecks_time20:45:411
2103989992823,3cyclictest2176826-21sh22:10:010
2103989992821,4cyclictest2225351-21munin-run23:30:010
2103989992820,6cyclictest2230242-21sh23:36:220
2103989992819,7cyclictest2226836-21switchtime23:31:000
2103989992819,7cyclictest2107268-21aten_r2power_cu19:20:380
2103989992819,7cyclictest2107268-21aten_r2power_cu19:20:380
2103989992818,8cyclictest2247305-21fw_packets00:05:440
2103989992817,4cyclictest2139971-21diskmemload22:45:460
2103991992722,2cyclictest0-21swapper/200:35:392
2103991992720,5cyclictest1001-21mta-sts-daemon00:07:532
2103990992720,4cyclictest1000-21mta-sts-daemon19:30:511
2103990992718,6cyclictest421-21dbus-daemon23:08:171
2103989992721,4cyclictest2144168-21ld21:15:520
2103989992721,4cyclictest2141568-21timerwakeupswit21:11:060
2103989992721,4cyclictest2134482-21fschecks_count20:50:410
210398999271,20cyclictest665-21snmpd23:43:550
210398999271,20cyclictest665-21snmpd23:43:550
2103991992621,2cyclictest0-21swapper/220:40:012
2103991992620,2cyclictest0-21swapper/220:51:012
2103991992619,4cyclictest2253715-21ssh00:15:492
2103991992619,4cyclictest2124970-21idleruntime-cro20:20:012
2103991992618,5cyclictest2160224-21unixbench_singl21:41:022
2103991992618,5cyclictest2160224-21unixbench_singl21:41:022
2103990992620,4cyclictest0-21swapper/120:10:551
2103989992613,7cyclictest2115325-21ntp_kernel_err19:45:480
2103991992520,2cyclictest0-21swapper/220:00:012
2103991992519,3cyclictest1001-21mta-sts-daemon20:45:092
2103990992520,3cyclictest0-21swapper/119:36:001
2103990992519,3cyclictest0-21swapper/121:05:491
2103990992518,4cyclictest1003-21mta-sts-daemon19:28:471
210399299244,13cyclictest0-21swapper/322:20:553
210399299244,12cyclictest0-21swapper/321:30:493
210399299244,11cyclictest0-21swapper/300:15:523
210399299243,14cyclictest0-21swapper/322:15:523
2103991992418,3cyclictest0-21swapper/219:40:462
2103990992419,2cyclictest0-21swapper/119:20:591
2103990992419,2cyclictest0-21swapper/119:20:591
2103990992418,3cyclictest1003-21mta-sts-daemon20:30:411
2103990992417,4cyclictest2092780-21kworker/u8:8-xprtiod19:10:021
2103990992414,7cyclictest2114450-21vmstat19:41:001
2103989992418,4cyclictest623-21sshd21:04:390
2103989992418,3cyclictest2103997-21kworker/u8:9-xprtiod19:30:410
2103989992417,5cyclictest2122992-21proc_pri20:10:550
2103989992417,5cyclictest2105934-21cpuspeed19:15:410
210399299234,11cyclictest0-21swapper/322:25:403
210399299234,10cyclictest0-21swapper/322:50:473
210399299233,12cyclictest0-21swapper/322:55:523
210399299233,12cyclictest0-21swapper/322:55:523
210399299233,12cyclictest0-21swapper/319:15:553
210399299232,12cyclictest0-21swapper/323:45:493
2103990992318,2cyclictest0-21swapper/120:00:521
2103989992318,3cyclictest1000-21mta-sts-daemon20:21:110
2103989992318,2cyclictest0-21swapper/019:30:000
2103989992317,4cyclictest2114409-21unixbench_singl19:40:590
2103989992315,5cyclictest0-21swapper/021:05:580
210399299224,11cyclictest0-21swapper/323:55:493
210399299223,13cyclictest0-21swapper/323:35:513
210399299223,12cyclictest0-21swapper/323:30:523
210399299222,13cyclictest0-21swapper/321:35:573
210399299222,12cyclictest0-21swapper/300:10:543
210399299222,12cyclictest0-21swapper/300:10:543
210399299222,11cyclictest0-21swapper/321:15:563
210399299222,11cyclictest0-21swapper/300:05:503
2103991992215,3cyclictest0-21swapper/220:30:592
2103989992214,3cyclictest2109964-21kworker/u8:10-nfsiod20:10:020
210398999221,16cyclictest2116957-21ntpq19:50:530
210399299214,8cyclictest0-21swapper/320:50:493
210399299213,9cyclictest0-21swapper/322:40:483
210399299213,12cyclictest0-21swapper/323:15:503
210399299213,12cyclictest0-21swapper/322:00:513
210399299213,11cyclictest0-21swapper/323:50:533
210399299213,11cyclictest0-21swapper/322:05:503
210399299213,10cyclictest0-21swapper/321:10:543
210399299212,12cyclictest0-21swapper/323:40:543
210399299212,12cyclictest0-21swapper/323:40:543
210399299212,12cyclictest0-21swapper/321:40:483
210399299212,12cyclictest0-21swapper/321:40:483
210399299212,12cyclictest0-21swapper/321:20:503
210399299212,11cyclictest0-21swapper/321:55:483
210399299212,10cyclictest0-21swapper/322:10:393
210399299205,9cyclictest0-21swapper/323:05:533
210399299203,9cyclictest0-21swapper/320:10:483
210399299203,9cyclictest0-21swapper/319:50:473
210399299203,9cyclictest0-21swapper/319:30:513
210399299203,10cyclictest0-21swapper/323:20:393
210399299202,11cyclictest0-21swapper/323:25:513
210399299202,11cyclictest0-21swapper/321:45:473
210399299202,11cyclictest0-21swapper/319:40:533
210399299202,10cyclictest0-21swapper/300:25:523
2103989992016,3cyclictest2103996-21kworker/u8:5-xprtiod19:10:020
210399299196,9cyclictest0-21swapper/300:30:503
210399299193,12cyclictest0-21swapper/323:10:553
210399299193,11cyclictest0-21swapper/321:25:533
210399299193,11cyclictest0-21swapper/319:25:513
210399299192,9cyclictest0-21swapper/320:25:473
210399299192,9cyclictest0-21swapper/320:25:473
210399299192,8cyclictest0-21swapper/300:20:393
210399299192,12cyclictest0-21swapper/323:00:543
210399299192,11cyclictest0-21swapper/300:00:503
210399299191,11cyclictest0-21swapper/322:45:373
210399299191,11cyclictest0-21swapper/300:35:583
210399299191,10cyclictest0-21swapper/320:45:533
210399299183,9cyclictest0-21swapper/321:50:493
210399299183,8cyclictest0-21swapper/320:30:503
210399299183,8cyclictest0-21swapper/319:20:493
210399299183,8cyclictest0-21swapper/319:20:493
210399299183,7cyclictest0-21swapper/322:35:383
210399299183,7cyclictest0-21swapper/320:00:493
210399299182,9cyclictest0-21swapper/320:05:483
210399299182,10cyclictest0-21swapper/322:30:433
210399299182,10cyclictest0-21swapper/320:35:533
210399299174,8cyclictest0-21swapper/320:55:523
210399299174,8cyclictest0-21swapper/320:20:503
210399299172,9cyclictest0-21swapper/320:15:493
210399299172,9cyclictest0-21swapper/319:10:523
210399299171,9cyclictest0-21swapper/321:05:463
210399299171,9cyclictest0-21swapper/320:40:493
210399299164,9cyclictest0-21swapper/321:00:483
210399299162,8cyclictest0-21swapper/319:55:483
210399299154,7cyclictest0-21swapper/319:45:473
210399299152,7cyclictest0-21swapper/319:35:473
210399299132,8cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional