You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 01:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri Apr 24, 2026 00:43:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
237529995244,5cyclictest322072-21systemd-run22:10:012
237529995040,7cyclictest315230-21ssh22:00:422
237528995042,5cyclictest439-21systemd-logind21:25:011
237527995025,10cyclictest12450irq/33-eth023:20:420
237527995025,10cyclictest12450irq/33-eth023:20:420
237528994838,6cyclictest305910-21systemd-run21:50:011
237528994741,3cyclictest0-21swapper/121:40:011
237527994632,11cyclictest313975-21grep22:00:000
237527994626,12cyclictest12350irq/32-eth021:15:480
237529994535,7cyclictest390842-21sh23:35:012
237528994436,5cyclictest415770-21aten_r2power_po00:05:311
237528994436,5cyclictest375857-21sh23:15:421
237528994436,5cyclictest250425-21sshd19:50:421
237528994434,6cyclictest420-21avahi-daemon23:46:201
237529994335,5cyclictest293779-21systemd-run21:35:012
237529994233,6cyclictest343699-21wc22:35:492
237529994233,6cyclictest343699-21wc22:35:492
237529994232,6cyclictest420-21avahi-daemon21:51:172
237529994134,4cyclictest366590-21cut23:05:012
237529994130,8cyclictest285742-21ssh21:25:012
237528994132,6cyclictest516-21in:imuxsock22:40:011
237528994132,6cyclictest516-21in:imuxsock22:40:011
23752799413,34cyclictest392016-21ssh23:35:410
237529994031,5cyclictest411065-21kworker/u8:11-xprtiod00:15:432
237528994033,4cyclictest0-21swapper/120:20:001
237528994032,5cyclictest1-21systemd22:45:011
237527994028,9cyclictest240528-21idleruntime-cro19:20:010
237529993934,2cyclictest0-21swapper/223:55:392
237529993934,2cyclictest0-21swapper/200:14:422
237529993931,5cyclictest345119-21kworker/u8:7+xprtiod23:20:012
237529993931,5cyclictest1-21systemd19:15:012
237529993930,7cyclictest362524-21awk23:00:012
237528993932,4cyclictest0-21swapper/100:25:011
237528993930,6cyclictest421-21dbus-daemon22:50:011
237527993930,6cyclictest420-21avahi-daemon23:30:450
237527993930,6cyclictest420-21avahi-daemon19:30:460
237528993832,3cyclictest0-21swapper/121:20:011
237528993830,5cyclictest423993-21fschecks_time00:15:331
237528993830,5cyclictest2619226-21cups-browsed19:20:291
237528993829,6cyclictest286652-21/usr/sbin/munin21:25:371
237528993828,7cyclictest411033-21dpkg-db-backup23:59:591
237528993827,8cyclictest323744-21timerandwakeup22:10:541
237527993832,3cyclictest431314-21cut00:25:010
237527993830,6cyclictest0-21swapper/000:30:350
237527993828,7cyclictest290941-21/usr/sbin/munin21:30:420
237529993731,3cyclictest0-21swapper/221:40:402
237529993731,3cyclictest0-21swapper/221:40:402
237529993729,5cyclictest440650-21ntp_kernel_pll_00:35:462
237529993728,5cyclictest412357-21ntp_kernel_pll_00:00:452
237528993729,5cyclictest1-21systemd22:20:021
237527993730,4cyclictest411124-21systemd-run00:00:010
237527993725,9cyclictest0-21swapper/021:45:000
237527993725,9cyclictest0-21swapper/021:45:000
237529993630,3cyclictest0-21swapper/200:10:002
237529993629,4cyclictest256981-21kworker/u8:14+rpciod20:15:012
237529993628,5cyclictest695-21cron22:50:002
237529993628,5cyclictest421-21dbus-daemon20:45:012
237529993628,5cyclictest297814-21sendmail21:40:012
237529993627,7cyclictest368418-21sh23:05:582
237529993627,6cyclictest420-21avahi-daemon23:45:472
237529993627,5cyclictest421-21dbus-daemon21:14:142
237528993630,3cyclictest0-21swapper/100:05:021
237528993629,4cyclictest309934-21debian-sa121:55:001
237528993627,6cyclictest548-21gdbus19:10:491
237527993628,5cyclictest264544-21systemd-run20:40:010
237527993626,7cyclictest317352-21sshd22:03:420
237529993527,5cyclictest2619226-21cups-browsed22:20:382
237529993525,7cyclictest330312-21latency_hist22:20:032
237528993529,3cyclictest0-21swapper/120:40:011
237528993529,3cyclictest0-21swapper/119:20:011
237528993528,4cyclictest390822-21cron23:35:001
237528993525,6cyclictest420-21avahi-daemon23:51:281
237527993529,3cyclictest427252-21sendmail00:20:010
237527993528,5cyclictest421-21dbus-daemon19:10:010
237527993528,5cyclictest421-21dbus-daemon19:10:010
237527993527,5cyclictest327956-21sh22:15:560
237527993525,8cyclictest0-21swapper/021:25:010
237527993521,11cyclictest0-21swapper/022:45:360
237529993428,3cyclictest0-21swapper/221:20:012
237529993428,3cyclictest0-21swapper/221:05:012
237529993428,3cyclictest0-21swapper/220:40:012
237529993427,3cyclictest0-21swapper/200:25:312
237528993427,4cyclictest1051-21mta-sts-daemon22:20:471
237528993427,3cyclictest0-21swapper/121:00:011
237528993426,5cyclictest300232-21ssh21:41:561
237528993426,5cyclictest300232-21ssh21:41:561
237528993424,7cyclictest421-21dbus-daemon22:02:071
237528993424,5cyclictest0-21swapper/122:05:441
237528993422,9cyclictest311626-21timerandwakeup21:55:551
237527993428,4cyclictest0-21swapper/022:15:010
237527993427,5cyclictest307717-21sed21:50:540
237527993427,5cyclictest0-21swapper/021:40:020
237527993426,5cyclictest0-21swapper/019:40:410
237527993422,8cyclictest396069-21ssh23:40:400
237527993420,9cyclictest12350irq/32-eth022:40:430
237529993327,3cyclictest0-21swapper/222:55:012
237529993325,5cyclictest372112-21systemctl23:10:522
237529993324,7cyclictest244387-21ntp_kernel_pll_19:30:432
237529993324,4cyclictest0-21swapper/221:45:472
237529993323,7cyclictest383220-21/usr/sbin/munin23:25:392
23752999332,26cyclictest379970-21sshd23:20:422
23752999332,26cyclictest379970-21sshd23:20:422
237528993326,4cyclictest0-21swapper/100:35:011
237528993324,5cyclictest379323-21aten_r2power_po23:20:301
237528993324,5cyclictest379323-21aten_r2power_po23:20:301
237528993322,8cyclictest371282-21cpu23:10:341
237528993322,8cyclictest256321-21needreboot20:10:391
237527993323,7cyclictest440417-21latency00:35:390
237529993226,3cyclictest0-21swapper/200:20:442
237529993225,4cyclictest263011-21ls20:35:012
237529993224,5cyclictest322727-21cpu22:10:332
237529993222,7cyclictest2619226-21cups-browsed21:27:272
237528993225,3cyclictest0-21swapper/122:30:011
237528993224,6cyclictest273506-21diskmemload22:50:331
237528993223,6cyclictest420-21avahi-daemon23:40:061
237527993224,5cyclictest415949-21fschecks_count00:05:340
237527993223,6cyclictest275341-21unixbench_multi21:10:540
237527993222,7cyclictest368384-21unixbench_multi23:05:580
23752799321,24cyclictest331526-21ssh22:20:490
23752999315,23cyclictest261-21systemd-journal00:35:012
237529993125,3cyclictest0-21swapper/223:50:522
237529993125,3cyclictest0-21swapper/223:40:422
237529993124,3cyclictest0-21swapper/222:25:372
237528993126,3cyclictest0-21swapper/123:40:011
237528993125,3cyclictest0-21swapper/123:05:371
237528993124,4cyclictest3728875-21kworker/u8:4+rpciod19:10:011
237528993124,4cyclictest3728875-21kworker/u8:4+rpciod19:10:011
237528993124,4cyclictest1055-21mta-sts-daemon21:05:411
237528993124,4cyclictest0-21swapper/122:30:531
237528993123,5cyclictest2619226-21cups-browsed21:34:581
237528993122,6cyclictest624-21lldpd23:00:541
237528993122,6cyclictest1051-21mta-sts-daemon19:35:451
237527993123,6cyclictest1051-21mta-sts-daemon23:03:180
237527993123,5cyclictest261627-21latency_hist20:30:020
237527993120,9cyclictest0-21swapper/000:15:010
237527993119,9cyclictest287597-21unixbench_singl21:25:540
237527993119,9cyclictest0-21swapper/023:15:510
237527993113,16cyclictest151ktimers/020:15:440
237529993024,4cyclictest1056-21mta-sts-daemon19:35:412
237529993024,3cyclictest0-21swapper/222:45:022
237529993024,3cyclictest0-21swapper/221:55:392
237529993023,4cyclictest257374-21aten_r2power_po20:15:312
237529993022,5cyclictest253944-21unixbench_singl20:00:522
237529993022,5cyclictest243541-21systemd-run19:30:022
23752999302,22cyclictest339589-21ssh22:30:462
237529993021,5cyclictest268991-21cron20:55:002
237528993024,3cyclictest0-21swapper/123:25:361
237528993022,5cyclictest411063-21cups-browsed00:39:061
237528993019,7cyclictest2619226-21cups-browsed22:55:411
237527993025,2cyclictest0-21swapper/023:45:320
237527993025,2cyclictest0-21swapper/019:55:010
237527993024,4cyclictest411627-21aten_r2power_cu00:00:320
237527993024,4cyclictest343478-21ntpq22:35:420
237527993024,4cyclictest343478-21ntpq22:35:420
237527993022,6cyclictest261539-21kworker/u8:3+rpciod20:45:010
237527993022,6cyclictest12350irq/32-eth022:50:380
237527993021,7cyclictest404806-21unixbench_multi23:50:530
237527993021,7cyclictest257010-21/usr/sbin/munin20:15:010
237527993019,7cyclictest360239-21timerwakeupswit22:55:540
237527993017,10cyclictest0-21swapper/019:55:530
237529992922,3cyclictest0-21swapper/221:00:002
237529992921,5cyclictest255550-21latency_hist20:10:012
237529992921,5cyclictest255550-21latency_hist20:10:012
237528992921,5cyclictest433053-21timerwakeupswit00:25:551
237528992921,5cyclictest254558-21gltestperf20:05:341
237528992921,5cyclictest254558-21gltestperf20:05:341
237528992919,7cyclictest274748-21sh21:10:421
23752799296,13cyclictest432587-21ssh00:25:440
237527992922,4cyclictest151ktimers/020:00:420
237527992921,6cyclictest273250-21timerandwakeup21:05:530
23752999283,18cyclictest273506-21diskmemload23:35:412
237529992819,7cyclictest240510-21munin-plugin-st19:20:022
237528992821,4cyclictest244899-21unixbench_multi19:30:541
237528992820,5cyclictest247900-21unixbench_multi19:40:521
23752799287,14cyclictest1051-21mta-sts-daemon19:25:470
237527992823,3cyclictest1055-21mta-sts-daemon21:45:470
23752799281,22cyclictest339476-21ssh22:30:420
23752999273,21cyclictest250425-21sshd19:50:422
237529992720,4cyclictest1056-21mta-sts-daemon19:20:442
237529992720,4cyclictest1051-21mta-sts-daemon19:40:462
237529992719,5cyclictest249465-21sed19:45:522
237529992711,12cyclictest421-21dbus-daemon20:20:412
237528992721,3cyclictest0-21swapper/120:40:401
237528992720,4cyclictest1056-21mta-sts-daemon20:26:441
237528992719,5cyclictest262933-21unixbench_singl20:30:521
237528992717,6cyclictest421-21dbus-daemon20:55:011
237527992721,3cyclictest318822-21fschecks_count22:05:330
237527992714,10cyclictest0-21swapper/022:30:010
23752799271,21cyclictest384086-21ssh23:25:460
23752799271,13cyclictest364864-21kworker/u8:0+rpciod23:10:430
237529992620,2cyclictest0-21swapper/219:55:422
237529992619,5cyclictest0-21swapper/221:10:022
237528992618,5cyclictest420100-21kernelversion00:10:371
237527992621,2cyclictest0-21swapper/021:00:010
237527992619,4cyclictest241059-21fschecks_time19:20:320
23752799260,5cyclictest12350irq/32-eth021:00:430
237529992517,5cyclictest267430-21grep20:45:532
237529992517,5cyclictest261468-21unixbench_singl20:25:532
237528992517,5cyclictest439-21systemd-logind19:30:011
237527992519,4cyclictest249517-21cut19:50:010
237527992518,4cyclictest1051-21mta-sts-daemon20:05:390
237527992518,4cyclictest1051-21mta-sts-daemon20:05:390
237527992516,2cyclictest0-21swapper/020:50:510
237528992418,3cyclictest0-21swapper/119:55:421
237528992415,6cyclictest1-21systemd20:05:011
237527992414,7cyclictest245850-21ntp_kernel_err19:35:410
23752799241,18cyclictest266905-21sshd20:45:420
237528992317,4cyclictest266314-21aten2.4-expect20:45:311
237527992316,4cyclictest238175-21kernelversion19:10:360
237528992216,3cyclictest0-21swapper/121:00:481
237528992214,6cyclictest259590-21sendmail_mailqu20:20:491
237527992216,4cyclictest262886-21tail20:30:510
23752799221,18cyclictest259387-21ntpq20:20:410
23752899213,8cyclictest171rcu_preempt19:45:361
23753099181,8cyclictest0-21swapper/322:45:463
23753099171,9cyclictest0-21swapper/321:30:453
23752999172,12cyclictest237540-21unin-run19:10:012
23752999172,12cyclictest237540-21unin-run19:10:012
23753099152,8cyclictest0-21swapper/300:20:443
23753099152,6cyclictest0-21swapper/322:00:423
23753099152,5cyclictest0-21swapper/322:20:453
23753099152,5cyclictest0-21swapper/321:05:443
23753099151,8cyclictest0-21swapper/323:00:393
23753099151,8cyclictest0-21swapper/322:55:423
23753099143,5cyclictest0-21swapper/323:05:523
23753099142,5cyclictest0-21swapper/323:40:403
23753099141,7cyclictest0-21swapper/323:15:443
23753099141,7cyclictest0-21swapper/320:05:423
23753099141,7cyclictest0-21swapper/320:05:423
23753099141,7cyclictest0-21swapper/319:50:443
23753099141,7cyclictest0-21swapper/300:25:423
23753099141,6cyclictest0-21swapper/322:40:433
23753099141,6cyclictest0-21swapper/320:15:433
23753099141,6cyclictest0-21swapper/319:20:413
23753099141,5cyclictest0-21swapper/319:55:403
23753099132,7cyclictest0-21swapper/323:35:433
23753099132,6cyclictest0-21swapper/322:35:433
23753099132,6cyclictest0-21swapper/322:35:433
23753099132,5cyclictest0-21swapper/300:00:323
23753099131,7cyclictest0-21swapper/322:25:403
23753099131,7cyclictest0-21swapper/322:05:373
23753099131,7cyclictest0-21swapper/300:35:463
23753099131,7cyclictest0-21swapper/300:05:453
23753099131,6cyclictest0-21swapper/320:00:393
23753099131,6cyclictest0-21swapper/300:10:413
23753099131,5cyclictest0-21swapper/320:25:413
23753099131,5cyclictest0-21swapper/320:20:403
23753099131,5cyclictest0-21swapper/319:40:393
23753099122,5cyclictest0-21swapper/321:40:313
23753099122,5cyclictest0-21swapper/321:40:313
23753099122,5cyclictest0-21swapper/321:15:423
23753099122,5cyclictest0-21swapper/300:15:353
23753099121,6cyclictest0-21swapper/323:30:313
23753099121,6cyclictest0-21swapper/322:50:383
23753099121,6cyclictest0-21swapper/321:50:433
23753099121,6cyclictest0-21swapper/321:45:373
23753099121,6cyclictest0-21swapper/321:20:303
23753099121,6cyclictest0-21swapper/321:00:463
23753099121,6cyclictest0-21swapper/320:55:383
23753099121,6cyclictest0-21swapper/319:10:413
23753099121,6cyclictest0-21swapper/300:30:363
23753099121,5cyclictest0-21swapper/319:45:393
23753099121,5cyclictest0-21swapper/319:30:423
23753099121,5cyclictest0-21swapper/319:15:403
23753099113,6cyclictest0-21swapper/321:25:403
23753099113,5cyclictest0-21swapper/322:20:013
23753099112,6cyclictest0-21swapper/323:55:323
23753099112,6cyclictest0-21swapper/320:35:423
23753099111,6cyclictest0-21swapper/323:50:353
23753099111,6cyclictest0-21swapper/323:45:423
23753099111,6cyclictest0-21swapper/323:20:423
23753099111,6cyclictest0-21swapper/323:20:423
23753099111,6cyclictest0-21swapper/322:30:313
23753099111,6cyclictest0-21swapper/322:10:353
23753099111,5cyclictest0-21swapper/323:25:313
23753099111,5cyclictest0-21swapper/323:10:323
23753099111,5cyclictest0-21swapper/319:35:423
23753099103,5cyclictest0-21swapper/321:40:013
23753099102,6cyclictest0-21swapper/321:55:333
23753099101,6cyclictest0-21swapper/321:10:303
23753099101,5cyclictest0-21swapper/320:50:313
23753099101,5cyclictest0-21swapper/320:10:323
23753099101,5cyclictest0-21swapper/319:25:473
2375309991,5cyclictest0-21swapper/320:45:323
2375309991,5cyclictest0-21swapper/320:45:013
2375309981,4cyclictest0-21swapper/320:30:303
2375309971,3cyclictest0-21swapper/319:10:003
2375309971,3cyclictest0-21swapper/319:10:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional