You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 11:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Fri Aug 21, 2026 00:42:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3257840997057,8cyclictest3336681-21munin-run22:20:011
325784099553,9cyclictest3368463-21ssh23:11:011
325784099553,9cyclictest3368463-21ssh23:11:011
3257839995443,8cyclictest1452915-21cups-browsed21:31:000
3257840995344,4cyclictest0-21swapper/100:00:011
3257840995344,4cyclictest0-21swapper/100:00:011
3257839995338,12cyclictest3292355-21systemd-run21:05:010
3257840995243,6cyclictest3339741-21cron22:25:001
3257840995243,6cyclictest3339741-21cron22:25:001
3257840994835,10cyclictest265-21systemd-journal21:56:181
3257840994823,21cyclictest3327537-21munin-plugin-st22:05:011
3257839994841,5cyclictest421-21dbus-daemon22:20:020
325783999483,39cyclictest3298033-21ssh21:15:590
3257839994828,14cyclictest12650irq/32-eth022:55:580
3257840994725,19cyclictest3320068-21unixbench-2d21:51:111
325784199463,35cyclictest3331752-21ssh22:11:002
3257841994632,9cyclictest3362553-21systemctl23:01:072
3257840994529,13cyclictest0-21swapper/121:25:561
3257840994523,18cyclictest0-21swapper/122:56:031
3257841994434,7cyclictest3317109-21head21:46:132
3257841994434,7cyclictest3317109-21head21:46:132
3257841994433,7cyclictest1-21systemd00:00:022
3257841994433,7cyclictest1-21systemd00:00:022
3257840994437,4cyclictest0-21swapper/122:40:011
3257840994432,7cyclictest1452915-21cups-browsed21:22:331
3257840994428,13cyclictest0-21swapper/123:40:011
325784099442,12cyclictest3374604-21ssh23:21:011
3257841994332,7cyclictest265-21systemd-journal00:18:182
325784099433,8cyclictest3313454-21ssh21:41:021
325784099433,8cyclictest3313454-21ssh21:41:021
3257840994333,7cyclictest3340773-21missed_timers22:25:561
3257840994319,21cyclictest3290820-21systemctl21:00:001
3257840994318,21cyclictest3345854-21idleruntime-cro22:35:011
325784099431,16cyclictest3420499-21ssh00:36:021
325784099422,15cyclictest3414382-21ntpq00:26:021
325784099422,13cyclictest3398797-21irqstats00:00:531
325784099422,10cyclictest3264680-21ntpq19:30:561
3257840994220,18cyclictest0-21swapper/120:21:041
3257841994135,3cyclictest0-21swapper/219:40:012
3257841994133,5cyclictest3324466-21munin-run22:00:012
325784099416,19cyclictest3316597-21ssh21:46:041
325784099416,19cyclictest3316597-21ssh21:46:041
325784099413,8cyclictest3278228-21ntpq20:15:591
325784099412,6cyclictest3272298-21ntpq19:56:041
3257840994126,12cyclictest0-21swapper/119:20:011
325784099412,15cyclictest3411279-21ntpq00:20:591
3257840994120,17cyclictest3299949-21cron21:20:011
3257840994120,17cyclictest0-21swapper/100:06:051
3257840994119,19cyclictest1452915-21cups-browsed23:49:231
325784099411,15cyclictest3328807-21ntp_states22:06:061
325784099411,10cyclictest3386886-21ntpq23:41:021
3257841994032,5cyclictest3378135-21unixbench_multi23:26:102
3257841994032,5cyclictest3341539-21grep22:26:122
3257841994032,5cyclictest3293695-21tail21:06:102
3257841994032,5cyclictest3292349-21systemd-run21:05:012
3257841994031,5cyclictest3304202-21/usr/sbin/munin21:25:592
3257841994030,6cyclictest1452912-21cupsd21:12:282
325784099402,14cyclictest3377572-21ssh23:25:581
325784099401,14cyclictest3405199-21ntpq00:11:021
325784099401,11cyclictest3285687-21ntpq20:40:581
3257839994033,4cyclictest3385626-21munin-run23:40:010
3257839994031,5cyclictest151ktimers/022:26:000
3257839994026,11cyclictest3324454-21ssh22:00:010
3257839994017,5cyclictest171rcu_preempt23:41:020
325784199393,28cyclictest665-21snmpd21:21:032
3257841993930,6cyclictest3315348-21systemd-run21:45:022
3257841993930,6cyclictest3315348-21systemd-run21:45:022
3257841993929,6cyclictest3420972-21unixbench-2d00:36:122
3257841993928,7cyclictest1001-21mta-sts-daemon22:21:222
3257841993928,7cyclictest1001-21mta-sts-daemon22:21:222
325784099399,3cyclictest0-21swapper/119:21:011
3257840993927,9cyclictest3295410-21timerandwakeup21:11:101
325784099392,10cyclictest3417388-21ntpq00:30:591
3257840993920,16cyclictest1-21systemd23:10:011
3257840993918,18cyclictest0-21swapper/119:45:011
3257840993918,18cyclictest0-21swapper/119:45:011
3257840993917,18cyclictest0-21swapper/122:46:021
3257840993914,22cyclictest0-21swapper/122:40:591
325784099391,28cyclictest3331882-21ssh22:11:051
325784099391,11cyclictest3276776-21ntp_states20:11:011
325784099391,10cyclictest3273747-21ntpq20:01:011
3257840993910,7cyclictest421-21dbus-daemon19:46:041
3257841993828,6cyclictest421-21dbus-daemon19:10:012
3257841993828,6cyclictest3268356-21systemd-run19:45:012
3257841993828,6cyclictest3268356-21systemd-run19:45:012
3257840993831,4cyclictest3266821-21munin-plugin-st19:40:001
3257840993830,5cyclictest3370896-21aten_r2power_fr23:15:501
3257840993829,6cyclictest3380112-21aten_r2power_fr23:30:491
3257840993817,16cyclictest0-21swapper/121:06:001
325784099381,14cyclictest3356163-21ssh22:50:581
3257839993829,6cyclictest0-21swapper/023:50:590
3257841993725,7cyclictest3344249-21mailstats22:31:082
3257840993717,13cyclictest421-21dbus-daemon19:11:001
325784099371,10cyclictest3291810-21ntpq21:01:031
3257840993710,5cyclictest3394192-21sh23:53:081
3257839993731,3cyclictest1000-21mta-sts-daemon20:55:580
3257839993728,6cyclictest3400944-21cron00:05:000
3257839993728,6cyclictest3260955-21latency_hist19:20:020
3257839993727,8cyclictest3358075-21basename22:55:010
3257839993720,13cyclictest12650irq/32-eth023:46:010
3257841993629,4cyclictest3290821-21awk21:00:002
3257841993628,5cyclictest3373352-21munin-plugin-st23:20:002
3257841993627,6cyclictest439-21systemd-logind00:10:022
3257841993627,6cyclictest3385626-21munin-run23:40:012
3257841993626,6cyclictest3375092-21sed23:21:102
3257841993626,4cyclictest0-21swapper/220:10:592
3257841993625,7cyclictest3265341-21systemd-run19:35:012
3257841993623,10cyclictest3278619-21timerwakeupswit20:16:082
325784099364,8cyclictest3362389-21ntpq23:01:021
3257840993613,2cyclictest0-21swapper/121:30:181
325784099361,11cyclictest3408251-21ntpq00:16:021
325784099361,11cyclictest3282729-21ntpq20:31:001
325784099361,11cyclictest3274867-21/usr/sbin/munin20:05:531
325784099361,11cyclictest3270784-21ntpq19:51:031
3257839993627,7cyclictest3263842-21sh19:30:010
325783999362,29cyclictest3347068-21ssh22:36:010
3257841993527,4cyclictest1002-21mta-sts-daemon19:16:032
3257841993526,6cyclictest421-21dbus-daemon22:40:012
3257841993525,8cyclictest3390393-21timerwakeupswit23:46:122
325784099351,9cyclictest3281430-21/usr/sbin/munin20:26:061
325784099351,12cyclictest3312312-21munin-run21:40:021
325783999354,23cyclictest665-21snmpd21:06:000
3257839993527,6cyclictest420-21avahi-daemon23:05:530
3257839993520,3cyclictest171rcu_preempt23:55:590
3257839993520,3cyclictest171rcu_preempt23:55:590
325783999350,4cyclictest3301246-21ssh21:21:020
3257841993427,3cyclictest0-21swapper/223:41:062
3257841993426,5cyclictest3398686-21fschecks_time00:00:522
3257841993426,3cyclictest0-21swapper/222:46:022
3257841993425,5cyclictest3336661-21cron22:20:012
3257841993424,6cyclictest0-21swapper/200:35:012
3257841993423,7cyclictest3397917-21cupsd00:23:512
325784199342,24cyclictest3298125-21ssh21:16:022
3257841993415,14cyclictest421-21dbus-daemon23:51:022
3257840993428,4cyclictest0-21swapper/120:40:011
3257840993411,3cyclictest0-21swapper/120:50:521
3257839993428,3cyclictest0-21swapper/021:30:010
3257839993427,4cyclictest3333626-21latency_hist22:15:010
3257839993425,6cyclictest421-21dbus-daemon00:17:260
3257839993424,7cyclictest3416871-21aten_r2power_vo00:30:500
3257839993423,4cyclictest171rcu_preempt19:30:570
325783999341,23cyclictest3368532-21ssh23:11:040
325783999341,23cyclictest3368532-21ssh23:11:040
3257841993327,3cyclictest0-21swapper/220:40:012
3257841993325,5cyclictest3309227-21systemd-run21:35:012
3257841993325,5cyclictest3289319-21munin-plugin-st20:55:002
3257841993325,5cyclictest0-21swapper/220:35:012
325784099331,11cyclictest3287229-21ntpq20:46:011
3257839993328,3cyclictest3273644-21needreboot20:00:560
3257839993326,5cyclictest1-21systemd20:35:010
3257839993325,6cyclictest3374656-21/usr/sbin/munin23:21:030
3257839993324,6cyclictest3351985-21ssh22:45:010
3257839993321,10cyclictest0-21swapper/020:45:010
3257839993320,3cyclictest171rcu_preempt00:35:580
325783999331,3cyclictest0-21swapper/020:50:580
3257839993311,4cyclictest171rcu_preempt20:46:000
325784199323,21cyclictest1452915-21cups-browsed21:50:592
3257841993222,7cyclictest1003-21mta-sts-daemon19:26:572
3257841993213,14cyclictest420-21avahi-daemon20:06:022
3257839993224,6cyclictest1003-21mta-sts-daemon20:06:050
325783999321,17cyclictest3373566-21sort23:20:020
3257839993210,17cyclictest3328927-21ssh22:06:090
3257841993124,4cyclictest3324967-21acpi22:00:492
3257841993124,3cyclictest0-21swapper/222:55:012
3257841993123,4cyclictest1000-21mta-sts-daemon22:55:062
3257841993121,6cyclictest1003-21mta-sts-daemon23:31:342
325784099311,10cyclictest3263850-21munin-run19:30:011
325783999317,14cyclictest665-21snmpd21:11:010
325783999314,21cyclictest3313594-21ssh21:41:080
325783999314,21cyclictest3313594-21ssh21:41:080
3257839993125,4cyclictest3414967-21head00:26:120
3257839993125,4cyclictest3327723-21latency_hist22:05:030
3257839993124,4cyclictest3284948-21latency_hist20:40:020
3257839993124,3cyclictest0-21swapper/022:30:530
3257839993119,10cyclictest3268326-21cron19:45:000
3257839993119,10cyclictest3268326-21cron19:45:000
3257839993114,12cyclictest151ktimers/023:28:460
3257839993110,17cyclictest421-21dbus-daemon20:26:020
325784199306,15cyclictest627-21lldpd22:06:082
3257841993023,3cyclictest0-21swapper/220:00:012
3257841993022,4cyclictest0-21swapper/221:35:592
3257841993019,7cyclictest3397920-21cups-browsed00:10:062
325784199292,19cyclictest3350037-21ssh22:40:572
3257841992921,5cyclictest3258286-21cpuspeed19:10:492
3257841992910,11cyclictest421-21dbus-daemon20:50:012
325783999297,3cyclictest171rcu_preempt00:11:050
325783999294,5cyclictest171rcu_preempt19:10:580
3257839992923,4cyclictest1003-21mta-sts-daemon21:47:190
3257839992923,4cyclictest1003-21mta-sts-daemon21:47:190
3257839992923,3cyclictest0-21swapper/023:00:520
3257839992919,7cyclictest3266167-21ntp_kernel_err19:35:580
3257839992913,14cyclictest3245148-21kworker/u8:0-xprtiod19:23:060
3257841992819,4cyclictest0-21swapper/223:14:042
3257841992819,4cyclictest0-21swapper/223:14:042
3257841992816,9cyclictest3397920-21cups-browsed00:25:092
3257839992822,4cyclictest3401517-21aten2.4-expect00:05:510
3257839992820,5cyclictest3278756-21unixbench_singl20:16:090
3257839992814,10cyclictest191rcuc/020:10:560
325783999280,4cyclictest151ktimers/021:35:530
3257841992719,5cyclictest3367247-21cron23:10:012
3257841992718,6cyclictest694-21cron20:25:012
3257841992712,11cyclictest1002-21mta-sts-daemon20:01:012
325783999278,6cyclictest171rcu_preempt19:46:000
325783999272,9cyclictest421-21dbus-daemon23:32:090
3257839992723,2cyclictest0-21swapper/022:45:550
3257839992720,5cyclictest3280078-21timerandwakeup20:21:080
3257839992720,3cyclictest0-21swapper/021:50:590
325783999270,3cyclictest171rcu_preempt19:56:010
3257841992620,3cyclictest0-21swapper/220:30:012
3257841992620,3cyclictest0-21swapper/219:55:012
3257841992619,4cyclictest3262179-21unixbench-2d19:21:092
3257840992618,5cyclictest3257844-21systemd-run19:10:011
3257839992620,4cyclictest3271309-21cron19:55:010
3257841992515,6cyclictest420-21avahi-daemon20:45:022
325783999252,17cyclictest3337971-21ntpq22:21:040
325783999252,17cyclictest3337971-21ntpq22:21:040
3257839992419,3cyclictest1003-21mta-sts-daemon00:22:220
3257841992315,5cyclictest3269845-21systemd-run19:50:012
325784299181,8cyclictest0-21swapper/321:31:033
3257839991811,4cyclictest3257846-21kthreadd19:10:010
325784299174,6cyclictest0-21swapper/322:35:593
325784299173,8cyclictest0-21swapper/300:21:033
325784299172,8cyclictest0-21swapper/322:45:593
325784299172,8cyclictest0-21swapper/321:21:033
325784299171,8cyclictest0-21swapper/319:30:593
325784299164,6cyclictest0-21swapper/322:26:073
325784299162,9cyclictest0-21swapper/322:11:033
325784299162,8cyclictest0-21swapper/322:41:053
325784299162,8cyclictest0-21swapper/321:46:023
325784299162,8cyclictest0-21swapper/321:46:023
325784299162,8cyclictest0-21swapper/321:36:003
325784299162,6cyclictest0-21swapper/322:31:003
325784299161,8cyclictest0-21swapper/323:30:593
325784299161,8cyclictest0-21swapper/319:56:033
325784299161,8cyclictest0-21swapper/300:26:033
325784299161,7cyclictest0-21swapper/321:06:023
325784299160,8cyclictest0-21swapper/320:50:573
325784299153,8cyclictest0-21swapper/321:15:493
325784299153,7cyclictest0-21swapper/323:55:593
325784299153,7cyclictest0-21swapper/323:55:593
325784299153,7cyclictest0-21swapper/321:41:083
325784299153,7cyclictest0-21swapper/321:41:083
325784299153,6cyclictest0-21swapper/321:26:063
325784299153,5cyclictest0-21swapper/300:10:543
325784299152,8cyclictest0-21swapper/323:41:023
325784299152,7cyclictest0-21swapper/323:25:473
325784299152,7cyclictest0-21swapper/322:55:573
325784299152,7cyclictest0-21swapper/322:06:043
325784299152,7cyclictest0-21swapper/321:51:003
325784299151,9cyclictest0-21swapper/323:15:563
325784299151,7cyclictest0-21swapper/320:01:043
325784299151,6cyclictest0-21swapper/320:55:583
325784299151,6cyclictest0-21swapper/319:21:033
325784299143,7cyclictest0-21swapper/320:06:023
325784299142,8cyclictest0-21swapper/323:45:593
325784299142,7cyclictest0-21swapper/321:10:513
325784299141,8cyclictest0-21swapper/323:00:583
325784299141,7cyclictest0-21swapper/323:21:033
325784299141,7cyclictest0-21swapper/323:11:023
325784299141,7cyclictest0-21swapper/323:11:023
325784299141,7cyclictest0-21swapper/319:36:043
325784299141,6cyclictest0-21swapper/320:35:573
325784299141,6cyclictest0-21swapper/320:25:593
325784299133,6cyclictest0-21swapper/322:15:483
325784299132,7cyclictest0-21swapper/323:06:023
325784299132,7cyclictest0-21swapper/322:50:533
325784299132,7cyclictest0-21swapper/300:16:023
325784299132,6cyclictest0-21swapper/319:11:003
325784299131,7cyclictest0-21swapper/323:50:593
325784299131,7cyclictest0-21swapper/322:01:063
325784299131,7cyclictest0-21swapper/320:15:023
325784299131,7cyclictest0-21swapper/300:06:083
325784299131,6cyclictest0-21swapper/320:46:053
325784299131,6cyclictest0-21swapper/319:41:023
325784299131,6cyclictest0-21swapper/319:41:023
325784299131,5cyclictest0-21swapper/319:15:583
325784299124,4cyclictest0-21swapper/300:31:023
325784299123,4cyclictest0-21swapper/320:21:003
325784299122,6cyclictest0-21swapper/323:35:543
325784299122,6cyclictest0-21swapper/300:35:493
325784299122,6cyclictest0-21swapper/300:00:503
325784299122,5cyclictest0-21swapper/322:20:533
325784299122,5cyclictest0-21swapper/322:20:533
325784299122,5cyclictest0-21swapper/321:55:473
325784299121,6cyclictest0-21swapper/319:46:043
325784299121,6cyclictest0-21swapper/319:25:583
325784299101,8cyclictest0-21swapper/319:10:023
325784299101,6cyclictest0-21swapper/321:01:013
32578429994,3cyclictest0-21swapper/320:30:183
32578429991,5cyclictest0-21swapper/320:40:523
32578429984,3cyclictest0-21swapper/320:16:073
32578429982,5cyclictest0-21swapper/319:51:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional