You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-19 - 11:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Aug 19, 2026 00:42:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3618424995647,6cyclictest3747507-21proc_pri23:41:092
3618423995443,6cyclictest421-21dbus-daemon00:00:011
3618423995141,6cyclictest3633428-21systemd-run20:00:011
361842299518,26cyclictest3661860-21ssh21:21:060
3618424994928,17cyclictest3654410-21diskmemload23:00:492
3618423994940,6cyclictest0-21swapper/122:30:011
3618422994931,11cyclictest12650irq/32-eth023:41:020
3618424994837,6cyclictest0-21swapper/200:01:062
3618424994739,5cyclictest3730951-21latency_hist23:15:022
3618424994726,17cyclictest0-21swapper/221:41:102
3618422994737,7cyclictest1883844-21cups-browsed22:50:230
361842299471,37cyclictest151ktimers/023:15:590
3618422994636,7cyclictest12650irq/32-eth000:16:030
3618424994539,3cyclictest0-21swapper/222:25:012
3618424994518,3cyclictest0-21swapper/222:00:592
3618424994518,3cyclictest0-21swapper/222:00:592
3618422994539,4cyclictest3773634-21munin-run00:25:010
3618422994536,6cyclictest3727814-21cron23:10:000
3618424994423,17cyclictest1000-21mta-sts-daemon22:05:132
361842499434,11cyclictest3671120-21ssh21:36:072
361842499432,31cyclictest3741226-21sshd23:31:002
361842499432,19cyclictest3658782-21ssh21:16:052
3618423994334,6cyclictest3738334-21proc_pri23:26:051
3618423994334,6cyclictest1883844-21cups-browsed23:21:011
3618424994232,6cyclictest3654879-21/usr/sbin/munin21:11:132
3618424994231,8cyclictest3701401-21needreboot22:25:562
3618424994219,3cyclictest1002-21mta-sts-daemon20:06:592
3618424994217,2cyclictest0-21swapper/222:56:092
361842399424,30cyclictest3689439-21ssh22:06:061
3618423994235,4cyclictest3669772-21munin-plugin-st21:35:011
3618423994234,5cyclictest3746126-21ssh23:40:011
3618423994233,6cyclictest3732168-21ntpq23:16:041
3618422994216,16cyclictest151ktimers/019:21:040
3618423994132,6cyclictest1003-21mta-sts-daemon21:49:591
3618422994134,4cyclictest3634938-21systemd-run20:05:010
3618422994119,11cyclictest151ktimers/023:35:580
361842499408,7cyclictest3725935-21ssh23:05:582
361842499401,19cyclictest3780909-21ssh00:35:592
3618423994031,6cyclictest3683040-21missed_timers21:55:561
3618423994031,6cyclictest3659169-21unixbench-2d21:16:111
3618423994031,4cyclictest0-21swapper/100:26:001
3618423994029,7cyclictest420-21avahi-daemon22:03:001
3618423994029,7cyclictest420-21avahi-daemon22:03:001
361842399400,5cyclictest171rcu_preempt23:11:011
3618422994026,4cyclictest0-21swapper/021:26:000
3618422994015,16cyclictest3692345-21sshd22:11:000
3618424993919,2cyclictest0-21swapper/222:50:492
361842399393,29cyclictest3692458-21sh22:11:051
3618422993932,4cyclictest3713882-21proc_pri22:46:060
3618422993920,13cyclictest12650irq/32-eth023:25:570
361842299391,32cyclictest3674028-21ssh21:41:010
361842499383,28cyclictest3637264-21modprobe20:10:592
361842499383,28cyclictest3637264-21modprobe20:10:592
3618424993830,5cyclictest3764656-21cat00:10:022
3618424993830,5cyclictest3764656-21cat00:10:022
3618424993829,6cyclictest3736981-21cron23:25:012
3618424993829,6cyclictest265-21systemd-journal21:10:012
3618423993832,4cyclictest3664906-21ntpq21:26:031
3618423993830,5cyclictest3672841-21sendmail21:40:011
3618423993830,5cyclictest3652935-21systemd-run21:05:011
3618423993827,7cyclictest3768477-21latency00:15:551
361842299384,28cyclictest3759630-21ssh00:01:030
3618422993825,7cyclictest12650irq/32-eth021:56:070
3618422993823,12cyclictest3625938-21latency_hist19:35:020
3618422993821,11cyclictest12650irq/32-eth022:26:060
3618422993813,6cyclictest14-21ksoftirqd/023:11:010
361842499375,16cyclictest3677200-21ntpq21:46:082
3618424993729,4cyclictest0-21swapper/222:35:522
361842499372,24cyclictest3753454-21ssh23:51:002
361842499372,24cyclictest3753454-21ssh23:51:002
361842499371,16cyclictest3777937-21ssh00:31:032
3618423993731,3cyclictest0-21swapper/120:30:011
3618423993729,5cyclictest3727838-21systemd-run23:10:011
3618422993731,4cyclictest3688756-21sh22:05:500
3618422993731,3cyclictest0-21swapper/000:30:520
3618422993727,8cyclictest3638210-21aten_r2power_fr20:15:490
3618422993727,8cyclictest3638210-21aten_r2power_fr20:15:490
3618422993727,7cyclictest0-21swapper/020:56:000
3618422993720,11cyclictest151ktimers/021:51:030
3618424993631,4cyclictest3618421-21cyclictest22:20:012
3618424993629,3cyclictest0-21swapper/222:15:012
3618424993627,5cyclictest1000-21mta-sts-daemon21:51:112
361842399364,27cyclictest3747286-21sshd23:41:001
3618423993627,3cyclictest0-21swapper/100:40:011
361842399362,28cyclictest3704500-21ssh22:30:571
3618424993529,3cyclictest0-21swapper/200:00:012
3618424993528,4cyclictest3774252-21aten_r2power_po00:25:512
3618424993527,4cyclictest0-21swapper/223:26:052
3618424993516,3cyclictest3637277-21kworker/u8:0-xprtiod21:35:022
3618423993530,3cyclictest0-21swapper/119:20:011
3618423993528,4cyclictest3760198-21unixbench_singl00:01:141
3618423993527,5cyclictest3622914-21cron19:25:011
361842399351,28cyclictest3698466-21sshd22:21:001
3618422993524,8cyclictest3762381-21kernelversion00:05:580
3618422993524,8cyclictest3762381-21kernelversion00:05:580
3618422993521,8cyclictest3670928-21sh21:35:590
361842299351,28cyclictest3704699-21ssh22:31:050
3618424993428,3cyclictest0-21swapper/220:40:012
3618424993427,3cyclictest0-21swapper/221:00:012
3618424993426,5cyclictest3649913-21systemd-run20:55:012
3618423993429,2cyclictest0-21swapper/122:36:001
3618423993428,3cyclictest3720721-21kworker/u8:14-xprtiod23:35:011
3618423993427,5cyclictest3648419-21true20:50:011
3618423993426,6cyclictest3777462-21fschecks_time00:30:531
3618423993426,3cyclictest0-21swapper/100:20:351
3618422993428,4cyclictest3648399-21cron20:50:010
3618422993427,4cyclictest3633436-21kworker/u8:120:00:010
3618422993419,12cyclictest0-21swapper/021:10:010
3618422993416,13cyclictest151ktimers/019:46:010
3618424993327,3cyclictest3644801-21kworker/u8:11-xprtiod20:45:012
3618424993326,3cyclictest0-21swapper/200:21:002
3618424993325,5cyclictest3769313-21unixbench_singl00:16:132
3618424993325,4cyclictest0-21swapper/222:41:012
3618424993324,6cyclictest3752268-21latency_hist23:50:022
361842399336,9cyclictest85-21hwrng22:41:021
3618423993326,4cyclictest3633954-21gltestperf20:00:521
3618423993324,6cyclictest439-21systemd-logind21:00:011
3618423993311,18cyclictest665-21snmpd20:31:021
3618422993328,3cyclictest3695941-21grep22:16:110
3618422993326,4cyclictest3740816-21fschecks_count23:30:520
361842299331,26cyclictest151ktimers/023:01:050
3618424993226,3cyclictest3165540-21kworker/u8:0-rpciod19:35:012
3618424993224,5cyclictest3621411-21munin-run19:20:012
361842399329,6cyclictest171rcu_preempt20:36:001
3618423993226,4cyclictest3765679-21ssh00:11:021
3618423993225,4cyclictest1000-21mta-sts-daemon00:06:071
3618423993225,4cyclictest1000-21mta-sts-daemon00:06:071
3618423993225,4cyclictest0-21swapper/121:10:011
3618423993224,5cyclictest0-21swapper/122:20:011
3618423993223,7cyclictest3719661-21missed_timers22:55:541
3618423993223,6cyclictest3661365-21fschecks_time21:20:551
3618423993223,6cyclictest3628936-21systemd-run19:45:011
3618423993222,4cyclictest0-21swapper/120:20:011
3618423993222,4cyclictest0-21swapper/120:20:011
3618422993225,3cyclictest0-21swapper/021:46:040
3618422993225,3cyclictest0-21swapper/021:11:100
3618422993222,7cyclictest3626421-21fschecks_count19:35:520
3618422993222,6cyclictest0-21swapper/000:30:010
361842299321,27cyclictest3756598-21ssh23:56:040
3618424993125,3cyclictest0-21swapper/223:40:012
3618424993124,4cyclictest3663633-21munin-plugin-st21:25:012
3618424993124,3cyclictest0-21swapper/222:00:012
3618424993121,6cyclictest485-21in:imuxsock22:35:012
3618423993124,4cyclictest3630920-21fschecks_count19:50:521
3618423993123,5cyclictest1883844-21cups-browsed20:51:341
3618423993121,8cyclictest3623759-21ntp_kernel_err19:25:551
3618423993121,6cyclictest1883844-21cups-browsed23:03:031
3618422993124,4cyclictest3686792-21tail22:01:120
3618422993124,4cyclictest3686792-21tail22:01:120
3618422993124,4cyclictest1000-21mta-sts-daemon22:36:230
361842299312,23cyclictest3780874-21ssh00:35:570
3618422993118,10cyclictest0-21swapper/021:35:020
361842299311,23cyclictest3654410-21diskmemload22:20:590
3618424993024,3cyclictest0-21swapper/219:40:012
3618424993024,3cyclictest0-21swapper/219:25:012
3618424993023,4cyclictest3643921-21kworker/u8:8-xprtiod20:50:012
3618424993023,4cyclictest1002-21mta-sts-daemon19:45:492
3618424993022,5cyclictest3619908-21idleruntime-cro19:15:012
3618424993021,6cyclictest0-21swapper/220:35:012
3618424993020,6cyclictest1883844-21cups-browsed23:17:542
361842399302,23cyclictest3654410-21diskmemload22:46:051
3618422993024,4cyclictest3765486-21missed_timers00:10:560
3618422993024,4cyclictest3645276-21tail20:36:110
3618422993023,5cyclictest3618433-21systemd-run19:10:020
3618422993023,4cyclictest3736991-21munin-plugin-st23:25:010
3618422993021,6cyclictest0-21swapper/021:05:010
3618422993017,2cyclictest0-21swapper/021:20:010
3618422993013,5cyclictest3750340-21sh23:46:000
3618424992923,3cyclictest0-21swapper/220:00:022
3618424992921,5cyclictest3639400-21munin-run20:20:002
3618424992921,5cyclictest3639400-21munin-run20:20:002
3618423992921,5cyclictest420-21avahi-daemon19:10:521
3618422992924,2cyclictest0-21swapper/020:15:010
3618422992924,2cyclictest0-21swapper/020:15:010
3618422992924,2cyclictest0-21swapper/019:20:010
3618422992923,4cyclictest3631891-21cron19:55:010
3618422992922,5cyclictest0-21swapper/023:55:010
3618422992922,5cyclictest0-21swapper/023:55:010
3618422992922,4cyclictest1-21systemd20:25:010
3618422992920,4cyclictest3649887-21cron20:55:010
361842499284,16cyclictest428-21mta-sts-daemon19:41:022
3618424992823,2cyclictest0-21swapper/222:50:012
3618424992820,5cyclictest1002-21mta-sts-daemon21:25:242
361842399283,20cyclictest3716744-21ssh22:50:581
361842399283,20cyclictest3646329-21sshd20:41:011
3618423992821,4cyclictest3683169-21kworker/u8:5-xprtiod23:47:011
3618423992819,7cyclictest3637418-21proc_pri20:11:061
3618423992819,7cyclictest3637418-21proc_pri20:11:061
3618423992819,5cyclictest0-21swapper/121:10:261
3618424992722,2cyclictest0-21swapper/219:55:022
3618424992721,3cyclictest0-21swapper/200:15:022
361842399273,20cyclictest3753370-21ssh23:50:571
361842399273,20cyclictest3753370-21ssh23:50:571
361842399272,20cyclictest3680157-21ssh21:51:011
361842399271,20cyclictest3674099-21ssh21:41:041
361842299279,11cyclictest12650irq/32-eth019:25:590
3618422992719,5cyclictest439-21systemd-logind20:45:010
3618422992717,8cyclictest0-21swapper/019:15:010
361842499264,14cyclictest428-21mta-sts-daemon20:21:002
3618424992618,5cyclictest3652439-21proc_pri21:01:052
3618424992618,4cyclictest3641279-21aten_r2power_po20:25:492
3618424992618,4cyclictest3618377-21/usr/sbin/munin19:10:022
3618423992622,3cyclictest28-21ksoftirqd/119:46:001
3618423992616,6cyclictest420-21avahi-daemon20:05:091
3618422992621,2cyclictest0-21swapper/019:45:010
361842299261,20cyclictest3710618-21ntpq22:40:570
3618424992516,6cyclictest420-21avahi-daemon19:30:012
3618423992518,4cyclictest3639876-21cpuspeed20:20:491
3618423992515,7cyclictest3625902-21sh19:35:011
3618422992518,5cyclictest0-21swapper/022:56:010
3618424992416,5cyclictest3634489-21systemctl20:01:062
3618422992417,5cyclictest0-21swapper/020:05:530
3618422992415,7cyclictest3642402-21cron20:30:010
361842299232,14cyclictest421-21dbus-daemon20:31:020
361842399222,15cyclictest3626876-21ntpq19:36:041
361842599202,9cyclictest0-21swapper/300:16:073
361842599201,11cyclictest0-21swapper/323:50:563
361842599201,11cyclictest0-21swapper/323:50:563
361842599193,9cyclictest0-21swapper/323:31:003
361842599192,8cyclictest0-21swapper/322:40:563
361842599192,8cyclictest0-21swapper/300:26:033
3618423991912,5cyclictest3618433-21systemd-run19:10:021
361842599183,7cyclictest0-21swapper/322:30:583
361842599183,6cyclictest0-21swapper/319:25:593
361842599182,9cyclictest0-21swapper/322:11:043
361842599182,9cyclictest0-21swapper/321:41:073
361842599181,10cyclictest0-21swapper/300:01:063
361842599173,8cyclictest0-21swapper/321:50:573
361842599173,8cyclictest0-21swapper/321:36:043
361842599172,9cyclictest0-21swapper/323:56:013
361842599172,9cyclictest0-21swapper/322:56:043
361842599172,8cyclictest0-21swapper/321:15:583
361842599172,8cyclictest0-21swapper/300:20:573
361842599172,7cyclictest0-21swapper/319:55:593
361842599172,6cyclictest0-21swapper/319:10:583
361842599171,9cyclictest0-21swapper/322:01:023
361842599171,9cyclictest0-21swapper/322:01:023
361842599171,9cyclictest0-21swapper/300:30:583
361842599171,7cyclictest0-21swapper/319:46:013
361842599164,7cyclictest0-21swapper/323:10:583
361842599162,9cyclictest0-21swapper/323:45:593
361842599162,9cyclictest0-21swapper/323:25:593
361842599162,8cyclictest0-21swapper/323:20:593
361842599162,8cyclictest0-21swapper/321:31:003
361842599162,8cyclictest0-21swapper/300:11:023
361842599162,8cyclictest0-21swapper/300:06:073
361842599162,8cyclictest0-21swapper/300:06:073
361842599162,7cyclictest0-21swapper/320:55:573
361842599161,9cyclictest0-21swapper/323:40:463
361842599161,8cyclictest0-21swapper/323:06:003
361842599161,7cyclictest0-21swapper/323:16:003
361842599154,7cyclictest0-21swapper/300:36:003
361842599153,7cyclictest0-21swapper/322:51:033
361842599153,7cyclictest0-21swapper/321:55:583
361842599153,7cyclictest0-21swapper/321:21:023
361842599153,6cyclictest0-21swapper/321:45:523
361842599152,6cyclictest0-21swapper/320:41:003
361842599152,6cyclictest0-21swapper/319:35:593
361842599151,8cyclictest0-21swapper/320:15:593
361842599151,8cyclictest0-21swapper/320:15:593
361842599144,7cyclictest0-21swapper/323:00:593
361842599144,7cyclictest0-21swapper/322:20:593
361842599143,7cyclictest0-21swapper/321:26:003
361842599142,7cyclictest0-21swapper/323:36:003
361842599142,7cyclictest0-21swapper/320:55:013
361842599142,5cyclictest0-21swapper/320:30:583
361842599141,8cyclictest0-21swapper/322:45:493
361842599141,7cyclictest0-21swapper/322:05:493
361842599141,7cyclictest0-21swapper/321:06:013
361842599134,5cyclictest0-21swapper/322:26:063
361842599132,6cyclictest0-21swapper/320:46:003
361842599132,6cyclictest0-21swapper/319:15:593
361842599131,7cyclictest0-21swapper/322:35:593
361842599131,6cyclictest0-21swapper/321:00:573
361842599124,6cyclictest0-21swapper/320:36:003
361842599123,6cyclictest0-21swapper/320:20:503
361842599122,7cyclictest0-21swapper/320:30:023
361842599121,8cyclictest0-21swapper/321:10:503
361842599121,7cyclictest0-21swapper/322:15:503
361842599121,7cyclictest0-21swapper/319:40:543
361842599121,6cyclictest0-21swapper/320:00:583
361842599121,6cyclictest0-21swapper/319:20:593
361842599111,6cyclictest0-21swapper/320:10:593
361842599111,6cyclictest0-21swapper/320:10:593
361842599102,6cyclictest0-21swapper/319:10:013
361842599102,5cyclictest0-21swapper/320:06:033
361842599102,4cyclictest0-21swapper/319:50:573
361842599102,4cyclictest0-21swapper/319:30:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional