You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 09:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Fri Jan 09, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3389941991121,3cyclictest0-21swapper/021:45:020
3389941998076,2cyclictest0-21swapper/021:50:020
3389941998072,2cyclictest0-21swapper/023:20:020
338994199801,75cyclictest0-21swapper/022:10:020
3389941997975,2cyclictest0-21swapper/023:10:010
3389952997724,19cyclictest3403265-21cat23:25:023
3389952997422,18cyclictest3393801-21ls20:25:023
3389952997116,16cyclictest3393505-21munin-run20:20:003
3389946996728,35cyclictest274-21cron00:20:001
3389946996714,13cyclictest1-21systemd20:05:011
338994699662,15cyclictest3391411-21modprobe19:40:011
3389949996449,11cyclictest291rcuc/222:20:002
338994699632,15cyclictest3395841-21true21:05:011
3389952996237,18cyclictest238-21systemd-journal19:55:003
3389946996224,4cyclictest3395052-21latency_hist20:50:011
338994699613,18cyclictest238-21systemd-journal00:10:011
3389946996024,5cyclictest3392457-21latency_hist20:00:021
338994699602,19cyclictest3393520-21basename20:20:001
338994699602,17cyclictest3393005-21latency_hist20:10:021
338994699602,17cyclictest3393005-21latency_hist20:10:021
3389946996015,6cyclictest309-21in:imuxsock20:50:021
338994699593,19cyclictest3394573-21cat20:40:021
338994699591,18cyclictest3390230-21latency_hist19:15:011
3389946995910,13cyclictest1-21systemd22:25:011
338994699583,17cyclictest3399203-21grep22:10:001
3389946995827,19cyclictest23-21ksoftirqd/123:30:021
3389946995821,12cyclictest23-21ksoftirqd/122:20:001
3389946995817,10cyclictest238-21systemd-journal23:19:551
3389952995739,13cyclictest3395544-21cron21:00:003
3389952995737,16cyclictest3396069-21sshd21:09:553
338994699578,9cyclictest0-21swapper/119:45:021
3389946995734,17cyclictest3404875-21cron00:00:001
338994699573,18cyclictest3402053-21munin-run23:05:001
338995299566,45cyclictest3394524-21sh20:40:003
3389952995637,13cyclictest275-21dbus-daemon00:05:003
3389949995647,6cyclictest30-21ksoftirqd/220:15:012
338995299559,12cyclictest3396574-21cron21:17:003
3389952995537,14cyclictest3400719-21sshd22:38:093
3389952995537,13cyclictest3394248-21sshd20:32:543
338994999554,10cyclictest30-21ksoftirqd/221:40:002
338994999552,9cyclictest30-21ksoftirqd/221:10:012
3389949995512,5cyclictest131rcu_preempt21:23:012
338994699552,17cyclictest3390665-21sh19:24:591
338994699552,13cyclictest3406770-21modprobe00:34:591
3389952995440,10cyclictest3406236-21sshd00:23:553
3389952995439,10cyclictest3393216-21sshd20:13:543
3389952995437,13cyclictest3397114-21sshd21:28:013
3389952995436,15cyclictest3390642-21sshd19:22:453
338994699546,13cyclictest3404729-21grep23:55:011
3389946995423,4cyclictest3397919-21/usr/sbin/munin21:45:011
3389941995439,11cyclictest1-21systemd00:20:010
3389952995346,4cyclictest3391951-21munin-run19:50:013
3389952995346,4cyclictest3391951-21munin-run19:50:013
3389952995338,11cyclictest3403576-21sshd23:33:553
3389952995338,11cyclictest3400461-21sshd22:31:553
3389952995338,11cyclictest3398399-21sshd21:53:553
3389952995338,11cyclictest3398399-21sshd21:53:553
3389952995338,11cyclictest3396319-21sshd21:11:553
3389952995337,12cyclictest3405438-21sshd00:07:563
3389952995333,15cyclictest3399464-21munin-run22:15:013
338994699533,17cyclictest3396620-21munin-run21:20:011
3389946995323,4cyclictest3398857-21latency_hist22:00:021
3389946995321,12cyclictest23-21ksoftirqd/122:05:011
3389952995239,9cyclictest3395006-21cat20:45:023
3389952995239,9cyclictest3395006-21cat20:45:023
3389952995238,10cyclictest3406491-21sshd00:29:553
3389952995238,10cyclictest3406491-21sshd00:29:553
3389952995237,11cyclictest3401758-21sshd22:59:553
3389952995236,12cyclictest3406741-21sshd00:33:563
3389949995243,6cyclictest0-21swapper/219:10:012
3389949995243,6cyclictest0-21swapper/219:10:012
3389949995231,16cyclictest238-21systemd-journal19:20:012
3389949995231,16cyclictest238-21systemd-journal19:20:002
3389946995221,5cyclictest3401016-21sh22:45:011
3389952995144,4cyclictest3401022-21munin-run22:45:013
3389952995138,9cyclictest3392971-21cron20:10:003
3389952995138,9cyclictest3392971-21cron20:10:003
3389952995137,11cyclictest3402032-21sshd23:03:553
3389952995137,11cyclictest3398149-21sshd21:48:553
3389952995137,10cyclictest3399969-21sshd22:23:553
3389952995137,10cyclictest3391682-21sshd19:44:553
3389952995125,20cyclictest3397390-21munin-run21:35:003
3389952995125,19cyclictest3401526-21grep22:55:003
3389952995124,22cyclictest3403889-21unin-run23:40:023
3389952995124,22cyclictest3403889-21unin-run23:40:013
3389949995133,13cyclictest275-21dbus-daemon20:45:002
3389949995133,13cyclictest275-21dbus-daemon20:45:002
338994699514,15cyclictest3407020-21modprobe00:40:001
3389946995131,16cyclictest23-21ksoftirqd/120:45:001
3389946995131,16cyclictest23-21ksoftirqd/120:45:001
3389946995118,5cyclictest3390916-21latency_hist19:30:021
3389946995117,4cyclictest3393929-21latency_hist20:25:021
3389946995113,11cyclictest23-21ksoftirqd/123:15:011
3389941995134,12cyclictest1-21systemd23:35:010
3389941995119,23cyclictest151rcuc/022:40:010
3389941995111,36cyclictest238-21systemd-journal21:10:000
3389952995038,9cyclictest3400225-21sshd22:29:553
3389952995038,9cyclictest3400225-21sshd22:29:553
3389952995037,10cyclictest364-21snmpd20:50:233
3389952995037,10cyclictest3401237-21sshd22:45:563
3389952995036,11cyclictest364-21snmpd23:56:003
3389952995036,11cyclictest3395029-21sshd20:47:553
3389952995036,10cyclictest3390364-21sshd19:17:443
3389952995036,10cyclictest3390364-21sshd19:17:433
3389952995036,10cyclictest3390114-21sshd19:14:553
338994699504,19cyclictest3399452-21munin-run22:15:001
338994699503,19cyclictest3400505-21unin-run22:35:001
3389946995018,6cyclictest3401801-21sendmail-msp23:00:001
3389946995014,9cyclictest3403147-21cut23:25:021
3389946995010,10cyclictest23-21ksoftirqd/122:50:001
3389952994936,10cyclictest3406990-21sshd00:35:543
3389952994936,10cyclictest3393991-21sshd20:25:563
3389952994935,10cyclictest3392735-21munin-run20:05:003
3389952994934,10cyclictest3402567-21munin-run23:15:013
3389952994930,15cyclictest364-21snmpd23:27:273
3389952994929,16cyclictest364-21snmpd21:59:493
338994999496,7cyclictest30-21ksoftirqd/219:25:002
338994999493,9cyclictest30-21ksoftirqd/221:30:002
3389949994915,5cyclictest131rcu_preempt20:55:552
338994199493,41cyclictest3404901-21readlink00:00:000
3389952994836,9cyclictest3396855-21sshd21:20:553
3389952994835,10cyclictest3404423-21cat23:50:013
3389952994835,10cyclictest3392422-21sshd19:56:553
3389952994834,9cyclictest3405683-21sshd00:11:553
3389952994834,9cyclictest3405683-21sshd00:11:553
3389952994834,11cyclictest3402795-21sshd23:18:553
3389952994834,10cyclictest3399214-21munin-run22:10:013
338994999484,10cyclictest30-21ksoftirqd/200:35:012
3389949994812,6cyclictest131rcu_preempt22:09:552
3389946994830,12cyclictest1-21systemd23:50:011
3389946994817,10cyclictest275-21dbus-daemon19:55:001
3389941994840,5cyclictest16950irq/224-800000019:40:010
338994199482,8cyclictest311-21rs:main0
338994199482,8cyclictest311-21rs:main0
3389941994811,6cyclictest131rcu_preempt22:55:000
3389952994734,9cyclictest364-21snmpd23:51:003
3389952994732,11cyclictest364-21snmpd19:30:053
3389952994732,10cyclictest364-21snmpd19:35:093
3389952994731,12cyclictest3399699-21cron22:20:003
3389952994730,14cyclictest364-21snmpd21:02:293
338994999479,9cyclictest30-21ksoftirqd/223:15:012
338994999477,8cyclictest30-21ksoftirqd/219:30:022
338994999473,8cyclictest30-21ksoftirqd/223:55:002
338994699472,8cyclictest3396094-21munin-run21:10:001
3389946994719,6cyclictest23-21ksoftirqd/121:50:021
3389941994733,10cyclictest1-21systemd19:50:010
3389941994733,10cyclictest1-21systemd19:50:010
3389941994729,14cyclictest238-21systemd-journal19:15:000
3389952994635,7cyclictest3404128-21munin-run23:45:003
3389952994633,10cyclictest3402283-21sshd23:08:553
3389952994629,12cyclictest37-21ksoftirqd/321:40:013
3389952994629,11cyclictest3390906-21cron19:30:003
338994999466,4cyclictest131rcu_preempt20:25:012
3389949994629,12cyclictest1-21systemd00:00:032
338994999461,10cyclictest30-21ksoftirqd/220:30:002
338994699463,3cyclictest131rcu_preempt23:40:001
338994699463,3cyclictest131rcu_preempt23:40:001
3389941994639,4cyclictest16950irq/224-800000022:25:010
3389941994637,5cyclictest16950irq/224-800000000:40:000
3389941994630,11cyclictest12-21ksoftirqd/022:15:010
3389941994629,11cyclictest309-21in:imuxsock23:30:000
338994199461,7cyclictest3396912-21wc21:25:010
3389952994521,19cyclictest3390031-21latency_hist19:10:013
3389952994521,19cyclictest3390031-21latency_hist19:10:013
338994999457,12cyclictest30-21ksoftirqd/200:10:012
338994999455,8cyclictest30-21ksoftirqd/223:00:012
3389949994535,7cyclictest30-21ksoftirqd/221:55:002
3389949994535,7cyclictest30-21ksoftirqd/221:55:002
3389949994532,10cyclictest30-21ksoftirqd/219:40:012
3389941994535,6cyclictest16950irq/224-800000022:00:000
3389941994534,7cyclictest12-21ksoftirqd/020:10:010
3389941994534,7cyclictest12-21ksoftirqd/020:10:010
3389941994525,15cyclictest238-21systemd-journal20:30:000
3389941994513,28cyclictest3395939-21sort21:05:020
3389941994510,31cyclictest238-21systemd-journal20:57:570
338994999448,5cyclictest131rcu_preempt20:20:002
3389949994435,6cyclictest30-21ksoftirqd/221:35:012
3389949994429,11cyclictest275-21dbus-daemon22:25:012
3389946994423,4cyclictest3393233-21latency_hist20:15:021
338994199443,7cyclictest3392463-21munin-run20:00:000
3389941994437,4cyclictest16950irq/224-800000020:45:000
3389941994437,4cyclictest16950irq/224-800000020:45:000
3389941994437,2cyclictest0-21swapper/023:15:020
338994199442,37cyclictest3405641-21latency_hist00:10:030
3389941994422,14cyclictest238-21systemd-journal21:30:000
3389941994413,7cyclictest12-21ksoftirqd/023:45:020
3389952994331,9cyclictest364-21snmpd00:16:293
3389952994321,18cyclictest3399145-21latency_hist22:05:023
338994999433,8cyclictest30-21ksoftirqd/223:25:022
3389949994333,7cyclictest3405687-21sshd00:12:552
3389949994333,7cyclictest3405687-21sshd00:12:542
338994999431,11cyclictest0-21swapper/223:33:552
3389949994310,6cyclictest131rcu_preempt00:40:012
3389946994329,11cyclictest23-21ksoftirqd/120:35:001
3389952994229,10cyclictest364-21snmpd21:40:333
338994999425,9cyclictest30-21ksoftirqd/220:35:012
3389949994230,9cyclictest3398658-21cron22:00:002
3389949994229,9cyclictest30-21ksoftirqd/200:05:022
338994999422,7cyclictest30-21ksoftirqd/222:55:012
3389946994230,9cyclictest3393994-21sshd20:26:551
338994199426,21cyclictest3403058-21kworker/u8:400:25:020
338994199426,21cyclictest3403058-21kworker/u8:400:25:020
338994199422,8cyclictest3401784-21munin-run23:00:000
3389941994225,12cyclictest12-21ksoftirqd/020:20:000
338994199422,34cyclictest3391686-21cron19:45:000
338994199421,37cyclictest3396360-21systemd-run21:15:010
338994999418,8cyclictest30-21ksoftirqd/223:40:022
338994999418,8cyclictest30-21ksoftirqd/223:40:022
338994999416,5cyclictest131rcu_preempt23:10:002
338994999412,5cyclictest131rcu_preempt19:45:012
3389949994120,15cyclictest1-21systemd20:05:012
3389941994130,8cyclictest12-21ksoftirqd/023:50:010
3389941994126,11cyclictest12-21ksoftirqd/023:05:020
3389941994122,13cyclictest275-21dbus-daemon19:20:020
3389941994122,13cyclictest275-21dbus-daemon19:20:010
3389949994032,6cyclictest30-21ksoftirqd/220:40:012
3389949994024,11cyclictest30-21ksoftirqd/219:35:002
3389949994024,11cyclictest275-21dbus-daemon22:50:022
338994999401,7cyclictest30-21ksoftirqd/200:26:552
338994999401,7cyclictest30-21ksoftirqd/200:26:552
3389949994015,5cyclictest131rcu_preempt22:36:562
338994999401,10cyclictest30-21ksoftirqd/222:05:022
338994199404,5cyclictest131rcu_preempt20:25:020
3389941994026,10cyclictest238-21systemd-journal00:02:550
3389941994025,12cyclictest3406738-21sshd00:33:360
3389941994024,12cyclictest3405687-21sshd00:12:540
3389941994024,12cyclictest3405687-21sshd00:12:540
338994199402,34cyclictest3393222-21sshd20:14:560
3389941994016,20cyclictest12-21ksoftirqd/019:55:000
338994999396,8cyclictest30-21ksoftirqd/223:45:002
3389949993931,5cyclictest30-21ksoftirqd/221:40:552
3389949993929,8cyclictest30-21ksoftirqd/223:50:002
3389949993924,10cyclictest3392462-21systemctl20:00:002
338994999391,7cyclictest30-21ksoftirqd/223:30:022
338994699393,4cyclictest131rcu_preempt21:35:021
3389946993929,6cyclictest309-21in:imuxsock19:50:001
3389946993929,6cyclictest309-21in:imuxsock19:50:001
3389946993926,8cyclictest309-21in:imuxsock21:15:001
3389941993922,11cyclictest3396619-21run-parts21:20:000
338994999389,5cyclictest131rcu_preempt19:50:012
338994999389,5cyclictest131rcu_preempt19:50:012
338994999383,5cyclictest131rcu_preempt20:50:002
3389949993830,6cyclictest30-21ksoftirqd/222:45:012
338994999381,8cyclictest0-21swapper/200:18:552
3389946993826,8cyclictest238-21systemd-journal00:30:011
3389946993826,8cyclictest238-21systemd-journal00:30:001
3389946993824,11cyclictest23-21ksoftirqd/121:40:011
3389946993823,10cyclictest23-21ksoftirqd/121:00:001
3389946993818,14cyclictest1-21systemd21:25:001
338994199383,5cyclictest3397402-21munin-run21:35:010
3389941993831,4cyclictest0-21swapper/019:10:010
3389941993831,4cyclictest0-21swapper/019:10:010
3389941993826,7cyclictest274-21cron20:40:000
338994199382,5cyclictest0-21swapper/019:34:550
3389941993823,10cyclictest12-21ksoftirqd/019:28:550
3389941993822,11cyclictest275-21dbus-daemon20:05:010
3389949993728,6cyclictest3402805-21modprobe23:20:002
3389949993728,6cyclictest30-21ksoftirqd/219:14:332
3389949993724,8cyclictest291rcuc/222:25:482
3389949993724,8cyclictest291rcuc/222:25:482
3389949993724,10cyclictest275-21dbus-daemon19:55:002
3389949993721,12cyclictest30-21ksoftirqd/221:20:002
3389949993720,13cyclictest1-21systemd22:35:002
338994999371,6cyclictest30-21ksoftirqd/220:10:022
338994999371,6cyclictest30-21ksoftirqd/220:10:022
3389949993711,6cyclictest131rcu_preempt22:11:552
338994699373,4cyclictest3400739-21modprobe22:40:001
3389946993725,8cyclictest3405150-21sshd00:04:561
3389946993723,8cyclictest238-21systemd-journal21:30:001
3389941993726,6cyclictest3397627-21cron21:40:000
338994199372,3cyclictest0-21swapper/022:20:000
3389941993722,12cyclictest275-21dbus-daemon22:35:000
338994199371,33cyclictest0-21swapper/023:24:420
338994999362,7cyclictest30-21ksoftirqd/223:00:492
3389949993621,11cyclictest3395807-21sshd21:02:552
3389946993620,12cyclictest238-21systemd-journal19:20:011
3389946993620,12cyclictest238-21systemd-journal19:20:011
3389941993625,9cyclictest12-21ksoftirqd/021:55:010
3389941993625,9cyclictest12-21ksoftirqd/021:55:010
3389941993624,9cyclictest12-21ksoftirqd/023:50:040
3389941993623,9cyclictest1-21systemd22:45:010
3389941993618,14cyclictest275-21dbus-daemon00:25:010
338994999352,9cyclictest30-21ksoftirqd/221:50:002
338994699353,4cyclictest131rcu_preempt23:45:001
3389946993518,13cyclictest1-21systemd00:25:011
3389941993527,6cyclictest12-21ksoftirqd/019:25:010
3389941993523,9cyclictest12-21ksoftirqd/020:50:000
3389941993523,9cyclictest0-21swapper/022:05:020
3389949993425,6cyclictest3406256-21munin-plugin-st00:25:002
3389949993424,8cyclictest30-21ksoftirqd/220:55:002
338994699345,3cyclictest131rcu_preempt22:30:001
338994699345,3cyclictest131rcu_preempt22:30:001
3389941993424,8cyclictest12-21ksoftirqd/020:35:000
338994199341,5cyclictest131rcu_preempt23:40:020
338994199341,5cyclictest131rcu_preempt23:40:010
3389941993327,4cyclictest16950irq/224-800000020:55:000
3389949993222,7cyclictest238-21systemd-journal21:10:562
338994699324,3cyclictest131rcu_preempt21:55:021
338994699324,3cyclictest131rcu_preempt21:55:021
338994699322,7cyclictest0-21swapper/123:35:001
3389946993223,6cyclictest3401664-21latency_hist22:55:011
3389946993220,9cyclictest3391193-21latency_hist19:35:021
3389941993223,6cyclictest12-21ksoftirqd/022:50:020
3389946993123,5cyclictest3389873-21latency_hist19:10:011
3389946993123,5cyclictest3389873-21latency_hist19:10:011
3389946993120,6cyclictest309-21in:imuxsock23:10:001
338994699304,5cyclictest131rcu_preempt00:13:211
338994699304,5cyclictest131rcu_preempt00:13:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional