You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 00:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Tue Dec 09, 2025 12:45:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1734081991316,12cyclictest131rcu_preempt11:30:011
173407699123115,5cyclictest12-21ksoftirqd/012:00:030
17340919911646,6cyclictest131rcu_preempt08:15:033
17340889910719,6cyclictest131rcu_preempt10:10:022
17340889910719,6cyclictest131rcu_preempt10:10:012
1734088999889,6cyclictest30-21ksoftirqd/208:15:032
1734088999889,6cyclictest30-21ksoftirqd/208:15:032
1734091998173,6cyclictest37-21ksoftirqd/310:25:013
1734091998073,3cyclictest37-21ksoftirqd/310:10:023
1734091998073,3cyclictest37-21ksoftirqd/310:10:013
1734091997010,22cyclictest131rcu_preempt11:55:023
1734088997022,12cyclictest30-21ksoftirqd/208:24:542
1734088996523,8cyclictest1736925-21sed08:05:012
1734088996523,14cyclictest1740769-21sshd09:19:522
1734088996226,4cyclictest1750519-21latency_hist12:25:022
1734076996258,2cyclictest0-21swapper/008:05:020
1734076996038,17cyclictest1-21systemd11:40:010
1734076995912,6cyclictest131rcu_preempt07:42:290
1734076995910,28cyclictest131rcu_preempt10:25:010
1734081995726,25cyclictest23-21ksoftirqd/108:10:011
1734076995733,15cyclictest272-21dbus-daemon12:15:010
173407699564,7cyclictest12-21ksoftirqd/009:20:010
1734076995645,7cyclictest12-21ksoftirqd/011:30:000
1734076995639,13cyclictest12-21ksoftirqd/012:25:020
1734081995538,13cyclictest1-21systemd10:50:011
1734081995535,16cyclictest1-21systemd11:40:011
1734076995514,9cyclictest12-21ksoftirqd/009:50:000
1734088995435,14cyclictest235-21systemd-journal10:35:012
1734088995416,22cyclictest0-21swapper/209:05:022
173407699536,5cyclictest131rcu_preempt08:37:290
1734076995318,13cyclictest12-21ksoftirqd/008:55:000
173407699524,7cyclictest131rcu_preempt07:30:010
1734076995242,4cyclictest16950irq/224-800000011:35:020
1734081995135,12cyclictest1-21systemd11:05:011
173407699506,11cyclictest12-21ksoftirqd/009:00:010
173407699506,10cyclictest12-21ksoftirqd/008:35:000
173407699505,9cyclictest12-21ksoftirqd/008:20:010
173407699505,9cyclictest12-21ksoftirqd/008:20:000
1734076995017,6cyclictest131rcu_preempt10:40:000
173407699496,11cyclictest12-21ksoftirqd/011:52:300
173407699496,11cyclictest12-21ksoftirqd/009:05:000
173407699494,11cyclictest12-21ksoftirqd/012:35:020
1734076994933,11cyclictest235-21systemd-journal10:00:030
1734091994841,3cyclictest37-21ksoftirqd/311:10:023
173408899483,19cyclictest1727409-21kworker/u8:107:15:032
1734088994830,14cyclictest272-21dbus-daemon10:40:012
1734081994835,9cyclictest235-21systemd-journal10:40:011
1734081994833,12cyclictest1751231-21systemctl12:40:011
173407699489,11cyclictest12-21ksoftirqd/010:10:020
173407699489,11cyclictest12-21ksoftirqd/010:10:010
173407699485,9cyclictest12-21ksoftirqd/008:25:010
1734076994829,17cyclictest151rcuc/009:15:030
173407699482,12cyclictest12-21ksoftirqd/008:10:000
1734076994818,11cyclictest12-21ksoftirqd/007:25:010
1734076994816,6cyclictest131rcu_preempt12:25:010
173407699481,10cyclictest12-21ksoftirqd/009:30:010
1734091994728,14cyclictest1-21systemd10:00:003
1734081994732,12cyclictest1743649-21sshd10:15:021
1734081994732,12cyclictest1743649-21sshd10:15:011
173407699479,5cyclictest131rcu_preempt10:20:020
173407699476,36cyclictest12-21ksoftirqd/012:05:010
173407699475,9cyclictest12-21ksoftirqd/007:10:020
1734076994736,7cyclictest12-21ksoftirqd/011:50:000
1734076994736,6cyclictest16950irq/224-800000010:30:010
173407699472,8cyclictest1736616-21munin-run08:00:000
1734076994716,27cyclictest1738199-21systemd-run08:30:020
1734076994716,27cyclictest1738199-21systemd-run08:30:010
173407699471,16cyclictest1746731-21sshd11:14:300
173409199464,5cyclictest131rcu_preempt08:50:013
1734091994635,6cyclictest306-21in:imuxsock07:30:003
1734091994629,12cyclictest272-21dbus-daemon12:20:013
1734091994620,21cyclictest37-21ksoftirqd/309:10:013
1734081994631,12cyclictest272-21dbus-daemon11:15:011
173407699468,6cyclictest131rcu_preempt09:45:000
173407699465,14cyclictest12-21ksoftirqd/007:35:030
173407699465,12cyclictest12-21ksoftirqd/009:10:010
173407699464,9cyclictest12-21ksoftirqd/008:15:000
1734076994636,7cyclictest12-21ksoftirqd/012:10:000
1734091994532,7cyclictest306-21in:imuxsock11:30:013
1734091994530,6cyclictest1741314-21munin-run09:30:003
1734091994527,13cyclictest272-21dbus-daemon07:25:013
1734091994516,19cyclictest37-21ksoftirqd/311:45:023
173409199451,5cyclictest131rcu_preempt07:40:003
173408899452,6cyclictest131rcu_preempt11:40:002
1734081994530,12cyclictest272-21dbus-daemon09:45:021
173407699458,8cyclictest12-21ksoftirqd/010:55:010
173407699457,9cyclictest12-21ksoftirqd/009:35:000
1734076994514,6cyclictest131rcu_preempt10:45:020
1734076994513,6cyclictest131rcu_preempt07:15:000
1734091994433,6cyclictest1743900-21sendmail10:20:003
1734091994421,16cyclictest272-21dbus-daemon08:35:013
1734088994429,11cyclictest30-21ksoftirqd/210:20:012
1734081994428,12cyclictest272-21dbus-daemon07:40:001
1734081994426,14cyclictest272-21dbus-daemon07:55:001
1734081994415,20cyclictest221rcuc/111:45:011
173407699446,8cyclictest12-21ksoftirqd/011:20:000
173407699445,10cyclictest12-21ksoftirqd/011:05:000
1734076994434,7cyclictest12-21ksoftirqd/012:17:000
1734076994416,9cyclictest12-21ksoftirqd/011:10:010
1734076994414,5cyclictest131rcu_preempt09:40:010
1734076994410,6cyclictest131rcu_preempt10:50:020
173409199437,6cyclictest131rcu_preempt09:45:003
173409199434,6cyclictest131rcu_preempt11:20:013
1734091994325,12cyclictest235-21systemd-journal07:35:003
1734091994310,6cyclictest131rcu_preempt12:40:003
1734088994330,10cyclictest235-21systemd-journal10:27:302
1734088994327,11cyclictest272-21dbus-daemon11:00:012
1734081994332,6cyclictest306-21in:imuxsock09:10:011
173407699436,8cyclictest12-21ksoftirqd/007:50:010
173407699433,4cyclictest131rcu_preempt09:55:010
1734076994330,8cyclictest306-21in:imuxsock10:00:000
1734076994326,12cyclictest272-21dbus-daemon09:25:010
1734076994325,13cyclictest1-21systemd07:35:010
1734076994324,12cyclictest235-21systemd-journal11:45:000
173407699432,13cyclictest1743651-21sshd10:15:020
173407699432,13cyclictest1743651-21sshd10:15:010
173409199423,3cyclictest131rcu_preempt08:10:013
1734091994223,14cyclictest235-21systemd-journal07:15:013
1734091994217,20cyclictest37-21ksoftirqd/310:49:593
1734091994213,6cyclictest131rcu_preempt07:20:003
1734088994228,9cyclictest272-21dbus-daemon10:00:012
1734081994228,10cyclictest1741806-21cron09:40:001
1734081994226,13cyclictest1-21systemd12:35:011
1734081994222,15cyclictest272-21dbus-daemon10:45:011
173407699426,11cyclictest12-21ksoftirqd/007:55:010
173407699423,9cyclictest12-21ksoftirqd/007:20:000
1734076994210,12cyclictest12-21ksoftirqd/011:00:010
1734091994125,11cyclictest306-21in:imuxsock08:05:013
1734091994113,14cyclictest131rcu_preempt08:20:013
1734091994113,14cyclictest131rcu_preempt08:20:003
173408899417,4cyclictest131rcu_preempt09:30:012
1734088994123,13cyclictest272-21dbus-daemon07:35:012
1734081994131,7cyclictest23-21ksoftirqd/108:15:011
1734081994124,13cyclictest235-21systemd-journal08:05:021
173408199412,3cyclictest131rcu_preempt10:00:021
173409199404,11cyclictest37-21ksoftirqd/307:55:013
1734091994030,8cyclictest37-21ksoftirqd/309:15:013
1734091994029,6cyclictest1739992-21cron09:04:593
1734091994028,6cyclictest306-21in:imuxsock08:30:003
1734091994028,6cyclictest306-21in:imuxsock08:30:003
1734091994028,10cyclictest37-21ksoftirqd/310:37:293
1734091994026,10cyclictest0-21swapper/310:15:033
1734091994026,10cyclictest0-21swapper/310:15:033
1734091994025,11cyclictest37-21ksoftirqd/309:55:013
1734091994025,11cyclictest272-21dbus-daemon12:10:013
1734091994024,12cyclictest235-21systemd-journal09:35:013
1734091994024,10cyclictest272-21dbus-daemon08:45:013
1734091994021,13cyclictest235-21systemd-journal10:30:013
173409199401,11cyclictest131rcu_preempt07:50:023
173408899407,24cyclictest30-21ksoftirqd/211:40:032
1734088994028,9cyclictest1739726-21sh09:00:002
1734088994026,10cyclictest272-21dbus-daemon09:55:012
1734088994024,12cyclictest1-21systemd11:25:012
1734088994024,12cyclictest1-21systemd11:25:012
1734088994024,11cyclictest306-21in:imuxsock11:05:022
1734088994022,13cyclictest272-21dbus-daemon09:35:002
173408199403,3cyclictest131rcu_preempt09:20:011
1734081994024,13cyclictest235-21systemd-journal09:35:011
1734081994023,13cyclictest1-21systemd11:20:011
1734081994023,13cyclictest1-21systemd10:30:011
1734081994022,13cyclictest272-21dbus-daemon10:05:001
173408199401,3cyclictest131rcu_preempt08:31:301
173407699402,11cyclictest1738938-21sshd08:41:300
1734091993922,3cyclictest37-21ksoftirqd/309:00:013
1734091993917,4cyclictest37-21ksoftirqd/307:55:013
1734091993915,20cyclictest37-21ksoftirqd/312:15:033
173408899397,4cyclictest131rcu_preempt09:50:012
173408899394,9cyclictest30-21ksoftirqd/211:50:012
1734088993931,5cyclictest861385-21sendmail-mta09:10:002
1734088993927,9cyclictest30-21ksoftirqd/211:30:002
1734088993922,13cyclictest235-21systemd-journal12:40:012
1734088993919,16cyclictest235-21systemd-journal12:15:012
173408199393,3cyclictest131rcu_preempt08:49:591
1734081993921,14cyclictest272-21dbus-daemon10:35:001
173407699392,9cyclictest1747308-21latency_hist11:25:020
173407699392,9cyclictest1747308-21latency_hist11:25:010
1734076993911,5cyclictest131rcu_preempt08:50:010
1734091993824,10cyclictest1749091-21run-parts12:00:013
1734091993824,10cyclictest1-21systemd10:45:023
1734091993821,12cyclictest235-21systemd-journal11:10:023
1734088993829,7cyclictest30-21ksoftirqd/207:55:022
1734088993827,8cyclictest1745178-21sshd10:40:302
1734088993825,10cyclictest1739013-21sed08:45:012
1734088993824,9cyclictest235-21systemd-journal07:45:012
1734088993819,14cyclictest272-21dbus-daemon07:40:002
173408899381,5cyclictest131rcu_preempt10:05:002
173408199383,3cyclictest131rcu_preempt12:25:011
173408199382,5cyclictest131rcu_preempt09:05:011
1734081993822,12cyclictest272-21dbus-daemon12:15:021
1734081993820,13cyclictest1735293-21sshd07:33:311
1734076993832,3cyclictest0-21swapper/010:30:030
173409199378,5cyclictest131rcu_preempt09:49:573
173409199376,5cyclictest131rcu_preempt09:25:013
173409199375,24cyclictest131rcu_preempt11:00:013
173409199373,6cyclictest131rcu_preempt10:55:003
1734091993728,6cyclictest1738818-21cat08:40:013
173409199372,5cyclictest131rcu_preempt11:35:023
1734091993724,9cyclictest37-21ksoftirqd/311:05:013
1734091993722,10cyclictest37-21ksoftirqd/312:35:013
173408899379,5cyclictest131rcu_preempt09:45:032
1734088993725,9cyclictest30-21ksoftirqd/209:15:022
1734088993722,12cyclictest272-21dbus-daemon10:55:002
1734088993720,11cyclictest1741842-21basename09:40:012
173408199374,3cyclictest131rcu_preempt08:25:021
1734081993710,5cyclictest131rcu_preempt08:19:291
1734081993710,5cyclictest131rcu_preempt08:19:281
1734081993710,25cyclictest1734072-21cyclictest08:45:031
173409199366,9cyclictest37-21ksoftirqd/312:20:033
173409199366,6cyclictest131rcu_preempt09:20:013
1734091993622,8cyclictest306-21in:imuxsock11:40:003
1734091993615,6cyclictest131rcu_preempt07:10:023
173408899368,3cyclictest131rcu_preempt11:55:002
173408899365,5cyclictest131rcu_preempt08:30:002
173408899365,5cyclictest131rcu_preempt08:29:592
173408899364,7cyclictest30-21ksoftirqd/209:20:022
1734088993611,5cyclictest131rcu_preempt07:57:282
1734081993623,10cyclictest1738690-21cron08:40:001
1734081993621,11cyclictest272-21dbus-daemon08:00:011
1734081993619,12cyclictest272-21dbus-daemon11:30:001
1734076993610,9cyclictest131rcu_preempt12:40:020
173409199351,5cyclictest131rcu_preempt10:35:023
1734091993514,3cyclictest37-21ksoftirqd/312:30:013
173408899355,3cyclictest131rcu_preempt12:35:012
173408899355,3cyclictest131rcu_preempt07:30:022
1734088993527,5cyclictest1739219-21latency_hist08:50:022
1734088993527,5cyclictest1735001-21latency_hist07:25:012
1734088993525,7cyclictest1738761-21latency_hist08:40:022
173408899351,3cyclictest131rcu_preempt11:19:592
1734081993527,5cyclictest1741027-21sshd09:20:311
173408199352,5cyclictest131rcu_preempt10:19:591
1734081993525,7cyclictest1734481-21latency_hist07:15:021
173408199352,4cyclictest131rcu_preempt12:00:001
173408199351,4cyclictest131rcu_preempt07:25:021
173409199344,5cyclictest131rcu_preempt08:54:593
173408899343,3cyclictest131rcu_preempt10:50:022
1734088993425,6cyclictest1744196-21latency_hist10:25:022
1734088993424,8cyclictest30-21ksoftirqd/208:15:002
1734088993420,10cyclictest306-21in:imuxsock12:25:032
1734088993420,10cyclictest0-21swapper/211:35:022
1734081993427,4cyclictest271-21cron10:25:001
1734081993424,7cyclictest23-21ksoftirqd/109:00:011
1734081993424,7cyclictest1749583-21sshd12:05:301
1734081993422,9cyclictest23-21ksoftirqd/109:15:001
1734081993418,12cyclictest1-21systemd11:00:001
1734081993412,6cyclictest131rcu_preempt12:05:001
1734091993323,8cyclictest37-21ksoftirqd/312:05:003
173409199331,7cyclictest0-21swapper/308:25:013
1734091993317,11cyclictest0-21swapper/309:40:013
1734088993324,7cyclictest30-21ksoftirqd/212:00:012
1734088993322,8cyclictest30-21ksoftirqd/211:05:022
1734088993320,9cyclictest30-21ksoftirqd/212:20:002
1734088993320,10cyclictest1-21systemd07:50:012
1734088993318,11cyclictest1738410-21sshd08:32:312
173408899331,22cyclictest291rcuc/208:10:022
173408199335,3cyclictest131rcu_preempt11:55:021
1734081993323,8cyclictest23-21ksoftirqd/107:45:031
1734081993322,9cyclictest23-21ksoftirqd/112:20:021
1734081993322,8cyclictest23-21ksoftirqd/111:50:011
1734081993317,11cyclictest0-21swapper/108:55:011
173408199331,3cyclictest131rcu_preempt07:50:011
173408199331,3cyclictest131rcu_preempt07:20:031
173409199322,8cyclictest37-21ksoftirqd/311:25:033
173409199322,8cyclictest37-21ksoftirqd/311:25:023
1734091993225,5cyclictest37-21ksoftirqd/311:50:003
1734088993222,7cyclictest30-21ksoftirqd/212:10:022
173408199324,3cyclictest131rcu_preempt08:30:011
173408199324,3cyclictest131rcu_preempt08:30:001
173408199323,3cyclictest131rcu_preempt12:30:021
1734081993223,7cyclictest23-21ksoftirqd/109:30:011
1734081993222,6cyclictest1747286-21cron11:25:011
1734081993222,6cyclictest1747286-21cron11:25:001
1734081993215,15cyclictest23-21ksoftirqd/110:55:011
173408199321,4cyclictest131rcu_preempt11:10:021
173409199311,5cyclictest0-21swapper/310:05:003
173408899313,23cyclictest1749367-21cut12:05:002
173408899312,9cyclictest1746808-21cat11:15:022
1734088993118,9cyclictest30-21ksoftirqd/208:55:012
173408899311,4cyclictest131rcu_preempt07:15:002
173408899311,3cyclictest131rcu_preempt07:10:012
173408199315,4cyclictest0-21swapper/109:55:001
1734081993125,4cyclictest1735123-21latency_hist07:30:011
1734091993020,6cyclictest37-21ksoftirqd/307:45:003
1734088993023,4cyclictest1743635-21/usr/sbin/munin10:15:012
1734088993023,4cyclictest1743635-21/usr/sbin/munin10:15:012
1734081993015,12cyclictest272-21dbus-daemon09:50:011
173408199294,3cyclictest131rcu_preempt10:05:021
173408199294,3cyclictest131rcu_preempt10:05:021
173408199272,3cyclictest131rcu_preempt07:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional