You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 02:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Sat Nov 15, 2025 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29563749911443,4cyclictest23-21ksoftirqd/112:00:011
29563749911443,4cyclictest23-21ksoftirqd/112:00:011
2956376996610,11cyclictest30-21ksoftirqd/210:05:012
2956376996610,11cyclictest30-21ksoftirqd/210:05:002
295636699666,5cyclictest131rcu_preempt11:10:010
2956366996561,2cyclictest0-21swapper/007:15:010
295637699617,11cyclictest30-21ksoftirqd/210:00:002
2956366996127,30cyclictest12-21ksoftirqd/008:10:010
295637699608,10cyclictest30-21ksoftirqd/209:40:022
295636699602,48cyclictest2960985-21systemctl08:40:010
2956366995913,43cyclictest382-21ntpd11:40:010
2956366995837,15cyclictest1-21systemd08:50:000
2956376995737,17cyclictest1-21systemd09:00:002
2956376995714,8cyclictest30-21ksoftirqd/209:20:012
295637699571,15cyclictest30-21ksoftirqd/207:35:002
295636699577,3cyclictest131rcu_preempt08:25:020
2956374995624,25cyclictest2956268-21kworker/u8:107:20:021
295638199558,5cyclictest131rcu_preempt09:45:013
295637699556,10cyclictest30-21ksoftirqd/211:00:012
2956376995512,8cyclictest30-21ksoftirqd/208:55:032
2956376995512,8cyclictest30-21ksoftirqd/208:55:022
2956381995436,13cyclictest272-21dbus-daemon11:15:013
2956381995420,6cyclictest131rcu_preempt07:40:023
295637699547,11cyclictest2972427-21munin-run12:20:002
2956376995442,8cyclictest0-21swapper/207:10:012
2956376995413,10cyclictest30-21ksoftirqd/211:30:002
295637699541,12cyclictest30-21ksoftirqd/210:10:022
2956376995410,12cyclictest30-21ksoftirqd/208:15:022
295638199536,9cyclictest37-21ksoftirqd/308:04:593
295638199527,9cyclictest37-21ksoftirqd/309:30:013
2956381995240,8cyclictest37-21ksoftirqd/307:50:013
2956381995240,8cyclictest37-21ksoftirqd/307:50:013
295638199521,9cyclictest37-21ksoftirqd/308:25:013
295637699529,10cyclictest30-21ksoftirqd/212:25:032
295637699525,10cyclictest30-21ksoftirqd/210:45:022
295637699525,10cyclictest30-21ksoftirqd/210:45:012
295637699524,7cyclictest30-21ksoftirqd/210:55:022
295637699522,8cyclictest30-21ksoftirqd/210:50:002
2956376995212,8cyclictest0-21swapper/212:30:012
2956376995211,12cyclictest30-21ksoftirqd/209:55:022
295636699522,24cyclictest131rcu_preempt09:20:020
2956381995113,6cyclictest131rcu_preempt11:00:003
295637699519,10cyclictest30-21ksoftirqd/211:45:012
295637699516,9cyclictest30-21ksoftirqd/207:25:022
295637699516,8cyclictest30-21ksoftirqd/211:55:012
295637699515,8cyclictest30-21ksoftirqd/211:25:032
295637699515,8cyclictest30-21ksoftirqd/211:25:022
295637699512,11cyclictest30-21ksoftirqd/209:35:002
2956376995112,10cyclictest30-21ksoftirqd/212:10:012
2956366995110,35cyclictest2956822-21modprobe07:20:000
295638199506,10cyclictest37-21ksoftirqd/310:25:013
2956381995012,6cyclictest131rcu_preempt10:00:013
2956381995011,10cyclictest131rcu_preempt10:30:013
2956381995011,10cyclictest131rcu_preempt10:30:003
295636699507,7cyclictest131rcu_preempt12:34:490
2956366995033,12cyclictest235-21systemd-journal07:37:490
2956366995016,20cyclictest12-21ksoftirqd/007:45:020
2956366995016,20cyclictest12-21ksoftirqd/007:45:020
295637699498,10cyclictest30-21ksoftirqd/212:15:032
295637699496,8cyclictest30-21ksoftirqd/211:40:022
295637699495,8cyclictest30-21ksoftirqd/208:20:002
295637699492,10cyclictest30-21ksoftirqd/212:04:592
2956376994915,9cyclictest30-21ksoftirqd/208:10:012
2956376994913,8cyclictest30-21ksoftirqd/208:25:002
295637699491,14cyclictest30-21ksoftirqd/211:05:012
2956376994911,10cyclictest30-21ksoftirqd/208:00:012
2956374994931,14cyclictest272-21dbus-daemon10:00:001
2956366994927,16cyclictest272-21dbus-daemon12:40:010
2956366994927,16cyclictest272-21dbus-daemon12:40:010
295638199489,8cyclictest37-21ksoftirqd/310:20:023
2956381994835,9cyclictest1-21systemd08:40:023
2956381994832,12cyclictest235-21systemd-journal12:10:013
295638199483,10cyclictest37-21ksoftirqd/309:20:013
295638199482,9cyclictest37-21ksoftirqd/310:05:013
295638199482,9cyclictest37-21ksoftirqd/310:05:003
2956376994834,9cyclictest1-21systemd10:20:012
295637699482,9cyclictest0-21swapper/208:50:012
295637699482,11cyclictest30-21ksoftirqd/209:15:012
295637699481,8cyclictest2973269-21systemd-run12:35:012
295637699481,7cyclictest30-21ksoftirqd/210:35:012
2956376994810,9cyclictest30-21ksoftirqd/209:30:002
2956366994832,11cyclictest2970635-21grep11:45:010
2956366994831,13cyclictest12-21ksoftirqd/010:10:010
2956366994829,15cyclictest235-21systemd-journal12:30:010
295636699482,42cyclictest2971102-21sshd11:54:480
295638199478,6cyclictest131rcu_preempt08:10:023
2956381994723,6cyclictest37-21ksoftirqd/311:30:003
295638199471,9cyclictest2962022-21munin-run09:00:013
2956381994713,4cyclictest131rcu_preempt12:34:503
295637699477,8cyclictest30-21ksoftirqd/207:50:022
295637699477,8cyclictest30-21ksoftirqd/207:50:022
295637699477,5cyclictest30-21ksoftirqd/209:25:022
295637699476,9cyclictest30-21ksoftirqd/207:45:022
295637699474,10cyclictest30-21ksoftirqd/211:20:012
295637699473,7cyclictest30-21ksoftirqd/209:10:022
295637699473,7cyclictest30-21ksoftirqd/209:10:012
295637699473,10cyclictest30-21ksoftirqd/208:35:002
2956381994637,6cyclictest37-21ksoftirqd/308:45:003
2956381994614,5cyclictest131rcu_preempt10:10:013
295638199461,10cyclictest37-21ksoftirqd/311:25:023
295638199461,10cyclictest37-21ksoftirqd/311:25:013
295637699466,8cyclictest30-21ksoftirqd/208:45:012
295637699463,12cyclictest30-21ksoftirqd/209:05:012
295637699463,12cyclictest30-21ksoftirqd/207:55:012
2956376994626,15cyclictest272-21dbus-daemon08:30:022
2956376994610,9cyclictest30-21ksoftirqd/209:40:022
2956374994628,13cyclictest2967224-21systemctl10:40:001
295638199459,6cyclictest131rcu_preempt09:40:013
295638199455,6cyclictest131rcu_preempt12:20:003
2956381994533,9cyclictest37-21ksoftirqd/312:25:003
2956381994533,7cyclictest37-21ksoftirqd/311:20:003
295638199452,5cyclictest131rcu_preempt07:35:013
2956381994521,20cyclictest37-21ksoftirqd/311:40:023
295637699455,7cyclictest30-21ksoftirqd/210:40:022
295637699454,10cyclictest30-21ksoftirqd/210:15:012
2956376994537,5cyclictest30-21ksoftirqd/211:14:592
295637699453,10cyclictest30-21ksoftirqd/209:48:212
2956366994528,13cyclictest2958108-21sshd07:40:480
295636699451,3cyclictest131rcu_preempt09:05:020
295638199445,6cyclictest131rcu_preempt10:45:023
295638199445,6cyclictest131rcu_preempt10:45:023
295638199443,8cyclictest37-21ksoftirqd/308:15:003
2956381994434,7cyclictest37-21ksoftirqd/309:25:013
2956381994428,13cyclictest37-21ksoftirqd/308:00:013
295638199442,6cyclictest131rcu_preempt09:50:013
295638199441,9cyclictest37-21ksoftirqd/307:20:003
295638199441,9cyclictest2965061-21wc09:55:023
2956381994415,9cyclictest131rcu_preempt09:07:493
2956381994415,9cyclictest131rcu_preempt09:07:483
295637699448,7cyclictest30-21ksoftirqd/211:35:012
295637699444,10cyclictest30-21ksoftirqd/210:25:032
295637699441,9cyclictest2970990-21cut11:50:022
295637499441,3cyclictest131rcu_preempt10:05:011
295637499441,3cyclictest131rcu_preempt10:05:011
2956366994413,11cyclictest131rcu_preempt11:00:020
295638199434,11cyclictest37-21ksoftirqd/308:35:023
2956381994330,10cyclictest37-21ksoftirqd/311:54:593
2956381994327,11cyclictest272-21dbus-daemon11:45:003
295637699438,9cyclictest30-21ksoftirqd/207:40:022
295637699434,9cyclictest30-21ksoftirqd/208:05:012
295637699434,6cyclictest30-21ksoftirqd/207:30:022
295637699433,7cyclictest30-21ksoftirqd/212:40:012
295637699433,7cyclictest30-21ksoftirqd/212:40:002
295637699431,7cyclictest2971107-21cron11:55:032
295637699431,7cyclictest2971107-21cron11:55:022
2956374994329,11cyclictest235-21systemd-journal10:15:021
2956374994327,12cyclictest272-21dbus-daemon09:05:001
2956374994313,6cyclictest131rcu_preempt08:00:001
2956366994330,9cyclictest306-21in:imuxsock08:45:000
2956366994322,14cyclictest235-21systemd-journal08:30:000
295638199427,5cyclictest131rcu_preempt12:40:013
295638199427,5cyclictest131rcu_preempt12:40:003
295638199423,9cyclictest37-21ksoftirqd/309:31:483
2956381994232,7cyclictest37-21ksoftirqd/311:50:023
295638199422,8cyclictest37-21ksoftirqd/308:30:003
295638199421,9cyclictest37-21ksoftirqd/308:50:003
2956381994216,7cyclictest37-21ksoftirqd/312:00:033
2956381994216,7cyclictest37-21ksoftirqd/312:00:023
2956381994215,8cyclictest131rcu_preempt07:30:013
295637699424,10cyclictest30-21ksoftirqd/207:20:012
2956366994236,3cyclictest16950irq/224-800000007:30:020
295636699422,6cyclictest2959951-21munin-run08:20:000
295636699422,3cyclictest131rcu_preempt10:15:020
295638199417,5cyclictest131rcu_preempt11:35:023
2956381994132,6cyclictest37-21ksoftirqd/308:55:013
2956381994132,6cyclictest37-21ksoftirqd/308:55:013
2956381994114,20cyclictest131rcu_preempt11:05:023
2956381994113,5cyclictest131rcu_preempt10:34:593
2956381994112,20cyclictest361rcuc/307:10:013
2956381994111,5cyclictest131rcu_preempt12:30:033
295637699415,9cyclictest30-21ksoftirqd/207:15:002
295637699415,7cyclictest30-21ksoftirqd/208:39:482
2956376994134,5cyclictest30-21ksoftirqd/210:28:482
2956376994134,5cyclictest30-21ksoftirqd/210:28:482
295637699411,7cyclictest2968745-21modprobe11:05:022
2956374994125,12cyclictest235-21systemd-journal11:41:181
2956374994125,12cyclictest23-21ksoftirqd/107:40:001
2956374994122,13cyclictest1-21systemd12:34:481
295636699416,4cyclictest131rcu_preempt08:15:000
2956366994126,11cyclictest2961983-21sshd08:56:490
2956366994126,11cyclictest235-21systemd-journal09:10:010
2956366994126,11cyclictest235-21systemd-journal09:10:000
2956366994126,10cyclictest408-21lldpd10:52:370
2956366994124,13cyclictest2960696-21sshd08:31:470
295636699412,35cyclictest2962809-21true09:15:000
295636699411,5cyclictest2971893-21modprobe12:10:010
295636699411,36cyclictest2969559-21sshd11:23:480
295636699411,36cyclictest2969559-21sshd11:23:470
295638199409,8cyclictest131rcu_preempt07:55:003
295638199409,5cyclictest131rcu_preempt07:45:003
295638199408,6cyclictest131rcu_preempt08:20:013
295638199406,5cyclictest131rcu_preempt10:15:023
2956381994032,6cyclictest37-21ksoftirqd/309:15:003
2956381994032,5cyclictest37-21ksoftirqd/307:20:033
2956381994023,12cyclictest235-21systemd-journal12:15:003
2956381994012,6cyclictest131rcu_preempt10:55:003
2956374994032,5cyclictest2968280-21sendmail-msp11:00:011
2956374994029,8cyclictest2957577-21sshd07:30:491
2956374994029,8cyclictest23-21ksoftirqd/108:45:011
2956374994026,11cyclictest23-21ksoftirqd/111:40:011
2956374994024,12cyclictest235-21systemd-journal09:15:021
295637499401,3cyclictest131rcu_preempt12:25:001
2956366994032,4cyclictest306-21in:imuxsock09:30:000
2956366994030,6cyclictest306-21in:imuxsock11:05:010
2956366994024,13cyclictest2972120-21sshd12:10:470
295636699402,34cyclictest2957090-21sshd07:24:480
2956366994020,15cyclictest272-21dbus-daemon10:25:000
295636699401,3cyclictest131rcu_preempt07:10:010
2956381993927,9cyclictest37-21ksoftirqd/310:50:023
2956381993925,8cyclictest306-21in:imuxsock07:15:003
295637499397,26cyclictest272-21dbus-daemon10:30:021
295637499397,26cyclictest272-21dbus-daemon10:30:011
2956374993930,5cyclictest2964375-21sh09:45:001
2956374993928,6cyclictest2963051-21cron09:20:001
2956374993927,7cyclictest2959947-21modprobe08:19:591
2956374993924,11cyclictest0-21swapper/110:50:001
2956374993924,10cyclictest235-21systemd-journal09:55:011
2956374993923,12cyclictest272-21dbus-daemon11:30:011
2956374993922,13cyclictest2973515-21systemctl12:40:011
2956374993922,13cyclictest2973515-21systemctl12:40:001
2956374993922,13cyclictest1-21systemd12:15:011
2956374993921,13cyclictest2964086-21cron09:40:011
2956366993927,9cyclictest12-21ksoftirqd/012:16:590
2956366993925,10cyclictest2964857-21sshd09:51:470
2956366993925,10cyclictest272-21dbus-daemon09:35:010
295636699392,33cyclictest12-21ksoftirqd/011:29:590
2956366993922,12cyclictest272-21dbus-daemon09:50:000
295638199386,9cyclictest2962263-21sshd09:01:483
295638199385,6cyclictest131rcu_preempt10:40:013
2956381993810,6cyclictest131rcu_preempt11:10:013
2956374993825,9cyclictest23-21ksoftirqd/112:05:011
2956374993825,10cyclictest23-21ksoftirqd/108:50:021
295637499382,32cyclictest23-21ksoftirqd/108:25:021
2956374993818,4cyclictest23-21ksoftirqd/112:30:011
2956374993817,15cyclictest1-21systemd12:10:001
2956374993815,8cyclictest131rcu_preempt10:45:031
2956374993815,8cyclictest131rcu_preempt10:45:021
295637499381,3cyclictest131rcu_preempt08:30:011
2956366993831,4cyclictest16950irq/224-800000010:45:010
2956366993831,4cyclictest16950irq/224-800000010:45:010
2956366993828,5cyclictest306-21in:imuxsock08:55:010
2956366993828,5cyclictest306-21in:imuxsock08:55:010
295636699382,5cyclictest131rcu_preempt11:35:010
2956366993823,11cyclictest235-21systemd-journal07:30:190
2956366993822,11cyclictest272-21dbus-daemon09:40:000
2956366993817,18cyclictest0-21swapper/007:50:420
2956374993726,7cyclictest2960711-21cron08:35:011
2956374993725,9cyclictest2959393-21sshd08:05:381
2956374993725,8cyclictest23-21ksoftirqd/107:30:001
2956374993724,10cyclictest2957118-21systemctl07:25:011
2956374993723,10cyclictest272-21dbus-daemon07:55:011
2956374993710,4cyclictest131rcu_preempt08:05:001
295636699372,7cyclictest2970868-21munin-run11:50:010
2956366993724,10cyclictest2969256-21sshd11:16:140
295636699372,32cyclictest2964352-21sshd09:41:480
295636699372,31cyclictest2966826-21sed10:30:030
295636699372,31cyclictest2966826-21sed10:30:020
2956366993722,11cyclictest2971339-21sshd11:56:210
2956366993722,11cyclictest2971339-21sshd11:56:200
2956366993722,11cyclictest272-21dbus-daemon09:25:010
2956366993720,13cyclictest235-21systemd-journal11:15:010
295636699371,5cyclictest2958914-21tr08:00:010
295637499363,7cyclictest23-21ksoftirqd/107:45:011
2956374993625,8cyclictest2968794-21systemd-run11:10:011
2956374993625,8cyclictest23-21ksoftirqd/108:35:481
2956374993625,7cyclictest2971107-21cron11:55:001
2956374993624,8cyclictest23-21ksoftirqd/109:30:011
2956374993622,10cyclictest235-21systemd-journal10:09:591
2956374993619,13cyclictest2972434-21sendmail12:20:011
295637499361,3cyclictest131rcu_preempt08:15:021
2956366993626,7cyclictest2965393-21sshd10:02:470
2956366993626,7cyclictest2965393-21sshd10:02:470
2956366993626,7cyclictest235-21systemd-journal10:40:010
2956366993625,7cyclictest12-21ksoftirqd/012:25:010
2956366993621,11cyclictest272-21dbus-daemon10:00:010
295636699361,31cyclictest0-21swapper/010:15:430
2956366993610,5cyclictest131rcu_preempt10:35:000
2956374993520,11cyclictest23-21ksoftirqd/109:23:151
295637499351,3cyclictest131rcu_preempt10:55:011
2956381993410,5cyclictest131rcu_preempt12:05:003
295637499345,4cyclictest131rcu_preempt11:50:011
2956374993426,5cyclictest2969033-21latency_hist11:15:001
2956374993423,6cyclictest23-21ksoftirqd/107:15:011
2956374993420,11cyclictest2965993-21date10:15:021
2956374993418,12cyclictest23-21ksoftirqd/111:20:001
2956374993417,12cyclictest272-21dbus-daemon08:55:021
2956374993417,12cyclictest272-21dbus-daemon08:55:011
295637499341,3cyclictest131rcu_preempt09:50:021
2956366993421,10cyclictest235-21systemd-journal12:00:560
295636699341,3cyclictest131rcu_preempt08:05:010
295637499335,3cyclictest131rcu_preempt11:25:021
295637499335,3cyclictest131rcu_preempt11:25:011
2956374993319,9cyclictest0-21swapper/109:00:021
2956366993327,4cyclictest0-21swapper/010:46:070
2956374993224,5cyclictest2966961-21latency_hist10:35:021
2956374993223,6cyclictest2970068-21cron11:35:001
2956374993222,7cyclictest23-21ksoftirqd/110:25:011
2956374993221,9cyclictest23-21ksoftirqd/107:50:001
2956374993221,9cyclictest23-21ksoftirqd/107:50:001
2956374993221,6cyclictest306-21in:imuxsock09:35:001
2956374993216,12cyclictest272-21dbus-daemon11:05:001
2956374993124,4cyclictest2956392-21latency_hist07:10:011
2956374993019,8cyclictest1-21systemd09:10:011
2956374993019,8cyclictest1-21systemd09:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional