You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 17:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Apr 17, 2026 12:45:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
382532999889,6cyclictest151rcuc/007:40:010
382532998884,2cyclictest0-21swapper/007:25:020
382532998736,9cyclictest1-21systemd11:10:010
38254399851,61cyclictest120-21kswapd010:05:023
38253299809,57cyclictest445371-21ssh11:25:030
38253299771,55cyclictest0-21swapper/012:10:020
382532997124,15cyclictest407780-21ssh09:55:060
382532997054,12cyclictest0-21swapper/011:45:020
382532997054,12cyclictest0-21swapper/011:45:020
382532996937,6cyclictest12-21ksoftirqd/010:15:020
38253699688,6cyclictest131rcu_preempt11:10:011
382532996856,8cyclictest309-21in:imuxsock09:35:010
382532996624,37cyclictest441221-21date11:15:030
382532996624,12cyclictest406657-21ssh09:52:370
382532996624,12cyclictest406657-21ssh09:52:370
382532996617,13cyclictest396157-21ssh09:27:200
382543996112,5cyclictest131rcu_preempt10:25:023
38254399608,6cyclictest131rcu_preempt09:40:013
382543996011,6cyclictest131rcu_preempt11:17:453
382543995910,6cyclictest131rcu_preempt10:40:003
382536995914,12cyclictest23-21ksoftirqd/110:20:001
382532995829,13cyclictest418978-21ssh10:21:530
382543995710,5cyclictest131rcu_preempt12:30:013
382541995748,5cyclictest386404-21kworker/2:009:26:082
382536995712,9cyclictest23-21ksoftirqd/110:30:351
38253299571,27cyclictest131rcu_preempt09:36:480
38254399564,9cyclictest37-21ksoftirqd/311:41:143
38254399564,9cyclictest37-21ksoftirqd/311:41:143
382543995612,6cyclictest131rcu_preempt07:55:023
382541995632,16cyclictest275-21dbus-daemon11:50:002
382536995617,5cyclictest131rcu_preempt10:15:011
382532995629,9cyclictest420492-21ssh10:25:220
382532995627,14cyclictest448467-21ssh11:32:370
38254399559,7cyclictest131rcu_preempt08:29:593
38254399559,6cyclictest131rcu_preempt11:30:013
382543995512,5cyclictest131rcu_preempt12:09:073
382541995546,5cyclictest399249-21kworker/2:310:32:342
382541995540,6cyclictest453606-21kworker/2:212:37:292
38253699553,5cyclictest131rcu_preempt08:20:011
382532995528,12cyclictest456685-21ssh11:52:110
38253299552,29cyclictest131rcu_preempt10:05:020
38254399548,6cyclictest131rcu_preempt10:45:013
382543995430,16cyclictest37-21ksoftirqd/309:49:363
38254399542,6cyclictest131rcu_preempt10:06:133
382543995425,10cyclictest37-21ksoftirqd/311:50:003
382543995419,8cyclictest37-21ksoftirqd/307:40:013
382543995416,6cyclictest131rcu_preempt11:23:093
382543995415,10cyclictest37-21ksoftirqd/312:38:243
382543995413,6cyclictest131rcu_preempt10:19:253
382543995410,4cyclictest131rcu_preempt11:37:173
382541995446,5cyclictest453606-21kworker/2:212:00:492
382541995446,5cyclictest383268-21kworker/2:111:25:012
382541995445,5cyclictest399249-21kworker/2:310:13:252
38253699549,4cyclictest131rcu_preempt11:20:011
38253699543,11cyclictest469770-21rm12:23:271
382536995413,7cyclictest131rcu_preempt08:30:001
382536995412,6cyclictest131rcu_preempt11:00:001
382536995412,5cyclictest131rcu_preempt12:16:511
382536995411,9cyclictest23-21ksoftirqd/112:31:481
382532995427,11cyclictest435802-21ssh11:02:080
38253299541,38cyclictest423502-21ssh10:32:500
38253299541,37cyclictest441346-21ssh11:15:130
38254399532,8cyclictest37-21ksoftirqd/312:00:003
382543995325,11cyclictest37-21ksoftirqd/309:32:283
382543995312,6cyclictest131rcu_preempt09:43:403
382543995311,7cyclictest131rcu_preempt08:25:003
382541995346,4cyclictest399249-21kworker/2:310:01:102
382541995345,5cyclictest383268-21kworker/2:111:34:372
382541995345,5cyclictest383268-21kworker/2:110:45:012
382541995345,5cyclictest383268-21kworker/2:110:39:082
382541995344,6cyclictest383268-21kworker/2:109:18:002
382541995343,5cyclictest383268-21kworker/2:109:05:022
382541995343,5cyclictest383268-21kworker/2:109:05:012
38253699534,8cyclictest23-21ksoftirqd/110:25:081
382536995334,15cyclictest1-21systemd12:10:011
38253699532,12cyclictest23-21ksoftirqd/112:03:471
382536995311,5cyclictest131rcu_preempt09:20:001
38253299534,3cyclictest131rcu_preempt10:49:260
382532995326,12cyclictest475226-21ssh12:36:240
382532995325,13cyclictest402809-21ssh09:43:290
38254399529,5cyclictest131rcu_preempt10:54:043
38254399526,6cyclictest131rcu_preempt09:10:013
382543995217,9cyclictest37-21ksoftirqd/311:15:023
382543995214,7cyclictest37-21ksoftirqd/307:20:013
382543995214,7cyclictest131rcu_preempt11:35:013
382543995214,5cyclictest131rcu_preempt12:15:503
382543995213,6cyclictest131rcu_preempt11:09:493
382543995213,6cyclictest131rcu_preempt09:22:273
382543995212,6cyclictest131rcu_preempt10:50:003
382543995211,17cyclictest131rcu_preempt12:21:073
382543995210,6cyclictest131rcu_preempt10:58:353
382543995210,6cyclictest131rcu_preempt09:14:173
382543995210,6cyclictest131rcu_preempt09:14:173
382541995244,5cyclictest399249-21kworker/2:310:06:112
382541995244,5cyclictest383268-21kworker/2:111:07:382
382541995243,6cyclictest434835-21kworker/2:012:14:552
382541995243,5cyclictest434835-21kworker/2:012:24:452
382541995243,5cyclictest434835-21kworker/2:012:18:312
382541995242,6cyclictest386404-21kworker/2:009:31:222
382541995242,6cyclictest383268-21kworker/2:110:54:292
38253699526,6cyclictest131rcu_preempt09:00:001
382536995220,7cyclictest23-21ksoftirqd/107:15:021
382536995220,21cyclictest221rcuc/110:41:161
382536995212,5cyclictest131rcu_preempt10:48:191
38253699521,19cyclictest0-21swapper/112:37:051
382532995229,18cyclictest275-21dbus-daemon12:25:010
382532995225,23cyclictest467639-21ssh12:18:210
38254399518,4cyclictest131rcu_preempt12:33:293
382543995135,12cyclictest275-21dbus-daemon07:15:013
382543995113,6cyclictest131rcu_preempt12:13:493
382543995111,5cyclictest131rcu_preempt09:58:363
382541995143,5cyclictest399249-21kworker/2:310:00:022
382541995143,5cyclictest383268-21kworker/2:109:45:572
382541995142,5cyclictest399249-21kworker/2:310:25:322
382541995142,5cyclictest383268-21kworker/2:111:11:302
382541995142,5cyclictest383268-21kworker/2:109:44:222
382541995140,7cyclictest383268-21kworker/2:109:00:012
382541995137,6cyclictest453606-21kworker/2:211:57:222
38253699518,6cyclictest131rcu_preempt11:23:071
38253699512,8cyclictest23-21ksoftirqd/109:33:591
38253699511,5cyclictest131rcu_preempt09:15:021
38253699511,5cyclictest131rcu_preempt09:15:021
382536995115,5cyclictest131rcu_preempt11:40:041
382536995115,5cyclictest131rcu_preempt11:33:251
382536995111,5cyclictest131rcu_preempt10:25:021
382532995134,14cyclictest1250ksoftirqd/012:30:010
382532995134,12cyclictest275-21dbus-daemon11:40:010
382532995122,25cyclictest12-21ksoftirqd/009:45:090
38254399507,5cyclictest131rcu_preempt12:04:243
382541995043,4cyclictest434835-21kworker/2:011:52:242
382541995041,5cyclictest399249-21kworker/2:310:23:202
382541995041,5cyclictest383268-21kworker/2:111:15:312
382541995041,5cyclictest383268-21kworker/2:110:45:202
382541995041,5cyclictest383268-21kworker/2:109:40:002
382541995041,5cyclictest383268-21kworker/2:109:20:462
382541995040,6cyclictest383268-21kworker/2:111:36:192
38253699509,6cyclictest23-21ksoftirqd/109:55:001
38253699509,6cyclictest23-21ksoftirqd/109:55:001
38253699509,6cyclictest131rcu_preempt11:59:561
38253699508,5cyclictest131rcu_preempt11:41:101
38253699508,5cyclictest131rcu_preempt11:41:101
38253699504,6cyclictest23-21ksoftirqd/109:59:281
382536995015,8cyclictest131rcu_preempt12:27:411
382536995014,5cyclictest131rcu_preempt11:25:531
382536995010,6cyclictest131rcu_preempt10:36:001
382536995010,6cyclictest131rcu_preempt09:26:571
382536995010,5cyclictest131rcu_preempt10:05:021
38253299503,10cyclictest3467602-21systemd-journal09:00:000
38254399498,5cyclictest131rcu_preempt09:29:413
38254399496,6cyclictest131rcu_preempt07:25:023
38254399496,5cyclictest131rcu_preempt10:35:013
38254399494,20cyclictest37-21ksoftirqd/311:52:143
38254399492,10cyclictest37-21ksoftirqd/309:00:003
382543994914,5cyclictest131rcu_preempt10:12:013
382543994913,9cyclictest131rcu_preempt11:04:333
382543994913,9cyclictest131rcu_preempt10:25:223
382543994912,8cyclictest131rcu_preempt08:18:233
382541994941,5cyclictest453606-21kworker/2:212:32:392
382541994941,5cyclictest453606-21kworker/2:212:05:422
382541994941,5cyclictest399249-21kworker/2:310:15:472
382541994940,5cyclictest399249-21kworker/2:309:50:352
382541994940,5cyclictest399249-21kworker/2:309:50:352
382541994940,5cyclictest383268-21kworker/2:109:12:072
382541994940,5cyclictest383268-21kworker/2:109:12:072
38253699497,6cyclictest131rcu_preempt09:43:221
38253699497,11cyclictest23-21ksoftirqd/108:23:221
38253699496,17cyclictest131rcu_preempt10:54:001
38253699493,8cyclictest23-21ksoftirqd/111:53:281
38253699491,5cyclictest131rcu_preempt11:15:011
382532994941,4cyclictest16850irq/134-800000010:45:000
38254399486,6cyclictest131rcu_preempt09:54:253
38254399486,6cyclictest131rcu_preempt09:54:253
38254399483,8cyclictest37-21ksoftirqd/309:16:233
382543994811,5cyclictest131rcu_preempt07:10:023
382541994840,5cyclictest383268-21kworker/2:110:55:442
382541994840,5cyclictest381699-21kworker/2:207:14:222
38253699489,6cyclictest131rcu_preempt10:07:241
38253699489,5cyclictest131rcu_preempt07:55:011
38253699488,5cyclictest131rcu_preempt08:50:021
38253699486,13cyclictest23-21ksoftirqd/107:50:011
38253699484,5cyclictest131rcu_preempt09:50:021
382536994838,7cyclictest23-21ksoftirqd/111:00:131
382536994838,6cyclictest23-21ksoftirqd/107:10:011
382536994813,5cyclictest131rcu_preempt07:30:031
38253699481,17cyclictest0-21swapper/109:21:591
38253299485,5cyclictest131rcu_preempt09:10:000
38253299483,26cyclictest131rcu_preempt11:55:330
382532994823,6cyclictest0-21swapper/010:55:380
382532994821,12cyclictest465828-21ssh12:14:130
38254399478,4cyclictest131rcu_preempt08:51:223
38254399472,8cyclictest37-21ksoftirqd/308:10:013
382541994740,4cyclictest453606-21kworker/2:212:27:192
382541994739,5cyclictest383268-21kworker/2:111:03:402
382541994739,5cyclictest381699-21kworker/2:207:10:022
382541994732,11cyclictest1-21systemd07:25:012
38253699478,6cyclictest131rcu_preempt07:40:001
38253699478,5cyclictest131rcu_preempt07:30:031
38253699477,6cyclictest131rcu_preempt09:38:251
38253699473,7cyclictest131rcu_preempt08:44:021
382536994713,5cyclictest131rcu_preempt08:00:011
382536994713,5cyclictest131rcu_preempt08:00:011
382536994711,5cyclictest131rcu_preempt12:14:451
38253299473,3cyclictest131rcu_preempt10:10:020
38253299473,38cyclictest392917-21modprobe09:20:000
382532994725,7cyclictest425660-21ssh10:38:070
38254399462,6cyclictest131rcu_preempt08:50:023
382543994614,9cyclictest37-21ksoftirqd/308:40:013
382543994611,5cyclictest131rcu_preempt07:50:013
382541994638,5cyclictest434835-21kworker/2:011:43:302
382541994638,5cyclictest434835-21kworker/2:011:43:302
382541994637,6cyclictest383268-21kworker/2:111:27:312
382541994637,6cyclictest381699-21kworker/2:207:42:332
382541994637,6cyclictest381699-21kworker/2:207:42:332
382541994633,9cyclictest30-21ksoftirqd/208:05:012
382541994629,13cyclictest275-21dbus-daemon09:10:002
38253699464,11cyclictest23-21ksoftirqd/109:05:011
38253699464,11cyclictest23-21ksoftirqd/109:05:011
382532994628,15cyclictest384772-21sshd07:53:220
382532994628,14cyclictest389112-21ssh09:10:270
382532994628,14cyclictest389112-21ssh09:10:270
382532994622,9cyclictest455556-21ssh11:49:480
38253299461,3cyclictest131rcu_preempt10:50:230
382543994510,5cyclictest131rcu_preempt08:00:003
382543994510,5cyclictest131rcu_preempt07:59:593
382541994537,5cyclictest383268-21kworker/2:108:35:022
382541994537,5cyclictest383268-21kworker/2:107:54:182
382541994529,12cyclictest382985-21systemctl07:20:002
38253699459,6cyclictest131rcu_preempt11:45:161
38253699455,5cyclictest131rcu_preempt07:45:011
38253699455,5cyclictest131rcu_preempt07:45:011
382536994530,11cyclictest3467602-21systemd-journal08:10:021
382536994513,5cyclictest131rcu_preempt08:10:021
382536994512,5cyclictest131rcu_preempt09:10:001
382536994512,4cyclictest131rcu_preempt08:38:511
382532994525,15cyclictest1-21systemd08:25:000
382532994519,12cyclictest460852-21ssh12:02:090
38253299451,41cyclictest417348-21ssh10:18:080
38254399449,5cyclictest131rcu_preempt07:35:013
38254399449,4cyclictest131rcu_preempt09:05:023
38254399449,4cyclictest131rcu_preempt09:05:013
38254399443,11cyclictest37-21ksoftirqd/308:44:593
382543994411,5cyclictest131rcu_preempt07:43:233
382543994411,5cyclictest131rcu_preempt07:43:233
382541994436,5cyclictest381699-21kworker/2:207:38:062
38253699447,6cyclictest131rcu_preempt08:05:021
38253699446,5cyclictest131rcu_preempt08:55:021
38253699441,5cyclictest131rcu_preempt07:25:021
38253299449,6cyclictest131rcu_preempt08:00:010
38253299449,6cyclictest131rcu_preempt08:00:010
382532994434,7cyclictest19350irq/34-mmc107:35:000
382532994428,13cyclictest393489-21ssh09:20:490
382543994313,5cyclictest131rcu_preempt07:26:223
382543994313,5cyclictest131rcu_preempt07:26:213
382541994335,5cyclictest383268-21kworker/2:108:50:472
382541994328,11cyclictest3467602-21systemd-journal08:15:002
382532994329,10cyclictest387117-21cron08:40:000
382532994327,11cyclictest275-21dbus-daemon08:45:010
382532994326,14cyclictest446246-21ssh11:27:110
382532994324,14cyclictest1-21systemd08:10:010
382532994320,18cyclictest3467602-21systemd-journal08:15:260
38253299431,38cyclictest472939-21ssh12:30:480
382543994213,6cyclictest131rcu_preempt08:11:363
382543994210,5cyclictest131rcu_preempt08:00:033
38254199429,20cyclictest30-21ksoftirqd/208:40:022
382541994234,5cyclictest383268-21kworker/2:108:05:352
382541994234,5cyclictest383268-21kworker/2:108:00:032
382541994234,5cyclictest383268-21kworker/2:108:00:022
382541994232,6cyclictest309-21in:imuxsock08:45:012
382536994229,10cyclictest386908-21cut08:35:021
38253299421,23cyclictest131rcu_preempt09:03:550
38253299421,23cyclictest131rcu_preempt09:03:550
382541994112,5cyclictest131rcu_preempt08:26:222
382532994120,3cyclictest131rcu_preempt07:18:340
382532994111,6cyclictest131rcu_preempt08:55:010
38254399404,10cyclictest37-21ksoftirqd/308:35:013
382541994032,5cyclictest383268-21kworker/2:108:21:082
382541994029,8cyclictest383731-21sshd07:32:232
38254199402,8cyclictest0-21swapper/208:20:012
38253699404,5cyclictest131rcu_preempt07:20:011
382536993913,6cyclictest131rcu_preempt07:35:021
382532993923,12cyclictest275-21dbus-daemon08:30:010
38253299391,35cyclictest0-21swapper/008:15:010
38254199382,4cyclictest131rcu_preempt08:50:012
382541993819,12cyclictest30-21ksoftirqd/207:50:022
382532993724,9cyclictest12-21ksoftirqd/007:49:210
38253299371,6cyclictest384292-21munin-run07:45:010
38253299371,6cyclictest384292-21munin-run07:45:010
38253299371,32cyclictest0-21swapper/008:46:490
38254199361,32cyclictest30-21ksoftirqd/207:30:012
38254199361,32cyclictest30-21ksoftirqd/207:30:012
38253299362,5cyclictest386879-21munin-run08:35:010
382532993623,9cyclictest12-21ksoftirqd/007:10:020
382532993620,12cyclictest385302-21sshd08:02:220
382532993422,9cyclictest382713-21latency_hist07:15:020
38253299321,4cyclictest0-21swapper/007:26:590
38253299321,4cyclictest0-21swapper/007:26:590
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional