You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-22 - 23:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Mon Jun 22, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2438021710,5sleep33089399cyclictest09:26:113
393621130,5sleep23089299cyclictest10:59:022
551921010,10sleep2301ktimersoftd/212:19:132
3088899961,72cyclictest24230-21ssh10:05:170
30888998721,51cyclictest18166-21latency_hist09:20:070
288732840,2sleep30-21swapper/312:07:473
98662810,4sleep23089299cyclictest11:45:202
212022800,2sleep10-21swapper/112:37:401
163222790,3sleep229-21rcuc/209:17:212
217922780,2sleep00-21swapper/010:01:150
217922780,2sleep00-21swapper/010:01:150
3088899756,64cyclictest0-21swapper/011:55:200
30888997019,26cyclictest18278-21latency_hist08:05:070
3088899701,51cyclictest30308-21grep10:50:340
3088899682,54cyclictest0-21swapper/009:00:290
3088899682,54cyclictest0-21swapper/009:00:290
3088899671,55cyclictest13631-21cron09:15:060
3088899671,52cyclictest0-21swapper/010:20:340
211252660,2sleep30-21swapper/311:58:453
3088899653,48cyclictest0-21swapper/012:20:200
3088899652,47cyclictest750-21snmpd09:50:350
3088899651,51cyclictest21212-21latency_hist11:20:070
3088899651,47cyclictest15093-21grep11:50:340
3088899650,58cyclictest25161-21kworker/0:211:40:220
30888996411,8cyclictest0-21swapper/009:25:180
30888996411,38cyclictest0-21swapper/011:45:190
3088899631,52cyclictest9327-21fschecks_time11:05:230
3088899631,50cyclictest0-21swapper/007:55:060
3088899621,47cyclictest0-21swapper/012:10:170
135472620,2sleep20-21swapper/209:14:592
3089399615,19cyclictest8050irq/56-dwc_otg_08:15:193
3088899611,35cyclictest0-21swapper/008:40:170
3088899611,35cyclictest0-21swapper/008:40:170
30888996110,40cyclictest22279-21idleruntime-cro09:25:060
3088899602,46cyclictest0-21swapper/010:15:270
3088899601,50cyclictest0-21swapper/010:40:220
3088899601,50cyclictest0-21swapper/008:15:120
3088899601,46cyclictest27084-21tr09:30:190
3088899591,47cyclictest0-21swapper/007:40:220
3088899591,46cyclictest9142-21ssh10:25:310
192632590,1sleep20-21swapper/209:20:312
30888995824,7cyclictest8050irq/56-dwc_otg_12:00:180
3088899581,49cyclictest0-21swapper/011:03:350
3088899581,45cyclictest29954-21aten_rbpower_po11:30:220
201842580,2sleep20-21swapper/212:35:482
3088899572,44cyclictest0-21swapper/009:45:300
3088899572,27cyclictest2299-21ntpq11:35:300
3088899562,43cyclictest3700-21sh09:40:270
3088899562,38cyclictest31539-21grep09:35:230
30888995616,5cyclictest0-21swapper/008:45:190
30888995615,28cyclictest0-21swapper/008:20:320
3088899561,44cyclictest25626-21grep10:45:270
3088899557,13cyclictest0-21swapper/008:25:200
30888995528,4cyclictest0-21swapper/009:05:340
3088899551,41cyclictest0-21swapper/012:25:240
30888995412,32cyclictest0-21swapper/007:50:070
30888995412,32cyclictest0-21swapper/007:50:070
173152540,2sleep10-21swapper/109:19:051
3089199533,19cyclictest8050irq/56-dwc_otg_09:10:191
3088899532,43cyclictest0-21swapper/008:55:590
3088899532,43cyclictest0-21swapper/008:50:210
3088899532,42cyclictest0-21swapper/011:10:250
30888995318,4cyclictest526-21fschecks_count07:15:230
3088899521,43cyclictest0-21swapper/012:35:460
3088899521,42cyclictest0-21swapper/007:11:350
3088899514,32cyclictest15148-21kernelversion12:30:260
3088899512,13cyclictest0-21swapper/012:05:310
3088899511,35cyclictest2862050ssh10:10:230
3088899511,27cyclictest8263-21proc_pri07:35:290
3088899506,34cyclictest9-21ksoftirqd/010:55:290
3088899505,4cyclictest29636-21gpgv11:30:080
3088899505,4cyclictest29636-21gpgv11:30:080
3088899501,40cyclictest17287-21grep10:35:280
237112500,12sleep28050irq/56-dwc_otg_10:43:362
3089399495,10cyclictest7950irq/56-dwc_otg_08:20:163
3089199497,28cyclictest0-21swapper/108:40:191
3089199497,28cyclictest0-21swapper/108:40:191
30891994910,9cyclictest0-21swapper/112:20:181
30888994922,5cyclictest2140-21cut07:20:200
3088899491,42cyclictest0-21swapper/012:17:590
3088899491,34cyclictest0-21swapper/011:20:330
298192494,10sleep00-21swapper/007:05:300
30888994813,17cyclictest18934-21/usr/sbin/munin08:05:280
30888994811,16cyclictest16274-21latency_hist10:35:060
3089199476,24cyclictest18362-21ssh09:20:191
30891994715,10cyclictest7850irq/56-dwc_otg11:05:171
3088899472,33cyclictest0-21swapper/007:25:430
3088899472,20cyclictest0-21swapper/009:55:200
30888994717,21cyclictest6205-21ls07:30:260
3088899471,38cyclictest0-21swapper/008:30:490
3089199467,26cyclictest0-21swapper/107:10:201
3089199467,25cyclictest0-21swapper/108:15:111
3089199465,10cyclictest0-21swapper/110:25:181
3089199451,7cyclictest7950irq/56-dwc_otg_11:50:171
3088899451,33cyclictest0-21swapper/008:10:310
3089399436,23cyclictest0-21swapper/307:30:193
30891994315,10cyclictest221ktimersoftd/111:20:261
3088899431,36cyclictest0-21swapper/007:59:350
3088899431,35cyclictest0-21swapper/008:35:160
3089399427,22cyclictest0-21swapper/311:50:193
3089399426,23cyclictest0-21swapper/308:50:203
3089399426,21cyclictest0-21swapper/310:00:193
3089399426,21cyclictest0-21swapper/310:00:193
3089399421,6cyclictest12128-21timerwakeupswit07:45:343
3089399421,6cyclictest12128-21timerwakeupswit07:45:343
3089199428,24cyclictest0-21swapper/109:05:201
3089199427,23cyclictest0-21swapper/109:25:121
3089199427,18cyclictest7850irq/56-dwc_otg08:45:181
3089199424,8cyclictest7850irq/56-dwc_otg11:25:161
3089199424,8cyclictest7850irq/56-dwc_otg11:25:161
3089199424,15cyclictest24237-21ssh10:05:181
30891994219,7cyclictest727-21runrttasks10:30:221
306642420,2sleep30-21swapper/307:10:063
3089399412,5cyclictest0-21swapper/312:00:183
3089299411,12cyclictest8050irq/56-dwc_otg_09:55:182
3089199417,23cyclictest0-21swapper/107:40:181
30891994122,9cyclictest0-21swapper/111:10:231
30891994113,14cyclictest26051-21ssh12:05:071
3089399403,30cyclictest8050irq/56-dwc_otg_07:10:183
3089299407,24cyclictest0-21swapper/207:25:202
3089199406,22cyclictest0-21swapper/107:20:201
3089199402,29cyclictest12771-21ssh09:50:341
303892400,1sleep20-21swapper/210:50:412
128002400,1sleep30-21swapper/309:13:393
3089399391,31cyclictest8050irq/56-dwc_otg_09:20:163
3089199393,12cyclictest0-21swapper/112:05:341
3089299383,28cyclictest0-21swapper/207:15:172
30892993811,11cyclictest8050irq/56-dwc_otg_11:35:062
3089199389,9cyclictest7850irq/56-dwc_otg11:40:191
3089199382,29cyclictest0-21swapper/112:10:181
30891993818,5cyclictest21-21rcuc/110:37:401
64282370,1sleep20-21swapper/211:40:282
30893993719,5cyclictest18023-21latency_hist09:20:073
30893993716,5cyclictest19472-21sh10:38:243
3089399371,27cyclictest0-21swapper/312:10:203
3089299371,27cyclictest8050irq/56-dwc_otg_09:05:182
30892993712,5cyclictest14851-21/usr/sbin/munin12:30:222
30892993712,4cyclictest2232-21date08:50:072
3089199376,7cyclictest7950irq/56-dwc_otg_08:05:241
3089199376,22cyclictest24893-21ssh10:45:191
3089199374,7cyclictest30886-21cron09:35:071
3089199373,9cyclictest23-21ksoftirqd/107:50:341
3089199372,5cyclictest7675-21fschecks_time09:45:231
30891993718,4cyclictest21-21rcuc/110:15:061
30891993716,5cyclictest29881-1kworker/u9:210:21:551
30891993713,8cyclictest0-21swapper/108:35:281
3089199371,16cyclictest1589750ssh09:55:181
3089399363,3cyclictest8050irq/56-dwc_otg_10:15:163
3089299367,7cyclictest7850irq/56-dwc_otg09:00:062
30892993625,8cyclictest0-21swapper/209:45:332
3089299361,24cyclictest12000-21ls07:45:332
3089299361,24cyclictest12000-21ls07:45:332
30892993610,7cyclictest7850irq/56-dwc_otg11:20:152
3089199367,13cyclictest8050irq/56-dwc_otg_09:40:061
3089199367,13cyclictest21-21rcuc/108:25:271
30891993620,9cyclictest7770-21fschecks_count07:35:221
3089199361,21cyclictest8739-21ssh11:05:071
30891993611,5cyclictest8050irq/56-dwc_otg_08:55:061
49542350,10sleep3381ktimersoftd/311:00:203
3089399354,16cyclictest8050irq/56-dwc_otg_08:25:203
30893993527,6cyclictest381ktimersoftd/309:30:163
3089399351,7cyclictest15884-21unixbench_singl07:55:353
3089299354,12cyclictest0-21swapper/212:00:192
3089299354,11cyclictest20105-21ssh10:00:182
3089299354,11cyclictest20105-21ssh10:00:182
30892993527,6cyclictest22613-21ssh09:25:092
30892993518,3cyclictest7950irq/56-dwc_otg_09:40:242
30892993514,5cyclictest965-21/usr/sbin/munin10:15:312
3089299351,2cyclictest0-21swapper/208:10:062
3089299351,21cyclictest16861-21cut08:00:242
3089299351,20cyclictest31037-21ssh09:35:082
3089199357,9cyclictest0-21swapper/111:45:171
3089199357,13cyclictest8050irq/56-dwc_otg_12:25:201
3089199356,20cyclictest29037-21ssh10:50:191
3089199354,16cyclictest7950irq/56-dwc_otg_10:00:311
3089199354,16cyclictest7950irq/56-dwc_otg_10:00:311
3089199351,6cyclictest7850irq/56-dwc_otg08:20:191
3089199351,25cyclictest21863-21timerwakeupswit10:40:331
259102350,2sleep20-21swapper/208:25:242
157882350,6sleep23089299cyclictest10:34:142
10312351,31sleep21040-21ssh12:13:392
30893993418,10cyclictest3663-21sed09:40:263
30893993417,10cyclictest8050irq/56-dwc_otg_11:35:313
30893993413,4cyclictest9476-21idleruntime-cro11:45:063
3089399341,15cyclictest7850irq/56-dwc_otg09:10:063
30892993416,9cyclictest0-21swapper/209:38:372
3089299341,4cyclictest0-21swapper/210:35:242
3089199342,6cyclictest27468-21/usr/sbin/munin08:30:281
30891993416,9cyclictest23-21ksoftirqd/110:15:261
3089199341,5cyclictest4186-21fschecks_time07:25:241
30891993415,5cyclictest23-21ksoftirqd/108:00:211
30891993411,4cyclictest0-21swapper/109:40:191
30891993411,11cyclictest0-21swapper/111:34:381
30893993319,5cyclictest31377-21ssh09:35:213
3089399331,4cyclictest8050irq/56-dwc_otg_07:25:353
30893993312,12cyclictest0-21swapper/312:35:323
3089299339,11cyclictest0-21swapper/211:14:312
3089299336,5cyclictest7950irq/56-dwc_otg_11:05:252
3089299333,8cyclictest0-21swapper/211:25:182
3089299333,8cyclictest0-21swapper/211:25:182
3089299331,28cyclictest8050irq/56-dwc_otg_10:25:182
3089299331,23cyclictest25841-21switchtime10:45:312
3089199335,6cyclictest7850irq/56-dwc_otg11:55:321
3089199332,22cyclictest0-21swapper/107:18:521
30891993317,6cyclictest21-21rcuc/110:57:301
30891993316,5cyclictest14928-21fschecks_time12:30:231
30891993313,5cyclictest7850irq/56-dwc_otg12:15:201
3089399327,5cyclictest7950irq/56-dwc_otg_08:45:193
3089399326,12cyclictest7850irq/56-dwc_otg11:30:113
3089399326,11cyclictest0-21swapper/310:55:203
30893993213,12cyclictest727-21runrttasks08:43:583
30893993213,12cyclictest727-21runrttasks08:43:583
3089399321,16cyclictest28463-21ls08:30:333
30893993210,4cyclictest0-21swapper/312:20:253
30892993214,5cyclictest301ktimersoftd/209:53:362
3089299321,20cyclictest10869-21ssh12:25:242
3089199325,5cyclictest23-21ksoftirqd/107:30:341
3089199322,4cyclictest7950irq/56-dwc_otg_07:55:211
290612325,18sleep229075-21apt-get07:05:162
3089399317,6cyclictest8050irq/56-dwc_otg_09:55:353
3089399313,4cyclictest8050irq/56-dwc_otg_09:55:073
3089399312,22cyclictest0-21swapper/311:14:283
30893993117,11cyclictest8050irq/56-dwc_otg_10:45:233
3089399311,13cyclictest8050irq/56-dwc_otg_11:25:183
3089399311,13cyclictest8050irq/56-dwc_otg_11:25:183
3089299318,6cyclictest7850irq/56-dwc_otg08:15:072
3089299317,5cyclictest7850irq/56-dwc_otg11:35:362
30892993115,5cyclictest727-21runrttasks11:58:352
3089299311,4cyclictest8050irq/56-dwc_otg_08:20:242
30892993113,5cyclictest29063-21sh12:08:082
3089299311,28cyclictest6028-21cut07:30:232
30892993112,4cyclictest0-21swapper/210:05:312
3089299311,22cyclictest32358-21ls10:15:072
3089199312,15cyclictest7850irq/56-dwc_otg11:38:341
3089199311,17cyclictest18343-21cat11:15:331
3089399308,14cyclictest0-21swapper/310:30:003
3089399307,4cyclictest39-21ksoftirqd/310:05:313
3089399306,5cyclictest7950irq/56-dwc_otg_09:45:333
3089399304,5cyclictest526-21latency_hist10:55:073
3089399303,5cyclictest8050irq/56-dwc_otg_11:20:243
3089399302,6cyclictest8050irq/56-dwc_otg_08:35:203
3089399302,5cyclictest0-21swapper/307:15:273
3089399301,5cyclictest18117-21ssh12:34:353
30893993015,4cyclictest727-21runrttasks08:11:183
30893993012,11cyclictest11608-21sh11:07:543
30893993010,11cyclictest0-21swapper/310:23:423
3089299309,10cyclictest0-21swapper/211:00:412
3089299302,16cyclictest0-21swapper/211:15:242
30892993017,6cyclictest13923-21ssh11:50:172
30892993016,3cyclictest8050irq/56-dwc_otg_12:20:302
3089299301,24cyclictest0-21swapper/208:35:202
3089299301,17cyclictest0-21swapper/210:22:212
3089199302,5cyclictest7850irq/56-dwc_otg08:10:221
30891993013,10cyclictest4416-21grep08:55:241
3089399297,6cyclictest7850irq/56-dwc_otg11:45:203
3089399293,5cyclictest28662-21tune2fs10:10:233
3089399292,8cyclictest7850irq/56-dwc_otg07:50:323
3089399291,6cyclictest8050irq/56-dwc_otg_11:15:253
3089399291,6cyclictest8050irq/56-dwc_otg_10:40:283
30893992914,4cyclictest4344-21fschecks_count08:55:233
30893992911,11cyclictest381ktimersoftd/307:40:223
3089299294,5cyclictest8050irq/56-dwc_otg_08:40:252
3089299294,5cyclictest8050irq/56-dwc_otg_08:40:252
3089299293,4cyclictest8050irq/56-dwc_otg_09:05:072
3089299293,4cyclictest8050irq/56-dwc_otg_09:05:072
3089299292,4cyclictest0-21swapper/207:40:152
3089299291,7cyclictest7850irq/56-dwc_otg08:30:232
3089299291,7cyclictest22083-21aten_rbpower_cu08:15:192
3089199292,15cyclictest8050irq/56-dwc_otg_09:00:101
3089199292,15cyclictest8050irq/56-dwc_otg_09:00:101
3089399285,14cyclictest0-21swapper/307:40:073
3089399281,6cyclictest2687-21seq12:15:253
30893992810,3cyclictest0-21swapper/310:32:433
3089299281,7cyclictest32094-21/usr/sbin/munin07:10:352
30891992821,5cyclictest11428-21fschecks_time07:45:251
30891992821,5cyclictest11428-21fschecks_time07:45:251
303592283,16sleep10-21swapper/107:06:491
3089399271,5cyclictest18548-21df08:05:223
30893992713,10cyclictest0-21swapper/312:25:253
30893992711,9cyclictest3565-21taskset07:24:053
3089399271,10cyclictest1652750/usr/sbin/munin08:00:343
3089299271,8cyclictest0-21swapper/207:50:172
30892992718,6cyclictest16474-21latency_hist08:00:072
3089399261,4cyclictest8050irq/56-dwc_otg_09:00:213
3089399261,4cyclictest8050irq/56-dwc_otg_09:00:213
3089299264,4cyclictest8075-21missed_timers07:35:262
3089299262,17cyclictest0-21swapper/208:54:252
3089299261,6cyclictest2095-21/usr/sbin/munin07:20:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional