You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 19:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Fri Mar 27, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16798999318,65cyclictest25618-21sed09:00:380
16798999220,51cyclictest24678-21wget12:20:240
16798998621,56cyclictest18167-21fschecks_count10:15:280
16798998522,48cyclictest2541-21unixbench_singl09:55:390
16798998521,46cyclictest9096-21cut09:25:270
16798998321,49cyclictest8643-21missed_timers08:15:310
16798998321,49cyclictest8643-21missed_timers08:15:310
16798998315,35cyclictest0-21swapper/011:00:170
16798998215,33cyclictest61-21kswapd010:55:240
16798998118,51cyclictest29526-21/usr/sbin/munin09:50:350
16798998118,51cyclictest29526-21/usr/sbin/munin09:50:350
16798998118,48cyclictest7191-21/usr/sbin/munin11:20:340
16798997920,39cyclictest29224-21latency_hist09:50:100
16798997824,42cyclictest4416-21chrt08:05:160
16798997714,34cyclictest28250-21cut09:10:230
16798997518,35cyclictest13387-21idleruntime-cro10:10:100
69192740,8sleep2301ktimersoftd/208:10:332
16798997423,11cyclictest8676-21ssh10:04:110
16798997416,31cyclictest0-21swapper/012:00:230
16798997415,41cyclictest0-21swapper/010:29:420
16798997321,11cyclictest12133-21apt-get12:05:220
16798997317,37cyclictest0-21swapper/010:35:260
16798997316,33cyclictest23782-21ssh11:40:180
16798997224,34cyclictest16772-21rm10:52:210
16798997218,30cyclictest22636-21/usr/sbin/munin08:55:410
16798997217,43cyclictest31790-21rm12:28:450
16798997217,29cyclictest19805-21seq12:14:390
16798997216,35cyclictest18329-21ssh09:35:380
16798997216,35cyclictest0-21swapper/010:41:000
16798997214,37cyclictest2248-21ssh09:17:040
16798997124,43cyclictest0-21swapper/011:15:130
16798997118,11cyclictest816-21ssh11:11:570
16798997118,11cyclictest816-21ssh11:11:570
16798997117,11cyclictest6924-21ntpq08:10:330
115412710,2sleep30-21swapper/310:06:523
16798997021,36cyclictest1616-21taskset11:51:450
16798997019,36cyclictest6646-21seq09:22:160
16798997015,35cyclictest6770-21ssh12:37:100
16798996818,33cyclictest23639-21ssh10:21:090
1679899673,14cyclictest13726-21ssh10:10:230
16798996722,42cyclictest21080-21fschecks_count08:50:260
16798996722,42cyclictest21080-21fschecks_count08:50:260
16798996714,37cyclictest0-21swapper/009:44:400
16798996713,34cyclictest0-21swapper/010:46:380
16798996619,12cyclictest15741-21df11:30:240
16798996619,12cyclictest15741-21df11:30:240
16798996616,33cyclictest0-21swapper/012:31:010
16798996616,33cyclictest0-21swapper/012:31:010
16798996614,28cyclictest4609-21ntpq11:55:340
16798996514,32cyclictest0-21swapper/007:20:360
16798996425,11cyclictest22780-21ssh12:17:290
16798996417,39cyclictest0-21swapper/011:29:010
16798996417,30cyclictest21988-21ssh11:37:280
16798996416,35cyclictest0-21swapper/007:41:550
16798996416,35cyclictest0-21swapper/007:41:550
16798996415,30cyclictest29518-21ssh11:46:020
187902630,3sleep337-21rcuc/309:36:203
16798996317,15cyclictest0-21swapper/007:50:230
16798996313,35cyclictest0-21swapper/011:06:160
16798996312,38cyclictest0-21swapper/007:26:150
16798996310,14cyclictest0-21swapper/008:00:190
16798996216,39cyclictest0-21swapper/007:33:550
16798996216,31cyclictest27476-21taskset09:06:020
163342625,47sleep016335-21sshd07:07:430
121302620,2sleep012129-21kernelversion08:25:290
16798996117,27cyclictest19981-21chrt07:16:250
16798996113,34cyclictest0-21swapper/010:32:330
16800995916,16cyclictest0-21swapper/209:10:202
16798995917,33cyclictest16560-21chrt08:36:480
16798995915,33cyclictest0-21swapper/007:39:000
16798995915,32cyclictest11270-21chrt08:23:230
16798995913,35cyclictest0-21swapper/007:46:420
16798995910,41cyclictest0-21swapper/009:30:560
16798995835,5cyclictest0-21swapper/008:30:230
16798995814,30cyclictest0-21swapper/007:57:080
16798995813,34cyclictest0-21swapper/007:13:340
1679999577,31cyclictest0-21swapper/111:30:231
1679999577,31cyclictest0-21swapper/111:30:231
1679999577,31cyclictest0-21swapper/108:45:151
16798995615,30cyclictest0-21swapper/008:47:210
1680099557,22cyclictest0-21swapper/208:05:152
1679999556,30cyclictest0-21swapper/108:25:221
16798995521,31cyclictest18672-21chrt08:45:050
1680199545,18cyclictest7950irq/56-dwc_otg_07:40:243
1680199545,18cyclictest7950irq/56-dwc_otg_07:40:243
1679999537,30cyclictest0-21swapper/109:40:231
1680199527,27cyclictest0-21swapper/308:15:233
1680199527,27cyclictest0-21swapper/308:15:233
1680099522,8cyclictest14518-21ssh10:50:222
1680199517,26cyclictest0-21swapper/309:55:233
16800995110,8cyclictest0-21swapper/209:50:212
16800995110,8cyclictest0-21swapper/209:50:212
1679999507,6cyclictest0-21swapper/110:10:241
1680199497,26cyclictest0-21swapper/307:15:163
1679999496,9cyclictest20439-21ssh12:15:221
16799994915,24cyclictest0-21swapper/108:38:281
1680199488,25cyclictest0-21swapper/312:10:233
1680199487,5cyclictest7950irq/56-dwc_otg_10:10:243
1680199486,28cyclictest0-21swapper/307:50:243
1679999484,26cyclictest0-21swapper/107:35:211
1680199477,25cyclictest917-21snmpd11:50:233
1680199467,23cyclictest0-21swapper/308:25:223
1679999467,25cyclictest0-21swapper/112:25:241
1679999467,23cyclictest0-21swapper/109:00:241
1679999464,23cyclictest21-21rcuc/112:00:461
1679999463,8cyclictest0-21swapper/111:05:231
1680199458,26cyclictest39-21ksoftirqd/309:44:413
1680199458,25cyclictest0-21swapper/309:25:373
1679999457,26cyclictest27835-21diskmemload10:40:221
1679999456,25cyclictest0-21swapper/110:25:231
1679999456,24cyclictest0-21swapper/107:45:231
1679999453,12cyclictest0-21swapper/108:20:211
1680199447,24cyclictest0-21swapper/307:30:233
1680199441,7cyclictest7950irq/56-dwc_otg_10:45:213
16801994414,18cyclictest6350-21cpu10:40:233
1680099446,24cyclictest27835-21diskmemload12:30:252
1680099446,24cyclictest27835-21diskmemload12:30:252
1679999446,24cyclictest0-21swapper/107:50:241
1679999443,9cyclictest30364-21ssh10:30:241
84202430,1sleep30-21swapper/312:00:293
78362430,2sleep20-21swapper/210:02:402
1680199437,22cyclictest0-21swapper/308:45:153
1680199435,24cyclictest15527-21ssh11:30:233
1680199435,24cyclictest15527-21ssh11:30:233
1680199431,4cyclictest0-21swapper/309:30:233
1680099431,18cyclictest0-21swapper/208:35:242
1679999436,23cyclictest0-21swapper/110:00:231
1679999435,6cyclictest21-21rcuc/111:00:381
1679999432,26cyclictest0-21swapper/112:20:211
16799994310,5cyclictest23-21ksoftirqd/111:15:131
1680199427,8cyclictest39-21ksoftirqd/310:15:293
1680199427,22cyclictest0-21swapper/308:05:163
1680199427,10cyclictest39-21ksoftirqd/312:28:473
1680199426,9cyclictest39-21ksoftirqd/311:26:323
16801994227,10cyclictest7950irq/56-dwc_otg_11:05:233
16801994224,3cyclictest0-21swapper/310:30:233
1679999427,23cyclictest0-21swapper/107:15:161
1679999424,7cyclictest17523-21unixbench_multi12:10:381
1679999423,21cyclictest7950irq/56-dwc_otg_09:35:531
16799994214,22cyclictest7850irq/56-dwc_otg_10:50:241
214432410,2sleep216797-21cyclictest11:36:282
1679999416,22cyclictest0-21swapper/108:15:231
1679999416,22cyclictest0-21swapper/108:15:231
1679999412,10cyclictest0-21swapper/107:10:231
16799994117,9cyclictest221ktimersoftd/112:06:121
16799994117,4cyclictest18436-21kernelversion10:15:311
212402402,7sleep30-21swapper/312:15:363
1680199405,9cyclictest37-21rcuc/310:50:303
1680099407,14cyclictest7950irq/56-dwc_otg_09:40:382
1679999409,5cyclictest21-21rcuc/111:40:371
1679999407,15cyclictest0-21swapper/109:45:241
1679999407,11cyclictest0-21swapper/108:30:231
1679999405,6cyclictest21-21rcuc/111:20:321
1679999404,24cyclictest7750irq/56-dwc_otg10:20:391
1679999403,8cyclictest23-21ksoftirqd/109:50:341
1679999403,8cyclictest23-21ksoftirqd/109:50:341
1679999402,4cyclictest0-21swapper/109:25:381
1679999402,4cyclictest0-21swapper/109:10:101
16799994017,11cyclictest7850irq/56-dwc_otg_08:40:271
1679999401,5cyclictest0-21swapper/109:30:231
16799994013,7cyclictest221ktimersoftd/108:50:241
16799994013,7cyclictest221ktimersoftd/108:50:241
121162400,3sleep112114-21ssh11:25:321
48042391,34sleep24816-21ssh09:59:432
1680199398,15cyclictest39-21ksoftirqd/311:15:103
1680199398,15cyclictest39-21ksoftirqd/311:15:103
1680199394,21cyclictest381ktimersoftd/308:10:323
16801993911,6cyclictest0-21swapper/309:15:103
16800993921,4cyclictest25732-21idleruntime-cro07:35:102
1679999397,17cyclictest221ktimersoftd/111:49:181
1679999395,17cyclictest0-21swapper/108:00:191
1679999394,14cyclictest7950irq/56-dwc_otg_07:45:111
1679999394,14cyclictest7950irq/56-dwc_otg_07:45:111
1679999393,8cyclictest31225-21ssh11:10:211
1679999393,8cyclictest31225-21ssh11:10:211
16799993911,7cyclictest221ktimersoftd/111:40:111
16799993910,9cyclictest23-21ksoftirqd/108:10:111
16799993910,15cyclictest221ktimersoftd/110:55:281
1680199387,4cyclictest7750irq/56-dwc_otg11:20:283
1680199387,28cyclictest0-21swapper/308:55:243
1680199384,14cyclictest37-21rcuc/310:35:213
1680199383,24cyclictest37-21rcuc/307:36:483
1680199383,12cyclictest381ktimersoftd/310:30:113
1680099386,23cyclictest0-21swapper/209:25:232
1680099386,21cyclictest4391-21ssh09:20:222
1680099383,5cyclictest0-21swapper/208:50:212
1680099383,5cyclictest0-21swapper/208:50:212
16800993831,5cyclictest32672-21aten_rbpower_vo09:15:242
1680099382,22cyclictest31-21ksoftirqd/212:00:212
16800993821,5cyclictest28048-21sh10:26:372
1679999387,6cyclictest23-21ksoftirqd/110:35:231
1679999385,2cyclictest0-21swapper/107:30:211
1679999385,19cyclictest21-21rcuc/110:10:111
1680199374,20cyclictest37-21rcuc/309:17:273
1680199371,24cyclictest29021-21ls11:45:373
1680099377,25cyclictest0-21swapper/208:40:232
1680099373,21cyclictest0-21swapper/211:45:092
1680099371,24cyclictest13573-21ls10:10:112
1679999377,23cyclictest0-21swapper/108:55:241
1679999373,25cyclictest221ktimersoftd/109:15:571
1680199368,9cyclictest39-21ksoftirqd/312:20:203
1680199366,16cyclictest0-21swapper/309:50:233
1680199366,16cyclictest0-21swapper/309:50:233
1680199361,5cyclictest24612-21ntpq11:40:333
1680199361,23cyclictest9374-21ls10:05:113
16800993627,6cyclictest301ktimersoftd/209:05:212
16800993617,4cyclictest2663-21apt-key11:15:112
16800993617,4cyclictest2663-21apt-key11:15:112
1679999369,12cyclictest7850irq/56-dwc_otg_09:23:001
1679999368,23cyclictest0-21swapper/107:55:231
1679999367,16cyclictest7950irq/56-dwc_otg_09:55:381
1679999364,8cyclictest0-21swapper/112:35:231
1679999363,6cyclictest32146-21ssh11:50:231
16799993617,5cyclictest21-21rcuc/110:46:341
265632350,6sleep31680199cyclictest09:45:383
1680199357,17cyclictest39-21ksoftirqd/309:20:123
1680199356,9cyclictest39-21ksoftirqd/308:50:233
1680199356,9cyclictest39-21ksoftirqd/308:50:233
1680199356,14cyclictest0-21swapper/307:45:233
1680199354,16cyclictest7850irq/56-dwc_otg_08:00:033
1680199353,13cyclictest7850irq/56-dwc_otg_11:57:223
1680199352,6cyclictest7750irq/56-dwc_otg11:00:433
16800993520,5cyclictest7750irq/56-dwc_otg11:33:192
16800993520,5cyclictest7750irq/56-dwc_otg11:33:192
16800993515,5cyclictest18693-21sh09:36:102
16800993514,4cyclictest25636-21timerandwakeup12:20:372
1680099351,26cyclictest28506-21ssh11:45:302
1680099351,24cyclictest26086-21ls09:05:102
1680099351,23cyclictest31723-21ls07:50:292
1679999355,4cyclictest7750irq/56-dwc_otg08:10:311
1680199347,24cyclictest0-21swapper/308:00:223
1680199347,17cyclictest39-21ksoftirqd/310:55:263
1680199343,22cyclictest11-21rcu_preempt09:08:513
16801993411,8cyclictest783-21ssh12:30:293
16801993411,8cyclictest783-21ssh12:30:293
1680099346,5cyclictest7850irq/56-dwc_otg_08:45:352
16800993427,4cyclictest30900-21idleruntime-cro11:10:102
16800993420,4cyclictest31-21ksoftirqd/210:35:242
16800993417,4cyclictest908-21runrttasks10:24:522
16800993415,6cyclictest7850irq/56-dwc_otg_11:00:262
16800993413,6cyclictest908-21runrttasks11:30:112
16800993412,4cyclictest0-21swapper/210:15:412
1680099341,20cyclictest0-21swapper/207:12:432
16800993411,10cyclictest27835-21diskmemload10:49:322
16800993410,5cyclictest0-21swapper/212:15:352
1679999348,9cyclictest7950irq/56-dwc_otg_12:30:271
1679999348,9cyclictest7950irq/56-dwc_otg_12:30:271
1679999345,5cyclictest21-21rcuc/109:11:531
16799993411,10cyclictest1665-1kworker/u9:211:55:341
1680199337,14cyclictest0-21swapper/309:00:243
1680199337,10cyclictest7950irq/56-dwc_otg_10:20:263
1680199336,17cyclictest0-21swapper/312:05:243
1680199331,21cyclictest39-21ksoftirqd/312:37:383
16800993327,4cyclictest27849-21aten_rbpower_po07:40:252
16800993327,4cyclictest27849-21aten_rbpower_po07:40:252
1680099332,12cyclictest7950irq/56-dwc_otg_07:30:102
16800993317,5cyclictest908-21runrttasks12:14:232
16800993314,7cyclictest15005-21unixbench_singl10:10:392
16800993311,5cyclictest6664-21if_eth010:40:272
1679999338,5cyclictest7850irq/56-dwc_otg_07:24:021
1680199324,8cyclictest37-21rcuc/308:31:453
16801993223,6cyclictest381ktimersoftd/308:38:283
1680199321,5cyclictest37-21rcuc/311:37:503
1680199321,12cyclictest381ktimersoftd/311:15:123
1680099328,4cyclictest0-21swapper/210:30:312
1680099327,12cyclictest7850irq/56-dwc_otg_09:45:332
1680099325,18cyclictest12695-21ls12:05:302
1680099325,13cyclictest7950irq/56-dwc_otg_07:15:242
16800993217,8cyclictest9852-21/usr/sbin/munin08:20:312
16800993211,7cyclictest13935-21switchtime09:30:392
1679999329,4cyclictest0-21swapper/107:25:111
16801993111,9cyclictest185032chrt08:42:543
1680099318,5cyclictest7750irq/56-dwc_otg11:15:342
1680099317,14cyclictest0-21swapper/212:35:232
1680099315,22cyclictest7950irq/56-dwc_otg_11:20:212
16800993110,10cyclictest0-21swapper/211:55:412
154352316,11sleep20-21swapper/207:05:312
1680099309,4cyclictest0-21swapper/210:55:142
1680099304,12cyclictest7950irq/56-dwc_otg_12:25:302
16801992913,7cyclictest381ktimersoftd/307:10:193
16801992912,9cyclictest238092chrt07:29:043
1680099296,16cyclictest0-21swapper/211:52:052
1680099296,11cyclictest7950irq/56-dwc_otg_07:45:282
1680099291,5cyclictest23729-21grep08:55:392
1680099291,21cyclictest3053-21latency08:00:282
1680099284,10cyclictest7850irq/56-dwc_otg_07:23:492
162582285,15sleep1505-21rs:main1
149652284,12sleep30-21swapper/307:05:163
1680199273,9cyclictest7950irq/56-dwc_otg_07:22:113
1680199271,6cyclictest0-21swapper/308:20:393
1680099274,13cyclictest0-21swapper/207:35:332
1680099271,5cyclictest7850irq/56-dwc_otg_08:30:402
16800992619,5cyclictest957-21fschecks_count07:55:272
1680099261,15cyclictest0-21swapper/208:15:302
1680099261,15cyclictest0-21swapper/208:15:302
1680099258,14cyclictest0-21swapper/208:25:272
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional