You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 22:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue May 26, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1660121370,2sleep1319999cyclictest09:50:261
494021040,6sleep1319999cyclictest09:36:131
3197991017,75cyclictest0-21swapper/011:00:180
119122930,2sleep00-21swapper/010:24:350
319799917,16cyclictest0-21swapper/009:35:100
3197999031,41cyclictest7950irq/56-dwc_otg_12:10:340
3197998610,65cyclictest32532-21fschecks_time08:30:200
319799841,54cyclictest0-21swapper/011:35:170
3197998334,14cyclictest0-21swapper/009:00:160
3197998221,3cyclictest0-21swapper/009:30:320
3197997924,40cyclictest0-21swapper/010:50:290
319799791,51cyclictest0-21swapper/010:35:160
283732790,4sleep0101ktimersoftd/008:20:030
3197997720,45cyclictest0-21swapper/012:25:120
3197997618,11cyclictest0-21swapper/009:16:460
96172750,1sleep00-21swapper/008:55:260
96172750,1sleep00-21swapper/008:55:260
3197997518,9cyclictest1133-21runrttasks08:35:170
71982740,1sleep30-21swapper/310:18:343
319799744,22cyclictest0-21swapper/011:45:160
243852740,5sleep2320099cyclictest11:56:442
226762740,3sleep322669-21ssh11:55:203
319799721,26cyclictest8229-21grep10:20:050
319799721,19cyclictest20266-21/usr/sbin/munin09:55:200
319799713,12cyclictest0-21swapper/008:25:160
3197997119,5cyclictest18200-21aten_rbpower_cu11:50:180
3197997019,11cyclictest0-21swapper/007:50:140
3197997015,5cyclictest11948-21chrt07:34:230
3197996919,10cyclictest0-21swapper/012:22:110
319799691,24cyclictest20417-21cat07:55:300
31772690,2sleep10-21swapper/112:10:281
319799681,19cyclictest8350-21swap09:40:290
319799681,14cyclictest0-21swapper/011:07:200
153522680,4sleep20-21swapper/209:10:292
319799672,4cyclictest0-21swapper/011:30:040
319799672,4cyclictest0-21swapper/011:30:040
3197996718,11cyclictest0-21swapper/008:06:120
319799671,25cyclictest0-21swapper/010:33:500
3197996617,10cyclictest0-21swapper/008:22:190
319799661,15cyclictest0-21swapper/010:25:120
319799661,15cyclictest0-21swapper/010:25:120
10122660,2sleep30-21swapper/308:31:213
319799656,15cyclictest0-21swapper/011:40:560
319799652,40cyclictest20341-21grep12:30:330
3197996516,8cyclictest0-21swapper/010:11:510
3197996514,11cyclictest0-21swapper/011:59:020
319799651,19cyclictest0-21swapper/009:10:200
319799651,18cyclictest0-21swapper/011:32:590
319799651,13cyclictest0-21swapper/008:10:250
319799651,13cyclictest0-21swapper/008:10:250
3201996413,4cyclictest8050irq/56-dwc_otg_10:45:163
3197996416,4cyclictest18485-21timerandwakeup11:10:310
3197996319,25cyclictest0-21swapper/010:05:180
319799631,4cyclictest31112-21ssh09:30:030
3197996314,11cyclictest0-21swapper/010:45:180
319799622,7cyclictest27230-21latency_hist12:40:050
319799622,19cyclictest0-21swapper/007:45:140
3197996216,4cyclictest0-21swapper/008:03:310
3197996214,3cyclictest0-21swapper/009:45:260
319799621,16cyclictest0-21swapper/011:18:420
319799621,15cyclictest0-21swapper/009:05:160
3197996210,13cyclictest0-21swapper/007:13:320
319799619,17cyclictest0-21swapper/009:52:380
319799614,42cyclictest8962-21ssh11:00:050
319799614,42cyclictest8962-21ssh11:00:050
319799612,19cyclictest0-21swapper/010:44:560
319799611,5cyclictest0-21swapper/012:16:160
319799611,33cyclictest31023-21fschecks_time12:05:220
3197995922,6cyclictest7850irq/56-dwc_otg09:25:040
3197995918,5cyclictest7850irq/56-dwc_otg12:04:490
319799591,17cyclictest0-21swapper/011:24:220
3197995818,4cyclictest0-21swapper/010:03:360
37592570,2sleep20-21swapper/210:14:552
319999577,10cyclictest23-21ksoftirqd/109:55:171
319999561,12cyclictest7850irq/56-dwc_otg12:35:151
3197995626,6cyclictest7865-21proc_pri08:50:280
319799561,21cyclictest0-21swapper/007:37:000
3197995522,21cyclictest0-21swapper/008:45:040
60882540,13sleep123-21ksoftirqd/111:35:211
3197995424,20cyclictest7850irq/56-dwc_otg07:25:170
274202540,2sleep20-21swapper/210:04:042
3197995315,20cyclictest6320-21/usr/sbin/munin08:45:310
320099518,28cyclictest14911-1kworker/u9:011:10:312
3197995131,4cyclictest7850irq/56-dwc_otg07:20:150
319999498,27cyclictest0-21swapper/108:35:181
319999493,7cyclictest7950irq/56-dwc_otg_08:20:151
320099484,5cyclictest7850irq/56-dwc_otg09:35:152
320099482,3cyclictest0-21swapper/211:20:162
319999488,26cyclictest22343-21ssh11:55:111
319999484,5cyclictest7950irq/56-dwc_otg_12:15:161
320199471,6cyclictest8050irq/56-dwc_otg_08:50:153
319799471,14cyclictest0-21swapper/007:15:310
3200994624,5cyclictest13265-21apt-get11:05:182
319999456,25cyclictest0-21swapper/108:00:091
3199994510,13cyclictest0-21swapper/110:45:241
3197994522,10cyclictest0-21swapper/007:41:450
3197994522,10cyclictest0-21swapper/007:41:450
320099444,5cyclictest8050irq/56-dwc_otg_08:35:182
319999446,23cyclictest0-21swapper/108:05:201
3199994424,5cyclictest1133-21runrttasks10:40:181
159902440,29sleep10-21swapper/111:46:451
320199431,4cyclictest8050irq/56-dwc_otg_08:10:163
320199431,4cyclictest8050irq/56-dwc_otg_08:10:163
3199994318,13cyclictest25004-21cut08:10:191
3199994318,13cyclictest25004-21cut08:10:191
320199427,24cyclictest0-21swapper/309:35:173
3201994222,6cyclictest22677-21idleruntime-cro09:20:043
320099427,15cyclictest0-21swapper/208:00:342
319999427,26cyclictest0-21swapper/109:25:171
275024223,12sleep00-21swapper/007:07:460
320199417,22cyclictest0-21swapper/308:00:093
320199414,22cyclictest25720-21ssh11:20:163
320099411,14cyclictest0-21swapper/212:10:322
319999419,9cyclictest22244-21apt-config11:55:051
319999418,5cyclictest8050irq/56-dwc_otg_12:05:181
319999412,18cyclictest8050irq/56-dwc_otg_10:20:111
3199994114,10cyclictest0-21swapper/111:15:181
320199406,16cyclictest8050irq/56-dwc_otg_11:40:193
3201994018,9cyclictest21916-21sh11:54:593
320199401,19cyclictest21339-21runrttasks09:55:343
320099408,24cyclictest0-21swapper/207:15:162
320099405,26cyclictest8050irq/56-dwc_otg_12:20:152
320099403,11cyclictest19091-21sh12:30:152
3200994010,5cyclictest0-21swapper/210:20:212
3199994018,10cyclictest7463-21aten_rbpower_en09:40:171
137932400,2sleep20-21swapper/209:47:092
114042400,1sleep20-21swapper/209:00:272
320199394,8cyclictest7950irq/56-dwc_otg_08:35:173
3201993928,8cyclictest8050irq/56-dwc_otg_12:00:153
320099396,17cyclictest8050irq/56-dwc_otg_09:55:052
320199387,29cyclictest0-21swapper/307:15:163
320099385,20cyclictest9901-21ssh11:40:182
3200993818,11cyclictest24596-21sh12:35:372
3200993814,4cyclictest26065-21sh09:23:332
3200993813,13cyclictest0-21swapper/211:25:172
3200993813,13cyclictest0-21swapper/211:25:172
319999381,3cyclictest7850irq/56-dwc_otg07:30:171
319999381,10cyclictest22823-21sh09:20:051
3201993716,15cyclictest39-21ksoftirqd/310:10:143
3201993710,17cyclictest0-21swapper/310:41:493
3200993717,5cyclictest19786-21sh11:51:072
3200993717,4cyclictest28307-21/usr/sbin/munin10:05:272
320099371,5cyclictest8171-21ntpq09:40:262
3200993712,5cyclictest11033-21wc07:30:262
319999377,5cyclictest5363-21idleruntime10:55:201
319999377,5cyclictest5363-21idleruntime10:55:201
319999373,24cyclictest0-21swapper/110:10:171
319999371,24cyclictest10290-21ls07:30:041
183312370,2sleep20-21swapper/212:29:292
320199362,6cyclictest8050irq/56-dwc_otg_10:25:203
320199362,6cyclictest8050irq/56-dwc_otg_10:25:203
320199361,23cyclictest8980-21ls11:00:053
320199361,23cyclictest8980-21ls11:00:053
320199361,17cyclictest2612-21ls11:30:313
320099366,12cyclictest7950irq/56-dwc_otg_08:30:042
320099362,5cyclictest8050irq/56-dwc_otg_07:45:172
3200993619,4cyclictest30389-21sh12:05:042
319999369,16cyclictest0-21swapper/111:06:421
319999364,20cyclictest19027-21ssh12:30:111
319999363,9cyclictest0-21swapper/107:40:151
319999363,9cyclictest0-21swapper/107:40:151
319999362,14cyclictest18644-21unixbench_multi11:10:321
3199993617,6cyclictest20844-21aten_rbpower_cu10:35:191
3199993616,4cyclictest1133-21runrttasks10:50:531
3199993610,5cyclictest141-21systemd-journal11:05:031
161142360,1sleep30-21swapper/309:50:203
320099356,6cyclictest7850irq/56-dwc_otg12:15:272
320099354,14cyclictest8050irq/56-dwc_otg_10:55:162
320099354,14cyclictest8050irq/56-dwc_otg_10:55:162
320099353,6cyclictest0-21swapper/211:00:342
320099352,15cyclictest0-21swapper/211:15:182
3200993517,7cyclictest29-21rcuc/209:30:152
320099351,17cyclictest0-21swapper/210:33:372
3200993511,5cyclictest18932-21cat09:15:192
3200993511,4cyclictest3816-21fschecks_time08:40:212
319999359,6cyclictest2684950wget08:15:171
319999357,5cyclictest8050irq/56-dwc_otg_10:05:221
319999357,16cyclictest0-21swapper/108:50:171
319999357,14cyclictest8050irq/56-dwc_otg_07:45:331
319999355,5cyclictest8050irq/56-dwc_otg_11:25:231
319999355,5cyclictest8050irq/56-dwc_otg_11:25:231
319999353,4cyclictest7950irq/56-dwc_otg_09:10:321
319999352,7cyclictest5069-21apt08:45:041
319999352,7cyclictest26948-21sh09:25:041
3199993516,9cyclictest14097-21diskmemload10:25:251
3199993516,9cyclictest14097-21diskmemload10:25:251
3199993512,7cyclictest0-21swapper/108:25:321
170622351,2sleep30-21swapper/311:10:043
115572350,2sleep10-21swapper/109:00:301
320199346,4cyclictest7950irq/56-dwc_otg_12:25:273
320199346,16cyclictest0-21swapper/312:30:103
320199343,4cyclictest8050irq/56-dwc_otg_11:50:043
320199341,17cyclictest0-21swapper/312:08:493
3201993411,11cyclictest1117-21kworker/3:312:10:193
320099346,6cyclictest7950irq/56-dwc_otg_07:35:302
320099346,23cyclictest8050irq/56-dwc_otg_07:50:142
320099345,13cyclictest8050irq/56-dwc_otg_08:20:242
3200993412,6cyclictest27208-1kworker/u9:009:55:212
3200993410,17cyclictest0-21swapper/211:35:342
3199993418,8cyclictest0-21swapper/111:40:181
3199993417,11cyclictest23-21ksoftirqd/111:20:151
319999341,14cyclictest3980-21wc07:10:281
319999341,13cyclictest0-21swapper/109:34:331
320199339,13cyclictest0-21swapper/311:29:273
320199339,13cyclictest0-21swapper/311:29:273
320199332,5cyclictest39-21ksoftirqd/309:30:153
320199332,16cyclictest8050irq/56-dwc_otg_09:21:253
320099338,3cyclictest0-21swapper/209:10:042
320099338,10cyclictest8050irq/56-dwc_otg_08:05:242
320099337,10cyclictest8050irq/56-dwc_otg_10:25:202
320099337,10cyclictest8050irq/56-dwc_otg_10:25:202
320099336,13cyclictest0-21swapper/210:40:242
320099333,5cyclictest798-21apt-get10:50:152
320099331,7cyclictest0-21swapper/211:30:312
3200993310,4cyclictest0-21swapper/210:36:512
319999339,4cyclictest0-21swapper/108:55:131
319999339,4cyclictest0-21swapper/108:55:131
319999338,5cyclictest7850irq/56-dwc_otg10:30:201
319999335,5cyclictest11609-21ssh09:45:161
319999334,5cyclictest7850irq/56-dwc_otg10:20:041
319999333,6cyclictest8050irq/56-dwc_otg_12:00:181
319999333,5cyclictest16080-21unixbench_multi12:25:301
320199324,16cyclictest8050irq/56-dwc_otg_08:40:233
320199324,13cyclictest8050irq/56-dwc_otg_08:25:143
320199322,5cyclictest7950irq/56-dwc_otg_11:35:353
320199321,6cyclictest14083-21kworker/u8:412:35:333
320199321,24cyclictest0-21swapper/309:25:263
320199321,23cyclictest7709-21uniq09:40:203
320199321,16cyclictest26846-21rm10:03:013
3201993210,15cyclictest0-21swapper/310:50:313
320099325,5cyclictest7950irq/56-dwc_otg_11:45:222
320099325,11cyclictest8050irq/56-dwc_otg_08:00:042
320099323,6cyclictest7850irq/56-dwc_otg10:18:552
320099322,9cyclictest7950irq/56-dwc_otg_08:15:032
320099322,9cyclictest7950irq/56-dwc_otg_08:15:032
3200993214,4cyclictest30188-21timerandwakeup10:45:312
319999329,13cyclictest0-21swapper/107:15:141
319999325,14cyclictest8050irq/56-dwc_otg_07:35:321
319999323,6cyclictest7850irq/56-dwc_otg09:05:161
319999322,10cyclictest0-21swapper/111:30:311
319999321,9cyclictest17776-21aten_rbpower_vo07:50:171
232692320,6sleep3320199cyclictest11:16:153
320199316,5cyclictest14723-21tr07:40:253
320199316,5cyclictest14723-21tr07:40:253
320199316,5cyclictest0-21swapper/307:55:263
320199313,4cyclictest8050irq/56-dwc_otg_10:30:303
320199313,4cyclictest7950irq/56-dwc_otg_12:15:153
3201993122,6cyclictest11316-21ssh12:20:253
320199311,5cyclictest32121-21cron10:10:033
3201993115,9cyclictest29164-21load08:20:223
320099316,12cyclictest0-21swapper/212:05:522
320099312,6cyclictest7950irq/56-dwc_otg_07:20:182
320099311,6cyclictest14722-21grep07:40:252
320099311,6cyclictest14722-21grep07:40:252
320099311,4cyclictest0-21swapper/208:45:182
3200993112,9cyclictest8066-21cat08:50:312
319999319,7cyclictest9450irq/48-DMA10:20:151
319999316,16cyclictest0-21swapper/112:20:181
319999314,4cyclictest8050irq/56-dwc_otg_10:00:231
319999312,7cyclictest32649-21if_wlan008:30:221
320199307,5cyclictest39-21ksoftirqd/309:05:263
320199307,15cyclictest0-21swapper/309:46:283
320199306,10cyclictest8050irq/56-dwc_otg_10:35:303
3201993023,4cyclictest21357-21latency_hist11:15:043
320199302,10cyclictest8050irq/56-dwc_otg_07:50:293
320199301,6cyclictest141-21systemd-journal09:10:293
320199301,5cyclictest11885-21grep09:00:333
320199301,4cyclictest7950irq/56-dwc_otg_11:00:193
3201993014,8cyclictest9782-1kworker/u9:010:21:273
320199301,19cyclictest23381-21cpuspeed08:05:213
320099302,11cyclictest0-21swapper/209:25:172
319999306,14cyclictest0-21swapper/107:20:181
319999304,4cyclictest8050irq/56-dwc_otg_08:45:281
320199295,6cyclictest8050irq/56-dwc_otg_07:20:183
3201992912,10cyclictest10978-21latency07:30:253
320099292,5cyclictest31-21ksoftirqd/208:30:232
19302295,11sleep30-21swapper/307:05:263
320199285,7cyclictest8050irq/56-dwc_otg_08:45:183
320199284,14cyclictest8050irq/56-dwc_otg_07:10:293
3201992811,11cyclictest283222chrt08:19:383
320099285,5cyclictest7950irq/56-dwc_otg_07:15:042
320099283,10cyclictest7950irq/56-dwc_otg_08:18:492
105052280,3sleep3381ktimersoftd/309:00:033
105052280,3sleep3381ktimersoftd/309:00:033
320199279,9cyclictest17254-21taskset07:48:073
320199271,3cyclictest8050irq/56-dwc_otg_07:25:153
3201992710,9cyclictest381ktimersoftd/307:35:273
320099277,3cyclictest0-21swapper/208:57:342
320099277,3cyclictest0-21swapper/208:57:342
320099275,16cyclictest0-21swapper/207:25:332
26392265,14sleep10-21swapper/107:06:251
20802265,10sleep20-21swapper/207:05:292
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional