You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 23:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Fri Dec 26, 2025 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1374521010,5sleep10-21swapper/107:10:111
1377099907,15cyclictest0-21swapper/011:10:210
13770998214,55cyclictest15431-21ls10:10:360
1377099796,53cyclictest0-21swapper/007:40:210
1377099788,49cyclictest5422-21cut08:15:270
1377099778,57cyclictest28701-21latency_hist09:20:090
1377099767,41cyclictest0-21swapper/009:25:080
13770997614,38cyclictest7292-21fschecks_time08:20:260
13770997611,45cyclictest0-21swapper/010:35:080
136182767,54sleep00-21swapper/007:10:110
13770997510,61cyclictest26390-21grep10:40:380
1377099749,50cyclictest0-21swapper/010:35:300
1377099749,40cyclictest32580-21missed_timers08:00:300
13770997413,36cyclictest1454-21apt-get08:05:190
13770997412,38cyclictest9870-21sed08:25:370
1377099729,40cyclictest1742-21grep11:05:090
13770997214,42cyclictest28176-21latency_hist07:50:090
13770997116,38cyclictest0-21swapper/009:30:260
13770997012,12cyclictest29956-21apt07:55:080
1377699686,7cyclictest7750irq/56-dwc_otg09:55:201
13770996810,55cyclictest24096-21tune2fs12:05:250
1377099678,37cyclictest3351-21acpi08:10:180
13770996651,7cyclictest0-21swapper/009:00:140
13770996612,6cyclictest0-21swapper/012:25:080
1377099657,43cyclictest0-21swapper/009:35:360
1377099657,43cyclictest0-21swapper/009:35:360
197472640,3sleep319750-21aten_rbpower_en08:55:233
1377099648,44cyclictest0-21swapper/012:02:400
1377099646,39cyclictest0-21swapper/011:05:280
1377099646,39cyclictest0-21swapper/011:05:280
13770996412,34cyclictest6661-21taskset09:46:200
1377099639,37cyclictest26522-21chrt09:11:540
13770996312,36cyclictest21382-21taskset10:29:430
13770996312,36cyclictest21382-21taskset10:29:430
13770996310,11cyclictest32605-21latency_hist12:30:090
1377099616,40cyclictest0-21swapper/007:55:270
1377099616,40cyclictest0-21swapper/007:55:270
13770996120,37cyclictest14110-21unixbench_multi11:35:370
13770996111,37cyclictest0-21swapper/009:54:460
13770996019,37cyclictest1022-21ls12:30:300
13770996012,45cyclictest13696-21unixbench-2d10:05:360
13770996010,36cyclictest0-21swapper/008:33:370
13770995920,27cyclictest1125-21runrttasks12:15:190
13770995915,41cyclictest0-21swapper/007:35:090
13770995910,33cyclictest0-21swapper/010:00:090
1377099589,39cyclictest0-21swapper/009:41:010
1377099582,23cyclictest0-21swapper/011:15:220
13770995812,33cyclictest0-21swapper/008:35:480
13770995812,33cyclictest0-21swapper/008:35:480
1377099577,37cyclictest20909-21taskset07:30:000
1377099574,41cyclictest0-21swapper/008:48:590
13770995733,14cyclictest28226-21grep10:45:360
13770995724,5cyclictest19525-21unixbench_multi11:50:390
13770995720,4cyclictest16754-21unixbench_singl07:15:360
1377099571,46cyclictest0-21swapper/008:45:080
13770995711,36cyclictest30869-21taskset09:25:240
1377099571,12cyclictest0-21swapper/007:20:350
1377699567,24cyclictest0-21swapper/111:00:211
1377099569,34cyclictest0-21swapper/008:53:420
1377099567,38cyclictest0-21swapper/011:55:280
1377099565,35cyclictest0-21swapper/011:46:330
13770995642,3cyclictest7850irq/56-dwc_otg_11:25:270
1377099561,32cyclictest18153-21wget10:20:240
1377099561,25cyclictest7844-21ls11:20:260
13770995611,12cyclictest12475-21latency_hist10:05:090
1377099557,38cyclictest0-21swapper/007:37:370
1377099557,28cyclictest25741-21wget12:10:230
13770995512,23cyclictest21143-21taskset08:58:480
1377699546,20cyclictest0-21swapper/110:10:221
13770995429,6cyclictest7850irq/56-dwc_otg_09:05:370
1377099541,32cyclictest11278-21wget11:30:210
13770995410,36cyclictest128-21systemd-journal11:43:310
13770995410,36cyclictest128-21systemd-journal11:43:310
1377699534,25cyclictest0-21swapper/112:20:201
13776995312,26cyclictest0-21swapper/108:10:131
1377099539,34cyclictest0-21swapper/010:15:210
1377099522,37cyclictest0-21swapper/010:55:220
1377099515,37cyclictest0-21swapper/010:54:350
1378199504,22cyclictest0-21swapper/308:45:183
13781995019,5cyclictest677-21fschecks_time12:30:263
1378099505,29cyclictest7950irq/56-dwc_otg_10:05:282
1377699506,26cyclictest0-21swapper/111:05:151
1377699506,26cyclictest0-21swapper/111:05:151
1378099498,25cyclictest0-21swapper/208:45:142
1377699497,27cyclictest0-21swapper/108:35:231
1377699497,27cyclictest0-21swapper/108:35:231
13776994911,24cyclictest0-21swapper/112:10:221
1377699488,25cyclictest0-21swapper/111:10:211
1377699482,5cyclictest0-21swapper/108:25:211
1377099484,35cyclictest0-21swapper/012:35:390
1378099471,6cyclictest10869-21grep08:30:252
1377699472,23cyclictest0-21swapper/109:05:221
1377699467,24cyclictest0-21swapper/110:40:231
1377699462,25cyclictest0-21swapper/110:30:221
13776994622,10cyclictest0-21swapper/111:25:381
1377699461,6cyclictest0-21swapper/108:15:221
316542450,3sleep11377699cyclictest09:25:361
1377699456,24cyclictest0-21swapper/112:35:191
1378199447,23cyclictest0-21swapper/307:25:223
1377699447,28cyclictest0-21swapper/109:15:221
1377699446,24cyclictest0-21swapper/108:05:141
13776994421,4cyclictest1125150wget11:30:211
132792443,33sleep00-21swapper/007:08:290
132792443,33sleep00-21swapper/007:08:290
1378099436,24cyclictest0-21swapper/207:25:222
1377699436,22cyclictest0-21swapper/108:20:221
13776994323,5cyclictest21-21rcuc/111:24:131
1377699431,4cyclictest0-21swapper/110:20:221
1378199424,19cyclictest7950irq/56-dwc_otg_07:10:113
1377699427,21cyclictest0-21swapper/108:45:131
1377699425,19cyclictest21-21rcuc/108:55:201
13776994234,6cyclictest221ktimersoftd/109:00:201
1378099417,24cyclictest0-21swapper/209:00:152
13780994119,12cyclictest32701-21fschecks_count09:30:252
1378099411,25cyclictest0-21swapper/209:10:202
1377699417,24cyclictest0-21swapper/107:25:151
1377699416,22cyclictest0-21swapper/109:45:211
1377699414,19cyclictest0-21swapper/110:45:201
1377699413,18cyclictest0-21swapper/111:55:201
1377699412,7cyclictest0-21swapper/107:40:211
1378199407,15cyclictest7950irq/56-dwc_otg_09:05:233
1378099401,25cyclictest0-21swapper/208:50:222
1377699403,27cyclictest0-21swapper/111:50:221
1377699403,16cyclictest23-21ksoftirqd/111:35:341
13776994011,13cyclictest7750irq/56-dwc_otg09:30:221
1378099391,24cyclictest0-21swapper/209:35:222
1378099391,24cyclictest0-21swapper/209:35:222
1377699397,14cyclictest7850irq/56-dwc_otg_10:35:311
1377699395,21cyclictest7750irq/56-dwc_otg07:40:091
13776993931,6cyclictest221ktimersoftd/109:35:201
13776993931,6cyclictest221ktimersoftd/109:35:201
13776993916,5cyclictest23-21ksoftirqd/112:30:231
1377699391,32cyclictest19177-21ls07:25:091
13776993912,4cyclictest21-21rcuc/112:05:251
312442380,2sleep10-21swapper/110:55:281
291702380,8sleep1221ktimersoftd/109:20:241
209312380,5sleep31378199cyclictest10:25:353
209312380,5sleep31378199cyclictest10:25:353
1378199386,15cyclictest7950irq/56-dwc_otg_09:20:093
1378199381,26cyclictest6844-21ls08:20:093
1378099381,23cyclictest0-21swapper/210:20:232
1377699384,19cyclictest7950irq/56-dwc_otg_10:05:351
1377699383,3cyclictest7950irq/56-dwc_otg_10:00:211
1377699382,4cyclictest0-21swapper/109:50:201
13776993820,5cyclictest21-21rcuc/108:30:341
1377699381,26cyclictest7750irq/56-dwc_otg07:15:341
1377699381,25cyclictest17822-21latency_hist10:20:091
1378199377,5cyclictest10854-21latency_hist11:30:093
1378099371,3cyclictest7850irq/56-dwc_otg_08:55:202
1378099371,25cyclictest0-21swapper/212:00:212
13776993714,14cyclictest7750irq/56-dwc_otg07:55:091
13776993713,6cyclictest23-21ksoftirqd/108:00:261
1377699371,2cyclictest0-21swapper/108:40:231
1377699371,14cyclictest7850irq/56-dwc_otg_07:35:081
285962360,2sleep30-21swapper/312:15:363
1378199365,18cyclictest7950irq/56-dwc_otg_11:35:353
1378099361,25cyclictest0-21swapper/211:30:142
1377699364,21cyclictest7750irq/56-dwc_otg11:45:261
1377699363,21cyclictest221ktimersoftd/107:47:351
1377699362,23cyclictest7750irq/56-dwc_otg12:26:031
134082368,17sleep213468-21apt-config07:10:092
134082368,17sleep213468-21apt-config07:10:092
1378199356,15cyclictest0-21swapper/309:00:153
1378199355,14cyclictest7950irq/56-dwc_otg_10:30:353
1378199351,23cyclictest13188-21missed_timers10:05:293
1378099352,16cyclictest7950irq/56-dwc_otg_07:10:112
1378099351,24cyclictest0-21swapper/211:35:232
1378099351,24cyclictest0-21swapper/208:05:222
13780993511,15cyclictest29184-21fschecks_time09:20:242
1377699355,18cyclictest7950irq/56-dwc_otg_09:44:101
1377699353,19cyclictest7750irq/56-dwc_otg12:15:361
1377699351,20cyclictest7750irq/56-dwc_otg10:27:491
1377699351,20cyclictest7750irq/56-dwc_otg10:27:491
13776993510,6cyclictest23-21ksoftirqd/107:55:241
13776993510,6cyclictest23-21ksoftirqd/107:55:241
1378199342,3cyclictest7950irq/56-dwc_otg_11:50:243
1378199341,12cyclictest7850irq/56-dwc_otg_10:10:233
1378099341,5cyclictest9055-21munin-run11:25:082
13780993414,13cyclictest11087-21fschecks_count10:00:262
1378099341,26cyclictest9656-21switchtime08:25:352
1378099341,20cyclictest6001-21cut11:15:272
1378099341,18cyclictest0-21swapper/211:25:232
1378099341,17cyclictest3956-21grep11:10:212
1378099341,16cyclictest7950irq/56-dwc_otg_10:00:082
13776993416,11cyclictest14895-21aten_rbpower_po11:40:261
13776993416,11cyclictest14895-21aten_rbpower_po11:40:261
1377699341,3cyclictest7950irq/56-dwc_otg_11:15:221
1377699341,3cyclictest0-21swapper/110:50:211
134752345,16sleep313616-21gpgconf07:10:103
134752345,16sleep313616-21gpgconf07:10:103
1378199338,11cyclictest7950irq/56-dwc_otg_08:50:263
1378199337,5cyclictest7750irq/56-dwc_otg07:15:223
1378199331,22cyclictest26411-21ls07:45:093
1378199331,18cyclictest0-21swapper/310:00:093
1378199331,16cyclictest10530-21ls08:30:093
1378099337,15cyclictest0-21swapper/208:10:142
1378099335,15cyclictest31-21ksoftirqd/209:50:082
1378099331,26cyclictest29815-21ls12:20:292
1378099331,21cyclictest994-21timerandwakeup11:00:362
1378099331,21cyclictest17762-21cut07:20:252
1378099331,16cyclictest22342-21ntp_kernel_pll_10:30:312
1378099331,13cyclictest0-21swapper/210:41:592
1377699336,12cyclictest221ktimersoftd/112:00:211
24882320,3sleep20-21swapper/212:35:232
1378199327,5cyclictest7850irq/56-dwc_otg_08:10:083
1378199327,15cyclictest0-21swapper/308:10:143
1378199327,11cyclictest7950irq/56-dwc_otg_07:20:253
1378199327,10cyclictest7950irq/56-dwc_otg_11:15:223
1378199326,14cyclictest7950irq/56-dwc_otg_10:55:083
1378199323,20cyclictest0-21swapper/309:40:173
1378199323,15cyclictest7950irq/56-dwc_otg_11:10:243
1378199322,15cyclictest32414-21latency_hist09:30:093
13781993217,5cyclictest12370-21/usr/sbin/munin08:35:313
13781993217,5cyclictest12370-21/usr/sbin/munin08:35:313
1378199321,6cyclictest128-21systemd-journal12:10:083
1378199321,21cyclictest32699-21cpuspeed11:00:243
1378199321,19cyclictest27193-21latency_hist12:15:093
1378099325,5cyclictest7950irq/56-dwc_otg_09:30:082
1378099321,21cyclictest0-21swapper/210:15:232
1377699327,4cyclictest7750irq/56-dwc_otg08:50:361
125732326,14sleep112607-21/usr/sbin/munin07:05:341
125732326,14sleep112607-21/usr/sbin/munin07:05:341
1378199317,14cyclictest0-21swapper/311:30:143
1378199313,6cyclictest0-21swapper/307:50:233
13781993112,7cyclictest1125-21runrttasks09:23:353
1378199311,18cyclictest6982-21munin-plugin-st09:50:083
1378099316,22cyclictest19971-21latency_hist11:55:092
1378099314,14cyclictest7850irq/56-dwc_otg_08:40:272
1378099312,20cyclictest0-21swapper/212:05:292
13780993121,7cyclictest25616-21/usr/sbin/munin12:10:312
1378099311,27cyclictest0-21swapper/210:55:212
1378099311,24cyclictest31890-21ls08:00:092
1378099311,24cyclictest31890-21ls08:00:092
1378099311,22cyclictest0-21swapper/207:16:502
1378099311,21cyclictest0-21swapper/211:05:242
1378099311,21cyclictest0-21swapper/211:05:242
1378099311,13cyclictest18333-21latency_hist11:50:092
1378099311,12cyclictest21205-21wget07:30:202
1378199309,9cyclictest7950irq/56-dwc_otg_11:05:273
1378199309,9cyclictest7950irq/56-dwc_otg_11:05:273
1378199308,5cyclictest9550irq/80-mmc010:45:093
1378199303,3cyclictest7950irq/56-dwc_otg_10:45:193
1378199301,4cyclictest7850irq/56-dwc_otg_09:50:283
1378199301,21cyclictest7950irq/56-dwc_otg_12:20:193
1378199301,14cyclictest31606-21memory12:25:303
1378099307,12cyclictest7950irq/56-dwc_otg_09:16:582
1378099302,5cyclictest7850irq/56-dwc_otg_09:05:212
13780993013,11cyclictest301ktimersoftd/212:26:332
1378099301,20cyclictest0-21swapper/207:45:322
1378099301,16cyclictest0-21swapper/207:35:272
1378099301,15cyclictest21009-21grep10:25:362
1378099301,15cyclictest21009-21grep10:25:362
90702290,2sleep30-21swapper/311:25:083
1378199293,15cyclictest7950irq/56-dwc_otg_08:00:303
13781992923,4cyclictest30749-21aten_rbpower_en10:55:213
1378199292,19cyclictest0-21swapper/312:04:593
13781992919,7cyclictest23257-21fschecks_count07:35:273
13781992915,4cyclictest11892-21seq10:00:373
1378199291,3cyclictest7950irq/56-dwc_otg_11:45:283
1378199291,17cyclictest14982-21ls11:40:263
1378199291,17cyclictest14982-21ls11:40:263
1378099299,5cyclictest7950irq/56-dwc_otg_12:15:352
1378099297,5cyclictest31-21ksoftirqd/207:40:302
1378099295,4cyclictest7560-21tune2fs09:50:262
1378099292,6cyclictest0-21swapper/207:50:202
1378099291,5cyclictest7950irq/56-dwc_otg_10:45:222
1378099291,21cyclictest0-21swapper/208:15:272
1378099291,20cyclictest3782-21ls09:40:252
1378099291,18cyclictest21354-21head11:55:372
1378099291,16cyclictest14907-21aten_rbpower_vo11:40:262
1378099291,16cyclictest14907-21aten_rbpower_vo11:40:262
1378099291,16cyclictest0-21swapper/210:35:162
1378099291,13cyclictest0-21swapper/212:30:112
1378099291,12cyclictest0-21swapper/208:24:172
1377699294,11cyclictest7850irq/56-dwc_otg_09:14:341
1378199288,6cyclictest7850irq/56-dwc_otg_08:30:273
1378199288,4cyclictest7850irq/56-dwc_otg_09:30:283
1378199288,14cyclictest0-21swapper/307:55:273
1378199288,14cyclictest0-21swapper/307:55:273
1378199282,14cyclictest7950irq/56-dwc_otg_09:35:253
1378199282,14cyclictest7950irq/56-dwc_otg_09:35:253
1378199281,5cyclictest6980-21apt-get08:20:203
13781992812,6cyclictest26789-21latency_hist09:15:093
1378199281,15cyclictest0-21swapper/310:20:423
1378099282,10cyclictest0-21swapper/210:11:402
1378099281,7cyclictest1107-21/usr/sbin/munin08:05:082
1378199277,9cyclictest7950irq/56-dwc_otg_07:30:313
1378199274,21cyclictest26844-21uniq07:45:253
13781992720,5cyclictest23592-21grep10:35:243
1378199271,7cyclictest17925-21latency_hist10:20:093
1378099272,3cyclictest0-21swapper/208:35:232
1378099272,3cyclictest0-21swapper/208:35:232
1378199265,9cyclictest7750irq/56-dwc_otg11:55:283
1378199264,10cyclictest7850irq/56-dwc_otg_08:45:083
13780992617,4cyclictest29990-21unixbench_multi10:50:372
1378199255,4cyclictest7850irq/56-dwc_otg_12:35:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional