You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-17 - 21:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Dec 17, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2388399936,55cyclictest0-21swapper/010:55:170
2388399886,61cyclictest0-21swapper/008:40:160
281632820,2sleep20-21swapper/210:20:212
304832760,2sleep030481-21ntpq11:55:260
23883997318,10cyclictest0-21swapper/009:45:140
294752720,4sleep32388699cyclictest08:55:093
23883997124,12cyclictest0-21swapper/008:45:150
221272710,2sleep20-21swapper/210:05:032
23883996934,17cyclictest0-21swapper/010:00:140
23883996727,9cyclictest0-21swapper/011:40:130
23883996727,9cyclictest0-21swapper/011:40:130
23883996636,15cyclictest0-21swapper/011:05:160
23883996636,15cyclictest0-21swapper/011:05:160
2388399655,20cyclictest13-21rcuc/012:35:150
23883996418,18cyclictest17399-21idleruntime11:20:210
2388399636,54cyclictest0-21swapper/007:15:160
2388399632,31cyclictest0-21swapper/012:05:170
2388399631,17cyclictest9580-21aten_rbpower_en08:00:170
23883996311,10cyclictest0-21swapper/007:40:150
107422630,3sleep20-21swapper/211:00:302
23883996228,5cyclictest5220-21unixbench_multi07:45:300
23883996228,5cyclictest5220-21unixbench_multi07:45:300
23883996115,6cyclictest0-21swapper/010:10:150
23883996115,4cyclictest0-21swapper/010:25:130
2388599602,5cyclictest26272-21wget11:45:172
2388399601,38cyclictest393-21idleruntime-cro09:05:040
2388399601,38cyclictest393-21idleruntime-cro09:05:040
2388599591,5cyclictest4584-21grep09:15:202
23883995937,2cyclictest0-21swapper/010:45:090
23883995931,5cyclictest7649-21apt09:25:030
2388599581,12cyclictest0-21swapper/212:00:302
23885995711,6cyclictest7950irq/56-dwc_otg_11:06:172
23885995711,6cyclictest7950irq/56-dwc_otg_11:06:172
2388499572,8cyclictest0-21swapper/108:40:151
2388399571,22cyclictest0-21swapper/008:10:030
2388399565,32cyclictest0-21swapper/007:50:160
2388399565,23cyclictest0-21swapper/011:15:160
2388399564,8cyclictest0-21swapper/011:10:130
2388399564,8cyclictest0-21swapper/011:10:130
23883995625,16cyclictest9457-21munin-run09:30:030
23883995616,25cyclictest17032-21fschecks_count08:20:200
2388599551,9cyclictest7950irq/56-dwc_otg_10:25:272
2388399557,30cyclictest23268-21unixbench_multi08:35:310
2388399557,30cyclictest23268-21unixbench_multi08:35:310
2388399554,26cyclictest0-21swapper/010:50:140
2388399553,30cyclictest28298-21fschecks_time11:50:190
2388399552,35cyclictest9-21ksoftirqd/010:05:170
2388599546,5cyclictest7850irq/56-dwc_otg_07:40:262
2388599545,6cyclictest7950irq/56-dwc_otg_09:04:142
2388599545,6cyclictest7950irq/56-dwc_otg_09:04:142
2388599545,4cyclictest5140-21/usr/sbin/munin12:15:222
23883995423,13cyclictest7850irq/56-dwc_otg_10:30:170
23883995422,18cyclictest0-21swapper/012:15:170
2388399541,33cyclictest27080-21head10:15:310
2388399541,31cyclictest4209-21latency_hist09:15:040
2388399541,31cyclictest0-21swapper/008:15:140
2388599535,4cyclictest7850irq/56-dwc_otg_09:40:252
2388599533,5cyclictest20697-21aten_rbpower_cu11:30:172
2388599531,9cyclictest20297-21latency_hist08:30:042
2388599531,5cyclictest12079-21apt12:35:132
23885995314,4cyclictest632-21/usr/sbin/munin09:05:292
2388599531,3cyclictest0-21swapper/208:53:012
23885995313,4cyclictest15795-21ntp_states09:45:252
23883995317,4cyclictest10181-21chrt12:30:150
2388399531,30cyclictest3677-21sed10:40:310
278592520,2sleep10-21swapper/111:50:041
2388599529,6cyclictest0-21swapper/208:50:032
2388599527,4cyclictest15919-21timerwakeupswit08:15:302
2388599523,3cyclictest7750irq/56-dwc_otg12:23:472
2388599522,5cyclictest0-21swapper/208:58:572
2388599521,8cyclictest0-21swapper/209:20:522
2388599521,4cyclictest128-21systemd-journal08:33:252
2388399521,29cyclictest5986-21latency_hist09:20:040
2388399521,27cyclictest1014-21tune2fs10:35:200
2388399521,27cyclictest1014-21tune2fs10:35:200
2388599519,5cyclictest18449-21cron09:55:032
2388599519,5cyclictest18449-21cron09:55:032
2388599514,7cyclictest7850irq/56-dwc_otg_07:51:042
2388599511,9cyclictest0-21swapper/210:54:282
2388599511,8cyclictest7850irq/56-dwc_otg_07:16:352
2388599511,7cyclictest0-21swapper/212:09:002
23885995114,4cyclictest1125-21runrttasks07:55:252
2388399518,24cyclictest0-21swapper/007:55:310
23883995118,7cyclictest31156-21latency_hist09:00:040
2388599509,37cyclictest0-21swapper/210:35:252
2388599509,37cyclictest0-21swapper/210:35:252
2388599504,7cyclictest114842sleep212:32:072
2388599503,6cyclictest7950irq/56-dwc_otg_11:15:322
23883995026,4cyclictest7850irq/56-dwc_otg_07:25:200
2388399501,25cyclictest24673-21grep07:10:300
2388699492,9cyclictest7750irq/56-dwc_otg09:40:163
2388599494,3cyclictest7850irq/56-dwc_otg_11:21:132
2388599491,9cyclictest7950irq/56-dwc_otg_07:35:282
2388599491,5cyclictest7850irq/56-dwc_otg_10:57:492
2388599491,5cyclictest10992-21taskset09:31:592
2388599491,5cyclictest0-21swapper/207:45:442
2388599491,5cyclictest0-21swapper/207:45:442
2388499497,28cyclictest0-21swapper/109:05:181
2388499495,25cyclictest0-21swapper/111:10:131
2388499495,25cyclictest0-21swapper/111:10:131
2388399491,5cyclictest0-21swapper/011:25:150
23883994911,10cyclictest0-21swapper/009:50:150
23883994911,10cyclictest0-21swapper/009:50:150
2388599481,3cyclictest0-21swapper/211:55:172
2388599481,3cyclictest0-21swapper/211:39:162
23885994812,4cyclictest301ktimersoftd/210:10:032
23883994819,16cyclictest9-21ksoftirqd/011:00:270
2388399481,25cyclictest22931-21kernelversion11:35:230
2388599472,9cyclictest7850irq/56-dwc_otg_09:57:102
2388599472,3cyclictest0-21swapper/208:00:262
2388599471,5cyclictest0-21swapper/209:26:232
23883994727,3cyclictest0-21swapper/009:55:150
2388399472,28cyclictest0-21swapper/008:30:040
2388699462,24cyclictest7950irq/56-dwc_otg_12:30:143
2388599461,9cyclictest7750irq/56-dwc_otg12:15:042
2388599461,7cyclictest0-21swapper/210:19:432
2388599461,14cyclictest0-21swapper/209:37:582
2388499466,26cyclictest0-21swapper/107:45:161
2388499466,26cyclictest0-21swapper/107:45:161
2388499464,12cyclictest0-21swapper/111:00:151
2388399463,27cyclictest28267-21latency10:20:220
23883994622,5cyclictest32338-21grep12:00:270
23883994619,13cyclictest7950irq/56-dwc_otg_09:40:300
2388399461,19cyclictest11259-21latency_hist09:35:040
2388699457,23cyclictest0-21swapper/308:30:173
2388599451,3cyclictest7850irq/56-dwc_otg_08:05:202
23885994512,4cyclictest9109-21awk12:25:292
2388499457,24cyclictest0-21swapper/109:55:161
2388499457,23cyclictest7750irq/56-dwc_otg11:10:051
2388499457,23cyclictest7750irq/56-dwc_otg11:10:051
2388499456,24cyclictest0-21swapper/112:35:081
23884994516,8cyclictest0-21swapper/110:25:131
2388399456,26cyclictest0-21swapper/012:20:100
2388399452,23cyclictest0-21swapper/009:10:040
23883994520,12cyclictest0-21swapper/008:10:040
23883994517,17cyclictest0-21swapper/008:54:280
2388399451,41cyclictest13-21rcuc/011:32:590
2388399451,24cyclictest1125-21runrttasks07:35:240
233242453,34sleep00-21swapper/007:06:260
216422451,6sleep321754-21tail11:30:303
2388699447,23cyclictest0-21swapper/311:55:163
2388699446,24cyclictest0-21swapper/307:50:163
2388599449,3cyclictest7950irq/56-dwc_otg_07:32:492
2388599441,8cyclictest0-21swapper/211:54:102
2388599441,5cyclictest0-21swapper/207:25:292
2388499443,25cyclictest0-21swapper/109:50:141
2388499443,25cyclictest0-21swapper/109:50:141
2388499442,35cyclictest7950irq/56-dwc_otg_10:55:251
2388399442,24cyclictest0-21swapper/012:10:300
2388599437,33cyclictest0-21swapper/211:10:152
2388599437,33cyclictest0-21swapper/211:10:152
2388599434,4cyclictest7850irq/56-dwc_otg_09:12:342
2388599431,9cyclictest0-21swapper/210:14:052
2388499438,23cyclictest7750irq/56-dwc_otg08:30:041
2388499437,25cyclictest0-21swapper/110:00:161
2388499437,22cyclictest0-21swapper/110:50:161
2388399432,21cyclictest0-21swapper/011:45:200
2388399431,21cyclictest0-21swapper/007:30:250
23883994310,4cyclictest13-21rcuc/009:40:040
2388699427,25cyclictest0-21swapper/312:15:143
23885994213,4cyclictest1125-21runrttasks08:37:342
23885994213,4cyclictest1125-21runrttasks08:37:342
2388499427,22cyclictest7750irq/56-dwc_otg09:20:211
2388499426,23cyclictest0-21swapper/109:45:161
2388499425,26cyclictest0-21swapper/108:05:151
23884994213,13cyclictest23-21ksoftirqd/109:00:141
23884994213,13cyclictest23-21ksoftirqd/109:00:141
2388499421,11cyclictest7750irq/56-dwc_otg10:05:141
2388399421,25cyclictest0-21swapper/012:25:110
2388399421,24cyclictest1125-21runrttasks07:22:080
2388499416,5cyclictest0-21swapper/110:40:161
2388499413,27cyclictest7750irq/56-dwc_otg09:30:031
23884994118,5cyclictest19450-21/usr/sbin/munin11:25:231
2388699402,5cyclictest4086-21munin-run10:45:033
2388499409,6cyclictest0-21swapper/112:25:161
2388499408,20cyclictest0-21swapper/111:40:141
2388499408,20cyclictest0-21swapper/111:40:141
2388499406,22cyclictest0-21swapper/112:20:101
2388499403,26cyclictest0-21swapper/108:15:161
2388499402,9cyclictest0-21swapper/111:15:161
2388499401,8cyclictest0-21swapper/112:15:121
2388699397,22cyclictest0-21swapper/309:35:163
2388699397,22cyclictest0-21swapper/308:45:173
23886993918,4cyclictest6368-21fschecks_time09:20:213
2388599391,12cyclictest0-21swapper/211:27:392
2388499397,22cyclictest0-21swapper/107:15:161
2388499395,4cyclictest7750irq/56-dwc_otg07:35:221
2388499392,8cyclictest0-21swapper/112:10:151
2388499392,7cyclictest22912-21grep08:35:271
2388499392,7cyclictest22912-21grep08:35:271
2388499391,25cyclictest9928-21cut09:30:191
23883993928,4cyclictest7850irq/56-dwc_otg_08:30:280
2388699386,10cyclictest0-21swapper/312:35:303
2388699381,27cyclictest20449-21ls10:00:043
2388699381,23cyclictest26250-21cut10:15:213
2388699381,22cyclictest8379-21ls10:55:223
2388499386,21cyclictest0-21swapper/112:30:301
23884993823,4cyclictest17403-21taskset11:20:211
2388499381,7cyclictest0-21swapper/110:15:161
2388699377,21cyclictest0-21swapper/307:35:183
2388499377,15cyclictest7950irq/56-dwc_otg_09:20:031
2388499373,7cyclictest30506-21head07:25:311
2388699364,6cyclictest0-21swapper/307:30:133
2388699362,5cyclictest0-21swapper/307:10:183
23886993624,7cyclictest381ktimersoftd/308:35:173
23886993624,7cyclictest381ktimersoftd/308:35:173
2388599361,9cyclictest0-21swapper/207:22:112
2388599361,12cyclictest0-21swapper/207:11:372
2388499367,5cyclictest7850irq/56-dwc_otg_08:55:321
2388499364,8cyclictest0-21swapper/112:05:151
2388499363,19cyclictest0-21swapper/110:10:151
23884993630,4cyclictest32214-21ls10:30:291
2388699351,25cyclictest586-21ls09:05:053
2388699351,25cyclictest586-21ls09:05:053
2388599351,13cyclictest0-21swapper/211:42:392
2388599351,13cyclictest0-21swapper/211:42:392
23884993511,10cyclictest5994-21munin-run10:50:031
2388699345,22cyclictest7950irq/56-dwc_otg_12:25:153
23886993426,5cyclictest381ktimersoftd/308:50:163
2388699341,22cyclictest1583-21switchtime10:35:283
2388699341,22cyclictest1583-21switchtime10:35:283
2388599347,14cyclictest0-21swapper/210:45:082
2388599341,22cyclictest24207-21cut08:40:192
2388599341,12cyclictest0-21swapper/208:20:342
2388499348,14cyclictest5969-21wget07:50:161
2388499345,9cyclictest7750irq/56-dwc_otg11:35:211
2388499344,22cyclictest7750irq/56-dwc_otg08:14:201
2388499342,5cyclictest0-21swapper/109:40:141
2388499341,25cyclictest27979-21cut10:20:191
2388499341,22cyclictest0-21swapper/107:40:151
2388699334,7cyclictest37-21rcuc/307:25:313
2388699334,14cyclictest7950irq/56-dwc_otg_07:20:203
2388699333,12cyclictest7950irq/56-dwc_otg_09:30:223
2388699332,17cyclictest22169-21latency_hist10:05:043
2388699331,4cyclictest7950irq/56-dwc_otg_12:00:143
2388599336,14cyclictest0-21swapper/210:30:162
2388499333,8cyclictest24582-21tr07:10:281
2388499333,4cyclictest28684-21unixbench_singl07:20:311
2388499331,14cyclictest7750irq/56-dwc_otg09:15:031
2388699327,11cyclictest7950irq/56-dwc_otg_07:50:033
2388699327,11cyclictest7950irq/56-dwc_otg_07:50:033
2388699322,7cyclictest16239-21grep09:45:303
2388699321,21cyclictest18315-21latency_hist08:25:043
2388699321,16cyclictest18609-21ls09:55:043
2388699321,16cyclictest18609-21ls09:55:043
2388599322,10cyclictest0-21swapper/208:11:502
2388499327,15cyclictest7850irq/56-dwc_otg_08:45:311
2388499326,5cyclictest7850irq/56-dwc_otg_08:00:171
2388499323,6cyclictest11-21rcu_preempt08:50:201
2388499323,16cyclictest7750irq/56-dwc_otg09:38:121
2388499322,20cyclictest7750irq/56-dwc_otg07:55:311
2388699316,15cyclictest0-21swapper/308:25:173
2388699316,13cyclictest7950irq/56-dwc_otg_07:55:213
2388699316,13cyclictest0-21swapper/310:05:163
2388699313,5cyclictest26497-21fschecks_time11:45:203
2388699313,14cyclictest7950irq/56-dwc_otg_09:15:243
2388699312,15cyclictest7950irq/56-dwc_otg_10:45:303
2388699312,12cyclictest7850irq/56-dwc_otg_11:08:423
2388699312,12cyclictest7850irq/56-dwc_otg_11:08:423
23886993117,5cyclictest28123-21if_err_eth010:20:203
2388699311,5cyclictest8199-21tune2fs09:25:213
2388699311,4cyclictest7950irq/56-dwc_otg_09:10:163
2388699311,4cyclictest0-21swapper/308:05:283
2388499319,6cyclictest7750irq/56-dwc_otg11:55:031
2388499318,15cyclictest0-21swapper/108:30:231
2388499317,14cyclictest0-21swapper/112:00:161
2388499315,6cyclictest7850irq/56-dwc_otg_11:59:121
23884993112,4cyclictest0-21swapper/107:30:211
2388499311,13cyclictest7850irq/56-dwc_otg_08:20:181
220522313,21sleep10-21swapper/107:05:141
2388699303,14cyclictest7950irq/56-dwc_otg_07:40:313
2388699302,9cyclictest1088-21if_wlan009:05:233
23886993024,4cyclictest14763-21ls08:15:043
2388699301,8cyclictest14529-21unixbench_multi11:10:293
2388699301,8cyclictest14529-21unixbench_multi11:10:293
2388699301,6cyclictest128-21systemd-journal10:35:043
2388699301,4cyclictest0-21swapper/310:50:243
2388699301,19cyclictest15497-21uniq11:15:213
2388699301,19cyclictest0-21swapper/312:24:573
2388499308,4cyclictest0-21swapper/111:33:161
23884993010,11cyclictest969-21fschecks_count10:35:201
23884993010,11cyclictest969-21fschecks_count10:35:201
257822291,4sleep30-21swapper/311:43:183
257822291,4sleep30-21swapper/311:43:183
2388699297,6cyclictest7950irq/56-dwc_otg_11:00:323
2388699292,19cyclictest0-21swapper/311:53:263
2388699291,6cyclictest2099-21unixbench_singl12:05:313
2388699291,12cyclictest7950irq/56-dwc_otg_11:35:313
2388699283,5cyclictest14936-21apt-get08:15:143
2388699282,20cyclictest0-21swapper/310:10:143
2388699282,15cyclictest0-21swapper/308:01:313
2388699281,8cyclictest2830-21aten_rbpower_po12:10:163
2388699281,7cyclictest19965-21grep11:25:303
2388699281,6cyclictest30611-21grep10:25:293
23885992821,5cyclictest301ktimersoftd/210:40:162
2388699273,14cyclictest0-21swapper/307:18:133
2388699271,23cyclictest17578-21ls11:20:233
234572274,15sleep30-21swapper/307:08:033
227652266,9sleep20-21swapper/207:05:312
2388699245,6cyclictest39-21ksoftirqd/308:40:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional