You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 18:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue Aug 11, 2026 12:43:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28899912414,58cyclictest0-21swapper/011:55:220
28899912414,58cyclictest0-21swapper/011:55:220
28909912229,26cyclictest7850irq/56-dwc_otg_10:20:171
2889991175,85cyclictest0-21swapper/011:25:170
2889991133,83cyclictest30891-21grep10:50:340
2889991095,84cyclictest0-21swapper/012:10:220
2889991095,84cyclictest0-21swapper/012:10:220
2892991083,19cyclictest7950irq/56-dwc_otg_11:55:213
2892991083,19cyclictest7950irq/56-dwc_otg_11:55:213
2892991062,13cyclictest7950irq/56-dwc_otg_11:50:163
28899910441,37cyclictest1180-21runrttasks11:50:190
1123621010,2sleep00-21swapper/010:26:300
28899910051,25cyclictest30197-21kworker/0:112:35:190
2889999923,43cyclictest13688-21diskmemload10:20:230
288999975,61cyclictest0-21swapper/012:15:050
288999972,77cyclictest10519-21tune2fs11:45:230
289299963,15cyclictest7950irq/56-dwc_otg_12:20:353
289299954,16cyclictest10468-21fschecks_count11:45:233
2892999510,67cyclictest7750irq/56-dwc_otg12:10:203
2892999510,67cyclictest7750irq/56-dwc_otg12:10:203
289299945,16cyclictest3552-21sed12:15:343
288999925,49cyclictest0-21swapper/012:20:330
2889999246,23cyclictest101ktimersoftd/012:25:200
289299913,13cyclictest19505-21ntp_states12:35:293
289299893,8cyclictest7950irq/56-dwc_otg_11:00:233
289299893,22cyclictest26274-21aten_rbpower_en11:25:213
289199893,60cyclictest10466-21ssh11:45:232
289299882,13cyclictest7950irq/56-dwc_otg_10:20:163
288999874,64cyclictest0-21swapper/011:00:240
289299845,72cyclictest0-21swapper/310:50:353
289299843,10cyclictest10788-21ssh12:25:213
2890998412,21cyclictest10784-21diskstats12:25:211
288999836,66cyclictest0-21swapper/009:55:100
288999836,37cyclictest5703-21rm09:40:130
288999836,37cyclictest5703-21rm09:40:130
289099814,52cyclictest0-21swapper/111:55:201
289099814,52cyclictest0-21swapper/111:55:201
245322810,2sleep10-21swapper/111:22:401
2891998011,12cyclictest29-21rcuc/211:55:262
2891998011,12cyclictest29-21rcuc/211:55:262
2889997727,24cyclictest14436-21ls09:10:220
2889997727,24cyclictest14436-21ls09:10:220
288999763,48cyclictest0-21swapper/010:34:260
130442750,5sleep3289299cyclictest10:29:593
289199748,22cyclictest29-21rcuc/212:25:202
2890997417,24cyclictest3653-21vmstat12:15:371
289299724,12cyclictest0-21swapper/310:34:263
288999727,62cyclictest0-21swapper/010:00:190
288999725,40cyclictest60-21kswapd008:50:100
2890997111,35cyclictest7950irq/56-dwc_otg_11:00:221
288999712,21cyclictest0-21swapper/011:20:150
289299701,2cyclictest0-21swapper/311:40:143
2890997013,26cyclictest10413-21rm11:45:211
309952690,3sleep20-21swapper/208:25:342
2892996913,13cyclictest7850irq/56-dwc_otg_10:55:153
2890996930,18cyclictest7750irq/56-dwc_otg11:25:181
14142695,9sleep30-21swapper/307:05:193
289199684,23cyclictest31-21ksoftirqd/212:20:332
2891996830,18cyclictest301ktimersoftd/210:20:232
289299674,5cyclictest7950irq/56-dwc_otg_09:40:043
288999672,43cyclictest25701-21runrttasks10:45:190
2892996615,5cyclictest15040-21grep11:10:333
289199665,40cyclictest26240-21ssh11:25:212
289099665,16cyclictest29699-21/usr/sbin/munin10:50:331
2890996616,39cyclictest0-21swapper/110:34:241
289299654,3cyclictest0-21swapper/309:00:173
289299654,10cyclictest7950irq/56-dwc_otg_11:35:323
289299653,5cyclictest7950irq/56-dwc_otg_07:35:283
2892996510,14cyclictest10842-21unixbench-2d11:05:333
288999652,41cyclictest19078-21sed11:15:340
288999652,38cyclictest9466-21sed07:25:330
288999641,52cyclictest30155-21fschecks_count10:10:200
289299633,13cyclictest7850irq/56-dwc_otg_07:40:293
289199633,32cyclictest0-21swapper/212:15:052
289199633,27cyclictest0-21swapper/212:35:292
2889996336,16cyclictest10663-21/usr/sbin/munin09:45:320
288999632,30cyclictest0-21swapper/008:10:160
2891996212,26cyclictest7950irq/56-dwc_otg_11:00:212
2889996242,5cyclictest1830-21uniq10:55:220
2889996211,35cyclictest0-21swapper/007:10:140
248182620,2sleep20-21swapper/208:10:212
289299611,20cyclictest2177950ssh11:20:043
289199614,40cyclictest0-21swapper/211:50:182
2890996127,18cyclictest7950irq/56-dwc_otg_11:50:161
2889996121,26cyclictest10124-21if_eth011:05:230
289299604,10cyclictest7950irq/56-dwc_otg_08:50:153
289299602,11cyclictest23345-21ssh11:20:333
289299601,7cyclictest0-21swapper/310:35:203
289099603,6cyclictest7750irq/56-dwc_otg09:20:171
2889996039,5cyclictest9450irq/48-DMA08:56:170
288999601,52cyclictest0-21swapper/007:55:180
288999601,25cyclictest0-21swapper/008:05:140
289299591,6cyclictest9963-21df09:45:203
2891995920,25cyclictest0-21swapper/208:55:172
289099594,14cyclictest7750irq/56-dwc_otg12:35:311
2890995913,10cyclictest7614-21grep12:20:321
288999593,32cyclictest0-21swapper/010:35:150
288999591,40cyclictest26501-21fschecks_time09:25:210
289299581,3cyclictest7850irq/56-dwc_otg_07:15:303
289199585,10cyclictest7950irq/56-dwc_otg_08:30:152
2891995810,29cyclictest30927-21ssh10:50:352
2889995818,23cyclictest26582-21grep10:05:240
288999580,29cyclictest13688-21diskmemload12:30:160
289299578,4cyclictest7950irq/56-dwc_otg_07:20:253
289299575,4cyclictest0-21swapper/309:17:543
289299575,10cyclictest7750irq/56-dwc_otg11:30:333
289299574,5cyclictest26131-21ssh09:25:153
289299572,8cyclictest27309-1kworker/u9:210:10:173
289299571,4cyclictest0-21swapper/310:43:393
289299571,4cyclictest0-21swapper/310:43:393
289299571,14cyclictest0-21swapper/309:40:353
289299571,14cyclictest0-21swapper/309:40:353
289199573,10cyclictest0-21swapper/210:25:162
289099572,7cyclictest0-21swapper/108:15:151
289099572,7cyclictest0-21swapper/108:15:151
2889995730,5cyclictest2926-21ssh09:35:360
288999572,33cyclictest0-21swapper/009:20:030
289299567,4cyclictest0-21swapper/309:31:393
289099566,15cyclictest7750irq/56-dwc_otg08:10:151
289099562,10cyclictest7750irq/56-dwc_otg11:35:151
288999563,33cyclictest0-21swapper/012:05:240
2889995628,6cyclictest4135-21grep08:40:310
2889995613,25cyclictest0-21swapper/007:20:310
288999561,25cyclictest22657-21latency09:20:240
288999561,19cyclictest0-21swapper/008:55:160
289299559,5cyclictest13688-21diskmemload10:46:433
289299551,6cyclictest18424-21sed07:50:323
289299551,5cyclictest7850irq/56-dwc_otg_08:18:523
289299551,5cyclictest7850irq/56-dwc_otg_08:18:523
289199552,10cyclictest0-21swapper/211:10:172
2891995510,24cyclictest0-21swapper/209:20:172
289099553,31cyclictest1452-21kworker/u8:012:10:221
289099553,31cyclictest1452-21kworker/u8:012:10:221
289099551,10cyclictest7950irq/56-dwc_otg_08:40:161
288999554,38cyclictest21484-21/usr/sbin/munin10:40:290
288999554,38cyclictest21484-21/usr/sbin/munin10:40:290
2889995536,8cyclictest7850irq/56-dwc_otg_09:50:220
288999553,29cyclictest0-21swapper/007:50:300
288999552,33cyclictest3165-21grep11:35:330
2889995514,27cyclictest0-21swapper/009:30:290
23642553,11sleep10-21swapper/107:06:471
289299545,13cyclictest7950irq/56-dwc_otg_08:55:193
289299542,7cyclictest0-21swapper/309:53:443
289199541,24cyclictest0-21swapper/209:10:222
289199541,24cyclictest0-21swapper/209:10:222
2890995428,9cyclictest7850irq/56-dwc_otg_11:05:281
2889995426,12cyclictest7750irq/56-dwc_otg10:15:340
289299531,4cyclictest0-21swapper/312:05:203
289299531,19cyclictest0-21swapper/310:15:113
289199533,13cyclictest0-21swapper/212:05:192
288999539,25cyclictest0-21swapper/008:15:200
288999539,25cyclictest0-21swapper/008:15:200
288999532,31cyclictest23132-21ntpq12:00:280
289299528,8cyclictest7850irq/56-dwc_otg_08:05:163
289299523,14cyclictest0-21swapper/310:05:133
289299521,6cyclictest17423-21apt-get09:55:063
289199527,11cyclictest7850irq/56-dwc_otg_07:15:162
2890995231,10cyclictest7750irq/56-dwc_otg07:30:341
2890995231,10cyclictest7750irq/56-dwc_otg07:30:341
288999525,5cyclictest7750irq/56-dwc_otg08:30:150
288999522,28cyclictest6094-21grep11:40:170
289299511,14cyclictest0-21swapper/309:20:423
289099518,27cyclictest0-21swapper/108:50:091
2890995124,14cyclictest7750irq/56-dwc_otg09:40:121
2890995124,14cyclictest7750irq/56-dwc_otg09:40:121
2890995114,12cyclictest7850irq/56-dwc_otg_07:35:221
288999514,30cyclictest140-21systemd-journal07:20:030
289299508,10cyclictest7950irq/56-dwc_otg_12:30:293
289299501,17cyclictest0-21swapper/307:45:313
289199505,11cyclictest7850irq/56-dwc_otg_09:30:152
289099508,29cyclictest0-21swapper/109:35:201
2890995026,10cyclictest7850irq/56-dwc_otg_09:48:131
288999506,25cyclictest9-21ksoftirqd/007:35:220
315882490,2sleep20-21swapper/210:11:212
289199492,28cyclictest7950irq/56-dwc_otg_12:10:222
289199492,28cyclictest7950irq/56-dwc_otg_12:10:222
289099483,31cyclictest7750irq/56-dwc_otg12:00:271
2890994826,6cyclictest2198050wget10:00:201
288999482,31cyclictest0-21swapper/008:39:280
288999481,32cyclictest0-21swapper/007:45:090
288999481,30cyclictest11894-21wc07:35:040
288999481,30cyclictest11894-21wc07:35:040
289299477,26cyclictest0-21swapper/308:20:173
289299471,30cyclictest0-21swapper/307:55:183
289199479,23cyclictest29-21rcuc/209:25:162
2890994721,11cyclictest7850irq/56-dwc_otg_10:07:011
2889994729,5cyclictest7950irq/56-dwc_otg_08:00:200
2889994725,5cyclictest1601-21latency_hist11:35:030
19892470,2sleep20-21swapper/209:35:242
289299463,9cyclictest22383-21ssh12:00:163
289299463,14cyclictest0-21swapper/310:00:503
289199467,28cyclictest0-21swapper/209:55:092
288999462,32cyclictest0-21swapper/008:29:050
23362463,13sleep00-21swapper/007:06:260
103392460,3sleep2301ktimersoftd/211:05:262
289099459,13cyclictest7950irq/56-dwc_otg_10:25:451
2890994519,9cyclictest7850irq/56-dwc_otg_07:45:241
2890994515,11cyclictest7850irq/56-dwc_otg_11:10:261
2890994515,11cyclictest7750irq/56-dwc_otg07:15:241
289099451,18cyclictest6604-21ls11:40:241
289099450,23cyclictest3870-21ssh10:17:411
288999452,24cyclictest0-21swapper/008:45:300
288999451,24cyclictest13266-21grep09:05:310
289299447,23cyclictest0-21swapper/307:10:163
289199447,11cyclictest31-21ksoftirqd/210:34:262
289199445,9cyclictest7850irq/56-dwc_otg_07:40:132
289099445,15cyclictest0-21swapper/112:30:181
289099442,26cyclictest11001-21ntp_states09:00:271
2890994420,15cyclictest1434-21cat09:35:031
2890994415,17cyclictest7950irq/56-dwc_otg_10:35:201
2889994421,5cyclictest7950irq/56-dwc_otg_08:20:180
289199439,16cyclictest30318-21aten_rbpower_en11:30:172
289099436,5cyclictest0-21swapper/109:10:151
289099436,5cyclictest0-21swapper/109:10:151
2890994322,11cyclictest7750irq/56-dwc_otg10:46:311
2889994318,12cyclictest0-21swapper/007:40:260
289199428,19cyclictest0-21swapper/211:15:172
289199426,27cyclictest7950irq/56-dwc_otg_10:40:142
289199426,27cyclictest7950irq/56-dwc_otg_10:40:142
289199425,23cyclictest22017-21ssh11:20:112
289099428,12cyclictest9356-21/usr/sbin/munin07:25:321
2890994213,8cyclictest7850irq/56-dwc_otg_07:40:201
2890994213,14cyclictest221ktimersoftd/109:50:551
289099421,26cyclictest0-21swapper/108:45:161
288999421,26cyclictest10905-21missed_timers09:00:250
2890994119,5cyclictest17711-21perl07:50:221
2890994118,9cyclictest7950irq/56-dwc_otg_10:12:021
289299401,24cyclictest0-21swapper/308:40:163
289099407,24cyclictest0-21swapper/108:30:161
289099407,24cyclictest0-21swapper/108:25:201
289099405,17cyclictest7850irq/56-dwc_otg_10:55:341
289099402,5cyclictest0-21swapper/109:55:291
2890994023,5cyclictest7950irq/56-dwc_otg_07:55:291
2890994021,9cyclictest7750irq/56-dwc_otg12:08:541
2890994018,9cyclictest7750irq/56-dwc_otg07:20:321
2890994013,14cyclictest7750irq/56-dwc_otg11:18:491
2890994012,7cyclictest221ktimersoftd/108:00:321
244032400,2sleep10-21swapper/108:10:031
289199397,5cyclictest7950irq/56-dwc_otg_12:00:352
289099398,4cyclictest3977-21unixbench-2d07:10:301
2890993910,7cyclictest7850irq/56-dwc_otg_09:15:301
289299380,28cyclictest0-21swapper/307:25:213
289199386,7cyclictest0-21swapper/208:15:152
289199386,7cyclictest0-21swapper/208:15:152
2891993818,7cyclictest8910-21missed_timers07:25:252
2890993818,7cyclictest7750irq/56-dwc_otg09:05:231
2890993812,12cyclictest221ktimersoftd/111:35:041
289199377,15cyclictest7950irq/56-dwc_otg_08:50:282
2891993717,5cyclictest16358-21processes07:45:292
2890993718,10cyclictest28878-21grep08:20:271
2890993717,6cyclictest7750irq/56-dwc_otg08:39:261
2890993717,11cyclictest7750irq/56-dwc_otg10:44:141
2890993717,11cyclictest7750irq/56-dwc_otg10:44:141
2890993711,10cyclictest28018-21ssh09:26:401
289299362,6cyclictest0-21swapper/308:25:183
289199367,5cyclictest7950irq/56-dwc_otg_08:25:022
289199364,19cyclictest14902-21fschecks_count12:30:222
289299358,16cyclictest0-21swapper/309:10:173
289299358,16cyclictest0-21swapper/309:10:173
289199358,6cyclictest7950irq/56-dwc_otg_09:00:322
289199357,13cyclictest7950irq/56-dwc_otg_10:15:182
289199355,5cyclictest7950irq/56-dwc_otg_08:35:252
2891993526,7cyclictest6313-21ssh09:40:232
2891993526,7cyclictest6313-21ssh09:40:232
289199352,16cyclictest7950irq/56-dwc_otg_09:10:042
2891993517,7cyclictest26635-21ssh10:45:322
289199351,10cyclictest7263-21sed11:40:332
289199345,6cyclictest7950irq/56-dwc_otg_10:00:272
289199344,5cyclictest7750irq/56-dwc_otg09:50:282
2891993410,11cyclictest7950irq/56-dwc_otg_10:05:282
289099340,17cyclictest9871-21rm08:57:461
10642349,13sleep20-21swapper/207:05:142
289199337,4cyclictest0-21swapper/209:45:352
289199335,5cyclictest7950irq/56-dwc_otg_11:35:282
289199333,6cyclictest7950irq/56-dwc_otg_09:15:232
289199332,7cyclictest17707-21fschecks_count10:35:212
289199331,18cyclictest7950irq/56-dwc_otg_08:05:312
289299329,20cyclictest0-21swapper/308:45:163
289299327,5cyclictest7950irq/56-dwc_otg_08:30:313
289299322,21cyclictest0-21swapper/308:00:293
289199327,13cyclictest7950irq/56-dwc_otg_08:02:462
289199327,11cyclictest31-21ksoftirqd/207:35:222
289199323,15cyclictest7950irq/56-dwc_otg_07:55:222
2891993217,4cyclictest7950irq/56-dwc_otg_07:10:232
289299316,22cyclictest12117-21taskset09:05:163
289299313,7cyclictest7950irq/56-dwc_otg_07:30:263
289299313,7cyclictest7950irq/56-dwc_otg_07:30:263
289199319,6cyclictest7950irq/56-dwc_otg_10:55:272
289199317,14cyclictest0-21swapper/207:30:172
289199317,14cyclictest0-21swapper/207:30:172
289199315,5cyclictest7950irq/56-dwc_otg_08:45:172
289199315,13cyclictest7950irq/56-dwc_otg_07:20:292
289199306,15cyclictest7950irq/56-dwc_otg_08:40:232
289199302,6cyclictest7750irq/56-dwc_otg07:50:182
289299294,6cyclictest7750irq/56-dwc_otg08:35:173
289299276,4cyclictest7950irq/56-dwc_otg_08:10:323
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional