You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 09:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Wed Jul 08, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30047991142,65cyclictest0-21swapper/000:00:280
30047991081,3cyclictest0-21swapper/023:00:260
3004799946,58cyclictest0-21swapper/023:45:280
3004799871,60cyclictest23887-21rm22:05:290
3004799861,65cyclictest13313-21tr23:50:290
208622860,3sleep30-21swapper/321:24:063
30047998425,43cyclictest0-21swapper/021:30:260
3004799836,23cyclictest0-21swapper/000:35:200
47382820,5sleep13005299cyclictest20:55:581
3004799823,39cyclictest0-21swapper/021:35:260
3004799781,60cyclictest3615-21wget20:55:290
30047997720,43cyclictest23424-21fschecks_time20:20:320
3004799762,52cyclictest22422-21sed20:15:450
30047997615,6cyclictest25578-21ssh23:25:380
3004799761,54cyclictest0-21swapper/021:40:350
3004799761,54cyclictest0-21swapper/021:40:350
3004799761,53cyclictest0-21swapper/023:05:590
30047997610,45cyclictest14561-21unixbench_multi21:15:440
3004799759,51cyclictest18952-21taskset20:06:230
3004799741,52cyclictest5722-21proc_pri19:30:380
3004799741,52cyclictest26214-21awk20:25:420
3004799731,6cyclictest9650irq/80-mmc019:25:290
3004799731,6cyclictest9650irq/80-mmc019:25:290
3004799731,58cyclictest9978-21ssh21:49:190
3004799731,52cyclictest0-21swapper/023:17:120
3004799731,50cyclictest31877-21latency_hist22:15:150
3004799731,50cyclictest31877-21latency_hist22:15:150
3004799722,59cyclictest0-21swapper/023:14:240
3004799722,59cyclictest0-21swapper/023:14:240
3004799711,19cyclictest1459-21ssh00:15:250
3004799711,11cyclictest0-21swapper/022:44:020
78512700,1sleep00-21swapper/022:25:070
318212700,2sleep30-21swapper/319:15:143
3004799701,52cyclictest704-21ssh22:15:430
3004799701,51cyclictest0-21swapper/000:34:540
30047997014,4cyclictest0-21swapper/023:20:250
3004799701,19cyclictest0-21swapper/000:20:370
3004799701,11cyclictest0-21swapper/021:54:350
30047996921,33cyclictest7850irq/56-dwc_otg00:25:320
3004799691,34cyclictest0-21swapper/020:40:270
30047996912,4cyclictest17191-21/usr/sbin/munin22:35:340
30047996912,4cyclictest17191-21/usr/sbin/munin22:35:340
3004799686,29cyclictest15149-21ssh21:55:280
3004799682,45cyclictest1086-21latency_hist19:20:140
3004799681,29cyclictest0-21swapper/020:30:260
3004799678,12cyclictest8050irq/56-dwc_otg_19:25:140
30047996724,32cyclictest8050irq/56-dwc_otg_20:45:410
30047996717,37cyclictest727-21runrttasks21:25:020
30047996711,43cyclictest0-21swapper/020:50:280
30047996620,30cyclictest0-21swapper/023:58:470
3004799661,49cyclictest8844-21cut19:40:310
3004799661,15cyclictest0-21swapper/023:43:470
3005299657,39cyclictest16629-21sh23:15:301
3004799654,16cyclictest8514-21ssh22:25:300
3004799652,43cyclictest0-21swapper/022:49:160
3004799651,60cyclictest29733-21ssh22:50:380
3004799651,19cyclictest0-21swapper/022:57:330
3004799651,19cyclictest0-21swapper/022:57:330
175982650,2sleep20-21swapper/221:20:262
30047996428,10cyclictest29756-21sh00:10:280
30047996422,40cyclictest25799-21idleruntime-cro21:30:140
30047996415,35cyclictest0-21swapper/023:39:300
3004799641,51cyclictest0-21swapper/021:14:150
30047996414,40cyclictest29518-21latency_hist00:10:150
30047996411,41cyclictest0-21swapper/021:00:510
30047996410,3cyclictest0-21swapper/020:10:290
94692630,3sleep39453-21ssh21:10:333
314832630,2sleep20-21swapper/223:32:462
3004799631,32cyclictest0-21swapper/019:10:260
3004799612,16cyclictest0-21swapper/021:06:110
30047996115,31cyclictest0-21swapper/019:58:080
30047996113,4cyclictest7850irq/56-dwc_otg23:30:230
3004799611,17cyclictest0-21swapper/022:00:250
30052996011,14cyclictest727-21runrttasks19:20:251
30047996012,40cyclictest0-21swapper/020:36:420
30047996012,40cyclictest0-21swapper/020:36:420
3004799601,12cyclictest0-21swapper/019:40:140
3004799591,6cyclictest8050irq/56-dwc_otg_22:30:280
30047995914,7cyclictest0-21swapper/020:00:240
30047995813,4cyclictest7950irq/56-dwc_otg_19:50:140
3006399579,27cyclictest0-21swapper/323:05:293
30057995447,5cyclictest12682-21sh23:10:282
30057995447,5cyclictest12682-21sh23:10:282
30047995426,16cyclictest0-21swapper/019:50:260
263242530,2sleep20-21swapper/223:26:062
3006399498,30cyclictest0-21swapper/320:45:203
30052994922,6cyclictest23-21ksoftirqd/121:30:281
293682494,10sleep00-21swapper/019:05:440
293682494,10sleep00-21swapper/019:05:440
3005299482,4cyclictest8050irq/56-dwc_otg_20:25:231
233332480,3sleep330039-21cyclictest20:20:293
30063994715,16cyclictest0-21swapper/322:45:363
30057994723,11cyclictest8050irq/56-dwc_otg_23:50:142
3006399461,9cyclictest8050irq/56-dwc_otg_19:25:283
3006399461,9cyclictest8050irq/56-dwc_otg_19:25:283
30057994621,6cyclictest15540-21sh21:55:332
3005299461,6cyclictest24822-21rm22:05:401
30052994612,3cyclictest0-21swapper/123:55:261
3006399457,25cyclictest0-21swapper/300:05:273
3005799455,25cyclictest7950irq/56-dwc_otg_00:35:142
3005299457,26cyclictest0-21swapper/119:15:301
3005299455,26cyclictest25609-21ssh00:05:271
136122450,7sleep13005299cyclictest23:10:421
136122450,7sleep13005299cyclictest23:10:421
30052994423,7cyclictest1251250wget22:30:281
3006399438,27cyclictest0-21swapper/320:10:283
30063994313,6cyclictest17463-21proc_pri23:15:423
30057994322,15cyclictest7950irq/56-dwc_otg_23:20:262
3005299434,28cyclictest8050irq/56-dwc_otg_22:55:251
3005299434,28cyclictest8050irq/56-dwc_otg_22:55:251
3005299431,5cyclictest8050irq/56-dwc_otg_20:15:271
3006399428,19cyclictest0-21swapper/323:35:283
30063994211,4cyclictest0-21swapper/323:25:153
3006399420,18cyclictest0-21swapper/323:10:383
3006399420,18cyclictest0-21swapper/323:10:383
30057994229,11cyclictest301ktimersoftd/221:35:262
3005299429,23cyclictest0-21swapper/119:25:281
3005299429,23cyclictest0-21swapper/119:25:281
30063994119,5cyclictest10239-21if_eth000:25:343
30063994117,12cyclictest30506-21sh21:35:263
3005799411,29cyclictest17618-21ls20:05:152
3005299418,28cyclictest0-21swapper/120:10:281
3005299418,20cyclictest0-21swapper/123:35:281
3005299412,10cyclictest0-21swapper/120:05:261
122312410,1sleep10-21swapper/100:27:491
320872400,1sleep10-21swapper/100:12:351
30063994012,4cyclictest28802-21apt-config23:30:153
3005299406,25cyclictest0-21swapper/119:55:281
3005299404,22cyclictest8050irq/56-dwc_otg_20:45:291
3006399397,20cyclictest0-21swapper/320:55:283
3006399396,19cyclictest37-21rcuc/300:35:213
3006399391,21cyclictest0-21swapper/319:45:263
3005799397,25cyclictest0-21swapper/219:45:192
3005799392,18cyclictest7950irq/56-dwc_otg_00:22:582
3005799391,7cyclictest5717-21awk23:00:402
30057993917,12cyclictest29-21rcuc/220:25:282
3005799391,27cyclictest29631-21ls20:35:412
3005799391,27cyclictest29631-21ls20:35:412
3005299399,5cyclictest0-21swapper/121:10:271
3005299397,25cyclictest0-21swapper/119:50:271
3005299396,14cyclictest0-21swapper/123:04:471
30052993912,18cyclictest0-21swapper/120:45:141
30052993910,15cyclictest1995-21awk21:40:141
3006399381,25cyclictest27030-21ls20:30:303
3005799389,26cyclictest0-21swapper/220:45:202
3005799388,25cyclictest0-21swapper/220:40:272
3005799388,25cyclictest0-21swapper/219:20:272
3005799387,22cyclictest0-21swapper/219:50:272
30057993814,6cyclictest31-21ksoftirqd/200:40:152
3005299388,18cyclictest0-21swapper/123:05:291
3005299386,20cyclictest0-21swapper/100:35:211
90712370,10sleep1221ktimersoftd/121:47:431
30063993715,5cyclictest8008-21latency_hist22:25:143
30063993713,6cyclictest26645-21ssh21:30:343
30063993713,10cyclictest8047-21run-parts21:05:393
30063993712,6cyclictest29372-21fschecks_time22:50:333
3005799377,22cyclictest0-21swapper/219:35:282
3005799377,20cyclictest0-21swapper/223:50:282
3005799374,3cyclictest0-21swapper/222:55:272
3005799374,3cyclictest0-21swapper/222:55:272
3005799372,21cyclictest31-21ksoftirqd/220:17:332
3005799372,21cyclictest27725-21ls22:10:152
3005799371,8cyclictest30074-21timerandwakeup22:50:422
30052993712,14cyclictest127032chrt21:51:381
314782360,5sleep23005799cyclictest22:14:382
314782360,5sleep23005799cyclictest22:14:382
3005799367,25cyclictest0-21swapper/200:10:282
3005799366,17cyclictest8050irq/56-dwc_otg_23:15:542
30057993613,7cyclictest0-21swapper/223:35:332
3005299366,18cyclictest0-21swapper/121:15:291
3005299362,5cyclictest7950irq/56-dwc_otg_20:20:251
3005299361,6cyclictest23-21ksoftirqd/122:20:141
3005299361,6cyclictest16807-21df_inode22:35:291
3005299361,6cyclictest16807-21df_inode22:35:291
3006399355,20cyclictest13338-21ssh21:15:293
3006399354,7cyclictest7950irq/56-dwc_otg_22:10:363
3006399354,7cyclictest7950irq/56-dwc_otg_22:10:363
3006399351,5cyclictest2071-21ls20:50:313
3006399351,4cyclictest12116-21sed21:50:433
30063993514,4cyclictest14102-21/usr/sbin/munin23:50:413
30063993513,5cyclictest26768-21latency_hist20:30:153
3006399351,27cyclictest5995-21ls00:20:303
3005799357,23cyclictest0-21swapper/219:10:262
3005799354,8cyclictest7950irq/56-dwc_otg_20:20:322
3005799351,4cyclictest8050irq/56-dwc_otg_20:30:282
30057993511,6cyclictest8050irq/56-dwc_otg_00:30:152
3005299354,20cyclictest7950irq/56-dwc_otg_00:30:251
3005299353,5cyclictest0-21swapper/122:50:411
3005299352,5cyclictest0-21swapper/100:05:151
3005299351,5cyclictest22367-21runrttasks22:03:361
30052993512,7cyclictest9550irq/48-DMA15:26:471
3005299351,26cyclictest25135-21cut22:45:331
3005299351,20cyclictest0-21swapper/123:40:431
264912351,30sleep30-21swapper/322:08:243
3006399346,5cyclictest8050irq/56-dwc_otg_00:15:393
3006399345,4cyclictest0-21swapper/323:45:373
3006399345,22cyclictest8050irq/56-dwc_otg_23:04:513
3006399343,25cyclictest8050irq/56-dwc_otg_23:55:273
3006399343,18cyclictest37-21rcuc/322:40:253
30063993414,4cyclictest8593-21basename19:40:273
30063993413,5cyclictest16130-21/usr/sbin/munin20:00:293
30063993412,6cyclictest16401-21head21:55:433
3005799347,22cyclictest0-21swapper/219:40:272
3005799344,15cyclictest7950irq/56-dwc_otg_20:05:372
3005799341,4cyclictest7950irq/56-dwc_otg_19:30:272
3005799341,25cyclictest7392-21cut21:05:302
3005799341,24cyclictest2561-21ssh00:15:422
3005799341,21cyclictest8958-21cpuspeed23:05:302
30057993410,4cyclictest7850irq/56-dwc_otg21:25:442
3005299348,5cyclictest0-21swapper/122:10:281
3005299348,5cyclictest0-21swapper/122:10:281
3005299347,13cyclictest8050irq/56-dwc_otg_23:45:441
3005299343,4cyclictest8050irq/56-dwc_otg_21:55:281
30052993414,11cyclictest8706-21diskmemload21:27:411
30052993413,12cyclictest0-21swapper/123:30:381
30052993412,15cyclictest0-21swapper/123:22:221
3006399337,5cyclictest7950irq/56-dwc_otg_20:20:153
3006399337,10cyclictest8050irq/56-dwc_otg_23:00:153
3006399337,10cyclictest8050irq/56-dwc_otg_23:00:153
30063993328,3cyclictest0-21swapper/320:05:333
30063993315,5cyclictest14504-21if_eth019:55:333
3006399331,4cyclictest0-21swapper/300:03:053
30063993311,4cyclictest0-21swapper/322:25:323
30063993311,12cyclictest0-21swapper/322:34:023
3005799337,6cyclictest7850irq/56-dwc_otg21:40:292
3005799337,6cyclictest7850irq/56-dwc_otg21:40:292
30057993322,4cyclictest8050irq/56-dwc_otg_00:05:352
30057993315,6cyclictest23291-21cron22:05:142
3005799331,21cyclictest7850irq/56-dwc_otg21:51:032
3005299339,4cyclictest0-21swapper/122:23:421
3005299335,13cyclictest8050irq/56-dwc_otg_00:15:381
3005299334,5cyclictest12118-21idleruntime-cro22:30:141
3005299331,7cyclictest7850irq/56-dwc_otg20:50:311
3006399326,6cyclictest0-21swapper/323:35:143
3006399326,3cyclictest0-21swapper/322:01:373
30063993218,10cyclictest0-21swapper/322:35:273
30063993218,10cyclictest0-21swapper/322:35:273
30063993212,5cyclictest26062chrt19:22:353
30063993212,10cyclictest0-21swapper/300:10:233
3006399321,17cyclictest0-21swapper/320:40:263
3005799328,4cyclictest9650irq/80-mmc022:20:262
3005799324,5cyclictest7850irq/56-dwc_otg22:45:142
3005799324,13cyclictest0-21swapper/221:00:392
3005799323,14cyclictest8050irq/56-dwc_otg_21:15:412
3005799322,19cyclictest727-21runrttasks20:15:022
30057993214,9cyclictest20462-21latency_hist22:40:152
30057993214,9cyclictest20462-21latency_hist22:40:152
30057993214,14cyclictest29-21rcuc/223:55:252
3005799321,2cyclictest0-21swapper/220:50:272
3005799321,21cyclictest7554-21ssh21:45:402
3005799321,19cyclictest14954-21ls19:55:402
3005299324,4cyclictest21516-21cron21:25:141
3005299321,27cyclictest5875-21uniq00:20:291
3006399319,4cyclictest381ktimersoftd/319:15:303
3006399318,4cyclictest0-21swapper/322:15:443
3006399318,4cyclictest0-21swapper/321:40:243
3006399318,4cyclictest0-21swapper/321:40:243
3006399313,23cyclictest8050irq/56-dwc_otg_19:30:263
3005799318,5cyclictest7850irq/56-dwc_otg21:10:322
3005799317,14cyclictest0-21swapper/223:40:272
3005799315,5cyclictest25364-21latency_hist00:05:152
3005799314,18cyclictest24875-21ssh22:45:292
3005799313,5cyclictest27429-21unixbench_singl21:30:452
3005299317,5cyclictest7950irq/56-dwc_otg_19:45:401
3005299317,11cyclictest0-21swapper/123:27:101
3005299315,15cyclictest8050irq/56-dwc_otg_19:10:381
3005299311,21cyclictest0-21swapper/119:35:331
3005299311,18cyclictest3295-21ssh21:40:401
3005299311,18cyclictest3295-21ssh21:40:401
30052993111,5cyclictest0-21swapper/122:40:271
30052993110,11cyclictest17492-21modprobe20:05:151
3006399309,10cyclictest0-21swapper/321:29:173
3006399304,7cyclictest5129-21ssh23:40:293
30063993010,5cyclictest13891-21/usr/sbin/munin00:30:353
3005799304,4cyclictest7850irq/56-dwc_otg22:15:292
3005799301,4cyclictest8050irq/56-dwc_otg_22:30:272
3005799301,20cyclictest8837-21ssh22:25:352
3005299309,10cyclictest21-21rcuc/123:50:281
3005299301,5cyclictest6551-21apt19:35:141
30052993014,4cyclictest221ktimersoftd/120:35:341
30052993014,4cyclictest221ktimersoftd/120:35:341
30052993012,11cyclictest221ktimersoftd/120:31:221
3006399299,4cyclictest0-21swapper/321:45:373
3006399296,14cyclictest0-21swapper/320:35:283
3006399296,14cyclictest0-21swapper/320:35:283
3006399295,4cyclictest8050irq/56-dwc_otg_21:00:313
3005799294,5cyclictest7950irq/56-dwc_otg_20:55:492
3005799293,8cyclictest8050irq/56-dwc_otg_19:25:262
3005799293,8cyclictest8050irq/56-dwc_otg_19:25:262
3005299294,7cyclictest7878-21ps21:05:371
3005299292,6cyclictest7950irq/56-dwc_otg_21:00:311
294822294,17sleep1750-21snmpd19:06:221
294822294,17sleep1750-21snmpd19:06:221
3006399289,8cyclictest37-21rcuc/319:37:453
3005799282,13cyclictest0-21swapper/219:15:272
3006399271,17cyclictest0-21swapper/319:50:243
297052273,15sleep30-21swapper/319:09:073
297052273,15sleep30-21swapper/319:09:073
295372273,15sleep20-21swapper/219:07:042
295372273,15sleep20-21swapper/219:07:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional