You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 16:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Jun 01, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
913424600,4sleep10-21swapper/107:05:261
2801921610,4sleep11053899cyclictest09:55:371
10532991041,56cyclictest0-21swapper/009:05:200
64972970,5sleep01053299cyclictest10:49:060
157722800,2sleep30-21swapper/307:25:073
64092780,3sleep10-21swapper/108:25:351
82582770,4sleep18249-21ssh11:30:221
1053299772,54cyclictest0-21swapper/011:30:190
99752750,2sleep10-21swapper/110:52:541
1053299753,25cyclictest0-21swapper/007:30:200
1053299732,51cyclictest0-21swapper/012:00:190
1053299701,55cyclictest3712-21timerandwakeup10:05:330
10532996920,27cyclictest0-21swapper/009:50:350
1053299691,58cyclictest0-21swapper/009:30:180
294932680,6sleep31054999cyclictest10:36:573
1053299687,49cyclictest0-21swapper/012:25:350
1053299687,49cyclictest0-21swapper/012:25:350
10532996842,7cyclictest28529-21latency_hist11:55:070
224022660,5sleep21054599cyclictest11:46:332
10532996629,22cyclictest19904-21ssh11:05:180
154552651,5sleep015463-21rm11:39:000
1053899653,18cyclictest7950irq/56-dwc_otg_07:30:201
10538996510,16cyclictest4776-21ssh12:05:191
26602640,2sleep30-21swapper/311:23:153
1053299631,47cyclictest0-21swapper/009:00:240
10532996246,5cyclictest14306-21grep12:15:350
1053299621,50cyclictest15596-21diskmemload11:00:080
1053299621,46cyclictest0-21swapper/008:45:360
10532996214,17cyclictest12201-21latency_hist07:15:070
10532996129,5cyclictest15566-21unixbench_multi08:50:340
1053299612,49cyclictest0-21swapper/010:27:190
1053299612,47cyclictest0-21swapper/009:15:320
1053299612,46cyclictest0-21swapper/012:20:350
1053299611,45cyclictest1133-21runrttasks09:45:240
1053299611,40cyclictest4910-21aten_rbpower_vo12:05:220
10532996045,7cyclictest8050irq/56-dwc_otg_09:10:240
10532996039,6cyclictest7950irq/56-dwc_otg_11:20:080
10532996039,6cyclictest7950irq/56-dwc_otg_11:20:080
1053299601,46cyclictest24470-21grep11:50:080
45322590,7sleep33245-21/usr/sbin/munin10:45:353
1053299591,46cyclictest0-21swapper/009:56:160
1053299591,45cyclictest17375-21switchtime11:40:320
1053299586,48cyclictest30811-21ssh12:35:350
1053299581,44cyclictest0-21swapper/009:25:190
129832570,3sleep10-21swapper/112:15:151
10532995739,6cyclictest24422-21ssh11:10:250
1053299572,43cyclictest0-21swapper/008:25:350
1053299562,42cyclictest31435-21timerwakeupswit08:05:370
1053299562,42cyclictest0-21swapper/010:17:500
1053299562,41cyclictest0-21swapper/010:00:140
1053299562,41cyclictest0-21swapper/009:20:200
1053299562,39cyclictest0-21swapper/009:40:190
1053299562,39cyclictest0-21swapper/009:40:190
1053299562,37cyclictest0-21swapper/010:52:100
1053299561,36cyclictest0-21swapper/010:20:250
1053299561,30cyclictest7334-21grep10:10:240
1053299561,30cyclictest7334-21grep10:10:240
195892550,23sleep21054599cyclictest11:05:072
10532995535,6cyclictest9450irq/48-DMA10:20:150
1053299542,40cyclictest0-21swapper/010:36:450
92592530,1sleep10-21swapper/112:10:241
1053299538,9cyclictest16858-21ls07:25:330
1053299532,40cyclictest0-21swapper/010:30:290
1053299532,37cyclictest0-21swapper/008:16:110
1053299531,45cyclictest0-21swapper/011:55:330
1053299531,45cyclictest0-21swapper/011:20:220
1053299531,42cyclictest0-21swapper/010:40:260
1053299524,9cyclictest0-21swapper/007:50:340
10532995235,7cyclictest8050irq/56-dwc_otg_08:55:300
10532995227,14cyclictest0-21swapper/009:40:070
10532995224,16cyclictest27355-21grep07:55:270
1053299522,39cyclictest0-21swapper/011:00:090
1053299521,42cyclictest14818-21missed_timers07:20:280
10538995124,4cyclictest27764-21fschecks_time10:35:251
1053299512,36cyclictest0-21swapper/007:47:240
1053299512,36cyclictest0-21swapper/007:47:240
1053299511,30cyclictest0-21swapper/012:10:210
1053299511,27cyclictest22165-21ps07:40:320
1053299502,33cyclictest0-21swapper/008:10:270
1053299501,41cyclictest0-21swapper/012:30:180
1053299501,39cyclictest0-21swapper/008:30:270
1053899491,25cyclictest0-21swapper/107:50:081
1053899491,25cyclictest0-21swapper/107:50:081
10532994917,17cyclictest30192-21modprobe08:05:070
1053299491,37cyclictest0-21swapper/008:35:190
77312480,2sleep30-21swapper/309:31:073
1054599482,8cyclictest8050irq/56-dwc_otg_10:20:182
1053299481,40cyclictest0-21swapper/007:16:450
315382470,24sleep18050irq/56-dwc_otg_11:58:081
313582460,1sleep30-21swapper/308:05:363
160672462,37sleep30-21swapper/311:40:073
1053899467,25cyclictest0-21swapper/109:30:201
10532994634,5cyclictest7950irq/56-dwc_otg_08:22:540
10532994634,5cyclictest7950irq/56-dwc_otg_08:22:540
1053299462,30cyclictest0-21swapper/008:40:100
101942462,36sleep00-21swapper/007:09:020
248492450,2sleep30-21swapper/311:10:313
1054999452,34cyclictest31556-21ssh10:40:173
10545994532,8cyclictest301ktimersoftd/211:05:192
1054599452,7cyclictest0-21swapper/207:55:202
1054599451,6cyclictest8050irq/56-dwc_otg_11:10:182
1053899457,24cyclictest0-21swapper/108:10:201
1053299454,25cyclictest0-21swapper/007:35:200
1054999442,6cyclictest26846-21sh09:55:203
1054599444,8cyclictest0-21swapper/207:35:202
91892430,6sleep2301ktimersoftd/207:05:272
1054599437,22cyclictest0-21swapper/212:10:202
1054599431,6cyclictest0-21swapper/212:05:192
10538994321,12cyclictest2633950ssh10:33:451
101672430,2sleep20-21swapper/210:53:152
261522420,1sleep10-21swapper/111:50:511
1053899421,27cyclictest0-21swapper/107:35:211
10538994123,9cyclictest5294-21ssh10:46:541
1054999405,22cyclictest4709-21ssh12:05:133
10549994017,4cyclictest10097-21sh10:14:183
10549994017,4cyclictest10097-21sh10:14:183
10538994019,4cyclictest1133-21runrttasks11:28:241
1053899401,30cyclictest0-21swapper/107:50:211
1053899401,25cyclictest0-21swapper/108:40:131
247332390,3sleep224730-21wget11:50:202
1054999398,25cyclictest0-21swapper/308:15:123
1054999397,25cyclictest0-21swapper/308:20:203
1054999397,25cyclictest0-21swapper/308:20:203
1054999397,22cyclictest30314-21ssh09:20:193
10538993918,11cyclictest175850ssh11:21:371
10538993914,13cyclictest0-21swapper/111:15:181
10538993914,13cyclictest0-21swapper/111:15:181
1053899391,29cyclictest14661-21ls08:50:211
1053899391,17cyclictest21544-21diskmemload09:55:071
1054999388,5cyclictest7850irq/56-dwc_otg11:50:353
1054599389,4cyclictest0-21swapper/209:27:142
1054599384,4cyclictest0-21swapper/210:00:192
1053899389,6cyclictest7950irq/56-dwc_otg_12:25:071
1053899382,24cyclictest0-21swapper/111:05:181
10538993822,10cyclictest531850ssh10:08:031
10538993818,4cyclictest2039-21/usr/sbin/munin09:25:221
10538993813,13cyclictest18648-21sh10:24:491
10538993811,4cyclictest0-21swapper/110:25:291
1054999372,10cyclictest523-21rs:main3
1054999371,6cyclictest15347-21latency_hist11:00:073
10549993716,7cyclictest7850irq/56-dwc_otg09:25:223
10549993712,3cyclictest0-21swapper/311:00:343
10549993711,4cyclictest39-21ksoftirqd/312:20:213
1054599378,13cyclictest8050irq/56-dwc_otg_12:40:072
10545993714,7cyclictest0-21swapper/209:45:292
1054599371,30cyclictest12681-21ls11:35:232
10538993713,4cyclictest1133-21runrttasks11:36:081
10538993710,11cyclictest0-21swapper/112:25:211
10538993710,11cyclictest0-21swapper/112:25:211
33622360,6sleep31054999cyclictest12:03:213
308822360,6sleep11053899cyclictest10:00:111
301762361,4sleep20-21swapper/208:05:002
1054999365,2cyclictest0-21swapper/312:30:203
1054999361,30cyclictest21310-21ssh12:25:203
1054999361,30cyclictest21310-21ssh12:25:203
1054999361,25cyclictest16900-21ls08:55:283
10545993623,5cyclictest301ktimersoftd/210:10:162
10545993623,5cyclictest301ktimersoftd/210:10:162
10545993618,8cyclictest21407-21/usr/sbin/munin07:40:232
1053899361,8cyclictest0-21swapper/110:40:191
10538993618,5cyclictest18436-21fschecks_count09:00:241
10538993618,12cyclictest2223450rm11:46:151
10538993617,12cyclictest253302chrt12:30:091
10538993615,10cyclictest22116-21ntp_states07:40:311
10538993614,5cyclictest1871-21sh12:00:381
1053899361,24cyclictest141-21systemd-journal11:44:301
1053899361,22cyclictest0-21swapper/108:15:121
1053899361,21cyclictest27002-21ls07:55:211
10538993611,13cyclictest0-21swapper/109:10:231
316592350,1sleep20-21swapper/211:58:212
1054999353,18cyclictest7950irq/56-dwc_otg_09:55:073
1054999352,7cyclictest1080050ssh09:35:283
10549993525,6cyclictest381ktimersoftd/310:15:193
10549993524,6cyclictest381ktimersoftd/309:15:183
1054999352,17cyclictest7950irq/56-dwc_otg_10:25:333
1054999351,20cyclictest13180-21latency08:45:293
1054599357,11cyclictest0-21swapper/211:31:042
10545993520,4cyclictest17536-21fschecks_time12:20:252
1054599351,23cyclictest0-21swapper/208:25:072
1054599351,23cyclictest0-21swapper/208:25:072
10545993511,16cyclictest0-21swapper/208:50:172
10545993510,4cyclictest0-21swapper/209:44:472
10545993510,4cyclictest0-21swapper/209:44:472
10538993520,5cyclictest24421-21fschecks_time11:10:251
10538993515,4cyclictest5172-1kworker/u9:010:15:291
10538993514,12cyclictest14600-21sh09:40:211
10538993514,12cyclictest14600-21sh09:40:211
1054999349,4cyclictest7850irq/56-dwc_otg11:25:283
1054999344,4cyclictest7850irq/56-dwc_otg08:40:243
1054999341,20cyclictest0-21swapper/308:35:193
1054599345,4cyclictest8050irq/56-dwc_otg_08:30:332
1054599345,20cyclictest6141-21ssh09:30:202
1054599341,5cyclictest28539-21fschecks_time11:15:242
1054599341,5cyclictest28539-21fschecks_time11:15:242
1054599341,30cyclictest22363-21chrt12:25:352
1054599341,30cyclictest22363-21chrt12:25:352
1053899341,4cyclictest0-21swapper/109:20:021
1053899341,24cyclictest1089-21ls12:40:081
1053899341,15cyclictest30382-21wc08:05:071
10538993410,9cyclictest0-21swapper/111:00:281
1054999339,5cyclictest21396-21kworker/3:209:10:323
1054999332,7cyclictest39-21ksoftirqd/311:55:253
1054999332,4cyclictest8162-21apt-get11:30:193
10549993317,4cyclictest17261-21grep11:40:313
1054999331,3cyclictest0-21swapper/311:45:173
1054999331,2cyclictest0-21swapper/309:40:193
1054999331,2cyclictest0-21swapper/309:40:193
1054999331,24cyclictest9076-21cpuspeed12:10:213
1054999331,18cyclictest23484-21cut07:45:243
1054999331,18cyclictest23484-21cut07:45:243
1054599336,6cyclictest7950irq/56-dwc_otg_08:15:252
1054599334,5cyclictest7850irq/56-dwc_otg12:00:242
10545993313,5cyclictest4146-21/usr/sbin/munin11:25:222
1053899338,6cyclictest7850irq/56-dwc_otg09:20:231
1053899332,7cyclictest17230-21ls08:55:331
1053899332,14cyclictest0-21swapper/110:12:271
1053899332,14cyclictest0-21swapper/110:12:271
10538993316,5cyclictest21232-21ssh09:48:051
102072333,23sleep30-21swapper/307:09:133
1054999329,17cyclictest39-21ksoftirqd/307:50:183
1054999326,9cyclictest0-21swapper/310:03:103
1054999321,3cyclictest0-21swapper/310:08:493
1054999321,24cyclictest21905-21latency07:40:273
1054999321,18cyclictest0-21swapper/309:00:243
10549993211,4cyclictest13127-21aten_rbpower_vo12:15:203
1054599327,6cyclictest0-21swapper/210:25:252
1054599326,14cyclictest0-21swapper/208:10:202
1054599325,10cyclictest7950irq/56-dwc_otg_12:30:332
1054599324,3cyclictest0-21swapper/209:23:222
1054599323,13cyclictest0-21swapper/210:59:112
1054599322,6cyclictest8050irq/56-dwc_otg_07:45:242
1054599322,6cyclictest8050irq/56-dwc_otg_07:45:242
1054599322,16cyclictest0-21swapper/210:09:572
1054599321,7cyclictest26797-21missed_timers09:15:282
1054599321,16cyclictest12020-21grep08:40:352
1053899324,5cyclictest9575-21chrt08:35:291
1054999316,5cyclictest7850irq/56-dwc_otg07:25:243
1054999316,15cyclictest0-21swapper/310:50:203
1054999315,5cyclictest39-21ksoftirqd/310:30:293
10549993122,6cyclictest23754-21latency_hist11:10:073
1054999311,6cyclictest7950irq/56-dwc_otg_10:20:223
1054599313,4cyclictest1133-21runrttasks10:36:512
1054599311,7cyclictest16847-21idleruntime-cro12:20:072
1054599311,6cyclictest20433-21latency_hist11:45:082
1054599311,19cyclictest7850irq/56-dwc_otg09:55:352
10545993111,3cyclictest0-21swapper/209:10:182
1054599311,10cyclictest0-21swapper/211:24:432
1053899315,14cyclictest15060-21sh10:59:381
1053899311,6cyclictest12505-21ssh09:37:151
10538993111,4cyclictest16797-21processes07:25:321
1054999306,14cyclictest0-21swapper/308:25:213
1054999303,4cyclictest7850irq/56-dwc_otg07:18:023
1054999301,5cyclictest29664-21aten_rbpower_po12:35:213
1054599307,5cyclictest8050irq/56-dwc_otg_07:10:242
1054599306,5cyclictest23327-21ssh10:30:212
1054599305,4cyclictest7850irq/56-dwc_otg10:15:252
1054599303,5cyclictest7850irq/56-dwc_otg07:20:342
1054599303,4cyclictest7850irq/56-dwc_otg09:50:272
1054599302,6cyclictest7850irq/56-dwc_otg09:00:292
10545993016,4cyclictest8050irq/56-dwc_otg_08:25:262
1054599301,5cyclictest7850irq/56-dwc_otg08:55:362
1054599301,3cyclictest7850irq/56-dwc_otg10:40:192
1054599301,19cyclictest13990-21ls07:20:072
10538993022,5cyclictest1133-21runrttasks08:05:371
1053899301,20cyclictest0-21swapper/108:30:311
1053899301,16cyclictest4693-21sed08:20:351
1053899301,16cyclictest4693-21sed08:20:351
1054999295,4cyclictest7950irq/56-dwc_otg_08:00:343
1054999291,20cyclictest357-21ls08:10:283
1054599296,10cyclictest8050irq/56-dwc_otg_08:35:252
1054599292,6cyclictest17837-21wget07:30:202
1054599291,5cyclictest7950irq/56-dwc_otg_09:35:222
1054599291,4cyclictest7850irq/56-dwc_otg10:45:272
1053899296,5cyclictest9450irq/48-DMA10:20:151
1053899292,15cyclictest14263-21aten_rbpower_po07:20:201
1053899291,4cyclictest13572-21ls08:45:351
1054999287,15cyclictest0-21swapper/309:05:203
1054999282,14cyclictest39-21ksoftirqd/307:55:253
10549992818,7cyclictest10862-21fschecks_count07:10:243
1054999281,7cyclictest28435-21ssh11:15:233
1054999281,7cyclictest28435-21ssh11:15:233
1054999281,16cyclictest7850irq/56-dwc_otg07:38:553
1054999281,13cyclictest7950irq/56-dwc_otg_08:50:323
1054999281,13cyclictest7850irq/56-dwc_otg08:32:493
1054599281,11cyclictest8050irq/56-dwc_otg_08:50:072
1053899281,6cyclictest10578-21apt-get07:10:181
1053899281,18cyclictest0-21swapper/107:18:571
10549992720,5cyclictest39-21ksoftirqd/307:30:283
1054599278,8cyclictest25953-21ls07:50:342
1054599276,10cyclictest8050irq/56-dwc_otg_09:05:302
1054599271,23cyclictest31429-21timerwakeupswit08:05:372
1054599256,4cyclictest7850irq/56-dwc_otg07:25:332
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional