You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 05:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sat Dec 20, 2025 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
146081212263,54sleep30-21swapper/319:05:433
40939421130,2sleep2421rcuc/223:15:182
14615528554,10sleep10-21swapper/119:06:361
14641228259,9sleep20-21swapper/219:09:542
14613928060,14sleep00-21swapper/019:06:230
2759042780,6sleep1275889-21idleruntime-cro21:09:591
146551993022,4cyclictest413213-21sleep23:20:003
146551992421,1cyclictest0-21swapper/320:05:003
146551992320,1cyclictest0-21swapper/319:55:173
146551992220,1cyclictest0-21swapper/319:35:003
146551992118,1cyclictest0-21swapper/300:10:263
14655199211,16cyclictest407662-21taskset23:11:433
146551992017,1cyclictest0-21swapper/321:30:233
4439032191,4sleep2443953-21ntpq23:45:212
14655199199,6cyclictest386727-21turbostat22:55:003
14655199196,6cyclictest323468-21taskset21:51:193
14655199192,11cyclictest392512-21taskset23:00:123
146551991916,1cyclictest0-21swapper/321:35:183
14655199191,13cyclictest0-21swapper/300:15:113
14655199190,15cyclictest486834-21taskset00:29:133
14655199188,8cyclictest376239-21turbostat22:49:553
14655199188,6cyclictest191178-21taskset19:46:413
14655199182,11cyclictest150822-21taskset19:10:193
146551991815,1cyclictest0-21swapper/323:45:173
146551991815,1cyclictest0-21swapper/323:35:243
146551991815,1cyclictest0-21swapper/323:25:143
146551991815,1cyclictest0-21swapper/321:10:173
146551991815,1cyclictest0-21swapper/321:05:133
146551991815,1cyclictest0-21swapper/321:00:153
146551991812,1cyclictest0-21swapper/322:25:153
14655199181,14cyclictest249203-21chrt20:40:363
14655199181,14cyclictest0-21swapper/320:15:143
14655199181,13cyclictest487180-21chrt00:30:143
14655199181,13cyclictest0-21swapper/300:35:113
14655199181,12cyclictest450145-21turbostat23:55:003
14655199180,14cyclictest0-21swapper/320:30:193
4694942173,9sleep2470050-21head00:10:182
3804172171,11sleep0380465-21ntpq22:45:180
3558652170,1sleep00-21swapper/022:25:150
14655199178,5cyclictest244210-21taskset20:39:443
14655199177,6cyclictest334230-21taskset22:04:493
14655199173,2cyclictest0-21swapper/320:20:153
146551991714,1cyclictest0-21swapper/323:20:243
146551991714,1cyclictest0-21swapper/319:40:163
146551991713,2cyclictest0-21swapper/322:40:233
146551991713,2cyclictest0-21swapper/321:25:143
146551991713,2cyclictest0-21swapper/319:30:143
146551991713,2cyclictest0-21swapper/319:30:003
14655199171,11cyclictest455720-21taskset00:00:123
14655199171,10cyclictest232503-21taskset20:25:173
14655199171,10cyclictest160119-21taskset19:20:133
14655199170,13cyclictest0-21swapper/321:55:203
2489612161,10sleep2249178-21users20:40:242
2344312161,10sleep0234526-21cut20:30:160
2169502161,10sleep1217010-21ld20:10:211
14655199161,9cyclictest366014-21taskset22:35:073
146551991613,1cyclictest0-21swapper/300:20:163
146551991612,2cyclictest0-21swapper/322:25:003
14655199161,13cyclictest0-21swapper/321:45:003
14655199161,12cyclictest0-21swapper/322:15:273
146551991611,2cyclictest0-21swapper/322:15:003
14655199161,12cyclictest0-21swapper/300:05:293
14655199161,11cyclictest0-21swapper/322:05:113
14655199161,10cyclictest455134-21taskset23:56:443
14655199161,10cyclictest391925-21taskset22:59:563
14655199161,10cyclictest391925-21taskset22:59:563
14655199161,10cyclictest365107-21taskset22:30:253
14655199161,10cyclictest260609-21taskset20:55:153
14655199161,10cyclictest212347-21taskset20:09:423
14655199161,10cyclictest196594-21taskset19:54:193
14655199160,14cyclictest0-21swapper/321:23:383
14655199160,13cyclictest0-21swapper/323:30:143
14655199160,13cyclictest0-21swapper/323:05:263
14655199160,13cyclictest0-21swapper/320:50:253
4859232151,4sleep2485945-21ntpq00:25:212
4658832151,9sleep0465901-21munin-run00:10:000
2437062151,10sleep2243870-21/usr/sbin/munin20:35:282
2221912151,10sleep1222245-21ld20:15:231
14655199151,9cyclictest217590-21taskset20:11:073
146551991513,1cyclictest0-21swapper/321:45:453
146551991512,2cyclictest0-21swapper/321:15:153
146551991512,1cyclictest0-21swapper/320:49:073
3822372140,6sleep2382526-21kthreadcore22:50:142
3171242140,6sleep0317214-21ldconfig21:45:190
3129692140,6sleep10-21swapper/121:45:001
14655199140,11cyclictest0-21swapper/319:15:153
4244852131,7sleep0425638-21kthreadcore23:30:160
4034742131,7sleep0403941-21cut23:10:160
3565032131,7sleep2358197-21cut22:25:172
3465922131,7sleep0348353-21kthreadcore22:15:170
14655199131,9cyclictest438664-21taskset23:40:203
4239832121,7sleep1424199-21/usr/sbin/munin23:30:131
4140342121,7sleep2415005-21kthreadcore23:20:162
4081862121,5sleep1408205-21apt-get23:15:141
3860282121,7sleep0386199-21timerwakeupswit22:50:240
3648132121,5sleep2364867-21ntpq22:30:222
3555492121,5sleep1355568-21apt-get22:25:121
3400352121,7sleep2340078-21fschecks_count22:10:132
3137992121,3sleep1313837-21kthreadcore21:45:161
2292202121,6sleep0229277-21kthreadcore20:25:150
1584262121,5sleep2158501-21awk19:15:232
1474292121,8sleep20-21swapper/219:10:152
4962432110,1sleep20-21swapper/200:35:192
4661832111,4sleep1466429-21df_inode00:10:131
4187172110,4sleep2418771-21cpuspeed_turbos23:25:132
4175522111,6sleep0417654-21processes23:20:220
3647902110,1sleep10-21swapper/122:30:211
3394942111,5sleep0339510-21munin-run22:10:000
3292672111,5sleep0329286-21apt-get22:00:100
3283002111,6sleep0328521-21unixbench_multi21:55:250
2962142112,4sleep1296258-21ntpq21:25:201
2273672111,5sleep1227415-21ntpq20:20:181
2071602111,4sleep1207180-21munin-run20:04:591
1975272110,7sleep0199160-21cut19:55:180
1701512111,6sleep0170383-21expr19:30:130
4576032101,5sleep1459439-21tr00:00:191
4543772101,5sleep1454423-21ntpq23:55:171
4333852101,5sleep2433423-21ntpq23:35:192
4242002101,5sleep2424407-21grep23:30:142
3976882101,4sleep0397707-21apt-get23:05:110
3964102101,5sleep2396456-21ld23:00:192
3924752101,4sleep0391939-21apt-get23:00:020
3545962100,6sleep1354789-21cat22:20:231
3397562101,3sleep00-21swapper/022:10:120
3342452101,4sleep1334271-21/usr/sbin/munin22:04:591
3193102101,4sleep0319893-21cut21:50:150
3069872101,4sleep1307058-21find21:35:261
3013902104,2sleep1321rcuc/121:30:211
3013652101,4sleep0301389-21ntpq21:30:200
2834312100,6sleep1285225-21kthreadcore21:15:181
2822762101,5sleep0283429-21tr21:15:170
2765002101,4sleep1276703-21diskstats21:10:131
2499752101,4sleep2250100-21diskstats20:45:132
2123712101,5sleep0212384-21apt20:10:000
2079232101,4sleep1207994-21irqstats20:05:161
2027602101,5sleep2203877-21tr20:00:162
2007452100,6sleep2200812-21needreboot19:55:202
1809592101,4sleep1180978-21apt-get19:40:111
1797062101,4sleep1179757-21ntpq19:35:231
1523612101,5sleep1153010-21cat19:15:011
1467142101,4sleep1146733-21apt-get19:10:101
496363291,4sleep0496411-21ntpq00:35:210
489749290,1sleep00-21swapper/000:30:200
485738291,4sleep0485879-21ntpq00:25:190
461562291,4sleep2463116-21kthreadcore00:05:162
460955291,3sleep1460974-21apt-get00:05:121
450415291,4sleep2450394-21/usr/sbin/munin23:55:112
446835291,4sleep0448624-21grep23:50:160
429337290,4sleep1428900-21apt-get23:35:021
391242291,4sleep2391370-21cat22:55:272
391242291,4sleep2391370-21cat22:55:272
386980290,3sleep00-21swapper/022:55:130
386980290,3sleep00-21swapper/022:55:130
324582291,4sleep2325426-21cut21:55:152
317271291,3sleep2317326-21ntpq21:45:212
307957291,2sleep00-21swapper/021:40:090
296172291,4sleep0296213-21gcc21:25:190
287587291,5sleep10-21swapper/121:20:151
281995291,5sleep2282172-21sed21:15:142
266078291,4sleep2266383-21cut21:00:162
255575291,4sleep2255916-21kthreadcore20:50:142
249267290,1sleep10-21swapper/120:41:241
224018291,4sleep2225199-21kthreadcore20:20:152
223737291,4sleep0223916-21tail20:20:140
206509291,4sleep0206772-21awk20:00:260
186771290,1sleep00-21swapper/019:45:160
180675291,4sleep2180869-21wc19:39:592
175665291,3sleep0175685-21apt-get19:35:140
165389291,4sleep0165649-21cut19:25:160
164891291,4sleep1165137-21expr19:25:121
487158281,2sleep20-21swapper/200:30:142
397427281,2sleep1397453-21munin-run23:04:591
385850281,3sleep1385977-21systemctl22:50:211
360809281,2sleep00-21swapper/022:30:120
329542281,4sleep2329569-21grep22:00:112
319895281,3sleep1321579-21grep21:50:161
302384280,1sleep20-21swapper/221:34:472
291925280,1sleep20-21swapper/221:24:452
275870281,3sleep0275898-21grep21:10:000
270829281,3sleep2270987-21cstates21:05:122
261020281,3sleep1262690-21cut20:55:171
260584281,3sleep2260608-21/usr/sbin/munin20:55:152
260310281,2sleep00-21swapper/020:55:120
249809281,2sleep0249828-21apt-get20:45:110
240131281,3sleep0241806-21kthreadcore20:35:170
218159281,2sleep00-21swapper/020:15:130
186469281,3sleep1186488-21apt-get19:45:121
496650271,2sleep1496813-21timerwakeupswit00:35:261
481335270,1sleep20-21swapper/200:20:562
477240271,2sleep0476020-21kworker/u16:0+events_power_efficient00:20:180
455698270,2sleep20-21swapper/200:00:122
444766270,1sleep00-21swapper/023:48:150
439509270,1sleep20-21swapper/223:43:012
434722270,1sleep00-21swapper/023:40:140
402424270,1sleep20-21swapper/223:06:582
398600270,3sleep1321rcuc/123:05:161
387863271,2sleep1389550-21tr22:55:181
387863271,2sleep1389550-21tr22:55:171
381159271,3sleep2381160-21sshd22:46:142
366593271,2sleep00-21swapper/022:35:160
340669271,2sleep10-21swapper/122:10:171
328826270,1sleep10-21swapper/121:57:591
303239271,2sleep00-21swapper/021:35:170
294333271,2sleep20-21swapper/221:25:172
291880270,1sleep00-21swapper/021:24:110
280449271,3sleep00-21swapper/021:10:200
254471270,5sleep10-21swapper/120:45:531
232955270,1sleep10-21swapper/120:25:251
228410270,1sleep20-21swapper/220:25:012
186128271,2sleep2185947-21apt-get19:45:012
181813271,2sleep2183481-21kthreadcore19:40:172
164407270,1sleep20-21swapper/219:22:252
164362270,1sleep00-21swapper/019:21:510
150584270,4sleep00-21swapper/019:10:170
481919260,1sleep10-21swapper/100:25:121
481393260,1sleep10-21swapper/100:21:391
476182260,1sleep20-21swapper/200:17:312
474700261,2sleep10-21swapper/100:15:181
466157261,2sleep00-21swapper/000:10:120
456276261,2sleep00-21swapper/000:00:180
454235260,1sleep00-21swapper/023:55:170
449908260,1sleep10-21swapper/123:52:101
445159261,2sleep20-21swapper/223:50:102
439318260,1sleep10-21swapper/123:40:351
418989261,2sleep00-21swapper/023:25:150
403943260,2sleep10-21swapper/123:10:171
371254260,1sleep20-21swapper/222:40:122
370088261,2sleep20-21swapper/222:35:232
367674261,2sleep10-21swapper/122:35:181
354310260,1sleep00-21swapper/022:20:180
350305260,4sleep20-21swapper/222:20:032
339142260,1sleep20-21swapper/222:05:332
323750260,3sleep20-21swapper/221:54:552
306289260,2sleep2306626-21strings21:35:192
270546260,2sleep10-21swapper/121:05:001
260028260,4sleep10-21swapper/120:54:461
255335261,2sleep00-21swapper/020:50:120
245119261,2sleep00-21swapper/020:40:140
239478261,2sleep10-21swapper/120:35:141
234528262,1sleep10-21swapper/120:30:171
213484261,2sleep00-21swapper/020:10:150
212876260,2sleep20-21swapper/220:10:022
207438261,2sleep20-21swapper/220:05:132
200959261,2sleep10-21swapper/119:55:231
196436260,1sleep20-21swapper/219:52:192
195751261,2sleep10-21swapper/119:50:221
193044261,2sleep00-21swapper/019:50:160
181263261,2sleep00-21swapper/019:40:150
174946260,1sleep10-21swapper/119:32:421
169759260,1sleep20-21swapper/219:28:372
164332260,1sleep10-21swapper/119:21:281
151744261,2sleep00-21swapper/019:15:010
490927250,1sleep10-21swapper/100:30:211
472913250,2sleep00-21swapper/000:15:170
443808250,1sleep10-21swapper/123:45:191
433483250,1sleep00-21swapper/023:35:200
419199250,1sleep10-21swapper/123:25:161
413457250,3sleep10-21swapper/123:20:071
405651250,1sleep20-21swapper/223:10:172
377071250,1sleep10-21swapper/122:45:151
375361250,2sleep1375484-21mailstats22:40:231
349112250,1sleep10-21swapper/122:15:221
345019250,3sleep20-21swapper/222:15:072
308235250,2sleep20-21swapper/221:40:122
280827250,1sleep20-21swapper/221:10:262
265837250,1sleep00-21swapper/021:00:140
236028250,1sleep20-21swapper/220:30:172
222246250,1sleep20-21swapper/220:15:242
174551250,1sleep20-21swapper/219:30:272
407920240,1sleep00-21swapper/023:15:010
371436240,1sleep00-21swapper/022:40:140
334495240,1sleep10-21swapper/122:05:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional