You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-21 - 21:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Thu May 21, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17173296017,9582sleep213925-21kworker/2:107:13:092
17230264717,613sleep317261-21/usr/sbin/munin08:54:243
17230264717,613sleep317261-21/usr/sbin/munin08:54:243
22332256412,535sleep122418-21perl12:34:201
26384248213,452sleep326425-21ntpq07:39:263
131342422125,204sleep2363-21snmpd12:04:292
131342422125,204sleep2363-21snmpd12:04:292
25663232095,205sleep2225-21systemd-journal07:39:012
9173227914,239sleep29235-21fschecks_time08:29:202
287262274107,58sleep20-21swapper/211:14:022
19087225032,201sleep219216-21timerwakeupswit08:59:322
25678224614,215sleep325739-21unin-run07:39:023
2559223411,202sleep22570-21modprobe08:09:012
2469022299,207sleep224716-21cut09:19:022
22703221257,141sleep222734-21fschecks_count07:29:202
6382220917,38sleep20-21swapper/211:44:142
21038220122,158sleep00-21swapper/007:24:090
2823218312,79sleep22025-21/usr/sbin/munin09:49:352
8040217915,72sleep28120-21fschecks_time11:49:192
30584217914,142sleep230714-21unixbench_multi09:35:302
9214217624,133sleep09236-21cron11:54:010
27175217074,80sleep227178-21cron11:09:012
4960216413,133sleep25109-21latency_hist09:59:032
24482216115,124sleep124531-21/usr/sbin/munin07:35:121
17508215810,40sleep20-21swapper/207:14:062
29525215686,26sleep229309-21apt-get09:34:042
23016215412,129sleep023054-21grep07:29:260
1065215216,69sleep21193-21latency_hist08:04:042
2247214720,28sleep10-21swapper/111:30:001
11378214723,107sleep20-21swapper/208:34:332
14485214645,77sleep214525-21sort07:04:212
398021446,127sleep225962-1kworker/2:2H08:12:202
2066321448,65sleep20-21swapper/212:29:162
17131214211,115sleep117147-21grep08:54:221
17131214211,115sleep117147-21grep08:54:221
11808214215,99sleep211951-21head11:59:322
2061521388,19sleep20-21swapper/209:04:332
2061521388,19sleep20-21swapper/209:04:332
14668213826,95sleep114737-21latency07:04:241
15186213722,19sleep20-21swapper/212:10:422
28641213612,110sleep223758-21sshd07:47:092
19200213630,94sleep219259-21fschecks_count12:24:192
18204213613,21sleep30-21swapper/312:19:323
95721346,15sleep20-21swapper/209:44:302
25572213416,101sleep025701-21timerwakeupswit09:19:330
1571521347,53sleep215763-21ntpq08:49:272
5754213019,78sleep00-21swapper/008:19:040
32018212742,32sleep232172-21gpgv11:24:032
31963212719,85sleep332086-21latency_hist07:59:053
10044212611,97sleep210062-21apt-get10:14:142
7141212533,80sleep27190-21cat10:04:292
592212513,94sleep2717-21timerwakeupswit11:24:352
14767212427,73sleep30-21swapper/307:04:263
17556212215,18sleep20-21swapper/210:38:492
10755212117,79sleep10-21swapper/110:14:491
24449212031,77sleep224332-21/usr/sbin/munin10:59:202
12721211921,77sleep012753-21tune2fs12:04:190
12721211921,77sleep012753-21tune2fs12:04:190
31487211827,27sleep30-21swapper/309:39:183
22272211514,82sleep322296-21/usr/sbin/munin09:09:343
22272211514,82sleep322296-21/usr/sbin/munin09:09:343
1484521155,17sleep20-21swapper/208:46:482
1335921155,12sleep20-21swapper/208:42:362
10271211516,28sleep30-21swapper/311:54:323
4928211425,75sleep20-21swapper/211:39:172
29078211426,76sleep229074-21apt-get07:49:152
26447211310,89sleep226488-21grep07:39:292
1598321128,14sleep20-21swapper/212:14:182
2054221115,18sleep20-21swapper/207:21:002
18048211134,11sleep20-21swapper/208:58:032
18048211134,11sleep20-21swapper/208:58:032
17835211111,21sleep20-21swapper/212:19:232
9541210915,43sleep29605-21dpkg10:14:012
7026210915,79sleep37028-21sshd08:21:093
31879210933,10sleep20-21swapper/207:59:002
23554210912,15sleep00-21swapper/012:39:030
1750210923,15sleep20-21swapper/211:29:322
7807210817,52sleep27882-21latency08:24:232
420721086,90sleep20-21swapper/211:34:492
4378210610,20sleep20-21swapper/208:14:142
22265210620,13sleep20-21swapper/209:09:332
22265210620,13sleep20-21swapper/209:09:332
17970210610,24sleep20-21swapper/210:39:172
1318021065,88sleep20-21swapper/210:24:112
5784210526,12sleep20-21swapper/209:59:322
22281210523,12sleep20-21swapper/212:34:162
21516210520,14sleep20-21swapper/207:24:272
13944210445,31sleep30-21swapper/312:09:023
13944210445,31sleep30-21swapper/312:09:023
1296821048,81sleep213081-21latency_hist10:24:032
31639210340,20sleep20-21swapper/209:39:212
25858210320,12sleep20-21swapper/209:19:392
20862210333,39sleep20-21swapper/210:48:012
11264210311,77sleep311327-21if_eth011:59:213
27660210219,10sleep20-21swapper/209:27:542
15208210222,13sleep20-21swapper/210:29:282
14800210215,70sleep114890-21sed12:09:321
14236210212,73sleep314298-21/usr/sbin/munin08:44:313
21292210120,14sleep20-21swapper/210:49:192
29086210017,15sleep20-21swapper/211:14:182
20482210022,55sleep020623-21unixbench_singl10:45:210
294329912,20sleep00-21swapper/008:09:230
2036529911,69sleep320377-21needreboot09:04:263
2036529911,69sleep320377-21needreboot09:04:263
616729820,54sleep00-21swapper/011:44:040
1781829718,60sleep317907-21munin-node07:14:113
2943329520,58sleep129522-21latency_hist09:34:031
2132529430,46sleep121412-21fschecks_time10:49:221
1662629412,68sleep316770-21latency_hist08:54:043
1230629422,53sleep112413-21fschecks_time08:39:201
94762937,26sleep20-21swapper/211:54:112
2306029321,52sleep123090-21idleruntime-cro09:14:011
2306029321,52sleep123090-21idleruntime-cro09:14:011
762429217,59sleep37684-21/usr/sbin/munin08:24:203
2471729113,60sleep324865-21wc09:19:033
2551428817,56sleep325570-21sed09:19:323
1009428816,28sleep00-21swapper/010:14:320
8762879,13sleep10-21swapper/111:24:551
3225728712,47sleep10-21swapper/109:39:321
2338828741,22sleep20-21swapper/210:54:312
1735928710,64sleep017419-21mailstats08:54:290
1735928710,64sleep017419-21mailstats08:54:290
2934328612,17sleep00-21swapper/009:34:020
2349128520,42sleep323638-21grep10:54:323
832128421,46sleep38352-21/usr/sbin/munin11:49:243
804428417,48sleep08152-21latency_hist10:09:030
2685928413,21sleep00-21swapper/009:24:250
23892849,61sleep00-21swapper/008:06:490
1964828420,40sleep319163-21apt-get10:44:113
3235928321,44sleep331885-21apt-get11:24:123
385628221,38sleep13251-21/usr/sbin/munin11:34:321
139782820,7sleep30-21swapper/310:24:313
359728116,45sleep33570-21/usr/sbin/munin09:54:163
2755228028,26sleep00-21swapper/011:09:190
1659028015,45sleep316693-21/usr/sbin/munin07:09:313
1239728014,51sleep112511-21cat12:04:031
95227910,56sleep00-21swapper/011:25:510
3131027920,28sleep00-21swapper/007:54:340
280402796,16sleep30-21swapper/309:29:173
2401027924,31sleep124094-21dpkg07:34:031
2175027925,30sleep30-21swapper/307:24:333
477627819,30sleep30-21swapper/308:14:273
377827820,36sleep03854-21mailstats11:34:310
2132227816,19sleep00-21swapper/009:09:030
2132227816,19sleep00-21swapper/009:09:030
698927756,11sleep20-21swapper/208:20:402
3220127715,34sleep30-21swapper/307:59:173
1595027717,41sleep015454-21apt-get12:14:120
2444327619,39sleep124332-21/usr/sbin/munin10:59:191
1143927620,32sleep30-21swapper/310:19:033
2292727522,21sleep00-21swapper/010:54:210
2141327529,25sleep121612-21rm09:09:041
2141327529,25sleep121612-21rm09:09:041
567027421,33sleep15699-21munin-run08:19:011
3078627412,46sleep130822-21/usr/sbin/munin11:19:221
1192827415,28sleep00-21swapper/010:19:250
935627323,29sleep10-21swapper/108:29:231
2810227325,29sleep30-21swapper/307:44:313
1871327318,36sleep10-21swapper/108:59:231
2075927213,27sleep30-21swapper/312:29:213
1834427211,46sleep318350-21ntp_states10:39:263
1052027210,12sleep00-21swapper/011:55:550
330427120,31sleep00-21swapper/009:54:010
2523827113,43sleep025361-21timerwakeupswit07:36:450
1820927121,28sleep10-21swapper/107:14:251
1163527119,26sleep10-21swapper/110:19:131
3037127022,22sleep30-21swapper/309:35:273
2994127013,48sleep00-21swapper/011:14:440
2722827012,42sleep027311-21latency_hist11:09:030
157527022,25sleep10-21swapper/108:04:241
649626925,18sleep10-21swapper/110:04:031
40422696,11sleep30-21swapper/308:13:103
269826919,27sleep10-21swapper/109:49:341
2184626920,27sleep00-21swapper/010:49:330
1607126915,22sleep00-21swapper/007:09:190
2058126812,44sleep3363-21snmpd07:21:303
1951026818,27sleep00-21swapper/012:24:250
1501826814,37sleep115040-21latency_hist08:49:011
1389326814,32sleep113895-21cron12:09:001
1389326814,32sleep113895-21cron12:09:001
3028526720,24sleep10-21swapper/111:19:011
2954626717,25sleep30-21swapper/307:49:323
916026610,15sleep00-21swapper/008:29:180
2607226614,34sleep126124-21cut07:39:201
14522669,11sleep00-21swapper/009:45:050
3241626511,32sleep132474-21/usr/sbin/munin07:59:231
1218326516,23sleep00-21swapper/008:39:110
9152648,11sleep00-21swapper/008:02:450
648226415,34sleep06495-21unin-run10:04:020
2384326417,22sleep10-21swapper/109:14:331
2276426422,25sleep30-21swapper/312:34:303
223126412,21sleep00-21swapper/011:29:590
1878826419,30sleep118800-21vmstat10:39:321
1063126418,28sleep310628-21apt-get08:34:133
2916926317,32sleep129176-21forks07:49:221
28562635,36sleep10-21swapper/108:09:201
2601126316,24sleep126105-21fschecks_time11:04:201
1640126319,24sleep30-21swapper/310:34:213
1598626318,28sleep116054-21/usr/sbin/munin07:09:161
1496726325,18sleep10-21swapper/110:29:231
1436926318,27sleep314427-21/usr/sbin/munin12:09:223
2753426219,22sleep10-21swapper/107:44:171
1489726214,30sleep014966-21/usr/sbin/munin10:29:220
647626114,22sleep00-21swapper/008:19:290
526826117,18sleep30-21swapper/309:59:213
499226111,26sleep10-21swapper/111:39:201
2484326112,27sleep30-21swapper/310:59:293
3063126020,24sleep10-21swapper/107:54:181
3035126014,22sleep10-21swapper/109:35:251
2779926018,27sleep127860-21latency_hist09:29:031
225826017,21sleep30-21swapper/309:49:233
2236626018,26sleep122375-21systemd-run07:29:031
2023226013,18sleep10-21swapper/110:45:181
167752600,7sleep00-21swapper/008:54:040
1643926012,21sleep30-21swapper/312:14:313
1090426015,21sleep10-21swapper/108:34:241
956825918,21sleep30-21swapper/308:29:293
884625916,18sleep10-21swapper/108:29:031
771625916,19sleep30-21swapper/311:49:033
2318525919,24sleep10-21swapper/110:54:261
1084025918,28sleep110937-21latency_hist11:59:031
865125815,26sleep18677-21netstat10:09:241
688625818,17sleep10-21swapper/110:04:231
3271425817,22sleep00-21swapper/009:44:040
2877025820,19sleep30-21swapper/311:14:033
2658525814,19sleep30-21swapper/309:24:203
2623525818,25sleep126246-21munin-run09:24:021
2566625815,26sleep025675-21munin-run11:04:020
2298625818,23sleep30-21swapper/307:29:253
1764625819,19sleep00-21swapper/010:39:030
1071125815,27sleep010718-21diskstats08:34:180
840025718,17sleep10-21swapper/111:49:261
76172575,17sleep10-21swapper/111:48:251
3036425718,17sleep30-21swapper/311:19:033
2753325716,25sleep027527-21apt-get07:44:160
1846525711,23sleep10-21swapper/112:19:341
79072568,11sleep00-21swapper/008:24:250
2412825615,19sleep00-21swapper/009:14:360
1987925617,19sleep00-21swapper/007:19:250
1921925618,19sleep30-21swapper/308:59:333
1595125610,20sleep10-21swapper/112:14:131
960725520,19sleep30-21swapper/310:14:023
653925519,16sleep10-21swapper/108:19:301
46072558,38sleep00-21swapper/009:55:420
2744325514,21sleep127461-21apt-get11:09:121
2197425517,19sleep00-21swapper/012:34:020
191382556,37sleep00-21swapper/010:43:510
175825518,23sleep1307-21in:imuxsock09:49:011
1572425515,25sleep015797-21cat07:09:030
1351425512,27sleep313518-21pluginstate08:44:023
1320525515,24sleep013201-21apt-get10:24:130
664425415,20sleep00-21swapper/011:44:240
3043225416,15sleep00-21swapper/009:35:280
2606625412,25sleep026071-21diskstats07:39:190
2180925416,22sleep021793-21apt-get09:09:190
2180925416,22sleep021793-21apt-get09:09:190
2147025412,24sleep321504-21irqstats10:49:243
1911025415,23sleep319107-21apt-get12:24:123
1636825412,19sleep10-21swapper/110:34:191
673825313,25sleep36732-21apt-get10:04:173
3186025314,23sleep01-21systemd11:24:010
172725315,23sleep31714-21apt-get11:29:173
154552539,16sleep30-21swapper/310:29:333
485625212,24sleep34852-21apt-get11:39:143
359425211,24sleep13588-21apt-get09:54:141
2645025217,19sleep30-21swapper/311:04:303
1954625215,21sleep119595-21df07:19:181
1865925214,21sleep018712-21irqstats08:59:220
970525112,23sleep09910-21cat10:14:030
364425115,16sleep30-21swapper/311:34:263
2109725113,23sleep121116-21ntp_states12:29:271
1768125114,16sleep00-21swapper/012:19:210
1107925112,24sleep011074-21apt-get11:59:140
3120825018,16sleep30-21swapper/311:19:323
1801525011,22sleep018018-21df_inode07:14:200
1418525013,21sleep014180-21apt-get12:09:150
1383525012,15sleep00-21swapper/008:44:200
130525015,21sleep31372-21df08:04:183
1616924711,17sleep10-21swapper/108:49:341
13102479,18sleep30-21swapper/309:44:353
1938024611,20sleep119385-21irqstats12:24:221
1206324611,18sleep30-21swapper/310:19:283
2916124514,15sleep00-21swapper/007:49:210
1320624511,17sleep10-21swapper/110:24:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional