You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 21:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Dec 29, 2025 12:44:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24974266366,577sleep225004-21idleruntime-cro08:19:032
753324549,404sleep27576-21awk10:49:262
376923760,119sleep23612-21cyclictest08:54:232
4531229843,173sleep24572-21grep10:39:332
962322767,253sleep29710-21tune2fs07:29:232
11525227420,238sleep111574-21unin-run11:04:031
20990225415,222sleep121053-21cpuspeed09:49:201
20990225415,222sleep121053-21cpuspeed09:49:201
826022466,205sleep28315-21cat07:24:262
17906224513,190sleep217928-21cpu09:39:192
26939222917,190sleep327024-21fschecks_time08:24:253
26939222917,190sleep327024-21fschecks_time08:24:253
7473221813,190sleep27485-21sh12:34:022
5695221527,175sleep25709-21grep10:44:212
31923221517,147sleep231926-21cron12:09:022
31923221517,147sleep231926-21cron12:09:022
14982221573,22sleep20-21swapper/211:14:222
14982221573,22sleep20-21swapper/211:14:222
22261221320,174sleep122329-21fschecks_time08:09:241
3444220714,177sleep33583-21latency_hist08:54:053
112802205123,22sleep20-21swapper/207:34:242
12461220013,126sleep212585-21timerwakeupswit11:04:332
19473218625,123sleep219500-21ntpq07:59:282
24456217616,140sleep024595-21sed08:14:370
2573217213,142sleep02581-21sshd08:49:300
14012217131,125sleep214129-21cat11:09:332
6224216744,108sleep26323-21latency_hist07:19:052
24643216776,16sleep20-21swapper/211:44:232
25978216376,23sleep20-21swapper/211:49:152
2318921628,140sleep223311-21cat09:54:342
1377721578,133sleep213841-21mailstats09:25:062
103221506,12sleep30-21swapper/310:29:243
27836214815,118sleep027960-21timerwakeupswit10:09:340
1979621488,82sleep219286-21apt-get11:29:162
999621448,83sleep20-21swapper/209:14:232
17218213712,109sleep217237-21modprobe07:54:032
8147213637,81sleep28294-21latency_hist09:09:062
15607213518,15sleep20-21swapper/209:29:382
1044213413,105sleep31083-21processes08:44:323
5121213327,76sleep05186-21latency_hist07:14:150
16601213351,20sleep20-21swapper/209:34:272
6973213114,100sleep17112-21latency_hist10:49:051
28381213130,24sleep20-21swapper/208:29:212
25294213149,44sleep225286-21apt-get08:19:192
28496212847,43sleep228545-21grep10:14:042
26114212815,61sleep20-21swapper/210:04:312
23102212828,17sleep20-21swapper/211:39:262
32213212549,62sleep20-21swapper/212:09:182
8611212419,29sleep10-21swapper/109:09:261
10775212333,76sleep210847-21mailstats10:59:362
9096212248,19sleep20-21swapper/210:54:312
7764212132,30sleep27788-21grep12:34:182
2688521218,102sleep0369-21snmpd10:08:490
22672212114,89sleep022708-21latency_hist11:39:040
15730212028,77sleep215848-21cat07:49:062
1828211921,74sleep11843-21modprobe08:49:021
8457211817,86sleep08602-21latency_hist10:54:050
3586211829,39sleep00-21swapper/007:09:180
24185211828,75sleep224216-21tune2fs09:59:222
24185211828,75sleep224216-21tune2fs09:59:222
13481211742,60sleep20-21swapper/207:39:532
2885211518,84sleep22968-21latency_hist12:19:042
27110211564,26sleep20-21swapper/208:24:282
27110211564,26sleep20-21swapper/208:24:282
27621211317,79sleep00-21swapper/011:54:220
8628211262,12sleep20-21swapper/209:09:272
3273821129,66sleep232753-21idleruntime-cro08:44:032
32048211218,23sleep10-21swapper/112:09:051
32048211218,23sleep10-21swapper/112:09:051
28293211123,68sleep328433-21unixbench_singl11:54:373
14660211112,83sleep114768-21sed11:14:051
18391211011,42sleep217995-21/usr/sbin/munin11:24:322
7753210816,77sleep17823-21latency_hist07:24:051
30688210828,23sleep230738-21awk10:19:262
5862210713,77sleep05929-21threads08:59:350
31252210519,66sleep131394-21cat08:39:051
21473210463,25sleep20-21swapper/209:49:332
21473210463,25sleep20-21swapper/209:49:332
16271210410,79sleep016423-21latency_hist11:19:040
5128210312,15sleep10-21swapper/112:24:281
28977210350,13sleep20-21swapper/210:14:252
28977210350,13sleep20-21swapper/210:14:252
22543210318,66sleep20-21swapper/208:09:302
16478210325,22sleep20-21swapper/211:19:192
11537210314,75sleep111584-21processes07:34:311
8361210228,53sleep17738-21/usr/sbin/munin12:34:331
7525210216,63sleep07497-21latency_hist12:34:040
3084521025,86sleep20-21swapper/208:34:462
2755021018,51sleep20-21swapper/211:54:202
2806210017,62sleep02829-21munin-run12:19:020
24433210030,33sleep10-21swapper/111:44:131
19449210018,68sleep119472-21netstat07:59:271
3106829815,68sleep031111-21/usr/sbin/munin12:04:280
3106829815,68sleep031111-21/usr/sbin/munin12:04:280
34182975,38sleep20-21swapper/210:34:482
3043229745,38sleep20-21swapper/212:04:052
2919729712,67sleep029217-21/usr/sbin/munin10:14:310
2919729712,67sleep029217-21/usr/sbin/munin10:14:310
1412829713,66sleep314130-21cron07:44:023
871229615,68sleep38743-21nfs_client09:09:283
217592965,10sleep20-21swapper/208:07:282
217592965,10sleep20-21swapper/208:07:282
2107329620,56sleep021086-21fschecks_count09:49:230
2107329620,56sleep021086-21fschecks_count09:49:230
1967029642,13sleep20-21swapper/209:44:242
1967029642,13sleep20-21swapper/209:44:242
288629521,62sleep32933-21timerandwakeup07:04:413
179722958,16sleep20-21swapper/207:54:312
2292229412,67sleep022956-21ntp_kernel_pll_09:54:280
1810929418,63sleep118137-21cut11:24:271
1242929225,42sleep312569-21apt-config07:39:053
114429127,30sleep30-21swapper/312:14:043
3060429020,47sleep030744-21tail08:34:360
2446729028,44sleep324075-21/usr/sbin/munin09:59:273
2446729028,44sleep324075-21/usr/sbin/munin09:59:273
3037728926,42sleep330393-21apt12:04:023
1933028912,57sleep30-21swapper/307:59:253
655228715,57sleep16589-21latency_hist09:04:051
2695328719,52sleep327032-21latency_hist10:09:053
1526428722,48sleep315298-21/usr/sbin/munin11:14:273
1526428722,48sleep315298-21/usr/sbin/munin11:14:273
1321528713,56sleep312733-21apt-get09:24:143
2754928615,46sleep10-21swapper/111:54:191
151028621,43sleep31025-21apt-get12:14:123
275728530,23sleep20-21swapper/207:04:362
259028519,53sleep12620-21awk07:04:311
2318928512,57sleep023267-21sed11:39:280
1002628513,55sleep010167-21sed10:59:050
581828413,52sleep00-21swapper/010:44:230
415628427,36sleep14189-21fschecks_count10:39:241
2878228420,18sleep00-21swapper/008:29:310
964528317,22sleep00-21swapper/009:14:050
558628310,19sleep00-21swapper/007:14:230
538528216,44sleep20-21swapper/212:24:332
1827328219,46sleep318309-21ntpq09:39:283
51002817,62sleep25105-21munin-run07:14:052
3157728117,46sleep131614-21fschecks_count08:39:251
283722817,44sleep30-21swapper/308:29:173
2600528126,31sleep30-21swapper/311:49:203
2004728114,50sleep320125-21timerandwakeup09:44:333
2004728114,50sleep320125-21timerandwakeup09:44:333
1524628118,30sleep00-21swapper/009:29:320
285428016,46sleep02883-21/usr/sbin/munin10:34:260
131052807,15sleep00-21swapper/009:24:070
1238428020,42sleep112459-21grep11:04:321
2637427912,52sleep026405-21/usr/sbin/munin11:49:290
118592797,57sleep211895-21ntpq09:19:282
869827815,42sleep10-21swapper/110:54:221
62152785,14sleep30-21swapper/310:44:353
1273427818,44sleep112784-21unin-run09:24:041
605227718,13sleep20-21swapper/208:59:372
2948027715,30sleep10-21swapper/111:59:281
2934727717,30sleep30-21swapper/310:14:343
2934727717,30sleep30-21swapper/310:14:343
443727623,33sleep10-21swapper/112:24:031
381427612,28sleep00-21swapper/012:19:340
34142767,12sleep20-21swapper/208:54:042
1770127625,31sleep10-21swapper/109:39:061
1212127617,25sleep00-21swapper/009:19:340
457227515,41sleep34581-21/usr/sbin/munin08:54:383
3312745,11sleep20-21swapper/210:24:552
2729027415,45sleep127383-21fschecks_time10:09:231
245452746,53sleep00-21swapper/011:44:210
2244427416,23sleep00-21swapper/008:09:280
1072727416,28sleep00-21swapper/010:59:340
4592735,11sleep20-21swapper/208:44:072
257127220,26sleep30-21swapper/310:34:213
2458627214,35sleep024666-21grep09:59:330
2458627214,35sleep024666-21grep09:59:330
40812716,11sleep20-21swapper/212:20:012
2300727120,33sleep123101-21perl11:39:251
1154627120,28sleep30-21swapper/309:19:213
214027017,30sleep10-21swapper/112:14:511
213132705,12sleep20-21swapper/211:34:092
21122705,12sleep20-21swapper/210:32:542
1939127027,22sleep30-21swapper/311:29:053
2187926917,36sleep321908-21munin-plugin-st08:09:023
2187926917,36sleep321908-21munin-plugin-st08:09:023
2013526923,22sleep30-21swapper/311:30:183
1883926916,30sleep30-21swapper/307:59:043
1697426923,23sleep30-21swapper/311:19:323
75372685,11sleep20-21swapper/207:21:562
492426817,30sleep00-21swapper/008:59:030
927926721,27sleep30-21swapper/310:54:363
3176026717,30sleep031466-21/usr/sbin/munin08:39:280
2389726715,38sleep123948-21basename08:14:251
1190826719,24sleep30-21swapper/311:04:213
257512669,14sleep00-21swapper/008:19:330
189082667,47sleep00-21swapper/009:40:470
699926519,27sleep37122-21/usr/sbin/munin12:29:353
303222657,48sleep00-21swapper/012:03:190
932726415,21sleep10-21swapper/107:29:061
2143026414,29sleep30-21swapper/311:34:223
1796426424,22sleep10-21swapper/107:54:301
1727326418,21sleep10-21swapper/107:54:061
961626320,24sleep10-21swapper/107:29:221
3274726312,30sleep10-21swapper/112:09:341
248526317,30sleep12508-21/usr/sbin/munin08:49:271
2270926315,25sleep10-21swapper/111:39:051
2193126323,24sleep10-21swapper/108:09:041
2193126323,24sleep10-21swapper/108:09:041
1846226321,26sleep10-21swapper/109:39:351
1567626320,22sleep30-21swapper/307:49:033
1122626321,19sleep30-21swapper/309:19:053
94626218,26sleep1943-21apt-get10:29:181
92126214,21sleep00-21swapper/010:29:170
654726221,25sleep00-21swapper/007:19:240
564526215,23sleep10-21swapper/107:14:241
3228726218,19sleep00-21swapper/012:09:200
3094326213,32sleep031053-21timerwakeupswit10:19:320
3007726217,25sleep10-21swapper/110:19:041
3007726217,25sleep10-21swapper/110:19:041
2501226221,20sleep30-21swapper/311:44:323
1114626221,25sleep111173-21munin-run09:19:031
754126116,22sleep30-21swapper/309:04:353
267322615,10sleep10-21swapper/110:06:491
2262326112,32sleep322725-21fschecks_time09:54:233
2012826118,17sleep00-21swapper/009:44:340
2012826118,17sleep00-21swapper/009:44:340
1981226115,25sleep10-21swapper/109:44:271
1981226115,25sleep10-21swapper/109:44:271
1340126113,21sleep10-21swapper/107:39:521
128982618,41sleep00-21swapper/011:06:490
1285126113,25sleep00-21swapper/007:39:140
857326015,28sleep38665-21users12:34:363
783826012,26sleep00-21swapper/012:34:200
375626015,29sleep13768-21fschecks_count08:54:231
2024226014,27sleep120272-21awk11:30:201
70325919,18sleep00-21swapper/010:29:050
2683425917,25sleep026830-21apt-get08:24:180
2683425917,25sleep026830-21apt-get08:24:180
233292595,11sleep30-21swapper/308:12:503
1107225916,25sleep011065-21apt-get07:34:180
1024125918,24sleep30-21swapper/310:59:173
844325810,35sleep3287-21dbus-daemon10:54:043
836025817,24sleep00-21swapper/007:24:280
807725817,26sleep38133-21tune2fs07:24:233
3168225818,24sleep131817-21latency_hist10:24:051
3144025816,27sleep331569-21uptime12:04:353
3144025816,27sleep331569-21uptime12:04:353
241542586,15sleep30-21swapper/311:43:513
2098725815,27sleep320982-21apt-get09:49:183
2098725815,27sleep320982-21apt-get09:49:183
1644925813,31sleep316503-21sendmail_mailqu07:49:313
735725716,27sleep17423-21sed09:04:331
679425710,24sleep10-21swapper/112:29:331
252672575,39sleep10-21swapper/110:02:491
252672575,39sleep10-21swapper/110:02:491
2432825716,18sleep30-21swapper/308:14:353
2386125719,19sleep10-21swapper/109:59:051
1678925712,31sleep316796-21sh09:34:323
1463525717,23sleep10-21swapper/107:44:241
1413225711,23sleep30-21swapper/311:09:343
976825613,27sleep09802-21irqstats07:29:250
700025612,17sleep00-21swapper/009:04:240
521825613,16sleep00-21swapper/012:24:310
1954525615,19sleep00-21swapper/011:29:060
1675525617,22sleep116842-21memory11:19:261
593125513,17sleep30-21swapper/312:29:023
357325517,19sleep30-21swapper/312:19:293
3038725514,25sleep330394-21diskstats10:19:183
1355925516,18sleep00-21swapper/011:09:230
524125412,25sleep35236-21apt-get08:59:183
423525416,23sleep34259-21ntp_states07:09:333
3156425413,23sleep331570-21df_inode08:39:233
3018925418,19sleep30-21swapper/308:34:263
2967925423,17sleep10-21swapper/108:34:041
1649325412,27sleep016551-21/usr/sbin/munin09:34:250
161212545,40sleep10-21swapper/111:18:331
161212545,40sleep10-21swapper/111:18:331
561025314,23sleep15605-21apt-get10:44:161
464725311,24sleep04737-21switchtime10:39:350
2063125313,23sleep020625-21apt-get08:04:190
2063125313,23sleep020625-21apt-get08:04:190
1647625315,22sleep016471-21apt-get11:19:170
1042725319,19sleep30-21swapper/307:29:363
69225216,22sleep3700-21systemd-run10:29:043
534325217,22sleep35171-21apt-get07:14:183
160402525,36sleep10-21swapper/109:34:011
1031525213,23sleep010338-21ntp_states09:14:300
269425116,18sleep10-21swapper/110:34:231
2600325112,22sleep125998-21apt-get11:49:181
1846225115,19sleep00-21swapper/011:24:340
262352505,34sleep10-21swapper/108:19:591
2551025012,22sleep325567-21irqstats08:19:283
1323925011,26sleep113234-21apt-get09:24:161
518824915,18sleep10-21swapper/107:14:161
3028724910,20sleep130306-21apt-get10:19:121
1750924911,21sleep017504-21apt-get07:54:170
1511224912,20sleep015164-21irqstats11:14:240
102724911,16sleep00-21swapper/008:44:290
770124813,18sleep37708-21modprobe07:24:033
1532324810,15sleep30-21swapper/309:29:353
414024713,18sleep30-21swapper/310:39:223
1916724712,19sleep00-21swapper/007:59:220
1479524714,16sleep00-21swapper/007:44:280
2762324312,16sleep30-21swapper/310:09:293
1604624311,16sleep00-21swapper/007:49:200
2708624111,16sleep10-21swapper/108:24:271
2708624111,16sleep10-21swapper/108:24:271
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional