You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 18:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Wed Jul 29, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
138062859618,8560sleep213296-21apt-get11:28:042
132032704814,7022sleep28092-21kworker/2:111:26:512
132032704814,7022sleep28092-21kworker/2:111:26:512
16391242426,4222sleep013839-21kworker/0:011:34:560
24380253621,499sleep324411-21/usr/sbin/munin10:18:133
24380253621,499sleep324411-21/usr/sbin/munin10:18:133
30421239612,360sleep130459-21ls10:38:091
209732323125,175sleep10-21swapper/110:08:091
209732323125,175sleep10-21swapper/110:08:091
131082289128,138sleep113264-21unixbench_singl07:58:221
8193223310,207sleep18348-21latency_hist09:27:531
10730223166,26sleep10-21swapper/111:18:221
30789222220,185sleep230859-21wc10:38:182
2412222246,164sleep12494-21cut12:33:151
14390220561,43sleep114425-21grep08:03:191
8965220112,149sleep10-21swapper/109:28:211
21667220115,170sleep1360-21ntpd11:53:101
5790220011,176sleep35798-21uname09:18:183
24471219511,111sleep124563-21cat12:02:511
24751218717,140sleep11-21systemd08:37:511
16027218728,23sleep10-21swapper/108:08:191
13834218530,130sleep10-21swapper/111:28:231
1874218420,134sleep10-21swapper/112:32:511
8740218113,106sleep18834-21fschecks_time11:13:091
585218017,32sleep1606-21munin-run10:47:501
10004217817,143sleep310087-21apt-config11:17:523
3267621779,121sleep132755-21cat07:18:251
2077021769,151sleep120891-21cat08:23:211
27658217410,151sleep127728-21cat10:28:201
2278921749,145sleep122403-21/usr/sbin/munin10:13:141
13470217320,17sleep10-21swapper/109:43:151
11822217121,134sleep311881-21/usr/sbin/munin09:38:123
28431216611,141sleep328480-21ntpq12:13:153
26629216511,141sleep126634-21grep12:08:161
19949216311,23sleep10-21swapper/108:22:531
3760215812,127sleep13814-21latency_hist10:57:511
27190215817,19sleep10-21swapper/108:43:201
25527215821,20sleep10-21swapper/108:38:191
29972215710,132sleep130123-21latency_hist07:12:521
1245215320,20sleep10-21swapper/110:48:141
30114214815,59sleep10-21swapper/112:18:171
283692145103,22sleep10-21swapper/107:07:521
3143721449,35sleep10-21swapper/112:23:061
22195214428,98sleep322350-21latency_hist10:12:523
22195214428,98sleep322350-21latency_hist10:12:523
861214224,92sleep2898-21proc_pri09:03:192
31299214111,106sleep10-21swapper/108:58:051
2743214110,119sleep12764-21munin-run07:27:501
182421400,13sleep01825-21modprobe12:32:490
14841213528,96sleep114895-21cat09:48:111
829521338,20sleep10-21swapper/111:11:561
3203621309,65sleep132211-21sshd07:17:531
28637213034,67sleep228656-21apt-get10:33:012
28637213034,67sleep228656-21apt-get10:33:012
812421297,45sleep10-21swapper/107:43:161
29730212912,96sleep029746-21fschecks_count12:18:080
1170021299,70sleep111769-21idleruntime09:38:101
583221278,16sleep10-21swapper/111:03:121
1669821268,103sleep116848-21date11:37:521
182621246,106sleep00-21swapper/010:48:500
4331212117,88sleep24345-21latency_hist07:32:522
24419212121,71sleep324439-21latency_hist12:02:493
1960121209,53sleep10-21swapper/110:03:131
4139211715,81sleep04180-21ntp_kernel_err09:13:310
2823921178,15sleep10-21swapper/108:48:081
10246211748,53sleep110264-21kernelversion09:33:131
196421158,53sleep10-21swapper/109:08:061
196421158,53sleep10-21swapper/109:08:061
1646621159,18sleep116483-21kernelversion09:53:121
925021148,51sleep10-21swapper/107:48:051
925021148,51sleep10-21swapper/107:48:051
376821146,14sleep10-21swapper/109:13:191
2866221147,14sleep10-21swapper/110:33:041
2866221147,14sleep10-21swapper/110:33:041
731621138,93sleep1364-21snmpd07:41:351
17526211215,79sleep029952-1kworker/0:2H09:57:510
22918211014,81sleep222915-21systemd-run11:57:512
1903021108,50sleep10-21swapper/111:43:211
27083210923,46sleep30-21swapper/310:28:023
1801621096,45sleep10-21swapper/109:58:121
2651121069,45sleep10-21swapper/110:23:561
2651121069,45sleep10-21swapper/110:23:561
1640210630,63sleep10-21swapper/109:07:381
16911210420,53sleep216930-21apt-get11:38:012
12263210417,64sleep012418-21grep09:38:210
23182210314,72sleep023392-21latency_hist08:32:530
5938210210,72sleep26010-21/usr/sbin/munin09:18:212
23794210214,73sleep323873-21cat08:33:373
3219121008,24sleep20-21swapper/210:43:152
29931210013,74sleep029971-21latency_hist07:12:510
22174210014,70sleep222193-21latency_hist10:12:512
22174210014,70sleep222193-21latency_hist10:12:512
892329913,69sleep09001-21sed11:13:120
211412996,42sleep10-21swapper/111:50:561
249802985,41sleep10-21swapper/112:03:111
251712978,11sleep20-21swapper/212:03:172
1731229616,62sleep317359-21ntp_kernel_pll_11:38:143
39392955,41sleep10-21swapper/107:28:271
2981629547,32sleep129923-21grep08:53:091
1817829528,56sleep10-21swapper/111:42:361
62372946,41sleep10-21swapper/109:18:241
52142946,42sleep10-21swapper/111:02:071
2346329412,67sleep023482-21apt-get08:33:010
1590429419,53sleep315918-21sh09:52:493
252302936,40sleep10-21swapper/110:22:181
252302936,40sleep10-21swapper/110:22:181
2239429316,64sleep022396-21sshd10:12:570
2004029325,42sleep220188-21grep10:03:222
130552935,41sleep10-21swapper/111:24:561
130552935,41sleep10-21swapper/111:24:561
1584829214,65sleep315917-21systemctl11:33:223
1474429225,41sleep014839-21fschecks_time09:48:100
3262029117,57sleep0307-21latency_hist09:02:530
1340228910,57sleep00-21swapper/009:43:140
549728731,45sleep10-21swapper/107:33:271
2832528731,41sleep10-21swapper/112:13:131
2780928616,59sleep00-21swapper/008:46:580
2377128620,21sleep20-21swapper/210:17:512
1861028419,47sleep218585-21/usr/sbin/munin08:18:052
327028319,42sleep23370-21apt-config09:12:522
327028319,42sleep23370-21apt-config09:12:522
2526328312,52sleep025341-21switchtime12:03:190
2166428313,55sleep00-21swapper/011:53:090
1821928330,42sleep10-21swapper/108:16:551
1171128330,41sleep10-21swapper/107:53:261
2914728217,49sleep329223-21mailstats10:33:193
2914728217,49sleep329223-21mailstats10:33:193
1056828214,18sleep00-21swapper/007:52:520
1056828214,18sleep00-21swapper/007:52:520
511728120,48sleep35119-21sshd11:00:563
2269728130,41sleep10-21swapper/108:29:271
3103628023,27sleep30-21swapper/312:22:523
1088428017,35sleep210776-21/usr/sbin/munin07:53:102
64927914,43sleep0805-21latency_hist10:47:530
373627816,45sleep33231-21apt-get09:13:013
2750127823,38sleep327523-21/usr/sbin/munin07:03:183
1829027816,36sleep318317-21modprobe08:17:503
1909427723,30sleep20-21swapper/211:43:222
706627616,26sleep30-21swapper/309:23:103
249527620,28sleep20-21swapper/212:33:162
1126327613,50sleep311300-21ps07:53:203
673027519,17sleep20-21swapper/207:38:212
49327529,26sleep00-21swapper/007:18:330
3099227514,35sleep031108-21grep10:38:200
2908627514,28sleep20-21swapper/207:08:172
1884527517,25sleep00-21swapper/008:18:110
3183427414,38sleep231835-21cron07:17:502
2647227419,39sleep326475-21dpkg12:08:023
1619627425,28sleep20-21swapper/209:53:052
1384127414,44sleep00-21swapper/011:28:260
836427315,32sleep28387-21munin-run11:12:502
530427317,17sleep00-21swapper/007:33:220
1045227322,33sleep29938-21apt-get11:18:032
757827215,26sleep00-21swapper/011:08:200
312427218,26sleep30-21swapper/307:28:123
287612729,12sleep30-21swapper/307:08:103
194727210,48sleep21984-21ps07:23:192
81222717,51sleep00-21swapper/009:27:350
3097827117,40sleep231007-21awk08:57:502
2836527122,30sleep228386-21munin-run10:32:492
1679827121,30sleep20-21swapper/208:12:522
63627025,27sleep30-21swapper/310:47:523
511327014,40sleep24556-21/usr/sbin/munin07:33:202
288927017,37sleep32912-21/usr/sbin/munin10:53:173
268142708,12sleep30-21swapper/310:27:493
268142708,12sleep30-21swapper/310:27:493
1199127010,44sleep30-21swapper/311:22:563
1199127010,44sleep30-21swapper/311:22:563
1010027017,33sleep310163-21fschecks_time09:33:113
847426912,22sleep30-21swapper/307:43:223
3179826920,28sleep00-21swapper/012:24:220
2841726915,23sleep00-21swapper/010:32:510
2216126914,41sleep322167-21ntp_states08:28:203
204502695,10sleep30-21swapper/310:06:063
3176426812,42sleep231818-21processes08:58:182
2671126817,22sleep30-21swapper/308:43:103
2632726816,34sleep026358-21awk08:42:500
203326817,34sleep32042-21/usr/sbin/munin09:08:083
203326817,34sleep32042-21/usr/sbin/munin09:08:083
179772685,11sleep20-21swapper/208:13:582
178726819,31sleep31810-21munin-run12:32:493
3221026731,19sleep30-21swapper/312:24:333
3126926711,41sleep331270-21sshd10:39:573
2985226723,26sleep20-21swapper/212:18:102
241626718,27sleep00-21swapper/009:08:180
241626718,27sleep00-21swapper/009:08:180
2349126718,31sleep223482-21apt-get08:33:072
1654026711,39sleep216542-21sshd11:36:522
439926613,35sleep24439-21grep10:58:172
34072668,45sleep00-21swapper/007:28:180
2030626615,23sleep00-21swapper/011:48:130
1741826625,24sleep00-21swapper/008:13:150
841026512,25sleep30-21swapper/309:28:033
834626517,23sleep20-21swapper/207:43:212
60282657,18sleep30-21swapper/307:37:533
57042657,14sleep00-21swapper/009:18:150
42826513,26sleep30-21swapper/307:18:323
2731226511,35sleep227403-21load07:03:132
2298726519,27sleep223030-21systemd-run08:32:512
870026416,21sleep20-21swapper/209:28:142
2570826412,24sleep20-21swapper/208:38:242
2023926419,20sleep20-21swapper/211:48:112
1624426418,25sleep30-21swapper/308:08:213
711026318,28sleep37105-21apt-get11:08:073
484826316,35sleep34906-21latency07:33:133
3189726310,39sleep231931-21/usr/sbin/munin12:24:242
2752526325,21sleep00-21swapper/007:03:190
1986626320,29sleep3225-21systemd-journal08:22:513
6922629,12sleep20-21swapper/207:18:352
2650526211,18sleep00-21swapper/012:08:070
2536326218,26sleep20-21swapper/210:22:522
2536326218,26sleep20-21swapper/210:22:522
2211326215,34sleep322185-21sensors11:53:203
2089226222,20sleep20-21swapper/208:23:222
2052526213,15sleep30-21swapper/308:23:153
1936026211,38sleep019389-21who11:43:240
2596926114,27sleep326020-21apt-config12:07:503
991926015,25sleep210001-21apt-config11:17:512
3025426011,33sleep030265-21ntp_states07:12:560
2775726016,29sleep227783-21munin-run12:12:492
1432326016,26sleep214358-21/usr/sbin/munin08:03:162
563625916,26sleep05643-21diskstats11:03:070
376925917,25sleep23735-21apt-get09:13:202
364625912,20sleep00-21swapper/012:37:510
2920525911,22sleep00-21swapper/008:49:570
291082597,11sleep00-21swapper/012:15:010
2444425911,30sleep024470-21munin-run12:02:500
1824625921,18sleep00-21swapper/011:42:520
1100725920,21sleep00-21swapper/007:53:120
724525816,19sleep20-21swapper/209:23:132
3030025813,26sleep30-21swapper/308:53:193
2816625814,27sleep228160-21apt-get08:48:052
2087725814,26sleep020870-21apt-get10:08:040
2087725814,26sleep020870-21apt-get10:08:040
198725823,18sleep30-21swapper/307:23:203
1774925811,21sleep30-21swapper/309:58:023
934625719,17sleep30-21swapper/307:48:113
934625719,17sleep30-21swapper/307:48:113
592625713,29sleep25956-21awk11:03:132
33492575,40sleep00-21swapper/010:53:300
2888825712,26sleep028930-21irqstats10:33:110
2888825712,26sleep028930-21irqstats10:33:110
270712576,16sleep20-21swapper/208:43:172
204582577,38sleep00-21swapper/010:06:110
14532576,11sleep20-21swapper/212:28:322
1382125717,22sleep30-21swapper/311:28:133
2813725614,18sleep20-21swapper/212:13:102
2398425619,16sleep20-21swapper/210:17:592
2398425619,16sleep20-21swapper/210:17:592
1370525617,18sleep30-21swapper/309:43:213
1264125618,20sleep00-21swapper/007:58:130
1134725612,17sleep00-21swapper/009:37:530
998025513,25sleep29973-21apt-get09:33:062
96162555,37sleep00-21swapper/009:31:550
71312556,38sleep00-21swapper/007:39:120
2863525514,22sleep30-21swapper/308:48:183
250952557,36sleep00-21swapper/010:20:360
250952557,36sleep00-21swapper/010:20:360
2146725510,32sleep021602-21latency_hist08:27:520
1637525512,24sleep016377-21users08:08:220
148625513,25sleep01481-21apt-get07:23:050
1362925513,21sleep20-21swapper/209:43:202
1328125513,23sleep013296-21apt-get11:27:500
1328125513,23sleep013296-21apt-get11:27:500
3183925416,20sleep30-21swapper/307:17:513
2562825415,19sleep00-21swapper/010:23:060
2562825415,19sleep00-21swapper/010:23:060
126125417,21sleep20-21swapper/210:48:152
1236625411,18sleep20-21swapper/207:58:032
1180125319,18sleep30-21swapper/311:22:523
427725212,15sleep00-21swapper/010:58:150
3182025214,21sleep331899-21mailstats08:58:203
2972025212,23sleep029714-21apt-get08:53:040
288125210,16sleep20-21swapper/210:53:162
1894725214,23sleep318969-21/usr/sbin/munin11:43:193
1508025214,22sleep215104-21ntp_states09:48:162
2322825111,15sleep20-21swapper/211:58:082
2267425112,23sleep322697-21irqstats10:13:123
2060425116,16sleep30-21swapper/311:48:203
1785825116,20sleep20-21swapper/209:58:092
1392825112,22sleep013922-21apt-get08:03:060
39425011,24sleep0389-21apt-get09:03:050
1815225016,16sleep00-21swapper/009:58:150
130925011,21sleep31335-21ntp_states10:48:163
970524912,22sleep29711-21ntp_states07:48:182
970524912,22sleep29711-21ntp_states07:48:182
239924912,21sleep32405-21df_inode12:33:133
1371124912,16sleep30-21swapper/308:02:523
1162424913,17sleep20-21swapper/209:38:072
46424812,22sleep3470-21df_inode09:03:083
2514324610,21sleep325152-21df_inode08:38:093
139262468,19sleep30-21swapper/308:03:053
1745424514,15sleep30-21swapper/309:57:503
1048524513,18sleep00-21swapper/011:18:090
77022449,18sleep00-21swapper/007:43:030
575124412,13sleep30-21swapper/311:03:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional