You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 13:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Thu Sep 17, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
168472918411,9156sleep225647-21kworker/2:223:57:502
3522489519,4864sleep124164-21kworker/1:119:38:041
250272391913,3893sleep123555-21kworker/1:220:55:101
17427225535,2532sleep017399-21apt-get23:58:160
28147285911,833sleep221236-21kworker/2:219:23:032
2523251624,474sleep02599-21/usr/sbin/munin19:43:200
322124808,433sleep33336-21latency_hist19:47:523
19151235617,323sleep119160-21/usr/sbin/munin22:18:211
19151235617,323sleep119160-21/usr/sbin/munin22:18:211
841228418,248sleep217472-21kworker/u8:221:22:492
29774228022,239sleep229781-21/usr/sbin/munin19:28:082
9435226811,203sleep39449-21apt-key20:07:513
21970226587,52sleep30-21swapper/322:28:253
369022630,34sleep322850-21cyclictest19:48:113
9165226291,120sleep30-21swapper/321:48:113
23547225449,192sleep323612-21awk19:08:183
23547225449,192sleep323612-21awk19:08:183
28833224144,143sleep328836-21cron22:52:483
30949224041,144sleep330967-21modprobe19:32:493
26661222964,23sleep30-21swapper/300:27:523
16929222975,74sleep33536-1kworker/3:2H23:57:513
3756222537,151sleep33829-21switchtime23:13:573
19565222447,54sleep30-21swapper/322:22:523
19565222447,54sleep30-21swapper/322:22:523
21488222337,93sleep321612-21timerandwakeup20:43:213
1815221636,51sleep30-21swapper/323:08:193
14442221644,134sleep314488-21ntp_offset23:48:143
12271221240,134sleep312231-21latency_hist23:42:523
23147220931,158sleep123163-21sort19:08:081
23147220931,158sleep123163-21sort19:08:081
606522017,182sleep36081-21uname21:38:133
30795220018,99sleep30-21swapper/322:58:103
13381219929,118sleep30-21swapper/320:18:133
15210219316,159sleep215268-21fschecks_count22:08:092
3275821919,167sleep3375-21sendmail_mailtr23:03:193
3275821919,167sleep3375-21sendmail_mailtr23:03:193
672821876,78sleep30-21swapper/319:58:103
873217628,109sleep3956-21mailstats19:38:193
1128621729,122sleep311390-21date20:12:523
17939216817,135sleep117940-21cron22:17:491
13696216730,103sleep313777-21grep22:03:093
4200216327,101sleep34203-21acpi21:33:003
20073216146,95sleep320113-21latency22:23:123
1796215931,77sleep30-21swapper/321:23:233
24967215872,47sleep30-21swapper/300:22:513
2034621569,136sleep0367-21snmpd20:39:460
14151215418,117sleep114225-21cat22:03:191
3203321515,15sleep10-21swapper/121:18:041
23617215012,122sleep023693-21sed19:08:190
23617215012,122sleep023693-21sed19:08:190
26103214823,83sleep30-21swapper/322:43:073
26103214823,83sleep30-21swapper/322:43:073
5585214622,106sleep15660-21grep19:53:191
28632214529,78sleep328698-21mailstats19:23:163
2579121458,124sleep325889-21date22:42:513
12312213912,19sleep30-21swapper/320:13:243
12312213912,19sleep30-21swapper/320:13:243
28951213810,85sleep30-21swapper/300:33:173
28951213810,85sleep30-21swapper/300:33:173
16884213828,47sleep30-21swapper/322:13:153
267621367,16sleep30-21swapper/321:28:033
5979213520,96sleep2231-21systemd-journal23:22:482
23712213510,81sleep30-21swapper/322:33:223
2928221317,39sleep30-21swapper/321:08:203
19369213024,81sleep00-21swapper/000:03:170
7617212825,39sleep30-21swapper/321:43:103
2954221287,16sleep30-21swapper/322:53:173
2954221287,16sleep30-21swapper/322:53:173
7435212718,85sleep07516-21fschecks_time21:43:070
16843212721,82sleep1492-21cron23:57:491
21873212618,82sleep10-21swapper/100:12:511
26955212412,94sleep10-21swapper/100:27:541
22804212323,80sleep122952-21grep00:13:201
3136211931,47sleep20-21swapper/223:13:032
1932321198,61sleep322850-21cyclictest20:38:043
2755521176,28sleep30-21swapper/319:18:563
568221165,27sleep30-21swapper/323:18:583
16819211621,74sleep016925-21timerandwakeup20:28:190
19053211518,83sleep219072-21uniq00:03:112
1727121155,28sleep30-21swapper/320:30:203
13794211521,72sleep013925-21unixbench_multi20:18:220
11675211517,78sleep011821-21grep23:38:210
2002211422,63sleep12052-21fschecks_count19:43:091
2213021118,59sleep30-21swapper/300:13:063
3821211016,74sleep13437-21/usr/sbin/munin19:48:141
22501211038,53sleep30-21swapper/320:48:093
27888210937,51sleep30-21swapper/300:28:243
26576210912,79sleep026644-21/usr/sbin/munin22:43:200
26576210912,79sleep026644-21/usr/sbin/munin22:43:200
24659210948,18sleep30-21swapper/320:53:223
9909210812,81sleep09976-21swap23:33:200
18768210828,42sleep318917-21latency_hist00:02:523
8432210745,15sleep30-21swapper/323:28:193
16927210613,75sleep117061-21/usr/sbin/munin20:28:211
679821056,14sleep00-21swapper/019:58:110
4806210413,76sleep14858-21fschecks_count23:18:071
30208210414,75sleep130239-21cat21:12:511
1549421047,61sleep315530-21ntpq22:08:143
3152021036,13sleep30-21swapper/321:15:153
29374210315,59sleep20-21swapper/221:08:232
21820210320,58sleep00-21swapper/000:12:500
10730210014,69sleep30-21swapper/323:37:523
635329915,68sleep26398-21fschecks_count23:23:092
687629820,63sleep16976-21timerandwakeup23:23:201
1218629712,70sleep212309-21timerwakeupswit20:13:232
1218629712,70sleep212309-21timerwakeupswit20:13:232
6162945,17sleep30-21swapper/321:19:553
2652829425,36sleep3367-21snmpd20:59:083
2554929419,45sleep125542-21apt-get20:58:051
135229422,41sleep10-21swapper/123:08:051
118142948,71sleep311724-21latency_hist21:57:523
510629341,35sleep30-21swapper/319:53:093
288442930,11sleep00-21swapper/022:52:480
169029312,65sleep01830-21cat19:42:520
1509129311,67sleep215093-21sshd23:50:112
89429116,60sleep0953-21date21:22:510
2854329115,59sleep20-21swapper/200:33:042
2854329115,59sleep20-21swapper/200:33:042
2273229137,33sleep322739-21modprobe19:07:493
91992900,7sleep00-21swapper/023:32:510
2593529015,64sleep30-21swapper/319:13:273
325032887,65sleep31-21systemd19:37:503
963828720,39sleep19773-21unixbench_singl21:48:211
2941928421,24sleep00-21swapper/021:08:250
280628413,30sleep00-21swapper/021:28:080
2160228429,29sleep00-21swapper/019:03:110
137492846,34sleep30-21swapper/323:47:493
2209328324,39sleep022236-21sed22:28:260
197652837,33sleep30-21swapper/300:03:243
1536628312,49sleep214628-21/usr/sbin/munin20:23:212
3184528216,49sleep131922-21grep19:33:201
190862828,50sleep30-21swapper/320:37:523
1490428213,54sleep215048-21latency_hist22:07:522
292582810,7sleep10-21swapper/100:33:221
292582810,7sleep10-21swapper/100:33:221
275828120,27sleep20-21swapper/219:43:222
1210528114,49sleep10-21swapper/120:13:221
1210528114,49sleep10-21swapper/120:13:221
866328024,26sleep10-21swapper/121:47:521
673928016,29sleep30-21swapper/323:23:183
1725028018,46sleep017324-21cat22:13:240
1040328014,42sleep09894-21/usr/sbin/munin20:08:250
2710227923,15sleep30-21swapper/321:03:033
2710227923,15sleep30-21swapper/321:03:033
2456727917,48sleep024659-21tune2fs22:38:070
213152798,31sleep30-21swapper/300:08:213
680427811,48sleep0367-21snmpd23:23:190
2774027813,49sleep127862-21cat21:03:221
2774027813,49sleep127862-21cat21:03:221
2654127823,37sleep20-21swapper/200:27:512
2156127814,31sleep20-21swapper/222:28:122
398927726,29sleep20-21swapper/221:32:512
2628227722,39sleep126286-21cron19:17:491
2063927721,25sleep10-21swapper/122:23:221
159827716,44sleep00-21swapper/021:23:200
2888427621,26sleep10-21swapper/122:52:501
2397127622,33sleep023967-21apt-get20:53:030
2303927622,33sleep223144-21grep20:48:212
1872327623,29sleep20-21swapper/200:02:512
1053327617,43sleep010609-21fschecks_time21:53:080
2708027514,45sleep227099-21apt-get21:03:022
2708027514,45sleep227099-21apt-get21:03:022
1279027521,16sleep30-21swapper/321:58:203
2570227416,42sleep125781-21sendmail_mailtr00:23:171
2373827422,31sleep10-21swapper/100:18:081
1334127320,30sleep20-21swapper/222:02:522
2520627223,32sleep00-21swapper/000:23:040
1550627220,12sleep30-21swapper/320:23:223
1224327217,34sleep112245-21systemd-run23:42:501
79192710,7sleep20-21swapper/220:02:522
620327117,24sleep20-21swapper/221:38:162
471727115,30sleep00-21swapper/019:52:490
2208427128,21sleep20-21swapper/219:03:212
12662710,7sleep10-21swapper/123:07:521
12662710,7sleep10-21swapper/123:07:521
108582719,16sleep20-21swapper/221:53:142
1031127117,23sleep10-21swapper/120:08:221
207062700,7sleep20-21swapper/222:23:222
2812926917,26sleep20-21swapper/222:48:192
2789626916,39sleep227910-21latency_hist19:22:512
95426820,22sleep10-21swapper/121:22:521
2717926817,21sleep00-21swapper/021:03:080
2717926817,21sleep00-21swapper/021:03:080
2462626820,26sleep00-21swapper/019:12:550
2973226716,25sleep00-21swapper/022:53:220
2973226716,25sleep00-21swapper/022:53:220
264672675,16sleep00-21swapper/020:58:240
2365426713,36sleep223721-21df00:18:052
2341526719,21sleep00-21swapper/000:17:510
2696426617,31sleep026969-21munin-node00:28:010
239432666,17sleep00-21swapper/022:34:560
1603926620,28sleep115100-21/usr/sbin/munin22:08:241
1397426616,18sleep00-21swapper/022:03:140
442826521,23sleep10-21swapper/121:33:101
3153326520,27sleep231610-21/usr/sbin/munin19:33:132
3042626516,18sleep00-21swapper/021:13:020
2217926518,25sleep20-21swapper/220:47:522
1596326518,21sleep00-21swapper/020:27:510
949526422,24sleep09648-21cat20:07:520
850826416,25sleep10-21swapper/120:03:131
617226436,15sleep10-21swapper/121:38:151
2135726417,24sleep10-21swapper/120:43:191
2133426414,28sleep00-21swapper/020:43:180
200552646,12sleep10-21swapper/100:05:411
18512648,13sleep00-21swapper/023:08:200
1062826419,31sleep11-21systemd23:37:501
434626314,22sleep00-21swapper/021:33:090
3201626311,22sleep00-21swapper/023:02:510
2915526319,22sleep00-21swapper/000:33:220
2915526319,22sleep00-21swapper/000:33:220
2876426316,28sleep128890-21/usr/sbin/munin19:23:181
2316826318,28sleep123181-21/usr/sbin/munin22:33:121
1558626318,27sleep015580-21apt-get23:53:030
139726315,32sleep21417-21kernelversion21:23:142
1240026312,22sleep00-21swapper/021:58:130
67262625,17sleep00-21swapper/021:38:300
3251526214,31sleep032531-21kernelversion23:03:110
3251526214,31sleep032531-21kernelversion23:03:110
2664526218,21sleep10-21swapper/122:43:211
2664526218,21sleep10-21swapper/122:43:211
1403926210,33sleep10-21swapper/123:48:031
1279126218,26sleep00-21swapper/020:17:490
1279126218,26sleep00-21swapper/020:17:490
951326121,19sleep00-21swapper/021:48:200
173972619,23sleep10-21swapper/123:58:011
1148126120,26sleep211552-21/usr/sbin/munin23:38:192
440726017,24sleep00-21swapper/023:17:490
2358426016,24sleep20-21swapper/222:33:212
2243626015,19sleep20-21swapper/200:13:132
2200826018,28sleep122083-21sensors_temp19:03:201
1713826019,27sleep217162-21proc_pri22:13:222
1403826017,27sleep014033-21apt-get23:48:020
907625912,22sleep10-21swapper/123:32:491
498925922,19sleep00-21swapper/023:18:100
3225425919,22sleep20-21swapper/223:03:052
3225425919,22sleep20-21swapper/223:03:052
2447825915,25sleep124475-21apt-get22:38:001
1821325915,28sleep118253-21ps20:33:171
1510825918,23sleep10-21swapper/120:23:161
135325918,25sleep21347-21apt-get23:08:062
3044825816,25sleep130445-21apt-get21:13:031
2502725816,22sleep20-21swapper/222:38:182
229702587,11sleep10-21swapper/120:48:201
153352588,21sleep00-21swapper/022:08:110
1061025811,18sleep10-21swapper/121:53:091
836825717,19sleep20-21swapper/223:28:172
521725712,26sleep05267-21/usr/sbin/munin19:53:100
2959525714,21sleep20-21swapper/222:53:202
2959525714,21sleep20-21swapper/222:53:202
266902575,15sleep20-21swapper/221:01:132
1959425715,22sleep20-21swapper/220:38:112
1899825717,17sleep00-21swapper/022:18:200
1899825717,17sleep00-21swapper/022:18:200
1825625718,24sleep218328-21sshd20:33:182
973625616,19sleep20-21swapper/223:33:142
789825619,19sleep10-21swapper/120:02:521
476825618,17sleep20-21swapper/219:52:522
422525614,25sleep24221-21apt-get21:33:032
311425616,24sleep13133-21apt-get23:13:021
1361925611,29sleep213715-21mailstats20:18:202
1271025615,25sleep112759-21irqstats23:43:111
61702557,38sleep20-21swapper/219:56:352
2677225517,17sleep20-21swapper/222:43:222
2677225517,17sleep20-21swapper/222:43:222
2657425510,18sleep10-21swapper/119:18:011
2529225512,18sleep20-21swapper/219:13:142
1980425513,24sleep019797-21apt-get22:23:050
1802525515,23sleep018118-21memory20:33:120
930825412,27sleep29350-21ntp_states21:48:142
860925415,24sleep28626-21apt21:47:492
3164425418,22sleep00-21swapper/019:33:150
2660225412,24sleep026593-21apt-get19:18:040
2373325413,22sleep023737-21diskstats00:18:070
1932125415,24sleep119316-21apt-get20:38:021
1257825317,15sleep10-21swapper/121:58:181
1248725315,22sleep012484-21apt-get23:43:020
3067225211,17sleep10-21swapper/122:57:591
3016525213,19sleep00-21swapper/019:28:180
1620425212,20sleep20-21swapper/220:28:052
2921225117,14sleep10-21swapper/122:53:111
2921225117,14sleep10-21swapper/122:53:111
2147325118,16sleep10-21swapper/122:28:111
1441625112,23sleep10-21swapper/120:22:511
1380325116,18sleep20-21swapper/223:47:522
982925011,20sleep29410-21apt-get20:07:542
93672505,10sleep20-21swapper/220:07:252
2814325011,22sleep028138-21apt-get19:23:010
2243725011,15sleep00-21swapper/020:48:060
2059725017,16sleep20-21swapper/200:08:082
3137424913,17sleep20-21swapper/222:58:212
274624917,17sleep10-21swapper/121:28:051
2704824913,18sleep20-21swapper/200:28:072
2382124910,20sleep223959-21uptime19:08:212
2382124910,20sleep223959-21uptime19:08:212
1825224910,22sleep218248-21apt-get22:18:022
1825224910,22sleep218248-21apt-get22:18:022
2458024515,15sleep20-21swapper/220:53:212
3044924311,17sleep20-21swapper/221:13:042
117024311,16sleep20-21swapper/219:38:222
3009824213,16sleep10-21swapper/119:28:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional