You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 13:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Aug 30, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
116542511514,5088sleep1315-21kworker/1:123:07:031
50162323250,3129sleep34922-21kworker/3:222:48:043
2992325188,491sleep330003-21fschecks_time22:24:143
10663243613,401sleep310755-21latency_hist21:23:553
10505240422,361sleep210523-21apt-get23:04:052
2777234115,255sleep32854-21latency_hist00:23:553
3264423280,8sleep20-21swapper/219:08:552
18991229820,257sleep119130-21grep23:29:231
7252223221,192sleep37309-21fschecks_count22:54:173
1184622209,190sleep00-21swapper/023:08:550
15558221214,180sleep215696-21latency_hist21:38:562
6203220712,110sleep36276-21fschecks_time00:34:133
435321989,168sleep34356-21cron21:03:533
523321959,34sleep35219-21munin-run19:23:553
397218973,32sleep30-21swapper/319:09:093
26942218916,85sleep30-21swapper/322:14:173
26942218916,85sleep30-21swapper/322:14:173
6104218715,154sleep26148-21ntpq19:24:222
27704218558,42sleep327708-21acpi20:34:023
2646221769,31sleep326511-21ntpq23:54:203
19927216250,96sleep31721-21munin-node20:09:043
1790521628,142sleep0367-21snmpd20:01:100
32166216157,42sleep332273-21latency_hist20:48:563
17051216183,62sleep317054-21cron21:43:543
3696215813,132sleep03761-21cat19:18:550
3696215813,132sleep03761-21cat19:18:550
2884215814,93sleep33035-21latency_hist20:58:573
18310215854,80sleep332641-21cyclictest23:29:053
6245215711,130sleep26334-21cat22:50:082
32533215515,125sleep032587-21if_eth000:14:150
14297215359,17sleep30-21swapper/321:34:133
3074821489,123sleep030767-21apt-get00:09:020
20817214615,77sleep320861-21ntp_kernel_pll_21:54:193
22569214349,83sleep3367-21snmpd20:16:183
30487214143,75sleep30-21swapper/300:08:533
25153214116,37sleep025232-21awk20:24:350
25153214116,37sleep025232-21awk20:24:350
8710213922,87sleep18730-21apt-get22:59:041
2412221399,115sleep324197-21cat22:04:243
20439213716,105sleep120515-21mailstats20:09:221
12695213556,59sleep30-21swapper/323:09:233
19947213151,65sleep319983-21expr23:34:133
13491213116,88sleep20-21swapper/219:48:572
22771212918,92sleep022776-21modprobe20:18:540
14190212813,98sleep314227-21proc_pri19:49:223
6335212730,69sleep30-21swapper/322:50:103
4620212652,61sleep30-21swapper/321:04:063
16478212617,89sleep016640-21unixbench_multi21:39:270
331212511,102sleep3365-21proc_pri00:14:233
8891212411,16sleep20-21swapper/219:34:112
401212477,26sleep30-21swapper/320:49:203
10003212447,65sleep310060-21systemd-run23:03:543
2352212348,64sleep32345-21apt-get19:14:103
2352212348,64sleep32345-21apt-get19:14:103
2298621237,93sleep30-21swapper/323:44:103
2298621237,93sleep30-21swapper/323:44:103
3469212118,20sleep30-21swapper/322:43:033
2238521198,97sleep322426-21ntp_kernel_err21:59:203
31873211515,83sleep331936-21latency22:29:273
19634211516,77sleep019779-21unixbench_singl21:49:280
2019021139,23sleep320200-21munin-run21:53:553
5936210914,79sleep25828-21cron00:33:562
2771121096,86sleep00-21swapper/020:34:030
4384210819,77sleep04395-21ntpq19:19:200
21709210815,75sleep121798-21/usr/sbin/munin23:39:161
23522210716,73sleep023529-21ntp_kernel_pll_20:19:220
23522210716,73sleep023529-21ntp_kernel_pll_20:19:220
23183210710,80sleep323284-21fschecks_time20:19:133
23183210710,80sleep323284-21fschecks_time20:19:133
18480210763,25sleep30-21swapper/320:04:123
31637210618,68sleep0367-21snmpd19:04:230
12181210634,56sleep30-21swapper/319:44:093
9849210519,17sleep30-21swapper/319:36:023
24531210532,46sleep224525-21apt-get23:49:062
14310210515,72sleep114403-21fschecks_time21:34:151
6209210410,19sleep30-21swapper/321:09:103
6209210410,19sleep30-21swapper/321:09:103
31160210322,59sleep00-21swapper/022:28:550
2612021025,16sleep30-21swapper/322:10:343
15806210224,18sleep30-21swapper/319:54:233
3398210111,67sleep30-21swapper/300:24:203
32584210134,31sleep10-21swapper/119:08:551
21687210118,59sleep121480-21/usr/sbin/munin20:14:151
14097210114,74sleep214105-21open_files23:14:222
1076821018,79sleep310774-21cut19:39:253
4505210016,65sleep14643-21unixbench_multi22:44:261
13932210017,66sleep013935-21cron21:33:540
11914210012,70sleep312065-21head21:25:173
3077229821,18sleep30-21swapper/300:09:063
293822975,12sleep30-21swapper/322:22:123
293822975,12sleep30-21swapper/322:22:123
72632967,73sleep30-21swapper/319:29:083
56812965,11sleep30-21swapper/300:32:033
2521229610,61sleep30-21swapper/323:49:233
1123929612,23sleep00-21swapper/021:25:020
41729511,69sleep0424-21ntpq20:49:210
15112955,11sleep30-21swapper/320:54:093
20272949,64sleep30-21swapper/322:38:553
753229317,59sleep17247-21/usr/sbin/munin19:29:181
2865229317,30sleep00-21swapper/022:19:190
2865229317,30sleep00-21swapper/022:19:190
846129215,55sleep18608-21/usr/sbin/munin21:14:251
2765329216,62sleep127675-21dpkg23:59:061
2679329210,23sleep326875-21cat20:29:263
2247029211,23sleep10-21swapper/121:59:221
70162917,72sleep37028-21munin-run19:28:553
2461229121,14sleep30-21swapper/320:24:053
2461229121,14sleep30-21swapper/320:24:053
2366429122,46sleep223796-21tail23:44:262
2366429122,46sleep223796-21tail23:44:262
179872908,66sleep317318-21/usr/sbin/munin21:44:263
133662900,7sleep10-21swapper/121:29:261
179812898,71sleep3367-21snmpd23:28:203
86802885,71sleep30-21swapper/321:14:393
366428821,44sleep23739-21cat20:59:242
2664728823,32sleep00-21swapper/022:14:060
2664728823,32sleep00-21swapper/022:14:060
165592889,15sleep30-21swapper/323:23:563
145632885,71sleep30-21swapper/323:14:403
1573428715,57sleep115741-21awk19:54:211
297472860,7sleep20-21swapper/222:23:562
2643028610,63sleep226477-21latency_hist22:13:562
2734128527,38sleep127368-21munin-run23:58:541
115472845,68sleep30-21swapper/323:05:443
461728325,24sleep10-21swapper/119:19:251
353528321,41sleep13553-21apt22:43:541
2255328320,11sleep30-21swapper/323:42:053
1365628319,11sleep30-21swapper/321:30:213
3231328215,48sleep032458-21/usr/sbin/munin22:29:360
293082828,18sleep30-21swapper/320:39:073
293082828,18sleep30-21swapper/320:39:073
2466128215,47sleep024742-21/usr/sbin/munin23:49:120
1076228221,32sleep00-21swapper/021:23:560
1001828217,46sleep110146-21/usr/sbin/munin21:19:251
361628122,42sleep23618-21cron19:18:532
361628122,42sleep23618-21cron19:18:532
35722810,12sleep00-21swapper/000:24:240
2879228020,10sleep30-21swapper/300:01:593
169312806,13sleep30-21swapper/319:59:113
236527923,29sleep20-21swapper/222:39:152
921027820,39sleep19245-21ntp_kernel_pll_19:34:181
2801627826,25sleep10-21swapper/122:18:551
2801627826,25sleep10-21swapper/122:18:551
2744327820,37sleep227457-21apt20:33:532
104427819,26sleep00-21swapper/019:09:310
1841027718,27sleep20-21swapper/223:29:122
1564227712,49sleep215679-21sed19:54:182
1548027713,42sleep00-21swapper/021:38:540
277127629,28sleep02772-21cron20:58:540
277127629,28sleep02772-21cron20:58:540
1315227620,34sleep213212-21find21:29:242
3203327517,31sleep20-21swapper/200:13:532
2823627527,29sleep128321-21mailstats20:34:211
2357027521,33sleep223567-21apt-get22:04:072
123032758,13sleep20-21swapper/223:09:142
66802744,11sleep00-21swapper/022:51:280
2273627415,35sleep222883-21unixbench_singl21:59:272
1339627411,47sleep013534-21latency_hist23:13:560
1059327419,36sleep210097-21apt-get19:39:002
934127320,24sleep00-21swapper/022:59:230
2827327313,26sleep20-21swapper/223:59:272
1966227315,42sleep119674-21sh20:08:541
1865327322,31sleep00-21swapper/021:48:550
1704027322,28sleep20-21swapper/223:24:192
777727212,42sleep07872-21cut19:29:250
703127222,24sleep10-21swapper/119:28:571
648627215,38sleep06527-21ntp_kernel_err00:34:180
3076927212,16sleep10-21swapper/100:09:041
1929027215,34sleep118869-21/usr/sbin/munin21:49:211
3112627120,29sleep00-21swapper/020:44:170
1877627119,34sleep218820-21ntpq20:04:192
1660227024,31sleep016618-21cat19:58:560
429826916,39sleep24332-21cut22:44:232
355326911,23sleep00-21swapper/020:59:220
293832697,16sleep20-21swapper/220:39:102
293832697,16sleep20-21swapper/220:39:102
2749526918,29sleep127623-21ls20:33:561
2471226913,40sleep124721-21diskstats20:24:121
2471226913,40sleep124721-21diskstats20:24:121
2255126910,45sleep222552-21sshd23:42:042
1957226932,21sleep00-21swapper/023:33:540
159526918,23sleep20-21swapper/220:54:122
159526918,23sleep20-21swapper/220:54:122
735726821,28sleep20-21swapper/219:29:152
2995526820,21sleep10-21swapper/100:04:271
2608726815,26sleep20-21swapper/223:54:112
2452826810,41sleep10-21swapper/123:49:051
1998526815,38sleep020071-21/usr/sbin/munin23:34:140
1791626816,22sleep20-21swapper/221:44:252
1725226821,21sleep20-21swapper/219:59:182
69826717,30sleep00-21swapper/022:34:080
213632677,50sleep00-21swapper/021:54:360
927626619,32sleep09308-21proc_pri19:34:210
1684426616,34sleep116851-21df_inode23:24:141
161672669,13sleep10-21swapper/121:39:201
313182658,43sleep131324-21open_files20:44:221
3054026518,26sleep230684-21latency_hist00:08:562
194926521,30sleep12012-21/usr/sbin/munin00:19:211
46526414,33sleep2562-21mailstats20:49:232
3106126416,30sleep231099-21modprobe22:28:532
2935626410,40sleep129382-21sh20:39:091
2935626410,40sleep129382-21sh20:39:091
2403326418,29sleep124055-21/usr/sbin/munin22:04:221
2080226418,20sleep20-21swapper/221:54:182
1519726417,25sleep10-21swapper/123:19:061
1256026419,32sleep112618-21sendmail_mailqu23:09:211
119926415,27sleep21277-21run-parts22:34:222
1111326416,31sleep111111-21apt-get21:24:061
888426319,24sleep20-21swapper/222:59:132
767226316,29sleep17766-21grep22:54:271
49042638,12sleep00-21swapper/022:46:390
3102026321,24sleep231079-21basename20:44:152
1503526315,33sleep115112-21cut19:53:561
2627626219,26sleep00-21swapper/020:29:150
18632627,10sleep10-21swapper/122:37:401
1180226225,18sleep20-21swapper/221:25:162
76292617,13sleep00-21swapper/022:54:260
329826118,22sleep10-21swapper/120:59:151
233842617,11sleep00-21swapper/023:44:200
233842617,11sleep00-21swapper/023:44:200
177292616,11sleep10-21swapper/121:44:201
54226019,20sleep20-21swapper/200:14:262
504526019,20sleep10-21swapper/100:29:231
301626017,25sleep23013-21apt-get00:24:072
2799026018,20sleep20-21swapper/220:34:142
1867726019,22sleep00-21swapper/020:04:170
178026016,27sleep10-21swapper/120:54:151
178026016,27sleep10-21swapper/120:54:151
1548826019,24sleep00-21swapper/023:19:170
64625921,24sleep1793-21sed20:49:261
48025914,26sleep20-21swapper/219:09:122
464425917,26sleep14641-21apt-get21:04:081
2420225917,18sleep00-21swapper/022:04:250
1077925914,30sleep010859-21/usr/sbin/munin19:39:260
1060825917,21sleep20-21swapper/221:23:542
910225816,21sleep00-21swapper/021:18:560
790025813,27sleep20-21swapper/221:14:142
2959725812,30sleep129745-21cat22:23:561
2959725812,30sleep129745-21cat22:23:561
2520925820,19sleep00-21swapper/022:09:140
2004825819,19sleep00-21swapper/020:09:130
1886825814,25sleep218887-21apt-get21:49:052
1710425817,23sleep20-21swapper/221:43:572
1372925818,26sleep213723-21apt-get19:49:092
685025712,19sleep20-21swapper/221:09:252
685025712,19sleep20-21swapper/221:09:252
393925711,30sleep24006-21df19:19:102
363625720,19sleep10-21swapper/100:24:261
240425717,24sleep12429-21/usr/sbin/munin19:14:121
240425717,24sleep12429-21/usr/sbin/munin19:14:121
2298425715,26sleep122979-21apt-get23:44:081
2298425715,26sleep122979-21apt-get23:44:081
2012825720,20sleep20-21swapper/223:34:172
1385625714,16sleep00-21swapper/023:14:160
3156125618,23sleep131077-21apt-get22:29:041
3148525610,15sleep20-21swapper/222:28:572
227825614,26sleep02273-21apt-get22:39:080
2175125616,17sleep20-21swapper/220:14:162
1559025619,20sleep20-21swapper/223:19:192
569025513,20sleep00-21swapper/019:24:040
476825510,27sleep04849-21fschecks_time21:04:150
2593225512,24sleep126024-21users22:09:271
2107325519,17sleep10-21swapper/121:54:251
155525515,23sleep01649-21perl00:19:120
1412825516,16sleep00-21swapper/019:49:190
3276425411,15sleep10-21swapper/100:14:211
2968925415,23sleep029695-21ntp_states00:04:220
2317225414,23sleep20-21swapper/220:19:122
2317225414,23sleep20-21swapper/220:19:122
557125314,21sleep15096-21apt-get22:48:581
522825315,16sleep20-21swapper/221:04:252
3123425318,15sleep30-21swapper/319:04:053
2767925312,24sleep027672-21apt-get23:59:090
2651425314,20sleep00-21swapper/023:54:210
132472538,37sleep00-21swapper/019:46:240
1064325313,23sleep10-21swapper/119:39:151
23902524,36sleep20-21swapper/200:19:342
2273425213,22sleep022744-21munin-run23:43:550
1424325216,16sleep10-21swapper/123:14:261
301102518,15sleep00-21swapper/020:39:240
301102518,15sleep00-21swapper/020:39:240
1995225111,22sleep219948-21apt-get20:09:072
2308725012,23sleep123082-21apt-get20:19:071
2308725012,23sleep123082-21apt-get20:19:071
2026325014,18sleep10-21swapper/123:34:201
1687925012,19sleep116922-21df19:59:091
639124915,18sleep00-21swapper/021:09:150
639124915,18sleep00-21swapper/021:09:150
2226424910,16sleep00-21swapper/021:59:170
70732484,32sleep10-21swapper/100:35:041
620724711,18sleep10-21swapper/121:09:081
620724711,18sleep10-21swapper/121:09:081
465824713,14sleep00-21swapper/000:29:140
39524712,21sleep1390-21apt-get19:09:071
2834024710,16sleep20-21swapper/222:19:132
1802324715,16sleep00-21swapper/023:28:540
1217824711,20sleep112175-21apt-get19:44:071
298812459,17sleep10-21swapper/122:24:121
266702459,17sleep20-21swapper/222:14:082
266702459,17sleep20-21swapper/222:14:082
246372459,18sleep20-21swapper/220:24:082
246372459,18sleep20-21swapper/220:24:082
45662439,16sleep20-21swapper/200:29:072
716624210,16sleep20-21swapper/222:54:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional