You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-15 - 03:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Sat Aug 15, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3008275417,720sleep23071-21/usr/sbin/munin00:30:022
29677262712,601sleep329681-21ntpq00:10:133
28644259437,510sleep128688-21ntpq22:25:131
22136240914,293sleep122230-21fschecks_time22:05:101
13840240938,262sleep113898-21fschecks_time23:21:051
2681623780,7sleep00-21swapper/000:00:200
10527235096,207sleep10-21swapper/119:45:151
13822233620,299sleep313665-21apt-get23:21:033
5111232910,306sleep15188-21cat00:35:181
5111232910,306sleep15188-21cat00:35:181
31322231130,199sleep131452-21grep20:50:201
2759227752,212sleep12780-21sort00:29:491
32713224617,115sleep1335-21latency_hist22:39:511
3466224538,28sleep10-21swapper/100:30:131
13498224437,189sleep113564-21sendmail_mailqu19:55:171
6462223717,202sleep16467-21cpu21:15:031
27430223373,81sleep127559-21latency_hist00:04:501
23930222815,190sleep10-21swapper/120:30:011
15282222220,181sleep115425-21unixbench_multi20:00:211
27878220928,162sleep027928-21/usr/sbin/munin00:05:090
1161622079,94sleep10-21swapper/123:14:511
2312220129,98sleep12357-21latency22:45:081
17476219411,99sleep117627-21latency_hist20:09:501
1184521886,167sleep01752-1kworker/0:0H23:14:580
8529217812,151sleep10-21swapper/121:20:161
406221788,97sleep14110-21processes22:50:161
25942217719,22sleep10-21swapper/120:35:181
32251217617,22sleep10-21swapper/120:55:051
11562217328,123sleep211586-21munin-plugin-st23:14:482
13251217095,16sleep10-21swapper/121:35:181
18030216315,132sleep018079-21unin-run23:34:480
1505215812,134sleep11571-21if_eth021:00:091
1505215812,134sleep11571-21if_eth021:00:091
19977215612,53sleep10-21swapper/123:40:071
1847621569,132sleep118539-21idleruntime23:35:101
3812215510,129sleep13971-21grep21:05:311
2461721559,133sleep124679-21grep23:55:041
2696121507,72sleep10-21swapper/122:20:081
2696121507,72sleep10-21swapper/122:20:081
13669214912,122sleep213163-21apt-get23:20:012
8652214815,112sleep18713-21latency19:40:101
8652214815,112sleep18713-21latency19:40:101
20358214812,121sleep220398-21uname23:40:162
19504214515,114sleep319582-21find21:55:193
2141214215,91sleep10-21swapper/119:20:001
6725213830,94sleep16809-21apt-config22:59:491
6725213830,94sleep16809-21apt-config22:59:491
3217521366,120sleep00-21swapper/019:12:360
3217521366,120sleep00-21swapper/019:12:360
30076213413,107sleep130090-21kernelversion22:30:101
29949213416,104sleep029953-21ntp_states19:05:130
1083213443,15sleep10-21swapper/119:15:191
570021329,109sleep15842-21grep21:11:401
10076213113,100sleep110230-21latency_hist23:09:501
24261212833,75sleep10-21swapper/122:10:191
4463212315,85sleep10-21swapper/119:25:211
24047212326,78sleep224087-21ps22:10:162
20666212144,21sleep10-21swapper/122:00:101
6642211819,80sleep06724-21true22:59:480
6642211819,80sleep06724-21true22:59:480
5355211517,79sleep25448-21fschecks_time19:30:062
363211519,79sleep0381-21open_files00:20:150
23643211421,76sleep023677-21idleruntime-cro20:29:480
17069211421,15sleep10-21swapper/121:49:481
18080211311,17sleep10-21swapper/123:34:491
15831211216,63sleep115957-21sed23:25:171
15521211132,64sleep115524-21cron21:44:471
11140211119,14sleep10-21swapper/121:30:011
25791211022,70sleep025842-21modprobe23:59:480
10894211023,14sleep10-21swapper/121:29:491
3454210915,79sleep00-21swapper/000:30:120
2653321090,7sleep30-21swapper/322:19:493
2653321090,7sleep30-21swapper/322:19:493
11550210920,14sleep10-21swapper/119:50:081
21202210822,27sleep00-21swapper/023:44:500
7168210723,58sleep36662-21apt-get23:00:003
24900210720,70sleep224934-21/usr/sbin/munin23:55:102
29983210617,13sleep10-21swapper/120:46:001
29983210617,13sleep10-21swapper/120:46:001
8518210411,61sleep10-21swapper/123:04:501
30163210412,83sleep00-21swapper/020:47:290
30163210412,83sleep00-21swapper/020:47:290
26156210421,12sleep10-21swapper/122:16:451
26156210421,12sleep10-21swapper/122:16:451
23072210419,69sleep323077-21entropy23:50:063
20139210411,79sleep020141-21sshd21:59:280
32509210311,14sleep10-21swapper/100:20:081
32397210322,23sleep10-21swapper/119:14:511
32397210322,23sleep10-21swapper/119:14:511
226221039,78sleep00-21swapper/022:45:070
2238121028,16sleep10-21swapper/120:25:001
29090210018,64sleep329070-21apt-get20:45:093
29090210018,64sleep329070-21apt-get20:45:093
2246729913,63sleep322477-21/usr/sbin/munin20:25:063
2179929913,73sleep121810-21needreboot23:45:111
1557229910,74sleep315631-21cat21:44:493
846929819,62sleep28534-21uniq19:40:072
846929819,62sleep28534-21uniq19:40:072
2680329815,60sleep226694-21/usr/sbin/munin22:20:052
2680329815,60sleep226694-21/usr/sbin/munin22:20:052
1975329715,62sleep319228-21/usr/sbin/munin20:15:173
195252976,13sleep10-21swapper/120:15:111
537029615,64sleep34866-21apt-get22:54:593
537029615,64sleep34866-21apt-get22:54:593
190022965,13sleep10-21swapper/121:55:091
159582969,68sleep216107-21awk23:25:182
1809229518,61sleep018244-21unixbench_singl21:50:190
11762940,7sleep30-21swapper/320:59:503
1268929317,64sleep312705-21cpu21:35:043
806429223,44sleep38095-21munin-run19:39:483
3155729117,58sleep031610-21/usr/sbin/munin22:35:090
444429025,42sleep04538-21sort21:09:490
1586629016,58sleep315867-21cron20:04:473
990928816,51sleep39910-21sshd23:08:273
282842887,70sleep10-21swapper/120:41:201
276132888,62sleep00-21swapper/022:20:190
276132888,62sleep00-21swapper/022:20:190
2728728816,57sleep326758-21apt-get20:40:003
1019128712,61sleep210235-21switchtime21:25:182
335828617,47sleep33483-21grep22:49:503
1712228612,54sleep317274-21latency_hist21:49:503
1691428518,31sleep30-21swapper/320:05:243
814428425,38sleep10-21swapper/119:39:501
58462849,59sleep25920-21users21:11:412
2653328417,30sleep30-21swapper/322:19:503
2653328417,30sleep30-21swapper/322:19:503
242462847,61sleep10-21swapper/123:54:481
129222835,10sleep10-21swapper/123:17:171
1111328312,19sleep00-21swapper/021:29:560
1027928313,54sleep010297-21/usr/sbin/munin23:09:580
301792827,42sleep10-21swapper/100:10:591
193028212,56sleep02001-21latency_hist19:19:500
183052826,43sleep10-21swapper/121:50:491
14562827,43sleep10-21swapper/122:40:561
308128117,22sleep30-21swapper/321:05:003
306982815,41sleep10-21swapper/100:14:511
3049128117,46sleep230622-21/usr/sbin/munin22:30:192
2949928131,30sleep229553-21sort19:05:032
292042810,6sleep00-21swapper/000:09:560
2369528125,28sleep20-21swapper/220:29:502
186222816,42sleep10-21swapper/120:10:271
1363328116,50sleep313750-21cat19:55:193
121928113,18sleep20-21swapper/200:24:492
682428025,25sleep20-21swapper/222:59:502
682428025,25sleep20-21swapper/222:59:502
2974328018,48sleep229847-21cat20:45:572
2974328018,48sleep229847-21cat20:45:572
276412806,41sleep10-21swapper/100:04:581
221262808,20sleep00-21swapper/022:05:090
218962807,40sleep10-21swapper/120:22:141
1648228012,42sleep00-21swapper/023:29:480
64522796,41sleep10-21swapper/119:33:571
2981727918,48sleep329876-21cpuspeed22:30:043
2344927812,47sleep023481-21proc_pri23:50:150
1252227814,45sleep212648-21/usr/sbin/munin23:15:162
107827816,37sleep20-21swapper/220:59:502
622427714,20sleep00-21swapper/021:14:500
3194927724,28sleep30-21swapper/320:54:503
1858727713,50sleep318668-21latency23:35:123
935327617,16sleep20-21swapper/221:24:502
485227623,37sleep04868-21modprobe22:54:480
3212227627,25sleep30-21swapper/300:19:503
3189727623,38sleep031910-21apt20:54:470
2883527616,43sleep229025-21cat20:44:522
198292766,11sleep00-21swapper/020:15:180
142602769,50sleep20-21swapper/221:40:002
1558827515,41sleep315628-21grep23:25:103
718827423,28sleep00-21swapper/023:00:010
501927418,17sleep20-21swapper/219:29:502
36852746,57sleep30-21swapper/319:24:583
2609227316,27sleep30-21swapper/323:59:593
2513527322,35sleep025171-21cpu22:15:020
2513527322,35sleep025171-21cpu22:15:020
2458227210,47sleep024584-21sshd22:11:270
1156527217,28sleep30-21swapper/319:50:103
850527111,24sleep00-21swapper/021:20:150
720427114,36sleep27189-21apt-get23:00:072
433027123,30sleep04462-21wc19:25:200
2673727115,36sleep326804-21http20:39:483
1320227121,31sleep313208-21munin-run23:19:493
49052707,19sleep30-21swapper/321:10:013
42527013,42sleep2481-21open_files19:14:562
261927015,38sleep02625-21ntp_states19:20:150
2183427018,32sleep20-21swapper/222:04:512
2835126916,36sleep328356-21/usr/sbin/munin22:25:073
1317326925,24sleep00-21swapper/021:35:170
49426820,30sleep2490-21apt-get22:40:002
356926814,37sleep33570-21munin-node22:49:583
3155426810,45sleep331594-21proc_pri19:10:213
3155426810,45sleep331594-21proc_pri19:10:213
1598026815,37sleep216116-21modprobe20:04:502
154526819,32sleep31555-21/usr/sbin/munin00:25:073
1284126812,32sleep00-21swapper/019:54:510
99826723,29sleep01022-21munin-run20:59:480
918526714,37sleep09208-21ntp_states23:05:130
3082426717,27sleep130870-21cyclictest19:09:501
211252674,10sleep30-21swapper/323:44:293
1840526720,23sleep00-21swapper/023:35:090
181402677,12sleep20-21swapper/223:34:512
180726620,28sleep30-21swapper/322:44:493
1716826619,31sleep317202-21ntp_kernel_pll_23:30:153
1375626622,21sleep00-21swapper/019:55:200
964126520,28sleep31-21systemd19:44:493
964126520,28sleep31-21systemd19:44:493
1745626518,31sleep017473-21latency_hist20:09:490
1118726517,32sleep211236-21unin-run19:49:482
277672646,12sleep00-21swapper/020:41:060
2429826412,23sleep20-21swapper/220:30:102
126622645,14sleep20-21swapper/221:34:592
66726318,23sleep30-21swapper/319:15:093
3101026313,17sleep20-21swapper/220:50:132
1516326315,35sleep015176-21vmstat21:40:220
141126322,20sleep00-21swapper/021:00:080
141126322,20sleep00-21swapper/021:00:080
1367226322,24sleep00-21swapper/023:20:030
1001626316,21sleep20-21swapper/223:09:472
347526214,24sleep30-21swapper/319:24:503
302142626,12sleep00-21swapper/022:30:130
2955426211,35sleep329573-21df_inode19:05:043
1026726214,29sleep30-21swapper/321:25:193
1039026117,29sleep310455-21awk19:45:133
91442607,11sleep30-21swapper/321:22:453
351626015,21sleep30-21swapper/300:30:153
2775626017,28sleep227814-21cut00:05:072
265212606,10sleep20-21swapper/200:00:142
2341726015,30sleep223440-21ntp_states23:50:132
1145726014,26sleep30-21swapper/323:13:283
646125917,25sleep06455-21apt-get21:15:020
2223825912,24sleep20-21swapper/222:05:112
1901225913,20sleep00-21swapper/020:14:500
525525817,26sleep35252-21apt-get19:29:593
2123625811,27sleep20-21swapper/220:20:132
2109625813,21sleep30-21swapper/322:00:203
2048525815,23sleep00-21swapper/022:00:050
203425814,25sleep20-21swapper/200:25:182
1762625815,20sleep30-21swapper/321:50:103
1496925816,21sleep20-21swapper/220:00:142
1198125816,24sleep312042-21/usr/sbin/munin21:30:203
321882576,10sleep20-21swapper/219:12:462
321882576,10sleep20-21swapper/219:12:462
3211425716,25sleep332261-21awk22:35:203
2553025715,27sleep20-21swapper/220:35:072
1735625717,23sleep217352-21apt-get21:50:012
3164525620,18sleep20-21swapper/222:35:112
2575325614,18sleep00-21swapper/020:35:130
197925612,19sleep30-21swapper/321:00:203
197925612,19sleep30-21swapper/321:00:203
94862555,10sleep00-21swapper/019:42:480
94862555,10sleep00-21swapper/019:42:480
2185825514,24sleep321890-21ntp_states23:45:133
212925517,23sleep20-21swapper/222:45:052
1770625515,22sleep317773-21df20:10:043
17582555,39sleep00-21swapper/022:44:460
1520525511,28sleep015279-21find20:00:200
1208325511,29sleep212183-21timerandwakeup19:50:212
79072544,15sleep20-21swapper/219:37:452
583025416,24sleep00-21swapper/019:30:160
3128125418,18sleep20-21swapper/200:15:162
1581025418,21sleep00-21swapper/021:45:030
155825420,18sleep00-21swapper/000:25:080
592925312,25sleep06080-21latency_hist00:39:490
592925312,25sleep06080-21latency_hist00:39:490
2813125312,25sleep328169-21grep00:05:163
2488625312,24sleep324895-21munin-run22:14:493
2052025316,17sleep20-21swapper/220:19:492
1941125315,19sleep20-21swapper/221:55:182
1770525312,23sleep217699-21apt-get20:10:032
1302125316,21sleep213015-21apt-get19:55:012
68772529,15sleep00-21swapper/019:35:050
518925216,19sleep20-21swapper/200:35:192
518925216,19sleep20-21swapper/200:35:192
455925212,27sleep34617-21cpuspeed00:35:043
455925212,27sleep34617-21cpuspeed00:35:043
359425211,24sleep23590-21apt-get22:50:012
3123325213,17sleep00-21swapper/000:15:140
2824325213,21sleep028262-21apt-get22:25:000
2204525216,16sleep00-21swapper/023:45:180
2091825215,21sleep320922-21diskstats20:20:063
189125217,19sleep20-21swapper/221:00:192
189125217,19sleep20-21swapper/221:00:192
686325118,19sleep30-21swapper/321:15:143
3236025111,23sleep332353-21apt-get00:20:023
2270025113,22sleep21-21systemd23:49:492
2883925015,15sleep20-21swapper/222:25:182
2513225013,22sleep225128-21apt-get22:15:002
2513225013,22sleep225128-21apt-get22:15:002
1588225014,21sleep215888-21df_inode21:45:052
2460324813,21sleep324610-21df_inode23:55:023
2401124813,20sleep024015-21diskstats20:30:040
234924813,17sleep20-21swapper/219:20:092
1146924810,21sleep211509-21memory21:30:112
2392724711,21sleep323923-21apt-get20:29:593
1006024715,16sleep00-21swapper/021:25:160
2261524614,14sleep30-21swapper/322:05:203
2081324511,19sleep020832-21apt-get20:19:590
1677524511,17sleep20-21swapper/223:30:062
311724211,15sleep20-21swapper/221:05:072
2966524211,15sleep20-21swapper/200:10:122
99232419,15sleep00-21swapper/019:44:580
66162409,12sleep20-21swapper/221:15:092
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional