You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 23:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun May 24, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7029288028,8761sleep21156-21kworker/2:107:27:072
240432293110,2903sleep06788-21kworker/0:211:44:210
240432293110,2903sleep06788-21kworker/0:211:44:210
10638210158,995sleep210674-21ntpq11:04:172
13031290026,838sleep213072-21ntpq09:29:152
17261276515,732sleep117262-21ntp_states07:59:211
17261276515,732sleep117262-21ntp_states07:59:211
6592262319,586sleep06720-21/usr/sbin/munin07:24:220
293772474179,200sleep229403-21modprobe12:03:492
1219724480,10sleep311784-21/usr/sbin/munin11:09:173
3788239037,267sleep23914-21latency_hist10:43:532
26626234635,216sleep226666-21fschecks_count11:54:122
26626234635,216sleep226666-21fschecks_count11:54:122
6711232525,281sleep16791-21cut12:34:091
2252323100,8sleep00-21swapper/009:59:120
18441230317,208sleep218479-21fschecks_count08:04:102
18441230317,208sleep218479-21fschecks_count08:04:102
3712225812,119sleep20-21swapper/208:59:232
26969224544,184sleep227063-21fschecks_time10:14:092
22290222813,200sleep322332-21ntp_offset11:39:153
16329222430,138sleep216402-21switchtime09:39:202
7727222124,148sleep27763-21latency07:29:132
7727222124,148sleep27763-21latency07:29:132
25875222115,184sleep20-21swapper/210:09:202
25875222115,184sleep20-21swapper/210:09:202
1728222040,139sleep21761-21latency08:54:142
15657221014,183sleep215720-21cut11:19:192
13068219410,91sleep213086-21modprobe11:13:502
2443219176,25sleep20-21swapper/207:13:582
19903218982,67sleep219956-21cpuspeed08:09:072
19903218982,67sleep219956-21cpuspeed08:09:072
20396218629,140sleep220518-21date09:53:542
5084218122,17sleep20-21swapper/209:04:402
3012217910,157sleep3363-21snmpd12:21:153
619121775,154sleep00-21swapper/010:49:210
13775217670,28sleep20-21swapper/207:49:122
29064217312,144sleep029138-21find10:19:200
10149217347,84sleep20-21swapper/209:19:222
2550621677,118sleep223689-1kworker/2:0H11:49:202
59521638,117sleep2758-21date07:08:532
771221616,142sleep27803-21cat09:13:532
771221616,142sleep27803-21cat09:13:532
2311521597,112sleep20-21swapper/208:19:082
8975215745,27sleep20-21swapper/210:59:162
2157421578,134sleep221635-21kworker/u8:311:38:502
7509215611,133sleep0363-21snmpd09:11:570
7509215611,133sleep0363-21snmpd09:11:570
658215615,125sleep3802-21latency_hist10:33:533
2138215335,23sleep20-21swapper/207:13:542
16921215052,19sleep20-21swapper/211:24:162
18491214818,109sleep318006-21apt-get11:29:013
18491214818,109sleep318006-21apt-get11:29:013
22357214763,17sleep20-21swapper/211:39:182
12064214229,102sleep2363-21snmpd09:25:192
595621409,55sleep25992-21ntpq10:49:142
24661214011,42sleep20-21swapper/208:24:092
28922213925,102sleep2363-21snmpd08:38:392
28922213925,102sleep2363-21snmpd08:38:392
29253213423,95sleep20-21swapper/208:39:042
29253213423,95sleep20-21swapper/208:39:042
25075213430,90sleep225095-21cat10:08:522
25075213430,90sleep225095-21cat10:08:522
14459213410,83sleep20-21swapper/211:14:392
20388213316,99sleep319877-21/usr/sbin/munin08:09:203
20388213316,99sleep319877-21/usr/sbin/munin08:09:203
9216213022,86sleep20-21swapper/207:34:132
20962213015,100sleep121060-21timerandwakeup11:34:451
8800212816,96sleep38833-21fschecks_count10:59:133
3689212616,85sleep20-21swapper/212:24:122
21672212617,11sleep20-21swapper/209:55:562
21672212617,11sleep20-21swapper/209:55:562
1891921259,89sleep20-21swapper/211:29:372
1891921259,89sleep20-21swapper/211:29:372
6268212421,81sleep16417-21tail10:49:221
5291212410,53sleep25311-21modprobe10:48:502
10858212324,23sleep20-21swapper/207:39:182
23528212212,93sleep123578-21fschecks_count11:44:101
23528212212,93sleep123578-21fschecks_count11:44:101
14253212221,15sleep20-21swapper/209:34:072
3135321217,81sleep231385-21tune2fs12:09:102
15853211817,18sleep00-21swapper/009:39:100
4059211612,89sleep34140-21cat12:24:223
27391211532,49sleep227394-21cron08:33:512
4956211420,64sleep10-21swapper/109:04:351
32216211322,47sleep20-21swapper/210:29:522
1276121129,88sleep212885-21timerwakeupswit07:44:232
23235211117,78sleep123279-21/usr/sbin/munin08:19:111
4828211014,24sleep20-21swapper/207:19:192
4828211014,24sleep20-21swapper/207:19:192
2215721109,88sleep00-21swapper/011:39:120
7430210914,79sleep37449-21apt-get07:29:013
7430210914,79sleep37449-21apt-get07:29:013
13315210813,81sleep113455-21sed09:29:221
543210727,65sleep1592-21latency_hist07:08:521
17300210633,45sleep117295-21apt-get09:44:041
17300210633,45sleep117295-21apt-get09:44:041
21242210512,75sleep021316-21cat09:54:490
21242210512,75sleep021316-21cat09:54:490
1638721037,60sleep20-21swapper/207:57:402
1638721037,60sleep20-21swapper/207:57:402
29980210214,71sleep130000-21apt10:24:011
19913210213,73sleep119915-21gpgv11:33:521
19913210213,73sleep119915-21gpgv11:33:521
17389210123,38sleep20-21swapper/209:44:092
17389210123,38sleep20-21swapper/209:44:092
30662210024,40sleep230690-21apt-config12:05:112
226692997,77sleep322671-21ntp_states09:59:163
3262529832,50sleep232712-21latency08:49:132
112129821,33sleep20-21swapper/210:34:122
2315429720,12sleep20-21swapper/209:59:432
486829621,42sleep30-21swapper/309:04:293
284502966,74sleep20-21swapper/210:19:022
2768029527,42sleep127674-21apt-get08:34:061
2768029527,42sleep127674-21apt-get08:34:061
2463729515,12sleep20-21swapper/211:48:022
2463729515,12sleep20-21swapper/211:48:022
227972947,45sleep222879-21latency_hist08:18:522
72012938,47sleep27211-21sshd10:54:072
340129320,49sleep03517-21timerwakeupswit07:14:200
614329217,11sleep20-21swapper/212:31:352
1777229248,25sleep20-21swapper/208:00:062
1777229248,25sleep20-21swapper/208:00:062
962929127,38sleep30-21swapper/309:19:113
87529110,65sleep0894-21apt-get10:34:020
312692908,43sleep231328-21systemctl10:28:512
3113129010,66sleep331133-21sshd10:27:063
950028917,54sleep09615-21/usr/sbin/munin10:59:260
85642898,66sleep28684-21cat09:14:212
19512888,37sleep20-21swapper/212:19:042
135982889,16sleep10-21swapper/111:14:051
1084228722,43sleep010910-21latency_hist09:23:520
48328633,35sleep01-21systemd07:08:510
472728616,23sleep30-21swapper/310:44:233
1485028616,53sleep014976-21timerwakeupswit09:34:220
70912858,40sleep27176-21mailstats12:34:182
2632928514,56sleep326474-21cat11:53:533
231782858,65sleep122400-1kworker/1:2H09:59:541
2257528515,54sleep122725-21sed11:39:211
1747828523,40sleep117609-21head11:24:271
688828425,26sleep30-21swapper/310:53:523
3257228314,55sleep332676-21latency_hist12:13:523
2690728315,47sleep30-21swapper/311:54:173
2690728315,47sleep30-21swapper/311:54:173
2415628315,54sleep124200-21ntpq10:04:151
2415628315,54sleep124200-21ntpq10:04:151
554228223,16sleep30-21swapper/312:29:183
323528222,28sleep10-21swapper/107:14:171
17082827,41sleep21715-21munin-run12:18:512
2060928114,50sleep120124-21apt-get09:54:011
2060928114,50sleep120124-21apt-get09:54:011
176928122,27sleep00-21swapper/012:18:530
1572128115,48sleep315792-21sensors_temp11:19:203
1126528111,56sleep311302-21idleruntime09:24:113
361328011,25sleep10-21swapper/108:59:221
3141228021,24sleep00-21swapper/010:28:530
2581228012,49sleep125874-21/usr/sbin/munin10:09:191
2581228012,49sleep125874-21/usr/sbin/munin10:09:191
332627915,43sleep33397-21ls07:14:193
2997727920,33sleep00-21swapper/008:39:220
2997727920,33sleep00-21swapper/008:39:220
2748427826,30sleep10-21swapper/110:14:201
1323927812,52sleep013314-21grep09:29:210
2042127712,16sleep30-21swapper/311:34:313
1475727716,48sleep314846-21switchtime09:34:213
794527613,42sleep08069-21cat12:38:520
105427617,45sleep11120-21sort10:34:111
567427520,25sleep00-21swapper/007:23:530
567427520,25sleep00-21swapper/007:23:530
278927419,37sleep32817-21latency_hist08:58:523
1026027420,31sleep30-21swapper/311:04:053
534327322,23sleep00-21swapper/010:48:520
2154027318,41sleep121628-21tune2fs08:14:111
1966427320,25sleep00-21swapper/008:08:530
1966427320,25sleep00-21swapper/008:08:530
1780527318,32sleep00-21swapper/009:44:190
1780527318,32sleep00-21swapper/009:44:190
1776427316,39sleep317801-21ps09:44:183
1776427316,39sleep317801-21ps09:44:183
1376427314,35sleep113653-21/usr/sbin/munin07:49:111
3166727223,30sleep00-21swapper/012:09:160
2821427214,27sleep00-21swapper/010:18:520
164672726,11sleep00-21swapper/007:58:390
164672726,11sleep00-21swapper/007:58:390
1249827221,24sleep30-21swapper/307:44:163
1034227218,23sleep10-21swapper/107:38:531
445227117,33sleep14532-21tune2fs07:19:101
445227117,33sleep14532-21tune2fs07:19:101
359127113,41sleep03596-21/usr/sbin/munin12:24:100
3087727119,32sleep130886-21/usr/sbin/munin08:44:071
2458827116,39sleep024590-21cpu08:24:070
386827012,42sleep13877-21/usr/sbin/munin12:24:151
1343927014,26sleep00-21swapper/007:48:530
1054127021,23sleep00-21swapper/011:04:150
799126911,28sleep10-21swapper/107:29:201
799126911,28sleep10-21swapper/107:29:201
43892690,10sleep30-21swapper/309:03:513
2981526917,25sleep30-21swapper/312:04:113
2513626919,32sleep125198-21/usr/sbin/munin11:49:101
2509626922,23sleep30-21swapper/310:08:533
2509626922,23sleep30-21swapper/310:08:533
90726820,21sleep30-21swapper/307:09:143
327262686,46sleep00-21swapper/008:49:150
3168226826,22sleep30-21swapper/307:04:063
3080326820,33sleep20-21swapper/208:44:042
2808126818,30sleep30-21swapper/308:34:163
2808126818,30sleep30-21swapper/308:34:163
1735726815,38sleep017477-21cat11:24:260
1422926815,22sleep00-21swapper/011:14:370
1036226821,26sleep110449-21fschecks_time11:04:121
759426711,16sleep00-21swapper/010:54:170
2744326721,26sleep00-21swapper/008:33:530
2087626717,31sleep30-21swapper/309:54:393
2087626717,31sleep30-21swapper/309:54:393
182826720,30sleep01861-21/usr/sbin/munin08:54:160
1493326717,36sleep014959-21munin-run07:53:500
122126720,24sleep10-21swapper/107:09:211
911326615,28sleep10-21swapper/110:59:191
3026626618,23sleep10-21swapper/112:04:211
2896626620,27sleep329060-21grep10:19:193
2674126618,29sleep326821-21timerandwakeup08:29:233
1289326623,19sleep30-21swapper/309:29:123
1058426617,25sleep00-21swapper/007:39:080
897126523,21sleep10-21swapper/109:15:501
373526517,33sleep33742-21modprobe10:43:503
2271926519,31sleep022775-21tr08:18:500
1239026513,24sleep00-21swapper/009:28:530
609526417,30sleep06103-21modprobe09:08:500
2952626418,24sleep30-21swapper/308:39:133
2952626418,24sleep30-21swapper/308:39:133
2868926416,19sleep00-21swapper/011:59:230
2507626419,23sleep00-21swapper/011:49:090
1879526420,19sleep00-21swapper/011:29:350
1879526420,19sleep00-21swapper/011:29:350
1418326415,33sleep114252-21df09:34:061
909426322,18sleep00-21swapper/007:34:110
1891226320,23sleep10-21swapper/108:04:211
1891226320,23sleep10-21swapper/108:04:211
121926316,33sleep11281-21sed08:53:531
83672627,10sleep10-21swapper/110:58:341
83026212,34sleep1852-21/usr/sbin/munin12:14:181
64826220,22sleep30-21swapper/308:49:233
1720326214,26sleep30-21swapper/307:59:183
1720326214,26sleep30-21swapper/307:59:183
1494526220,29sleep314886-21latency_hist11:18:523
2831026119,26sleep30-21swapper/311:59:143
2595026115,32sleep326078-21timerwakeupswit10:09:213
2595026115,32sleep326078-21timerwakeupswit10:09:213
846526022,20sleep30-21swapper/310:58:533
1640626017,20sleep30-21swapper/309:39:213
1582126016,26sleep115832-21df_inode09:39:071
305625924,18sleep10-21swapper/110:39:201
2711925918,21sleep10-21swapper/111:54:231
2711925918,21sleep10-21swapper/111:54:231
206225912,29sleep02163-21perl12:19:100
195025915,28sleep11947-21apt-get12:19:031
127352595,11sleep10-21swapper/111:09:291
652025816,23sleep16540-21grep09:09:361
652025816,23sleep16540-21grep09:09:361
3252125818,20sleep10-21swapper/112:13:501
247325817,25sleep02469-21apt-get10:39:030
1567125817,19sleep30-21swapper/307:54:183
1567125817,19sleep30-21swapper/307:54:183
1095125823,20sleep30-21swapper/307:39:193
900425721,19sleep30-21swapper/307:34:063
722125720,18sleep30-21swapper/307:28:533
5802575,11sleep10-21swapper/110:33:341
457225717,22sleep14641-21sensors_temp10:44:211
3104425715,26sleep00-21swapper/012:08:520
2313525711,28sleep023233-21fschecks_time08:19:100
627825618,20sleep10-21swapper/107:24:151
621925616,23sleep36361-21modprobe09:08:533
621925616,23sleep36361-21modprobe09:08:533
505525614,24sleep15049-21apt-get12:29:041
2704525621,19sleep00-21swapper/011:54:220
2704525621,19sleep00-21swapper/011:54:220
2541725617,20sleep00-21swapper/010:09:100
2541725617,20sleep00-21swapper/010:09:100
2351225617,24sleep323521-21df_inode11:44:083
2351225617,24sleep323521-21df_inode11:44:083
18182568,18sleep30-21swapper/310:34:253
171625612,32sleep31767-21latency_hist12:18:523
1135225621,18sleep10-21swapper/109:24:131
3126825514,21sleep00-21swapper/008:44:190
2944025515,19sleep10-21swapper/108:39:111
2944025515,19sleep10-21swapper/108:39:111
1545625514,16sleep00-21swapper/011:19:130
1268525512,21sleep10-21swapper/107:44:221
3184825414,24sleep031784-21apt-get10:29:420
1970425415,25sleep319753-21users09:49:233
1318925417,21sleep013275-21systemd-run11:13:520
2365125315,16sleep30-21swapper/308:19:223
2035925315,23sleep120365-21ntp_states08:09:181
2035925315,23sleep120365-21ntp_states08:09:181
186225315,18sleep10-21swapper/108:54:171
953325213,24sleep19528-21apt-get09:19:051
93625214,20sleep00-21swapper/007:09:150
685225213,23sleep36875-21irqstats12:34:113
2686825213,23sleep326865-21apt-get10:14:023
2387025212,22sleep023961-21perl10:04:090
2387025212,22sleep023961-21perl10:04:090
921625112,24sleep39244-21munin-run09:18:503
566125115,23sleep31-21systemd07:23:523
566125115,23sleep31-21systemd07:23:523
2891625114,22sleep128922-21ntp_states10:19:171
1945925112,19sleep10-21swapper/109:49:201
85892506,33sleep00-21swapper/007:32:130
85892506,33sleep00-21swapper/007:32:130
2794425012,25sleep027962-21kernelversion08:34:130
2794425012,25sleep027962-21kernelversion08:34:130
2485625015,19sleep30-21swapper/308:24:143
1715825013,23sleep317195-21ntp_states11:24:213
561624916,17sleep00-21swapper/012:29:190
400924919,15sleep00-21swapper/009:00:060
3206324911,15sleep30-21swapper/308:48:503
2838824912,19sleep10-21swapper/111:59:161
1990124913,19sleep00-21swapper/008:09:050
1990124913,19sleep00-21swapper/008:09:050
1926024911,16sleep00-21swapper/009:49:150
1534724914,17sleep10-21swapper/107:54:121
1534724914,17sleep10-21swapper/107:54:121
1527624911,22sleep115368-21perl11:19:101
2495624813,14sleep10-21swapper/108:24:161
1680124813,18sleep00-21swapper/007:59:070
1680124813,18sleep00-21swapper/007:59:070
813424717,18sleep00-21swapper/009:14:110
269812477,15sleep10-21swapper/108:29:251
2042924611,21sleep020435-21diskstats11:34:320
1848324512,18sleep30-21swapper/308:04:113
1848324512,18sleep30-21swapper/308:04:113
110102437,13sleep10-21swapper/107:39:211
136742406,16sleep30-21swapper/307:49:053
2185623910,17sleep30-21swapper/308:14:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional