You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-09 - 08:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Jun 09, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24565293613,906sleep324553-21/usr/sbin/munin20:47:563
3841264817,614sleep33846-21/usr/sbin/munin23:08:133
9627256315,533sleep39671-21ntpq19:58:193
20438227010,245sleep120593-21latency_hist22:17:571
3080722630,15sleep30-21swapper/319:23:223
30615225385,86sleep130660-21fschecks_count00:33:131
511422170,13sleep30-21swapper/321:28:413
16549221416,183sleep016637-21sed22:03:230
18198220716,177sleep118295-21timerwakeupswit22:08:251
587122040,17sleep125321-21cyclictest19:47:541
587122040,17sleep125321-21cyclictest19:47:541
17814219417,90sleep117944-21sort23:52:561
2965721930,6sleep10-21swapper/100:28:241
88221809,154sleep11036-21unixbench_singl21:13:251
768218013,153sleep0881-21cat21:13:240
28886217513,146sleep128895-21/usr/sbin/munin22:43:211
11530217419,129sleep111677-21unixbench_singl20:03:291
28197216671,81sleep128294-21date22:42:561
28030216613,137sleep328090-21fschecks_time20:58:143
28030216613,137sleep328090-21fschecks_time20:58:143
15866216611,140sleep015910-21ntpq20:18:180
27265216521,116sleep00-21swapper/022:38:190
9507216013,132sleep19654-21latency_hist21:42:571
9507216013,132sleep19654-21latency_hist21:42:571
31619215713,13sleep10-21swapper/122:53:141
12003215715,128sleep112047-21fschecks_count23:34:231
12003215715,128sleep112047-21fschecks_count23:34:231
65421558,16sleep10-21swapper/122:58:101
2404121557,114sleep00-21swapper/022:28:180
430215421,110sleep0580-21latency_hist22:57:570
29259215415,121sleep129292-21latency_hist21:02:561
29259215415,121sleep129292-21latency_hist21:02:561
2972121497,15sleep10-21swapper/121:03:161
2972121497,15sleep10-21swapper/121:03:161
2835214917,23sleep10-21swapper/123:03:271
1221214914,115sleep10-21swapper/119:32:571
1221214914,115sleep10-21swapper/119:32:571
8070214312,119sleep18117-21awk19:53:241
1560221439,122sleep1363-21snmpd22:01:471
11000214192,33sleep111003-21cron21:47:541
11000214192,33sleep111003-21cron21:47:541
25570213912,15sleep10-21swapper/122:33:151
802521386,12sleep10-21swapper/123:21:151
32126213814,109sleep132189-21sed19:27:571
3000213891,28sleep13003-21cron21:22:541
39402137107,14sleep10-21swapper/123:08:151
26456213612,38sleep126535-21ls00:18:241
30461213374,43sleep130670-21apt-key19:22:571
16638213279,39sleep116760-21cat22:03:241
1052421279,105sleep110606-21mailstats23:28:221
32114212615,20sleep00-21swapper/019:27:560
29502212514,91sleep30-21swapper/321:03:103
29502212514,91sleep30-21swapper/321:03:103
26537212513,33sleep226663-21sort00:18:252
2597921250,7sleep30-21swapper/319:08:203
1546421255,11sleep10-21swapper/120:17:591
6222212310,100sleep06109-21/usr/sbin/munin19:48:140
10595212215,74sleep20-21swapper/220:02:582
722221219,32sleep17299-21xargs21:33:251
2260321219,100sleep122648-21ntpq22:23:181
20145212115,93sleep120233-21grep23:58:251
26872211813,84sleep10-21swapper/120:53:231
26872211813,84sleep10-21swapper/120:53:231
28313211718,82sleep028349-21ntpq20:58:190
28313211718,82sleep028349-21ntpq20:58:190
2476121165,10sleep10-21swapper/122:29:401
22887211612,90sleep122956-21idleruntime00:08:141
11405211620,73sleep311524-21latency_hist23:32:563
24384211512,86sleep10-21swapper/100:13:151
24384211512,86sleep10-21swapper/100:13:151
1516121158,28sleep115213-21cat23:43:151
14084211312,87sleep214085-21sendmail-mta21:57:402
9528211179,15sleep10-21swapper/119:58:171
13619211112,87sleep113675-21interrupts23:38:191
13619211112,87sleep113675-21interrupts23:38:191
6381211015,74sleep20-21swapper/221:32:572
27519210916,78sleep227551-21tune2fs00:23:132
230621099,86sleep22394-21/usr/sbin/munin23:03:162
2283521096,13sleep00-21swapper/020:41:070
26506210811,15sleep10-21swapper/119:09:201
12320210811,80sleep312369-21ntp_states23:34:293
12320210811,80sleep312369-21ntp_states23:34:293
944221078,88sleep126958-21munin-node23:24:341
944221078,88sleep126958-21munin-node23:24:341
504721079,86sleep14566-21apt-get21:28:031
18551210717,74sleep018630-21sendmail_mailtr23:53:240
18551210717,74sleep018630-21sendmail_mailtr23:53:240
10012210732,41sleep00-21swapper/023:28:040
30832210610,13sleep10-21swapper/122:48:251
27646210611,15sleep10-21swapper/119:13:161
22275210668,20sleep10-21swapper/120:38:251
6282210471,15sleep10-21swapper/119:48:151
429221048,84sleep14319-21munin-run19:42:551
3589210314,73sleep03662-21cat19:38:250
18031210382,10sleep10-21swapper/120:24:251
12841210211,74sleep10-21swapper/121:53:071
2054921018,74sleep10-21swapper/120:33:211
30799210013,72sleep230319-21apt-get19:23:022
26130210015,72sleep326165-21latency_hist20:52:573
2092121007,14sleep10-21swapper/122:18:181
322672998,79sleep10-21swapper/121:12:121
322672998,79sleep10-21swapper/121:12:121
274342978,73sleep10-21swapper/100:23:081
455229612,70sleep14600-21apt-config21:27:551
64902955,10sleep10-21swapper/123:16:471
23732955,66sleep10-21swapper/119:33:301
1541529410,69sleep015447-21proc_pri23:43:230
1133629418,59sleep20-21swapper/223:32:552
186312937,74sleep10-21swapper/123:53:251
186312937,74sleep10-21swapper/123:53:251
1549829313,67sleep315547-21/usr/sbin/munin20:18:093
149992936,11sleep10-21swapper/120:15:321
149992936,11sleep10-21swapper/120:15:321
122362935,11sleep10-21swapper/120:07:571
2608029215,60sleep326209-21awk22:33:253
241942926,12sleep10-21swapper/120:43:421
112952927,34sleep10-21swapper/121:48:101
258662915,10sleep10-21swapper/120:50:121
211622906,72sleep10-21swapper/100:03:021
196872896,10sleep10-21swapper/120:30:461
2938228612,59sleep30-21swapper/300:28:193
2657028614,57sleep026646-21systemd-run22:37:540
2428728510,25sleep20-21swapper/200:13:082
2428728510,25sleep20-21swapper/200:13:082
610828314,52sleep36125-21/usr/sbin/munin19:48:053
1963428113,31sleep20-21swapper/223:58:062
166852810,29sleep316689-21fschecks_time23:48:133
2493228016,29sleep00-21swapper/000:13:260
2493228016,29sleep00-21swapper/000:13:260
2421428022,25sleep10-21swapper/119:03:191
757827918,37sleep27726-21grep23:18:262
2782927912,52sleep027840-21netstat19:13:190
2403327914,51sleep224114-21fschecks_time19:03:162
220127921,28sleep20-21swapper/219:33:272
1621327921,42sleep316247-21idleruntime-cro23:47:543
375127716,28sleep00-21swapper/021:23:220
2904127724,33sleep329038-21apt-get19:18:063
2645127722,35sleep326491-21sort20:53:133
2645127722,35sleep326491-21sort20:53:133
11362775,62sleep00-21swapper/019:32:280
11362775,62sleep00-21swapper/019:32:280
1102327721,33sleep21-21systemd21:47:552
1102327721,33sleep21-21systemd21:47:552
2335727620,30sleep20-21swapper/220:43:152
1437627624,27sleep20-21swapper/220:13:232
1437627624,27sleep20-21swapper/220:13:232
945427515,30sleep20-21swapper/221:42:542
945427515,30sleep20-21swapper/221:42:542
337227518,39sleep33377-21/usr/sbin/munin21:23:123
2789527521,34sleep30-21swapper/300:23:223
2509727518,21sleep00-21swapper/020:48:170
2050827517,43sleep020547-21ntp_kernel_err20:33:200
182927521,35sleep01841-21fschecks_count21:18:130
670327419,27sleep20-21swapper/221:33:132
580927415,43sleep25885-21mailstats23:13:212
204672737,43sleep30-21swapper/320:33:193
1931127319,34sleep219313-21cron23:57:542
1931127319,34sleep219313-21cron23:57:542
1136927316,36sleep011378-21/usr/sbin/munin21:48:130
682627222,38sleep00-21swapper/021:33:150
235172720,7sleep00-21swapper/020:43:160
1421427214,44sleep014313-21sshd23:38:300
1421427214,44sleep014313-21sshd23:38:300
815127115,29sleep00-21swapper/019:53:270
3125127116,38sleep331040-21/usr/sbin/munin21:08:193
3125127116,38sleep331040-21/usr/sbin/munin21:08:193
2649427114,24sleep00-21swapper/020:53:140
2649427114,24sleep00-21swapper/020:53:140
264852717,51sleep00-21swapper/019:09:030
1921227115,30sleep20-21swapper/220:28:252
1680627111,47sleep216829-21sed20:22:562
383427011,45sleep23840-21diskstats23:08:122
1712827017,25sleep30-21swapper/320:23:133
1218727014,41sleep012229-21sort23:34:260
1218727014,41sleep012229-21sort23:34:260
3058826920,20sleep30-21swapper/322:48:233
2995626914,27sleep00-21swapper/022:48:040
2576526924,27sleep20-21swapper/222:33:202
2102726911,43sleep321057-21awk22:18:203
40226830,19sleep30-21swapper/322:57:563
340126815,31sleep23010-21/usr/sbin/munin19:38:192
2418026822,18sleep00-21swapper/019:03:180
1013726813,28sleep30-21swapper/321:43:203
1013726813,28sleep30-21swapper/321:43:203
863426718,20sleep30-21swapper/323:23:153
863426718,20sleep30-21swapper/323:23:153
734826712,40sleep27349-21sshd19:52:202
489326712,20sleep00-21swapper/021:27:580
2753126720,29sleep20-21swapper/222:38:252
259542677,14sleep00-21swapper/000:18:100
2573826721,24sleep20-21swapper/219:08:162
1246026719,24sleep30-21swapper/320:08:163
772926618,18sleep30-21swapper/323:18:273
301682667,51sleep214626-21kworker/2:219:21:082
2402126612,22sleep30-21swapper/322:28:173
1987426611,37sleep019972-21users22:13:240
588626514,23sleep30-21swapper/323:13:223
305126513,23sleep30-21swapper/321:22:563
2688826520,27sleep326912-21cpu22:38:103
1686726514,38sleep216878-21netstat23:48:172
584426415,33sleep05867-21awk19:47:540
584426415,33sleep05867-21awk19:47:540
352126419,28sleep33591-21cut23:07:573
3089626413,31sleep20-21swapper/200:33:182
3036426411,25sleep00-21swapper/019:22:560
30326416,20sleep20-21swapper/221:13:142
2880926421,20sleep20-21swapper/219:17:572
2484626418,21sleep30-21swapper/319:03:293
1478926416,33sleep014813-21netstat21:58:180
921626312,20sleep20-21swapper/223:23:262
921626312,20sleep20-21swapper/223:23:262
58326320,23sleep30-21swapper/319:28:273
58326320,23sleep30-21swapper/319:28:273
278626319,24sleep00-21swapper/023:03:260
1750026315,18sleep20-21swapper/222:08:052
169326315,21sleep00-21swapper/019:33:170
1643726315,21sleep20-21swapper/222:03:212
1418226314,30sleep014246-21cat20:13:170
1418226314,30sleep014246-21cat20:13:170
2405126213,30sleep324185-21latency_hist00:12:573
2191726217,28sleep221944-21munin-run22:22:552
17682626,13sleep20-21swapper/223:00:582
138392628,43sleep00-21swapper/021:54:310
320342618,11sleep00-21swapper/021:09:130
320342618,11sleep00-21swapper/021:09:130
2486126111,22sleep30-21swapper/300:13:253
2486126111,22sleep30-21swapper/300:13:253
2236026120,22sleep20-21swapper/220:38:262
197062618,29sleep37444-1kworker/3:2H23:58:113
1895926116,26sleep30-21swapper/320:28:183
1707426116,23sleep00-21swapper/023:48:230
3115026017,19sleep20-21swapper/221:08:142
3115026017,19sleep20-21swapper/221:08:142
2773426014,22sleep30-21swapper/319:13:183
1573926021,22sleep20-21swapper/222:02:572
2181825922,21sleep30-21swapper/320:38:153
211322598,17sleep20-21swapper/220:34:312
133625918,23sleep00-21swapper/022:58:250
386225820,24sleep20-21swapper/221:23:242
2874425812,22sleep00-21swapper/000:27:550
218472586,12sleep00-21swapper/022:22:000
1773025818,19sleep30-21swapper/322:08:143
1752425815,27sleep017519-21apt-get22:08:070
3135525715,27sleep20-21swapper/222:52:582
2156025713,25sleep30-21swapper/300:03:203
2003525713,28sleep020055-21/usr/sbin/munin23:58:200
182452577,11sleep00-21swapper/020:27:100
1260625714,27sleep30-21swapper/321:52:563
1119025720,21sleep20-21swapper/220:03:222
87882568,37sleep30-21swapper/319:55:563
508525611,20sleep30-21swapper/319:43:253
508525611,20sleep30-21swapper/319:43:253
2995725618,24sleep229955-21ntp_states21:03:212
2995725618,24sleep229955-21ntp_states21:03:212
2532225620,20sleep20-21swapper/220:48:242
2265925613,28sleep322669-21ntp_states22:23:203
829425517,18sleep30-21swapper/321:38:153
829425517,18sleep30-21swapper/321:38:153
375925518,17sleep00-21swapper/023:08:100
3060925511,18sleep00-21swapper/000:33:120
2898125515,24sleep028975-21apt-get00:28:080
2127825523,18sleep20-21swapper/200:03:152
1804925515,23sleep318046-21apt-get23:53:063
1804925515,23sleep318046-21apt-get23:53:063
69125421,15sleep30-21swapper/321:13:233
3236425411,25sleep232357-21apt-get19:28:102
3236425411,25sleep232357-21apt-get19:28:102
2342125411,20sleep00-21swapper/000:08:260
125042549,34sleep20-21swapper/221:52:152
123882546,12sleep20-21swapper/223:34:332
123882546,12sleep20-21swapper/223:34:332
925825312,24sleep09253-21apt-get19:58:070
689225315,22sleep06888-21apt-get23:18:070
2852025317,19sleep20-21swapper/222:43:132
2802825315,23sleep228029-21diskstats20:58:122
2802825315,23sleep228029-21diskstats20:58:122
2663425317,22sleep20-21swapper/220:53:162
2663425317,22sleep20-21swapper/220:53:162
1907125313,24sleep319065-21apt-get22:13:073
1471825313,17sleep20-21swapper/223:42:562
1471825313,17sleep20-21swapper/223:42:562
1413125311,15sleep30-21swapper/320:13:163
1413125311,15sleep30-21swapper/320:13:163
1236425313,24sleep012360-21apt-get20:08:090
660925214,22sleep36612-21apt-get21:33:063
527925216,16sleep00-21swapper/023:13:030
3151625210,15sleep20-21swapper/222:53:062
310325213,23sleep33110-21df_inode19:38:123
303122528,17sleep00-21swapper/021:03:270
303122528,17sleep00-21swapper/021:03:270
2864625212,24sleep028702-21irqstats22:43:150
1507625216,17sleep30-21swapper/321:58:263
1369225211,22sleep313776-21memory23:38:213
1369225211,22sleep313776-21memory23:38:213
1089125213,24sleep310897-21df_inode20:03:153
1081825213,23sleep010812-21apt-get20:03:120
2842925112,23sleep328427-21apt-get22:43:063
2714525111,24sleep327168-21munin-run00:22:543
249672514,38sleep20-21swapper/222:32:192
1835625114,20sleep218326-21latency_hist20:27:572
110025117,18sleep30-21swapper/322:58:203
776025011,14sleep20-21swapper/219:53:132
2295725013,23sleep223008-21irqstats00:08:152
2220725017,18sleep00-21swapper/022:23:090
1618125015,20sleep316234-21interrupts22:03:143
1630724910,16sleep20-21swapper/223:47:572
3052724810,22sleep330524-21apt-get00:33:073
1950724811,15sleep20-21swapper/222:13:182
1859124812,20sleep018587-21apt-get20:28:070
149024812,20sleep31529-21df19:33:123
999924712,20sleep010013-21irqstats21:43:170
999924712,20sleep010013-21irqstats21:43:170
2898324715,15sleep20-21swapper/200:28:102
2220424610,21sleep222200-21apt-get22:23:072
2125924611,20sleep021270-21df_inode00:03:130
1284324414,17sleep30-21swapper/321:53:093
901924312,19sleep09088-21switchtime23:23:240
901924312,19sleep09088-21switchtime23:23:240
458724310,19sleep24654-21df19:43:122
458724310,19sleep24654-21df19:43:122
1236624210,17sleep20-21swapper/220:08:112
868324111,16sleep00-21swapper/021:38:240
868324111,16sleep00-21swapper/021:38:240
300824113,15sleep20-21swapper/221:22:552
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional