You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 14:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Thu Jul 23, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24697212806,1240sleep023909-1kworker/0:0H19:18:550
4043265487,550sleep34044-21ntp_states21:38:413
8361264443,589sleep08398-21ntpq20:08:400
28701253245,473sleep028788-21apt-config23:03:140
22918246616,389sleep00-21swapper/019:13:350
1610623139,120sleep020635-21cyclictest00:03:290
13730229912,272sleep213732-21sshd23:54:592
16233228393,148sleep016295-21processes20:33:430
178922807,262sleep01833-21ntpq23:18:400
7771227043,213sleep07921-21latency_hist20:08:160
32445222483,21sleep00-21swapper/023:13:350
23188221922,178sleep123336-21unixbench_singl00:28:451
23188221922,178sleep123336-21unixbench_singl00:28:451
2505221123,161sleep32668-21grep19:49:303
2505221123,161sleep32668-21grep19:49:303
28732220328,117sleep00-21swapper/021:13:430
15576220112,91sleep015614-21http00:03:120
8053219818,116sleep08068-21apt-key21:53:150
26659218452,25sleep00-21swapper/019:23:400
3082218115,152sleep23149-21sort23:23:342
11370217612,122sleep011511-21latency_hist22:03:160
11370217612,122sleep011511-21latency_hist22:03:160
2483821749,150sleep024884-21systemctl21:03:140
290421686,151sleep0364-21snmpd23:23:200
3479216611,88sleep00-21swapper/019:53:350
19093216563,88sleep019138-21latency_hist00:18:140
21994216313,131sleep00-21swapper/020:53:290
28215216115,103sleep00-21swapper/019:28:400
2291216115,95sleep00-21swapper/019:49:270
2291216115,95sleep00-21swapper/019:49:270
1121421520,34sleep211218-21fschecks_count20:18:302
11910215025,96sleep112054-21grep23:48:441
2643621488,99sleep00-21swapper/021:08:140
12380214812,118sleep012402-21awk20:23:140
17876213811,30sleep00-21swapper/020:38:440
2637021318,111sleep026438-21switchtime22:54:060
21363212829,20sleep00-21swapper/019:08:420
20196212723,79sleep320322-21latency_hist20:48:153
2138721256,106sleep00-21swapper/020:49:090
28633212318,87sleep328699-21apt-config23:03:133
2333721237,106sleep0364-21snmpd22:46:270
16104212217,87sleep315591-21apt-get00:03:283
1055321226,17sleep00-21swapper/023:43:510
23971212027,72sleep323489-21apt-get22:48:243
21927212013,91sleep221975-21systemd-run22:43:142
2498721195,11sleep00-21swapper/022:49:420
20895211940,56sleep00-21swapper/000:23:270
20895211940,56sleep00-21swapper/000:23:270
1729121195,11sleep00-21swapper/000:11:040
1729121195,11sleep00-21swapper/000:11:040
3182621186,17sleep00-21swapper/023:11:210
27049211817,85sleep327083-21idleruntime-cro22:58:133
1284321186,16sleep00-21swapper/022:08:040
1284321186,16sleep00-21swapper/022:08:040
2190921176,17sleep00-21swapper/022:43:000
3708211616,81sleep00-21swapper/021:38:340
2925521165,16sleep00-21swapper/023:03:380
1717211510,25sleep01732-21idleruntime-cro21:33:130
22416211415,79sleep322444-21/usr/sbin/munin22:43:473
13261211410,84sleep012702-21/usr/sbin/munin23:53:440
930721126,38sleep00-21swapper/023:42:460
930721126,38sleep00-21swapper/023:42:460
1885521127,11sleep00-21swapper/020:43:200
3206721116,11sleep00-21swapper/019:42:210
1053321115,40sleep00-21swapper/020:14:500
22418211010,18sleep00-21swapper/000:28:230
22418211010,18sleep00-21swapper/000:28:230
5568210919,69sleep25725-21tail19:58:462
982121089,84sleep09741-21latency_hist21:58:150
9719210816,66sleep19513-21/usr/sbin/munin20:13:341
4860210717,71sleep14861-21/usr/sbin/munin19:58:241
307221075,91sleep00-21swapper/021:35:430
376421068,13sleep20-21swapper/219:53:412
31077210623,58sleep1225-21systemd-journal21:23:131
26786210614,76sleep226788-21sshd22:54:492
234721067,13sleep10-21swapper/119:49:281
234721067,13sleep10-21swapper/119:49:281
585021046,85sleep00-21swapper/019:59:200
26742210423,66sleep00-21swapper/021:08:310
1369921049,17sleep00-21swapper/022:08:450
1369921049,17sleep00-21swapper/022:08:450
123121049,80sleep01257-21munin-run19:48:140
9246210312,76sleep19280-21idleruntime-cro20:13:131
25330210314,75sleep225387-21cat00:38:142
11448210238,16sleep00-21swapper/023:48:350
448421018,22sleep020635-21cyclictest23:28:290
1824921018,18sleep00-21swapper/022:23:380
1824921018,18sleep00-21swapper/022:23:380
653829910,14sleep00-21swapper/023:33:500
2137529928,23sleep10-21swapper/100:23:401
2137529928,23sleep10-21swapper/100:23:401
1398429929,48sleep114008-21munin-run23:58:131
298162988,15sleep00-21swapper/021:18:270
2112629817,66sleep121249-21timerwakeupswit20:48:501
3110829716,55sleep330686-21/usr/sbin/munin23:08:383
2550429714,64sleep225539-21/usr/sbin/munin21:03:442
207029723,51sleep22231-21unixbench_multi23:18:462
2420229613,64sleep324205-21/usr/sbin/munin00:33:353
2134029613,62sleep30-21swapper/319:08:413
1344229611,66sleep00-21swapper/020:23:460
807729515,66sleep38125-21fschecks_count20:08:353
201029515,66sleep22026-21cpu21:33:312
2923329417,54sleep129133-21/usr/sbin/munin23:03:311
239952947,12sleep00-21swapper/000:33:280
1458529433,21sleep00-21swapper/020:28:390
787629312,65sleep28031-21cat23:38:162
787629312,65sleep28031-21cat23:38:162
939629224,22sleep10-21swapper/123:43:151
939629224,22sleep10-21swapper/123:43:151
1805729212,60sleep218205-21grep20:38:462
53029121,46sleep3670-21sed23:13:473
51992917,16sleep00-21swapper/021:43:300
51992917,16sleep00-21swapper/021:43:300
170982917,11sleep00-21swapper/022:18:460
170982917,11sleep00-21swapper/022:18:460
557529017,55sleep15582-21ntp_states21:43:391
557529017,55sleep15582-21ntp_states21:43:391
2620729018,59sleep126263-21cpuspeed19:23:281
157592905,73sleep00-21swapper/022:15:390
1407229025,31sleep30-21swapper/323:58:153
2526228934,29sleep125325-21readlink22:53:141
2517528915,57sleep325190-21/usr/sbin/munin19:19:233
2462728715,55sleep224118-21apt-get19:18:252
199652876,10sleep00-21swapper/022:33:190
313682856,63sleep00-21swapper/021:23:280
304452856,12sleep00-21swapper/019:36:430
2252728517,53sleep322538-21awk00:28:303
2252728517,53sleep322538-21awk00:28:303
196692856,10sleep00-21swapper/022:32:420
1636328526,25sleep30-21swapper/322:18:333
1636328526,25sleep30-21swapper/322:18:333
489128414,51sleep14954-21latency_hist21:43:151
1430228436,30sleep30-21swapper/320:28:333
127128413,19sleep10-21swapper/119:48:151
203092835,10sleep00-21swapper/000:19:350
203092835,10sleep00-21swapper/000:19:350
1562828316,52sleep215783-21sed00:03:142
147328320,41sleep11488-21fschecks_count23:18:331
818028216,49sleep18375-21modprobe21:53:161
2084628132,28sleep30-21swapper/320:48:443
859928019,31sleep20-21swapper/220:08:472
38127913,46sleep3423-21/usr/sbin/munin19:43:413
2134327923,38sleep321475-21tail22:38:433
1944927922,16sleep30-21swapper/319:03:333
1032627920,34sleep20-21swapper/223:43:452
3056127817,17sleep10-21swapper/119:38:141
2522027821,33sleep10-21swapper/121:03:391
1397827823,34sleep114065-21wc20:28:151
1323627814,42sleep213120-21/usr/sbin/munin22:08:352
1323627814,42sleep213120-21/usr/sbin/munin22:08:352
705327716,45sleep37098-21grep21:48:383
2833127717,41sleep328438-21grep21:13:343
1992827620,24sleep10-21swapper/119:03:431
54127526,25sleep10-21swapper/121:28:341
3112727521,28sleep30-21swapper/321:23:153
2231627511,49sleep122320-21/usr/sbin/munin22:43:451
979227425,27sleep30-21swapper/321:58:143
1653827420,27sleep20-21swapper/222:18:362
1653827420,27sleep20-21swapper/222:18:362
1164527419,25sleep20-21swapper/222:03:302
1164527419,25sleep20-21swapper/222:03:302
1133927416,23sleep20-21swapper/223:48:332
3243227313,44sleep232502-21df19:43:322
3218227325,27sleep20-21swapper/219:43:152
2828927316,39sleep228361-21/usr/sbin/munin19:28:422
2737027317,30sleep10-21swapper/122:58:311
1248327327,25sleep312522-21cat23:53:143
3164927218,27sleep10-21swapper/119:38:461
1023727222,20sleep10-21swapper/123:43:441
328427123,24sleep30-21swapper/319:53:263
3202327117,37sleep132153-21latency_hist23:13:161
1176127123,19sleep10-21swapper/120:18:431
696626912,33sleep17000-21timerandwakeup23:33:591
326242699,32sleep10-21swapper/121:28:131
242626918,31sleep22504-21/usr/sbin/munin19:49:292
242626918,31sleep22504-21/usr/sbin/munin19:49:292
690026810,15sleep30-21swapper/320:03:433
3064226825,24sleep30-21swapper/319:38:163
2803426813,38sleep128183-21cut21:13:161
1941626814,25sleep10-21swapper/100:18:311
1941626814,25sleep10-21swapper/100:18:311
318432676,11sleep20-21swapper/221:23:422
2836226717,30sleep328473-21sort19:28:433
2566926710,16sleep30-21swapper/321:03:493
1243726717,22sleep30-21swapper/320:23:173
2609326624,22sleep10-21swapper/122:53:591
219126613,41sleep32237-21basename21:33:363
2097726625,22sleep30-21swapper/300:23:303
2097726625,22sleep30-21swapper/300:23:303
1970026618,35sleep319716-21netstat00:18:373
1970026618,35sleep319716-21netstat00:18:373
1585026623,19sleep30-21swapper/320:33:343
1477426620,21sleep20-21swapper/222:13:312
702126521,24sleep20-21swapper/220:03:462
3222826526,20sleep30-21swapper/319:43:163
2403726520,23sleep30-21swapper/320:58:413
2280626515,31sleep322824-21/usr/sbin/munin19:13:253
1217226519,23sleep30-21swapper/322:03:433
1217226519,23sleep30-21swapper/322:03:433
1169126515,32sleep111800-21perl22:03:331
1169126515,32sleep111800-21perl22:03:331
1130526514,31sleep111326-21munin-plugin-st22:03:131
1130526514,31sleep111326-21munin-plugin-st22:03:131
995626418,27sleep29989-21sshd20:13:392
963626412,24sleep30-21swapper/323:43:303
47512648,11sleep30-21swapper/323:28:363
291126419,21sleep10-21swapper/123:23:251
2427426416,15sleep20-21swapper/222:48:392
2238426413,25sleep10-21swapper/120:53:381
1937526424,20sleep20-21swapper/219:03:322
1907826416,32sleep219085-21modprobe22:28:122
1907826416,32sleep219085-21modprobe22:28:122
186462647,11sleep20-21swapper/200:13:442
2976626326,18sleep30-21swapper/323:04:063
2256726314,35sleep222649-21switchtime20:53:432
1997226334,13sleep20-21swapper/200:18:452
1997226334,13sleep20-21swapper/200:18:452
385826216,27sleep10-21swapper/119:53:441
2096126220,22sleep20-21swapper/219:08:322
1794826218,25sleep118056-21wc20:38:451
1040726219,26sleep20-21swapper/221:58:382
1040726219,26sleep20-21swapper/221:58:382
675226120,20sleep20-21swapper/223:33:562
647826119,17sleep20-21swapper/221:48:162
528026119,25sleep35301-21/usr/sbin/munin19:58:383
2670526118,23sleep20-21swapper/219:23:422
2479926119,19sleep10-21swapper/119:19:131
2202326122,18sleep10-21swapper/122:43:251
1774826120,24sleep10-21swapper/100:13:271
793526013,19sleep20-21swapper/221:53:132
462926018,23sleep10-21swapper/123:28:341
41326024,19sleep20-21swapper/223:13:462
2977126018,22sleep20-21swapper/219:33:432
266602609,21sleep30-21swapper/321:08:253
1758726024,19sleep10-21swapper/122:23:151
1758726024,19sleep10-21swapper/122:23:151
1752126013,33sleep317551-21ls20:38:343
149842606,12sleep30-21swapper/323:58:433
1363226010,32sleep313693-21sendmail_mailst22:08:443
1363226010,32sleep313693-21sendmail_mailst22:08:443
318592597,11sleep20-21swapper/223:11:482
2283525917,21sleep10-21swapper/119:13:311
1603325916,23sleep20-21swapper/220:33:372
1480825916,23sleep114884-21/usr/sbin/munin23:58:411
1291625916,24sleep212939-21irqstats20:23:362
2492625815,20sleep20-21swapper/221:03:162
1928825816,25sleep019284-21apt-get19:03:260
1908625816,26sleep319129-21munin-run22:28:133
1908625816,26sleep319129-21munin-run22:28:133
1809425814,27sleep118186-21/usr/sbin/munin22:23:351
1809425814,27sleep118186-21/usr/sbin/munin22:23:351
176825818,25sleep21846-21cat21:33:152
1014925813,29sleep310269-21timerandwakeup20:13:443
816125716,24sleep38168-21df_inode23:38:333
816125716,24sleep38168-21df_inode23:38:333
2790025716,20sleep10-21swapper/119:28:331
2245025715,24sleep222509-21users19:13:202
2044225714,18sleep20-21swapper/220:48:352
2014525716,24sleep120171-21munin-run20:48:131
1933725716,25sleep219343-21ntp_states20:43:382
1608925715,24sleep216101-21apt-get00:03:272
152252575,15sleep10-21swapper/120:30:011
2982025613,28sleep229869-21cpuspeed21:18:302
2034025615,25sleep220373-21munin-run22:38:132
1526625614,29sleep10-21swapper/122:13:421
138572569,19sleep30-21swapper/320:27:203
112332569,36sleep3364-21snmpd22:02:183
112332569,36sleep3364-21snmpd22:02:183
51032555,13sleep20-21swapper/221:43:242
365725514,24sleep23667-21diskstats21:38:322
3233625514,24sleep132342-21df_inode23:13:321
3177725512,17sleep30-21swapper/321:23:393
1799425511,18sleep30-21swapper/300:13:333
153252559,33sleep20-21swapper/220:31:192
1377725513,19sleep10-21swapper/122:08:461
1377725513,19sleep10-21swapper/122:08:461
2823425415,23sleep128228-21apt-get21:13:281
2402025414,16sleep10-21swapper/120:58:391
2334125416,17sleep20-21swapper/200:28:462
2334125416,17sleep20-21swapper/200:28:462
694825315,22sleep17037-21/usr/sbin/munin21:48:361
627025315,21sleep30-21swapper/323:33:283
2916925310,28sleep229171-21sshd21:14:062
2097025314,22sleep220976-21diskstats00:23:292
2097025314,22sleep220976-21diskstats00:23:292
1515125316,16sleep30-21swapper/322:13:403
2668525212,24sleep226679-21apt-get21:08:282
185325217,15sleep30-21swapper/323:18:433
1610825214,23sleep116101-21apt-get00:03:301
426825114,16sleep30-21swapper/323:28:163
2444025116,21sleep124451-21sh00:33:411
2129425116,17sleep10-21swapper/119:08:391
455325013,22sleep24562-21df_inode23:28:312
2663825012,22sleep326657-21ntp_states19:23:393
2180925019,15sleep30-21swapper/320:53:163
2836924917,17sleep20-21swapper/222:59:492
246424913,21sleep12470-21ntp_states21:33:421
896024811,21sleep28971-21ntp_states21:54:182
292432488,14sleep20-21swapper/223:03:322
1708224812,21sleep216601-21apt-get00:08:232
1708224812,21sleep216601-21apt-get00:08:232
1981924714,15sleep10-21swapper/122:33:161
1751224710,15sleep10-21swapper/100:13:151
1751224710,15sleep10-21swapper/100:13:151
513024611,17sleep30-21swapper/321:43:293
513024611,17sleep30-21swapper/321:43:293
4522469,20sleep3449-21apt-get21:28:273
2223324618,15sleep20-21swapper/222:43:422
644324513,18sleep10-21swapper/120:03:291
1583724513,15sleep10-21swapper/120:33:321
1974824214,16sleep30-21swapper/322:33:143
197082406,13sleep20-21swapper/222:33:132
204082377,14sleep30-21swapper/322:38:143
170842379,13sleep30-21swapper/300:08:243
170842379,13sleep30-21swapper/300:08:243
196102328,13sleep10-21swapper/122:31:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional