You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-24 - 04:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Aug 24, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27574211758,1155sleep212574-1kworker/2:0H21:04:382
2796525106,466sleep328001-21ntpq21:05:033
6350249916,468sleep26363-21unin-run19:59:172
240024118,388sleep32456-21fschecks_time23:09:353
32030232557,250sleep3231-21systemd-journal19:39:153
18611232312,198sleep318756-21head23:59:493
18611232312,198sleep318756-21head23:59:493
29206231154,199sleep329362-21unixbench_multi22:49:443
608122977,278sleep0367-21snmpd19:56:370
23948229328,20sleep324061-21latency_hist20:54:173
2756229112,259sleep32842-21store19:49:173
1512222527,196sleep315158-21ntpq23:49:393
9572223251,58sleep30-21swapper/321:49:363
22143222816,195sleep322188-21fschecks_count22:29:343
27065222211,17sleep30-21swapper/322:44:363
3262822217,201sleep332672-21ntpq21:19:413
7903221115,74sleep30-21swapper/321:44:353
7072221016,91sleep37161-21fschecks_time23:24:333
16389220824,133sleep316477-21fschecks_time23:54:363
506121948,131sleep35171-21timerandwakeup21:34:463
6364218953,123sleep30-21swapper/319:59:183
25528218714,117sleep325651-21latency_hist20:59:183
13060218416,153sleep213130-21/usr/sbin/munin21:59:452
11171218315,155sleep011185-21irqstats21:54:380
16813217731,95sleep316935-21sed20:29:473
1130821707,150sleep311344-21ntpq21:54:413
10660217011,124sleep310748-21switchtime23:34:463
22664216712,139sleep222742-21find22:29:452
6602216650,95sleep322361-21cyclictest19:59:303
6602216650,95sleep322361-21cyclictest19:59:303
20245216620,125sleep220390-21unixbench_multi20:40:072
20245216620,125sleep220390-21unixbench_multi20:40:072
25389216518,133sleep325431-21fstrim00:24:163
22401215951,29sleep30-21swapper/320:49:173
30303215839,97sleep30-21swapper/319:31:573
14196215517,79sleep314351-21apt-config20:24:173
19442215410,131sleep319504-21/usr/sbin/munin22:19:443
19442215410,131sleep319504-21/usr/sbin/munin22:19:443
12595215347,85sleep312635-21latency_hist20:19:173
12595215347,85sleep312635-21latency_hist20:19:173
22814215216,26sleep30-21swapper/319:09:403
2660215134,98sleep30-21swapper/321:29:173
22544214752,17sleep30-21swapper/300:14:313
217721410,7sleep20-21swapper/221:24:472
17379214134,63sleep322361-21cyclictest22:14:313
5535213917,102sleep30-21swapper/323:19:363
17723213863,20sleep30-21swapper/320:34:313
17723213863,20sleep30-21swapper/320:34:313
14573213811,112sleep014259-21/usr/sbin/munin22:04:560
23699213313,98sleep30-21swapper/322:34:343
3655213140,76sleep33702-21ntpq19:49:533
1156213116,46sleep21163-21modprobe19:44:152
2116221307,108sleep321287-21ls22:24:453
2116221307,108sleep321287-21ls22:24:453
1020821307,102sleep310276-21/usr/sbin/munin20:09:443
1020821307,102sleep310276-21/usr/sbin/munin20:09:443
4536212823,73sleep14698-21grep23:14:461
15345212814,21sleep30-21swapper/320:25:183
30119212511,97sleep330258-21latency_hist00:39:183
13131212443,27sleep30-21swapper/321:59:463
30540212214,91sleep30-21swapper/322:54:453
11998212221,79sleep112148-21unixbench_singl20:14:511
11998212221,79sleep112148-21unixbench_singl20:14:511
987421218,89sleep30-21swapper/323:34:173
813021178,45sleep30-21swapper/320:04:273
813021178,45sleep30-21swapper/320:04:273
7918211520,78sleep27966-21latency_hist20:04:182
7918211520,78sleep27966-21latency_hist20:04:182
27586211412,85sleep227695-21timerandwakeup22:44:472
15831211416,61sleep30-21swapper/322:09:313
4540211314,82sleep04598-21fschecks_count21:34:350
26100210945,14sleep30-21swapper/322:39:473
2059321095,40sleep30-21swapper/300:07:573
2059321095,40sleep30-21swapper/300:07:573
2192821085,38sleep30-21swapper/300:10:113
2192821085,38sleep30-21swapper/300:10:113
21544210717,69sleep221059-21/usr/sbin/munin20:44:432
1136721078,15sleep10-21swapper/121:54:431
2678210621,65sleep02836-21grep21:29:180
2048921065,38sleep30-21swapper/320:40:203
2048921065,38sleep30-21swapper/320:40:203
18645210521,66sleep218648-21cron22:19:152
32217210328,15sleep30-21swapper/322:59:463
26105210316,35sleep126108-21acpi19:19:271
1484021035,40sleep30-21swapper/322:05:033
30035210222,61sleep030075-21systemctl00:39:150
2848210221,58sleep03053-21latency_hist19:49:180
16379210111,60sleep10-21swapper/123:54:341
825029916,68sleep28299-21unin-run23:29:162
3012729811,66sleep00-21swapper/022:54:340
1360829811,71sleep213738-21/usr/sbin/munin20:19:482
1036629814,67sleep110451-21grep20:09:461
1036629814,67sleep110451-21grep20:09:461
438129628,43sleep30-21swapper/323:14:443
2573829619,15sleep30-21swapper/320:59:263
2961029412,48sleep30-21swapper/321:09:443
579029316,54sleep15941-21head19:55:041
33232936,46sleep30-21swapper/321:29:423
2636129211,44sleep30-21swapper/300:24:483
127272927,51sleep3231-21systemd-journal23:41:483
2899029110,30sleep329106-21cut19:29:183
254302908,26sleep20-21swapper/222:39:302
220732908,73sleep028689-1kworker/0:0H00:12:010
220732908,73sleep028689-1kworker/0:0H00:12:010
2300628910,62sleep023038-21/usr/sbin/munin00:14:440
474228823,40sleep10-21swapper/119:54:191
311322886,44sleep30-21swapper/321:14:443
311322886,44sleep30-21swapper/321:14:443
1950528821,48sleep119003-21apt-get20:39:251
1950528821,48sleep119003-21apt-get20:39:251
1134428820,54sleep011396-21tr23:39:150
3016428716,52sleep230169-21/usr/sbin/munin22:54:362
1606328715,58sleep016160-21date23:54:170
2889028519,46sleep128934-21latency_hist21:09:171
2418328523,16sleep30-21swapper/319:14:203
2743928453,14sleep30-21swapper/300:29:373
2389228315,52sleep123983-21sort00:19:171
2261528218,33sleep20-21swapper/220:49:282
158292828,17sleep10-21swapper/122:09:291
132092826,30sleep30-21swapper/323:44:313
3093928120,45sleep230959-21uname21:14:392
3093928120,45sleep230959-21uname21:14:392
3252828028,30sleep132593-21latency21:19:391
818727914,29sleep00-21swapper/021:44:410
3048627917,46sleep030488-21cron19:34:160
2828227912,51sleep028377-21sed19:24:470
262642799,52sleep026346-21mailstats20:59:430
1263927915,44sleep012786-21latency_hist20:19:180
858927817,19sleep00-21swapper/023:29:340
416427720,42sleep04195-21munin-plugin-st21:34:150
2473427714,48sleep024802-21/usr/sbin/munin20:54:470
2234427723,39sleep122345-21cron20:49:151
2924027610,16sleep10-21swapper/100:34:441
1904827615,40sleep219129-21grep22:19:352
1904827615,40sleep219129-21grep22:19:352
1451627622,28sleep10-21swapper/123:49:171
1142327618,33sleep20-21swapper/223:39:172
1957327536,23sleep10-21swapper/122:19:461
1957327536,23sleep10-21swapper/122:19:461
1071827514,47sleep110868-21latency_hist21:54:191
2873627411,47sleep128793-21awk22:49:341
2650127417,34sleep30-21swapper/319:19:413
2138527425,32sleep021423-21ntpq19:04:410
2099627414,43sleep121066-21df00:09:361
2099627414,43sleep121066-21df00:09:361
1593527417,43sleep26292-21kworker/u8:320:29:172
4692738,44sleep312-21rcu_sched19:39:483
3043527325,20sleep10-21swapper/121:14:181
256412738,13sleep10-21swapper/122:39:371
3226027213,42sleep232304-21/usr/sbin/munin21:19:322
46727120,33sleep0472-21modprobe23:04:150
299427118,31sleep13064-21fschecks_time21:29:361
1790527115,22sleep00-21swapper/020:34:360
1790527115,22sleep00-21swapper/020:34:360
1779627112,30sleep10-21swapper/120:34:331
1779627112,30sleep10-21swapper/120:34:331
662027010,36sleep00-21swapper/019:59:310
662027010,36sleep00-21swapper/019:59:310
3117227024,31sleep131208-21ntpq19:34:411
306752709,33sleep00-21swapper/021:14:330
306752709,33sleep00-21swapper/021:14:330
2421527016,26sleep20-21swapper/200:19:342
1292527021,26sleep10-21swapper/120:19:351
385726918,28sleep23924-21df23:14:302
3271926913,27sleep10-21swapper/119:39:421
3211226913,41sleep232239-21wc19:39:182
2543826921,25sleep00-21swapper/000:24:170
190762697,53sleep00-21swapper/000:03:170
190762697,53sleep00-21swapper/000:03:170
1375426918,29sleep013777-21munin-run22:04:150
130272699,14sleep20-21swapper/223:44:182
445826815,30sleep00-21swapper/023:14:450
280462688,12sleep20-21swapper/219:24:412
2769026817,19sleep10-21swapper/100:29:431
1297026815,37sleep113026-21sed23:44:171
973326716,27sleep20-21swapper/221:49:392
852426721,33sleep18131-21/usr/sbin/munin20:04:411
852426721,33sleep18131-21/usr/sbin/munin20:04:411
822226710,29sleep20-21swapper/220:04:342
822226710,29sleep20-21swapper/220:04:342
464626720,31sleep04514-21latency_hist19:54:180
2926326713,29sleep20-21swapper/200:34:452
2644026715,27sleep20-21swapper/220:59:452
19412675,11sleep30-21swapper/323:08:133
9462667,47sleep00-21swapper/019:41:310
757826619,24sleep10-21swapper/121:44:171
2544826621,29sleep11-21systemd20:59:161
1588926613,35sleep015906-21df_inode22:09:320
1521326620,22sleep20-21swapper/223:49:432
984026519,25sleep20-21swapper/223:34:162
745626527,28sleep30-21swapper/321:43:183
243212659,21sleep20-21swapper/220:54:372
2232926522,28sleep122347-21munin-run19:09:171
1778026516,29sleep00-21swapper/022:14:410
881526414,26sleep00-21swapper/020:04:480
881526414,26sleep00-21swapper/020:04:480
2829126420,21sleep10-21swapper/122:49:181
1638326412,22sleep10-21swapper/120:29:361
2989526317,24sleep20-21swapper/221:09:472
2234826313,33sleep222502-21latency_hist19:09:182
1983226315,21sleep00-21swapper/000:04:430
1983226315,21sleep00-21swapper/000:04:430
601826218,21sleep10-21swapper/123:19:461
578426219,25sleep15818-21munin-run21:39:161
372026219,20sleep20-21swapper/219:49:562
2745526214,30sleep027513-21/usr/sbin/munin22:44:450
26472625,13sleep10-21swapper/119:48:301
206682627,10sleep00-21swapper/020:42:350
206682627,10sleep00-21swapper/020:42:350
1780626218,22sleep20-21swapper/220:34:342
1780626218,22sleep20-21swapper/220:34:342
1215026217,22sleep20-21swapper/220:14:522
1215026217,22sleep20-21swapper/220:14:522
979826120,21sleep10-21swapper/123:34:151
421526121,22sleep20-21swapper/221:34:172
2589326115,22sleep00-21swapper/019:19:180
2181526116,24sleep20-21swapper/222:29:162
2181526116,24sleep20-21swapper/222:29:162
2150026113,32sleep021536-21ntp_states20:44:410
2087326119,22sleep10-21swapper/100:09:201
2087326119,22sleep10-21swapper/100:09:201
1461226118,26sleep114615-21apt-get20:24:281
47426012,33sleep1521-21munin-run23:04:161
312202608,41sleep10-21swapper/122:57:461
2767826014,28sleep027692-21/usr/sbin/munin21:04:560
2505326018,26sleep225198-21cat22:39:182
174462607,13sleep00-21swapper/023:57:480
96742597,17sleep20-21swapper/220:09:222
96742597,17sleep20-21swapper/220:09:222
72872596,13sleep00-21swapper/021:41:090
301712597,41sleep00-21swapper/021:11:300
2496325919,24sleep125052-21mktemp22:39:171
203925916,20sleep10-21swapper/121:24:471
779125817,24sleep20-21swapper/221:44:262
629625820,21sleep20-21swapper/221:39:502
2686025818,23sleep20-21swapper/219:19:482
2589325811,27sleep025925-21/usr/sbin/munin22:39:440
1476625816,19sleep10-21swapper/122:05:011
106162586,12sleep00-21swapper/020:09:490
106162586,12sleep00-21swapper/020:09:490
670125714,18sleep20-21swapper/223:24:152
3214025719,18sleep20-21swapper/222:59:452
2470625715,23sleep10-21swapper/100:19:461
239525714,27sleep22399-21/usr/sbin/munin23:09:342
2254325613,26sleep222537-21apt-get00:14:302
2048625616,24sleep020504-21apt-get22:24:260
2048625616,24sleep020504-21apt-get22:24:260
2011325614,20sleep20-21swapper/200:04:492
2011325614,20sleep20-21swapper/200:04:492
1360925622,16sleep00-21swapper/023:44:400
562625517,20sleep00-21swapper/023:19:370
2581325515,26sleep225839-21munin-run19:19:162
2321025519,19sleep00-21swapper/019:09:490
2290025517,20sleep10-21swapper/120:49:391
2057525511,32sleep120586-21df_inode22:24:311
2057525511,32sleep120586-21df_inode22:24:311
1852125519,19sleep20-21swapper/223:59:482
1852125519,19sleep20-21swapper/223:59:482
1310825516,20sleep00-21swapper/020:19:380
1045325518,23sleep110480-21awk23:34:401
936625413,25sleep09360-21apt-get21:49:290
79072546,15sleep10-21swapper/123:24:551
75925414,24sleep2754-21apt-get23:04:292
3204725414,23sleep10-21swapper/121:19:181
3204725414,23sleep10-21swapper/121:19:181
3077625416,20sleep00-21swapper/019:34:320
3002925413,22sleep230127-21users19:29:502
270525415,22sleep20-21swapper/219:49:162
1748325417,16sleep20-21swapper/220:34:182
148325412,26sleep01615-21unixbench_singl23:04:470
867625313,22sleep18722-21users21:44:501
33425318,19sleep00-21swapper/021:19:460
2968825313,17sleep10-21swapper/119:29:451
2944825314,26sleep229450-21sshd22:49:552
276832539,20sleep00-21swapper/000:29:420
2391325316,20sleep10-21swapper/119:14:181
2345825318,18sleep20-21swapper/219:09:512
2246325316,22sleep122496-21ntp_kernel_pll_22:29:401
1844325312,20sleep10-21swapper/123:59:471
1844325312,20sleep10-21swapper/123:59:471
1745825314,22sleep117462-21diskstats22:14:331
1615225314,23sleep016170-21apt-get20:29:270
1502325316,23sleep114729-21/usr/sbin/munin23:49:371
681225214,23sleep16869-21irqstats19:59:361
681225214,23sleep16869-21irqstats19:59:361
2808325212,23sleep228130-21users00:29:482
271522525,38sleep00-21swapper/019:21:120
2576425213,22sleep125758-21apt-get20:59:281
2153725214,19sleep20-21swapper/200:09:492
2153725214,19sleep20-21swapper/200:09:492
2027425213,17sleep00-21swapper/022:24:180
2027425213,17sleep00-21swapper/022:24:180
1428525213,25sleep214351-21sort22:04:332
279325115,22sleep02861-21systemctl23:09:440
230925112,22sleep12303-21apt-get23:09:291
172602517,10sleep20-21swapper/223:55:242
2101625014,18sleep30-21swapper/319:04:313
485824910,14sleep20-21swapper/221:34:412
1297424916,16sleep10-21swapper/121:59:441
1165324911,22sleep211652-21apt-get23:39:252
1090524911,22sleep210908-21apt-get21:54:282
228742488,14sleep00-21swapper/022:29:470
1588324812,15sleep00-21swapper/020:29:150
1123424811,21sleep011251-21apt-get20:14:290
83124711,20sleep1837-21df_inode23:04:321
698924710,18sleep00-21swapper/023:24:290
276792469,19sleep10-21swapper/119:24:321
2612724610,20sleep226133-21ntp_states00:24:432
1610124614,16sleep20-21swapper/222:09:382
1210124610,21sleep012112-21ntp_states23:39:390
265252456,13sleep00-21swapper/000:24:490
2567724510,19sleep125674-21apt-get00:24:301
243732458,13sleep10-21swapper/119:14:221
237072458,16sleep00-21swapper/022:34:350
2271324513,16sleep00-21swapper/020:49:340
2867224111,16sleep00-21swapper/022:49:330
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional