You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 01:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Thu Jul 23, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17131228678,2846sleep19280-21kworker/1:008:00:361
243092150036,1448sleep124314-21ntp_states08:23:411
1023327538,733sleep19337-21kworker/1:009:19:361
1023327538,733sleep19337-21kworker/1:009:19:361
24263265716,624sleep024308-21/usr/sbin/munin08:23:400
2990824088,388sleep1195-21jbd2/mmcblk2p2-12:08:171
506423668,345sleep25089-21awk07:23:362
506423668,345sleep25089-21awk07:23:362
23422228424,236sleep123445-21munin-plugin-st11:48:111
19251227369,111sleep219395-21grep09:48:432
6010224912,188sleep26158-21unixbench_singl12:33:412
10960223952,107sleep211021-21idleruntime09:23:332
10960223952,107sleep211021-21idleruntime09:23:332
6566222973,88sleep26711-21unixbench_multi09:08:432
1951422259,13sleep00-21swapper/011:33:450
14164222330,181sleep214186-21tr09:33:342
3412221511,48sleep20-21swapper/208:58:502
24555221014,133sleep224704-21unixbench_multi08:23:462
2219222050,6sleep30-21swapper/309:58:403
28203218060,69sleep20-21swapper/208:38:112
20715217712,148sleep320793-21find09:53:413
15709217247,110sleep20-21swapper/211:23:132
8503216916,69sleep28543-21processes07:33:412
8391216415,133sleep38455-21ps10:58:403
32761216238,71sleep232764-21cron12:18:112
20297216144,104sleep220326-21modprobe11:38:112
4941215925,54sleep20-21swapper/209:03:422
3214821588,138sleep232151-21diskstats07:08:302
167321577,134sleep2938-21/usr/sbin/munin08:53:452
24884215531,112sleep224966-21cut10:08:332
1887215532,84sleep21947-21cat10:38:322
1887215532,84sleep21947-21cat10:38:322
623721547,135sleep00-21swapper/012:34:280
21268215111,127sleep221392-21cat11:38:432
19038215116,119sleep319089-21processes09:48:403
22626214873,17sleep20-21swapper/211:43:402
9943214721,107sleep39979-21wc11:03:403
28576214742,52sleep20-21swapper/212:03:342
25525214750,76sleep20-21swapper/211:53:332
2673821449,120sleep226891-21sed08:33:152
30518214230,62sleep20-21swapper/208:44:372
13317213566,49sleep20-21swapper/211:14:172
2739213118,86sleep32238-21apt-get08:58:233
2216213023,89sleep02234-21sendmail08:58:110
21916213019,86sleep321930-21latency_hist11:43:123
3048421277,109sleep2364-21snmpd10:24:562
25011212712,66sleep21-21systemd11:53:122
6110212618,40sleep20-21swapper/210:53:142
4371212644,17sleep20-21swapper/212:28:402
29263212670,15sleep20-21swapper/210:23:152
1162821268,79sleep211671-21processes07:43:422
18803212019,76sleep10-21swapper/111:33:131
2680821195,101sleep222754-1kworker/2:2H11:58:212
9372211814,88sleep39314-21latency_hist07:38:143
7024211844,59sleep20-21swapper/207:29:372
1245211822,83sleep00-21swapper/010:36:200
1245211822,83sleep00-21swapper/010:36:200
932221177,36sleep20-21swapper/209:18:282
932221177,36sleep20-21swapper/209:18:282
4697211721,37sleep00-21swapper/009:03:360
22233211716,87sleep322240-21sh11:43:283
9251211630,45sleep30-21swapper/309:18:253
9251211630,45sleep30-21swapper/309:18:253
3233211439,57sleep20-21swapper/207:18:312
13025211446,50sleep20-21swapper/207:48:302
4240211316,12sleep20-21swapper/210:44:422
10382211310,86sleep110511-21awk07:38:441
26156211221,71sleep326271-21grep08:28:443
15082211222,66sleep015232-21unixbench_multi11:18:430
2674821116,56sleep226795-21ntpq10:13:382
21776211015,73sleep121841-21fschecks_time09:58:311
19133211011,84sleep319169-21/usr/sbin/munin11:33:363
1726821107,55sleep217421-21wc11:28:142
10738211049,47sleep210756-21awk11:08:112
26517210923,49sleep026537-21apt11:58:110
22784210918,67sleep00-21swapper/008:18:400
628210839,54sleep2723-21grep10:33:402
628210839,54sleep2723-21grep10:33:402
22057210823,68sleep222076-21latency_hist08:18:132
12638210820,65sleep312430-21apt-get07:48:153
27518210720,63sleep027676-21sed11:58:430
23975210744,18sleep20-21swapper/210:03:412
6353210610,78sleep36418-21/usr/sbin/munin09:08:403
29779210544,48sleep229827-21apt-config08:43:122
23615210430,24sleep20-21swapper/208:23:142
1409210433,51sleep20-21swapper/207:13:182
31028210338,25sleep20-21swapper/210:28:272
31028210338,25sleep20-21swapper/210:28:272
22079210212,73sleep322224-21date08:18:143
709210046,19sleep2292-21systemd-logind08:53:132
10790210019,66sleep010868-21latency_hist11:08:130
2947929915,71sleep029510-21grep10:23:290
2523129921,59sleep325249-21apt11:53:213
764229832,21sleep20-21swapper/209:13:232
530629813,70sleep35307-21sshd09:04:363
1794229829,57sleep217948-21ntp_states11:28:382
1475929843,44sleep214768-21grep07:53:352
3092929715,67sleep130942-21ntpq07:03:371
291029517,62sleep12996-21grep07:18:141
1875229519,60sleep318784-21idleruntime-cro11:33:113
1056429412,54sleep00-21swapper/009:23:140
1056429412,54sleep00-21swapper/009:23:140
496929332,47sleep05012-21idleruntime10:48:310
3079529314,66sleep330839-21awk08:44:423
1624329325,47sleep316367-21sed09:38:433
303572928,14sleep00-21swapper/008:44:030
862529034,15sleep20-21swapper/210:58:432
3181229033,14sleep20-21swapper/207:08:142
136529029,45sleep21509-21awk12:18:432
693028928,37sleep30-21swapper/307:29:343
603128916,55sleep06144-21date07:28:140
603128916,55sleep06144-21date07:28:140
283028921,43sleep32880-21sendmail07:18:123
2048228819,47sleep10-21swapper/109:53:351
1523328815,50sleep114376-21/usr/sbin/munin11:18:441
69628713,54sleep30-21swapper/312:18:283
265182870,6sleep10-21swapper/110:13:321
1762228710,20sleep10-21swapper/108:03:271
39028517,20sleep30-21swapper/310:33:333
39028517,20sleep30-21swapper/310:33:333
1992128528,38sleep20-21swapper/208:08:452
3119828412,56sleep331239-21systemd-run12:13:113
3018228429,14sleep20-21swapper/212:08:342
2168428417,13sleep20-21swapper/209:58:242
1091928420,31sleep30-21swapper/307:43:143
800928314,54sleep18028-21apt-get07:33:231
38328314,53sleep0456-21cat12:18:130
2964928320,46sleep329680-21munin-run12:08:113
1467028323,41sleep314704-21cat11:18:343
154452827,17sleep20-21swapper/209:38:222
213692815,63sleep20-21swapper/209:58:082
136592818,52sleep20-21swapper/209:33:132
182422805,32sleep20-21swapper/209:47:422
1642828018,13sleep20-21swapper/207:58:352
1457728037,25sleep30-21swapper/307:53:323
1344428017,46sleep013454-21/usr/sbin/munin07:49:050
1209928019,11sleep20-21swapper/211:10:462
753527918,13sleep20-21swapper/210:57:172
129232790,6sleep00-21swapper/011:13:450
296327821,37sleep0225-21systemd-journal10:43:110
296327821,37sleep0225-21systemd-journal10:43:110
1291427824,34sleep312430-21apt-get07:48:233
1773827715,11sleep20-21swapper/208:03:322
315027620,40sleep33203-21ls07:18:283
214927618,24sleep00-21swapper/012:23:220
1184227618,34sleep111990-21unixbench_multi07:43:451
3243627518,32sleep00-21swapper/007:08:360
3213327521,27sleep30-21swapper/308:48:343
3195127520,26sleep00-21swapper/008:48:310
2847827515,42sleep028485-21/usr/sbin/munin12:03:320
2828327521,30sleep00-21swapper/008:38:130
2763127525,27sleep30-21swapper/308:33:443
271827511,16sleep20-21swapper/212:23:402
830327417,28sleep00-21swapper/007:33:350
697027311,42sleep17097-21timerwakeupswit10:53:411
1205827213,42sleep11-21systemd09:28:121
1205827213,42sleep11-21systemd09:28:121
3029927125,28sleep130297-21apt-get08:43:231
2661327115,43sleep326656-21cat10:13:353
2528327118,24sleep30-21swapper/310:08:423
1958227114,26sleep30-21swapper/308:08:373
862027014,42sleep18629-21vmstat09:14:191
372127015,39sleep33774-21ls12:28:253
3003627019,28sleep00-21swapper/012:08:320
1111127021,23sleep00-21swapper/011:08:300
1926226921,21sleep10-21swapper/108:08:311
1418726914,29sleep30-21swapper/309:33:353
370226720,22sleep10-21swapper/107:18:421
2875726712,25sleep00-21swapper/008:38:340
218372676,50sleep016880-21kworker/0:211:42:410
1631526723,20sleep10-21swapper/111:23:361
924526623,20sleep10-21swapper/111:03:141
2595726619,18sleep10-21swapper/108:28:421
2588926624,21sleep00-21swapper/008:28:410
649126520,23sleep10-21swapper/109:08:421
2438926518,28sleep00-21swapper/011:48:420
2041426515,22sleep00-21swapper/009:53:340
185632657,12sleep00-21swapper/008:04:060
1833126511,27sleep10-21swapper/109:48:141
326526414,33sleep03332-21run-parts08:58:480
2801226419,30sleep128052-21sort10:18:291
204552645,11sleep10-21swapper/111:38:141
159526415,35sleep1225-21systemd-journal12:19:271
148726414,31sleep01499-21munin-node07:13:200
148152645,10sleep00-21swapper/009:33:480
1148926417,29sleep10-21swapper/111:08:391
689826317,22sleep00-21swapper/010:53:400
644926326,20sleep012-21rcu_sched07:28:180
3049326315,19sleep00-21swapper/007:03:280
2506726318,30sleep125134-21grep10:08:361
1944826318,23sleep10-21swapper/111:33:421
1425026314,31sleep114351-21latency_hist07:53:131
97942627,14sleep20-21swapper/207:38:332
3141626218,21sleep30-21swapper/310:28:363
3141626218,21sleep30-21swapper/310:28:363
3057326218,26sleep30-21swapper/307:03:313
2284326211,24sleep10-21swapper/108:18:411
1944226220,19sleep00-21swapper/008:08:340
1802226211,23sleep30-21swapper/308:03:373
470126118,26sleep00-21swapper/012:29:350
2970226117,24sleep10-21swapper/112:08:131
268426115,26sleep10-21swapper/110:39:361
268426115,26sleep10-21swapper/110:39:361
244226120,20sleep30-21swapper/312:23:333
219722615,43sleep00-21swapper/008:17:590
1777126112,31sleep017920-21/usr/sbin/munin09:43:450
1692826111,35sleep016930-21sshd11:24:360
614526016,27sleep16346-21apt-config07:28:151
614526016,27sleep16346-21apt-config07:28:151
392426018,20sleep00-21swapper/010:43:430
3246826020,23sleep10-21swapper/107:08:371
2698226019,25sleep10-21swapper/111:58:321
11832604,11sleep10-21swapper/110:35:331
11832604,11sleep10-21swapper/110:35:331
683125918,20sleep10-21swapper/107:29:321
652825914,29sleep36581-21/usr/sbin/munin10:53:313
404925914,24sleep00-21swapper/007:18:450
3272025918,23sleep132767-21users08:48:441
3215225913,19sleep30-21swapper/307:08:313
3159625920,17sleep00-21swapper/010:28:420
3159625920,17sleep00-21swapper/010:28:420
2984825922,17sleep10-21swapper/110:23:381
2982825917,24sleep329855-21munin-run08:43:133
2879025913,22sleep10-21swapper/108:38:351
2261025916,25sleep322670-21memory08:18:343
1865225913,29sleep118657-21/usr/sbin/munin09:48:301
1764025914,22sleep30-21swapper/309:43:443
1080025917,26sleep010794-21apt-get09:23:260
1080025917,26sleep010794-21apt-get09:23:260
339825819,21sleep10-21swapper/110:43:321
30725816,26sleep31-21systemd12:18:123
2385825816,17sleep10-21swapper/110:03:401
112942587,16sleep30-21swapper/311:08:333
900825721,17sleep30-21swapper/309:18:133
370025720,21sleep30-21swapper/310:43:383
3167025717,18sleep10-21swapper/110:28:431
3167025717,18sleep10-21swapper/110:28:431
2226925717,24sleep00-21swapper/009:58:420
1582725719,15sleep10-21swapper/107:58:141
133802575,12sleep10-21swapper/111:14:201
1283925716,21sleep30-21swapper/309:28:423
123225719,22sleep10-21swapper/108:53:361
2362025613,26sleep323664-21ntpq10:03:343
1592125616,23sleep30-21swapper/311:23:223
1332825614,26sleep313336-21ntpq11:14:183
961225518,17sleep00-21swapper/007:38:270
1604625528,15sleep10-21swapper/109:38:411
1568025515,21sleep00-21swapper/009:38:320
1243025513,26sleep012437-21diskstats09:28:310
112625520,18sleep30-21swapper/308:53:343
1116025515,24sleep011154-21apt-get07:43:280
903725414,20sleep00-21swapper/009:18:140
718825412,17sleep00-21swapper/009:13:130
605825413,24sleep30-21swapper/310:53:113
501325415,23sleep15065-21awk10:48:321
2371425416,25sleep123765-21sort11:48:271
177825415,23sleep31787-21diskstats10:38:293
177825415,23sleep31787-21diskstats10:38:293
1129725418,19sleep30-21swapper/309:23:423
1129725418,19sleep30-21swapper/309:23:423
2680925320,19sleep30-21swapper/311:58:223
2415525313,20sleep324166-21ntp_states11:48:373
1603825314,24sleep016041-21/usr/sbin/munin07:58:210
1523225321,18sleep10-21swapper/109:38:131
564725213,24sleep15725-21latency_hist09:08:141
232025219,18sleep10-21swapper/112:23:311
1280825212,23sleep112814-21ntp_states09:28:401
70525113,22sleep0709-21diskstats12:18:290
526825111,23sleep15262-21apt-get12:33:241
2945125112,16sleep30-21swapper/310:23:243
2643125114,23sleep026517-21cut10:13:320
2547525115,21sleep125524-21irqstats11:53:321
1749025112,23sleep017487-21apt-get11:28:240
991225012,23sleep19935-21ntp_states11:03:381
815325010,16sleep00-21swapper/010:58:330
333325015,21sleep10-21swapper/108:58:491
2745725011,15sleep10-21swapper/108:33:421
2385825012,21sleep323928-21df08:23:313
2324325014,17sleep00-21swapper/010:03:230
1655225015,19sleep30-21swapper/307:58:403
530124911,20sleep35340-21df12:33:263
2792424911,22sleep327920-21apt-get10:18:233
216942496,34sleep10-21swapper/108:14:331
1987724915,16sleep00-21swapper/009:53:130
597724819,16sleep30-21swapper/307:28:133
597724819,16sleep30-21swapper/307:28:133
46532485,32sleep10-21swapper/112:29:001
2521624613,19sleep025255-21ntp_states10:08:390
2258724613,16sleep00-21swapper/011:43:380
2850324511,15sleep00-21swapper/010:18:410
319512438,15sleep10-21swapper/112:13:381
1449524210,16sleep00-21swapper/007:53:290
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional