You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-08 - 21:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Sep 08, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
226902568514,5658sleep110896-21kworker/1:007:42:371
226902568514,5658sleep110896-21kworker/1:007:42:371
172372463513,4609sleep116387-21kworker/1:010:49:361
188962198514,1953sleep011173-21kworker/u8:109:13:370
15520275619,719sleep315530-21/usr/sbin/munin12:28:383
1821625197,498sleep218255-21ntpq10:53:452
1438524027,381sleep114427-21ntpq08:58:441
1107023656,348sleep2367-21snmpd10:30:272
17591233710,290sleep217696-21wc12:33:502
8862232920,293sleep08893-21/usr/sbin/munin10:23:430
13068230711,244sleep20-21swapper/208:53:492
18985229853,153sleep219047-21fschecks_time09:13:432
13361229415,262sleep313389-21apt07:13:273
25843227464,152sleep225863-21modprobe07:53:202
22423223213,203sleep322465-21awk09:23:463
16235222321,184sleep016386-21unixbench_multi09:03:520
16235222321,184sleep016386-21unixbench_multi09:03:520
30131220818,112sleep230221-21fschecks_time09:48:402
13684220626,89sleep20-21swapper/210:38:512
31699220433,132sleep231809-21apt-config11:38:222
23122219355,123sleep20-21swapper/207:43:392
15035218237,86sleep20-21swapper/207:18:232
27593217222,108sleep20-21swapper/211:23:442
22866217225,97sleep20-21swapper/211:08:422
721321678,144sleep27288-21cat10:18:402
721321678,144sleep27288-21cat10:18:402
28196216657,96sleep228199-21cron09:43:192
24968216430,78sleep225146-21apt-config09:33:222
3739216329,94sleep23886-21grep08:23:532
13060216333,109sleep313242-21grep07:13:233
11781215734,84sleep211848-21modprobe12:18:192
379321568,109sleep23794-21/usr/sbin/munin10:08:352
11703215617,123sleep017566-1kworker/0:0H07:08:310
9465215516,124sleep09537-21mailstats12:08:480
9465215516,124sleep09537-21mailstats12:08:480
29664215412,99sleep229711-21ntp_offset08:03:462
7800215257,57sleep20-21swapper/208:38:422
544121509,33sleep20-21swapper/210:13:392
20171214843,55sleep20-21swapper/210:58:532
20171214843,55sleep20-21swapper/210:58:532
9041214731,101sleep29067-21fschecks_count12:08:382
9041214731,101sleep29067-21fschecks_count12:08:382
15404214725,80sleep20-21swapper/210:43:522
458621458,39sleep20-21swapper/208:28:332
26031214516,52sleep226153-21timerwakeupswit09:33:582
717121427,97sleep27247-21sed12:03:212
13036214134,86sleep20-21swapper/207:13:222
25311214035,58sleep225369-21grep11:18:192
18126213819,94sleep30-21swapper/307:28:213
1257213811,111sleep11370-21latency_hist08:18:221
1257213811,111sleep11370-21latency_hist08:18:221
14570213423,22sleep20-21swapper/208:58:492
13858213420,92sleep013999-21unixbench_multi10:38:530
11166213431,82sleep111253-21systemd-run07:08:201
17353213221,30sleep00-21swapper/012:33:440
21563213045,39sleep20-21swapper/207:38:382
21563213045,39sleep20-21swapper/207:38:382
20566213016,59sleep20-21swapper/209:18:402
18861213036,70sleep318882-21apt-get09:13:323
17442212510,101sleep316955-21/usr/sbin/munin12:33:473
2621212415,90sleep32634-21/usr/sbin/munin10:03:433
21785212428,70sleep00-21swapper/009:23:220
2735211920,52sleep20-21swapper/211:48:362
16502211728,79sleep2367-21snmpd12:30:512
8721211626,76sleep28869-21latency_hist12:08:222
1182221168,15sleep30-21swapper/310:33:433
6344211529,41sleep20-21swapper/208:33:422
11254211544,50sleep211332-21wc07:08:212
1816021145,17sleep20-21swapper/209:10:442
8998211314,85sleep19084-21tr10:23:481
275421129,90sleep21-21systemd08:23:202
2412621108,73sleep10-21swapper/111:13:381
3005210915,80sleep03021-21kernelversion11:48:420
28971210931,45sleep2367-21snmpd11:28:412
1403821089,46sleep20-21swapper/212:23:402
27361210715,41sleep127393-21ntp_states09:38:461
10803210723,41sleep20-21swapper/208:48:322
27468210617,74sleep327560-21latency_hist07:58:223
16528210511,51sleep216532-21modprobe07:23:192
29744210422,39sleep20-21swapper/209:48:202
1594210421,58sleep01694-21fschecks_time08:18:420
3243221030,8sleep20-21swapper/211:38:452
21575210317,69sleep321577-21entropy07:38:393
21575210317,69sleep321577-21entropy07:38:393
30737210211,14sleep10-21swapper/111:33:491
12687210223,33sleep20-21swapper/212:19:282
10265210224,59sleep110339-21latency_hist12:13:221
10265210224,59sleep110339-21latency_hist12:13:221
19991210023,37sleep20-21swapper/207:33:372
2589629818,60sleep126036-21sed07:53:231
990129736,40sleep20-21swapper/208:43:512
3157929718,38sleep20-21swapper/209:53:332
6782969,39sleep20-21swapper/209:58:292
2599529611,71sleep126013-21netstat11:18:431
34112956,11sleep30-21swapper/308:23:463
3158329514,68sleep131632-21/usr/sbin/munin09:53:361
849029419,35sleep20-21swapper/210:23:322
2662029421,54sleep326635-21cron09:38:193
1668329410,42sleep20-21swapper/210:48:482
2533529329,34sleep10-21swapper/109:33:331
1314829311,68sleep313169-21dpkg10:38:323
210842925,12sleep30-21swapper/309:18:533
1828229213,66sleep118311-21proc_pri10:53:481
117552927,34sleep10-21swapper/107:08:531
42812918,48sleep24273-21apt-get11:53:362
2894829121,22sleep30-21swapper/309:43:493
1759229111,65sleep117663-21mailstats09:08:501
1176629110,36sleep20-21swapper/210:33:422
3211129021,47sleep032089-21latency_hist08:13:210
3211129021,47sleep032089-21latency_hist08:13:210
1377529016,59sleep013862-21fschecks_time07:13:400
200512889,62sleep120170-21timerwakeupswit10:58:521
200512889,62sleep120170-21timerwakeupswit10:58:521
1696928819,49sleep017120-21grep10:48:540
1210728826,31sleep10-21swapper/112:18:221
2871828619,37sleep10-21swapper/111:28:321
2404828620,45sleep324118-21sshd11:13:363
2350428614,55sleep323549-21processes07:43:483
1419628611,20sleep114257-21memory12:23:431
1021428623,27sleep30-21swapper/312:13:193
1021428623,27sleep30-21swapper/312:13:193
825328417,15sleep30-21swapper/310:23:213
825328417,15sleep30-21swapper/310:23:213
3272428421,18sleep20-21swapper/208:13:432
3272428421,18sleep20-21swapper/208:13:432
24442848,40sleep22471-21munin-run11:48:192
1351128421,32sleep10-21swapper/107:13:291
974428317,47sleep00-21swapper/008:43:490
819928312,56sleep18223-21munin-run10:23:191
819928312,56sleep18223-21munin-run10:23:191
76182836,15sleep10-21swapper/112:03:411
2536528316,47sleep00-21swapper/009:33:360
2208328328,43sleep222097-21df09:23:362
1997128320,30sleep00-21swapper/010:58:510
1997128320,30sleep00-21swapper/010:58:510
1809328311,57sleep018186-21memory10:53:430
2979728217,53sleep029832-21latency_hist09:48:220
1553928221,26sleep20-21swapper/209:03:342
1553928221,26sleep20-21swapper/209:03:342
1593228121,41sleep316050-21grep07:18:523
2693928016,45sleep326983-21/usr/sbin/munin09:38:363
195152805,63sleep20-21swapper/207:31:562
38127914,44sleep0231-21systemd-journal09:58:190
2221727914,27sleep022359-21sed07:38:520
2221727914,27sleep022359-21sed07:38:520
710827822,33sleep00-21swapper/010:18:380
710827822,33sleep00-21swapper/010:18:380
23872784,11sleep30-21swapper/311:47:363
2902427719,43sleep329146-21latency_hist08:03:223
319327615,42sleep10-21swapper/111:48:471
691727518,29sleep10-21swapper/108:33:531
553827522,35sleep15605-21modprobe11:58:201
3246427529,29sleep30-21swapper/311:38:473
1193927519,34sleep012103-21apt-key12:18:210
28792749,16sleep20-21swapper/210:03:502
2074927421,36sleep120780-21latency09:18:431
1210927415,44sleep012185-21systemctl10:33:520
3080727319,13sleep20-21swapper/208:08:342
3080727319,13sleep20-21swapper/208:08:342
2640427314,41sleep126469-21latency07:53:431
2141327318,35sleep321437-21/usr/sbin/munin11:03:473
2613327224,34sleep20-21swapper/207:53:362
2027127219,25sleep10-21swapper/109:18:221
1841727219,32sleep018505-21ls09:13:210
2688527122,37sleep226932-21systemctl11:23:202
249727120,26sleep00-21swapper/011:48:210
2074327125,25sleep30-21swapper/311:03:223
2074327125,25sleep30-21swapper/311:03:223
2028127118,22sleep00-21swapper/007:33:430
1749827119,32sleep117630-21grep07:23:511
1062027111,42sleep110768-21latency_hist08:48:221
2202427017,28sleep10-21swapper/109:23:351
1836027014,38sleep3289-21dbus-daemon09:13:203
107612707,40sleep10-21swapper/112:13:421
215952698,19sleep20-21swapper/211:03:502
1816526920,25sleep00-21swapper/007:28:220
1435526915,24sleep00-21swapper/008:58:430
1133826914,40sleep111358-21latency_hist10:33:211
1096826921,25sleep30-21swapper/312:13:483
3053626819,35sleep130569-21munin-run08:08:211
281912686,15sleep00-21swapper/009:43:170
2434926829,18sleep10-21swapper/107:48:221
2418826815,28sleep00-21swapper/009:28:510
1614526820,29sleep30-21swapper/309:03:513
1614526820,29sleep30-21swapper/309:03:513
119942686,11sleep00-21swapper/008:51:410
845826722,20sleep30-21swapper/308:38:563
589726713,25sleep30-21swapper/308:33:223
3206726722,22sleep30-21swapper/309:53:503
2785326719,11sleep20-21swapper/207:58:412
2538926713,38sleep025499-21cat11:18:210
208922676,12sleep30-21swapper/307:34:433
972026615,36sleep39742-21pluginstate08:43:483
66212669,26sleep00-21swapper/011:58:510
604026620,19sleep10-21swapper/111:58:401
3036226618,22sleep00-21swapper/011:33:390
1449826620,23sleep10-21swapper/110:43:221
585426511,30sleep30-21swapper/311:58:343
428426523,24sleep30-21swapper/311:53:373
3011126513,35sleep030120-21diskstats09:48:380
2779426516,27sleep10-21swapper/107:58:401
2517326522,19sleep30-21swapper/307:48:493
679126420,27sleep06915-21ls08:33:520
3003326416,28sleep30-21swapper/311:33:223
2726026414,36sleep327264-21entropy11:23:373
1757526415,27sleep30-21swapper/310:53:213
1025226415,32sleep010302-21if_eth010:28:430
40426313,27sleep30-21swapper/308:13:483
40426313,27sleep30-21swapper/308:13:483
3181026316,23sleep30-21swapper/311:38:233
2101826314,30sleep021027-21sshd11:03:360
1728326321,29sleep20-21swapper/207:23:482
1543626324,22sleep00-21swapper/012:28:350
1531726313,22sleep30-21swapper/309:03:223
1261626316,30sleep012666-21if_eth008:53:390
49782627,12sleep00-21swapper/008:28:430
2946726219,20sleep00-21swapper/011:28:510
275026218,30sleep12752-21cron08:23:191
2590226217,30sleep025994-21/usr/sbin/munin11:18:420
1864426216,18sleep00-21swapper/007:28:470
1837626212,22sleep10-21swapper/107:28:401
981626119,20sleep10-21swapper/108:43:501
790026114,34sleep07954-21if_err_eth008:38:440
662826115,26sleep31-21systemd10:18:213
3101826113,32sleep131070-21interrupts08:08:411
3101826113,32sleep131070-21interrupts08:08:411
2482426111,36sleep124826-21sshd09:33:161
2399226119,27sleep324028-21ntp_states09:28:463
2257426115,24sleep00-21swapper/011:08:360
1527426113,31sleep015341-21sshd07:18:370
460726014,27sleep34720-21users10:08:563
3262526014,21sleep00-21swapper/008:13:410
3262526014,21sleep00-21swapper/008:13:410
2848526016,27sleep128480-21apt-get09:43:341
1789526014,30sleep317901-21diskstats10:53:373
1548126021,24sleep115247-21/usr/sbin/munin07:18:421
1367126018,25sleep113708-21modprobe08:58:201
1106426015,27sleep30-21swapper/308:48:413
94725915,27sleep01009-21latency_hist11:43:210
585125919,21sleep10-21swapper/110:13:501
458525915,26sleep14580-21apt-get08:28:321
3216125921,21sleep10-21swapper/111:38:371
2790325914,27sleep028033-21/usr/sbin/munin11:23:500
2717025916,26sleep127167-21apt-get11:23:311
2249625916,24sleep122488-21apt-get11:08:331
1624725910,30sleep116339-21users12:28:521
1517825918,19sleep10-21swapper/110:43:461
1415925920,20sleep00-21swapper/012:23:420
124082595,11sleep30-21swapper/308:53:333
449425813,20sleep00-21swapper/011:53:420
2225825816,24sleep10-21swapper/111:08:211
165225811,34sleep1367-21snmpd09:59:171
84262574,16sleep30-21swapper/312:05:123
826525716,24sleep18338-21wc08:38:541
818325713,26sleep08310-21/usr/sbin/munin12:03:520
534025715,25sleep05359-21apt-get10:13:330
2458425716,26sleep024578-21apt-get07:48:330
216425715,21sleep30-21swapper/308:18:533
1831325713,21sleep10-21swapper/112:38:211
1291825717,18sleep00-21swapper/012:19:340
3051925619,19sleep00-21swapper/008:08:200
3012125617,17sleep10-21swapper/109:48:391
1991225615,25sleep119909-21apt-get07:33:321
175825618,17sleep10-21swapper/111:43:501
1364825612,23sleep30-21swapper/312:23:213
990325513,24sleep39898-21apt-get07:03:353
670925512,27sleep36787-21ls08:33:513
439425516,18sleep10-21swapper/110:08:541
234025519,20sleep10-21swapper/110:03:371
1083925512,25sleep310943-21users10:28:553
916825417,15sleep30-21swapper/310:23:503
2797725313,20sleep00-21swapper/007:58:430
2355425314,19sleep00-21swapper/007:43:490
189052539,16sleep30-21swapper/307:28:543
1038825318,18sleep00-21swapper/007:03:480
331225216,20sleep10-21swapper/108:23:441
2986625220,18sleep329950-21timerandwakeup08:03:513
2562425211,24sleep325620-21apt-get11:18:333
2289525217,18sleep30-21swapper/311:08:433
1519725214,17sleep30-21swapper/312:28:223
325425011,21sleep03311-21irqstats08:23:430
310912507,16sleep30-21swapper/308:08:433
310912507,16sleep30-21swapper/308:08:433
1554325017,20sleep115610-21df09:03:371
1554325017,20sleep115610-21df09:03:371
313824916,16sleep30-21swapper/311:48:453
244024912,17sleep00-21swapper/010:03:400
165424914,19sleep30-21swapper/309:59:193
2655024810,22sleep026587-21ntp_states07:53:460
1523024713,17sleep00-21swapper/010:43:500
123524711,16sleep00-21swapper/009:58:480
421924513,16sleep00-21swapper/010:08:490
1157524510,18sleep10-21swapper/110:33:331
3022524312,17sleep330011-21/usr/sbin/munin09:48:413
2202324311,16sleep00-21swapper/009:23:340
1512524312,17sleep30-21swapper/310:43:443
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional