You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 22:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Jul 24, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1182829717,952sleep211864-21ntpq11:33:322
6495259647,462sleep26641-21unixbench_multi09:33:392
6495259647,462sleep26641-21unixbench_multi09:33:392
27059248219,446sleep327098-21cron08:58:073
27059248219,446sleep327098-21cron08:58:073
24202244015,409sleep324208-21sshd12:13:253
19155224721,207sleep319216-21modprobe11:58:073
19155224721,207sleep319216-21modprobe11:58:073
11141222016,114sleep211165-21modprobe11:33:082
5280221936,96sleep25183-21/usr/sbin/munin11:13:262
30607221967,94sleep20-21swapper/212:33:232
31560221336,89sleep20-21swapper/207:28:212
31560221336,89sleep20-21swapper/207:28:212
28194221128,149sleep228290-21grep10:43:262
13379221033,163sleep213427-21ntpq11:38:332
15498220511,24sleep10-21swapper/110:03:291
1197220232,81sleep20-21swapper/207:33:422
5576220040,123sleep25701-21cat09:33:102
24679219917,97sleep224802-21latency_hist07:08:102
8876219349,72sleep20-21swapper/211:23:392
6932219172,29sleep20-21swapper/207:53:282
14333218917,157sleep214409-21mailstats09:58:382
32041218611,161sleep232135-21timerandwakeup10:53:402
14888217814,150sleep310842-21kworker/3:111:43:343
18757217731,19sleep20-21swapper/208:28:402
855217671,19sleep20-21swapper/209:18:092
13733217549,36sleep20-21swapper/208:13:462
13733217549,36sleep20-21swapper/208:13:462
25477216612,76sleep20-21swapper/212:18:092
25477216612,76sleep20-21swapper/212:18:092
15417216639,88sleep215481-21mailstats08:18:392
15417216639,88sleep215481-21mailstats08:18:392
16926216545,106sleep217015-21cat11:48:382
29058216424,103sleep229046-21apt-get12:28:222
24211215712,131sleep024254-21fschecks_count12:13:260
19965215669,20sleep220038-21tr11:58:392
19249214710,77sleep219391-21latency_hist11:58:102
19249214710,77sleep219391-21latency_hist11:58:102
24503214350,76sleep224501-21munin-run10:33:092
2591321410,29sleep125917-21unixbench_singl10:34:411
3698214045,15sleep20-21swapper/209:24:352
6662213746,77sleep26815-21date07:53:112
29079213715,96sleep20-21swapper/207:18:402
29079213715,96sleep20-21swapper/207:18:402
24833213743,75sleep20-21swapper/208:48:422
24833213743,75sleep20-21swapper/208:48:422
2200121316,113sleep222037-21ntpq10:23:322
2200121316,113sleep222037-21ntpq10:23:322
27909213014,27sleep20-21swapper/210:43:102
19235212918,89sleep10-21swapper/111:58:091
19235212918,89sleep10-21swapper/111:58:091
354121278,79sleep20-21swapper/207:43:102
31895212610,102sleep331410-21/usr/sbin/munin10:53:373
2853621265,11sleep20-21swapper/209:01:462
16101212650,41sleep216108-21munin-run11:48:092
10011212646,68sleep210003-21apt-get08:03:252
21642212510,91sleep20-21swapper/212:03:392
1658212537,65sleep20-21swapper/209:18:372
3141621235,46sleep20-21swapper/209:08:482
10197212352,57sleep210227-21awk09:48:082
10197212352,57sleep210227-21awk09:48:082
2412021227,74sleep20-21swapper/212:13:202
18502212212,22sleep20-21swapper/210:13:232
19548212136,43sleep10-21swapper/108:33:191
11266212116,90sleep111270-21modprobe08:08:081
3323212015,27sleep03325-21sshd11:07:340
29888212019,87sleep315366-1kworker/3:0H10:48:243
26316212035,39sleep20-21swapper/210:38:102
26705211718,75sleep20-21swapper/212:19:342
22220211711,94sleep00-21swapper/008:40:450
4153211517,82sleep34180-21netstat07:43:363
26974211216,77sleep127020-21fschecks_count07:13:301
2432211121,22sleep10-21swapper/109:23:081
3065921109,42sleep230782-21cat07:23:412
3065921109,42sleep230782-21cat07:23:412
23861211019,63sleep123994-21sed10:28:401
2113221105,70sleep20-21swapper/210:20:252
5347210713,17sleep10-21swapper/107:48:261
8916210619,68sleep38948-21proc_pri07:58:373
23351210618,75sleep023402-21cpuspeed07:03:250
8039210518,68sleep38169-21grep09:38:403
3784210511,20sleep20-21swapper/207:43:272
9396210412,76sleep19399-21ntp_states09:43:351
9396210412,76sleep19399-21ntp_states09:43:351
7559210417,72sleep17674-21cat07:53:431
30755210415,74sleep030771-21cut09:08:300
24980210417,68sleep324964-21apt-get10:33:263
19771210425,60sleep319783-21modprobe10:18:083
12347210411,24sleep20-21swapper/209:53:292
12347210411,24sleep20-21swapper/209:53:292
31097210312,21sleep20-21swapper/210:52:352
23433210347,40sleep323447-21grep07:03:283
26574210222,18sleep20-21swapper/208:53:422
26574210222,18sleep20-21swapper/208:53:422
24933210121,15sleep20-21swapper/207:08:222
3419210010,76sleep33545-21latency_hist11:08:103
3193429819,65sleep331979-21awk07:28:343
3193429819,65sleep331979-21awk07:28:343
300332975,11sleep20-21swapper/209:06:062
2918229720,55sleep329330-21head07:18:413
2918229720,55sleep329330-21head07:18:413
1695029719,57sleep116453-21/usr/sbin/munin08:23:381
33102966,11sleep20-21swapper/211:07:242
589329512,20sleep15897-21diskstats09:33:261
589329512,20sleep15897-21diskstats09:33:261
302642956,13sleep00-21swapper/010:48:370
299629511,69sleep03085-21memory09:23:320
2598529411,70sleep026000-21kernelversion12:18:310
2141529419,54sleep121562-21grep10:23:111
268112935,66sleep10-21swapper/110:38:291
175072938,19sleep20-21swapper/210:08:392
1508729314,56sleep015172-21latency11:43:390
789529213,65sleep17965-21mailstats09:38:381
2279129213,27sleep10-21swapper/112:08:311
2279129213,27sleep10-21swapper/112:08:311
481929118,58sleep14888-21grep09:28:371
821429017,20sleep00-21swapper/007:58:100
727729014,25sleep00-21swapper/011:18:490
91052899,66sleep29228-21timerwakeupswit07:58:402
67522899,66sleep26739-21apt-get11:18:302
16772895,11sleep20-21swapper/211:01:192
346328816,53sleep13616-21/usr/sbin/munin09:23:411
2978028815,55sleep129928-21sed07:23:111
2978028815,55sleep129928-21sed07:23:111
261482875,11sleep30-21swapper/307:12:353
115582879,19sleep20-21swapper/208:08:252
2902728619,51sleep328544-21apt-get12:28:193
93128517,53sleep3991-21latency07:33:353
2728828515,51sleep326863-21/usr/sbin/munin07:13:363
272692858,63sleep2364-21snmpd12:23:212
1164028516,51sleep111532-21/usr/sbin/munin08:08:281
316612846,11sleep10-21swapper/107:28:281
316612846,11sleep10-21swapper/107:28:281
2157028416,20sleep10-21swapper/112:03:381
3212528316,51sleep132016-21/usr/sbin/munin09:13:271
230262837,37sleep20-21swapper/212:08:362
2083128310,17sleep20-21swapper/208:38:082
1468528317,19sleep10-21swapper/108:18:091
1468528317,19sleep10-21swapper/108:18:091
680928218,38sleep36739-21apt-get11:18:353
2529928212,57sleep025317-21netstat07:08:330
166612825,65sleep20-21swapper/210:08:062
2123428123,25sleep10-21swapper/108:38:281
282572808,56sleep228405-21sed07:18:112
796627914,19sleep20-21swapper/209:38:392
2900627915,47sleep029015-21df_inode09:03:250
1810227911,53sleep018108-21diskstats08:28:250
1200827911,52sleep312090-21sed11:33:373
997227818,41sleep110061-21grep11:28:261
251182788,22sleep124911-21/usr/sbin/munin07:08:301
2334727825,32sleep123343-21apt-get07:03:211
2278127814,49sleep022861-21tune2fs08:43:270
2278127814,49sleep022861-21tune2fs08:43:270
2190227813,50sleep021960-21sed10:23:300
2190227813,50sleep021960-21sed10:23:300
1382127717,44sleep013839-21/usr/sbin/munin09:58:190
1674727615,46sleep316786-21expr08:23:323
1360927610,52sleep313688-21date09:58:103
1360927610,52sleep313688-21date09:58:103
533527511,52sleep0364-21snmpd09:30:420
2904927518,26sleep00-21swapper/012:28:200
166972759,51sleep20-21swapper/208:23:312
163722756,58sleep00-21swapper/010:04:250
107272757,56sleep210715-21apt-get09:48:242
3029727421,31sleep10-21swapper/110:48:381
283712746,14sleep10-21swapper/110:43:301
1477027416,42sleep114277-21apt-get11:43:201
278032735,12sleep00-21swapper/007:13:460
2435727312,46sleep024434-21cut08:48:320
2435727312,46sleep024434-21cut08:48:320
1944827322,29sleep30-21swapper/311:58:203
647227218,38sleep06524-21sendmail11:18:080
40872726,29sleep20-21swapper/211:08:332
1198427213,39sleep112014-21munin-run09:53:081
1028227222,29sleep30-21swapper/308:03:343
211452719,47sleep20-21swapper/208:38:202
191427119,33sleep01938-21munin-run07:38:080
1847927120,31sleep118485-21ntp_states08:28:351
1826427119,31sleep10-21swapper/110:13:101
1461227112,19sleep00-21swapper/008:18:080
1461227112,19sleep00-21swapper/008:18:080
309472707,53sleep00-21swapper/007:23:550
309472707,53sleep00-21swapper/007:23:550
3057527012,43sleep130658-21sed07:23:401
3057527012,43sleep130658-21sed07:23:401
2047527016,30sleep30-21swapper/308:33:413
1519427016,30sleep30-21swapper/310:03:113
1313127024,23sleep10-21swapper/111:38:271
63426915,37sleep1699-21/usr/sbin/munin10:58:271
2623126928,20sleep30-21swapper/312:18:383
242272699,17sleep10-21swapper/108:48:211
242272699,17sleep10-21swapper/108:48:211
238372695,11sleep20-21swapper/208:46:402
238372695,11sleep20-21swapper/208:46:402
1353026919,21sleep10-21swapper/109:58:081
1353026919,21sleep10-21swapper/109:58:081
491526815,30sleep11-21systemd11:13:081
284826811,15sleep30-21swapper/307:38:393
1850426820,25sleep30-21swapper/308:28:373
2894826715,34sleep029007-21/usr/sbin/munin07:18:380
2894826715,34sleep029007-21/usr/sbin/munin07:18:380
2808826714,26sleep30-21swapper/308:58:403
2702226717,29sleep10-21swapper/112:23:081
2588626718,26sleep00-21swapper/008:53:280
2588626718,26sleep00-21swapper/008:53:280
1701626724,26sleep317136-21sed11:48:393
1336126719,26sleep30-21swapper/308:13:353
1336126719,26sleep30-21swapper/308:13:353
785526616,25sleep00-21swapper/009:38:370
605626621,21sleep10-21swapper/111:13:411
385926611,39sleep03864-21df_inode07:43:290
297526619,30sleep03127-21sed07:38:400
1444526614,26sleep30-21swapper/311:43:103
501126522,20sleep30-21swapper/311:13:103
3194626518,26sleep10-21swapper/112:38:101
315482656,12sleep00-21swapper/012:34:010
3097426513,20sleep30-21swapper/312:33:323
269226519,24sleep30-21swapper/309:23:183
2102826510,29sleep00-21swapper/012:03:210
2079026519,21sleep10-21swapper/112:03:101
316492648,13sleep00-21swapper/007:28:270
316492648,13sleep00-21swapper/007:28:270
3077226410,17sleep10-21swapper/109:08:311
1847426416,32sleep118475-21munin-node10:13:181
1022926414,35sleep310264-21munin-run09:48:093
1022926414,35sleep310264-21munin-run09:48:093
902526324,22sleep00-21swapper/011:23:410
592726312,29sleep30-21swapper/307:48:403
2795126318,24sleep00-21swapper/012:23:400
2655426316,27sleep30-21swapper/310:38:233
2589226319,25sleep10-21swapper/108:53:291
2589226319,25sleep10-21swapper/108:53:291
1695526319,22sleep00-21swapper/010:08:240
938526218,29sleep09395-21ntp_states09:43:340
938526218,29sleep09395-21ntp_states09:43:340
2849226213,20sleep30-21swapper/310:43:323
2705726214,24sleep00-21swapper/010:38:360
2424826220,19sleep30-21swapper/308:48:273
2424826220,19sleep30-21swapper/308:48:273
1554826213,14sleep00-21swapper/008:18:410
1554826213,14sleep00-21swapper/008:18:410
120932628,15sleep00-21swapper/008:08:390
1188726216,28sleep011893-21ntp_states11:33:340
2577126120,21sleep00-21swapper/010:34:410
2216826113,21sleep30-21swapper/310:23:373
2216826113,21sleep30-21swapper/310:23:373
169432617,14sleep00-21swapper/008:23:370
701826014,27sleep17071-21/usr/sbin/munin11:18:421
3245626013,28sleep00-21swapper/009:13:350
2867426016,21sleep00-21swapper/010:43:370
2836626014,25sleep10-21swapper/108:59:351
834125914,27sleep38334-21apt-get11:23:243
601325912,29sleep06141-21/usr/sbin/munin07:48:410
3148525915,18sleep00-21swapper/010:53:260
312825913,18sleep10-21swapper/107:38:411
187225921,20sleep30-21swapper/311:03:093
138482599,32sleep10-21swapper/109:58:241
1295125915,25sleep30-21swapper/308:13:113
7212585,10sleep30-21swapper/309:16:283
248225815,20sleep30-21swapper/309:23:103
2169925815,30sleep321765-21wc08:38:383
2160825815,18sleep10-21swapper/110:23:191
2160825815,18sleep10-21swapper/110:23:191
2111925817,26sleep30-21swapper/312:03:283
1965725820,17sleep00-21swapper/008:33:260
1654125819,22sleep10-21swapper/111:48:291
852525712,20sleep00-21swapper/007:58:260
375625717,18sleep00-21swapper/011:08:270
3038525720,21sleep30-21swapper/307:23:333
3038525720,21sleep30-21swapper/307:23:333
2269325715,26sleep322688-21apt-get08:43:223
2269325715,26sleep322688-21apt-get08:43:223
2258025712,26sleep322571-21apt-get12:08:253
1597825716,24sleep316052-21cat10:03:393
1564325717,18sleep10-21swapper/108:18:421
1564325717,18sleep10-21swapper/108:18:421
841425610,17sleep10-21swapper/111:23:261
2336625616,23sleep00-21swapper/012:08:420
1975225614,16sleep00-21swapper/011:58:330
252092557,40sleep00-21swapper/012:15:360
252092557,40sleep00-21swapper/012:15:360
2511825521,18sleep30-21swapper/312:14:343
2511825521,18sleep30-21swapper/312:14:343
2139725517,19sleep00-21swapper/010:23:100
196225514,24sleep11993-21munin-run07:38:091
190312559,34sleep0364-21snmpd11:56:310
190312559,34sleep0364-21snmpd11:56:310
1076725514,24sleep310715-21apt-get09:48:543
581425410,30sleep35883-21df09:33:243
581425410,30sleep35883-21df09:33:243
3037025419,17sleep10-21swapper/112:33:091
2286425416,22sleep10-21swapper/108:43:281
2286425416,22sleep10-21swapper/108:43:281
1907525419,19sleep00-21swapper/010:13:370
1692725411,27sleep316946-21apt-get10:08:193
1519825417,19sleep30-21swapper/308:18:313
1519825417,19sleep30-21swapper/308:18:313
144425411,16sleep00-21swapper/009:18:310
1273425413,26sleep012740-21ntp_states09:53:390
1273425413,26sleep012740-21ntp_states09:53:390
3026025313,17sleep10-21swapper/109:08:111
2727025313,23sleep327266-21apt-get12:23:223
2318025313,23sleep023174-21apt-get10:28:230
17122536,37sleep00-21swapper/011:01:450
1039225217,18sleep30-21swapper/311:28:373
740425115,20sleep37461-21awk07:53:413
231925120,15sleep10-21swapper/111:03:281
1119525117,23sleep011279-21latency_hist11:33:100
3141225010,27sleep3364-21snmpd09:08:453
2806325011,22sleep128176-21users12:23:411
1632625012,17sleep00-21swapper/011:48:190
1189525015,17sleep10-21swapper/111:33:351
1123125011,23sleep011242-21ntp_states09:49:230
588524912,22sleep05892-21df_inode09:33:250
588524912,22sleep05892-21df_inode09:33:250
2325324918,17sleep30-21swapper/310:28:263
871424814,18sleep10-21swapper/107:58:311
3167524810,21sleep131702-21irqstats10:53:311
1301024810,20sleep00-21swapper/011:38:220
565924711,21sleep35667-21ntp_states11:13:353
421624714,18sleep10-21swapper/107:43:381
3019324714,17sleep330246-21ssh09:08:093
2737624712,18sleep00-21swapper/008:58:210
2012624510,20sleep320159-21df10:18:253
2540224415,15sleep10-21swapper/112:18:081
2540224415,15sleep10-21swapper/112:18:081
1001424010,16sleep00-21swapper/008:03:270
47292399,13sleep30-21swapper/309:28:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional