You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 15:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Mon Aug 17, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24072374911,3726sleep0895-21kworker/0:108:48:180
32557255526,513sleep032578-21/usr/sbin/munin08:40:060
1335252616,491sleep11366-21awk12:09:381
23121251811,492sleep123160-21ntpq08:10:031
869624976,445sleep3367-21snmpd09:05:533
22507242715,393sleep322549-21latency_hist08:09:413
3053023607,343sleep3367-21snmpd11:58:053
3194222469,227sleep119907-21kworker/u8:210:19:591
2563222852,117sleep30-21swapper/308:49:423
690322170,16sleep33194-21cyclictest10:44:393
43822157,193sleep3482-21ntpq12:05:043
21499221023,167sleep321468-21latency_hist09:49:413
4065220770,121sleep34088-21basename08:54:393
12508220219,26sleep30-21swapper/309:19:583
12508220219,26sleep30-21swapper/309:19:583
13674219811,170sleep313755-21mailstats07:40:093
13674219811,170sleep313755-21mailstats07:40:093
19650219227,18sleep30-21swapper/307:59:563
27318217817,146sleep127386-21apt-config11:49:391
8950217713,16sleep30-21swapper/307:25:073
6741217215,102sleep30-21swapper/307:19:423
5857216656,24sleep3314-21rs:main3
1042921627,145sleep3367-21snmpd09:13:083
682216011,133sleep2776-21sed10:24:402
4145215736,16sleep30-21swapper/307:10:133
11367215719,30sleep00-21swapper/007:34:410
17735215420,111sleep217949-21apt-config07:54:422
140562150119,13sleep30-21swapper/309:24:583
9615214841,85sleep30-21swapper/310:50:183
9369214717,115sleep09378-21diskstats09:09:580
24863214722,71sleep30-21swapper/309:59:543
2190321468,98sleep321956-21idleruntime09:49:573
652821457,91sleep36532-21find12:24:503
652821457,91sleep36532-21find12:24:503
31791214529,20sleep30-21swapper/310:19:493
1491821449,121sleep314920-21apt-get11:09:523
7526214144,19sleep30-21swapper/307:20:103
1596821409,23sleep30-21swapper/309:30:073
29994213911,74sleep30-21swapper/310:14:563
29994213911,74sleep30-21swapper/310:14:563
3050421369,115sleep130510-21cat08:34:501
23899213614,105sleep024001-21timerandwakeup09:55:110
23899213614,105sleep024001-21timerandwakeup09:55:110
527321310,8sleep20-21swapper/207:14:472
17541213115,98sleep317564-21dpkg07:54:393
27182213015,100sleep027194-21unin-run08:24:400
632212922,93sleep0664-21basename10:24:380
5493212920,85sleep15575-21fschecks_time07:14:591
29476212917,95sleep129481-21/usr/sbin/munin11:54:561
2506021286,21sleep30-21swapper/311:40:083
22753212813,92sleep00-21swapper/008:09:540
27713212510,97sleep027339-21apt-get11:49:420
27713212510,97sleep027339-21apt-get11:49:420
19845212513,97sleep019846-21cpu11:24:550
19845212513,97sleep019846-21cpu11:24:550
26590212411,26sleep30-21swapper/311:45:093
1386421249,103sleep013923-21sendmail_mailst11:05:080
12294212217,20sleep30-21swapper/311:00:073
12294212217,20sleep30-21swapper/311:00:073
18110212116,22sleep30-21swapper/311:19:413
1811021207,99sleep318175-21ls07:55:233
10984212022,72sleep311069-21latency_hist12:39:403
12096211829,74sleep30-21swapper/307:35:083
12096211829,74sleep30-21swapper/307:35:083
394321167,96sleep33962-21open_inodes12:15:503
394321167,96sleep33962-21open_inodes12:15:503
25864211413,40sleep30-21swapper/308:19:503
23880211214,49sleep30-21swapper/311:35:363
2005211214,76sleep22154-21/usr/sbin/munin08:45:132
1703621128,93sleep3367-21snmpd09:34:473
966211117,75sleep1231-21systemd-journal08:44:391
29709211114,82sleep129846-21awk08:30:101
27437211011,17sleep30-21swapper/308:24:553
6671210913,79sleep06750-21/usr/sbin/munin09:00:090
5307210910,17sleep30-21swapper/312:20:023
5307210910,17sleep30-21swapper/312:20:023
3164210813,71sleep03334-21apt-key12:14:420
3164210813,71sleep03334-21apt-key12:14:420
27713210843,51sleep30-21swapper/310:09:393
13499210713,36sleep30-21swapper/311:04:593
11709210616,72sleep211814-21fschecks_time07:34:592
11709210616,72sleep211814-21fschecks_time07:34:592
24421210515,70sleep10-21swapper/108:15:001
24421210515,70sleep10-21swapper/108:15:001
1866021059,84sleep318687-21awk11:20:013
30970210412,79sleep33194-21cyclictest11:59:553
30970210412,79sleep33194-21cyclictest11:59:553
3028321048,82sleep330317-21cat08:34:403
23288210411,15sleep30-21swapper/309:54:543
23288210411,15sleep30-21swapper/309:54:543
19608210315,71sleep219753-21sed11:24:412
2791921029,38sleep30-21swapper/311:49:573
2791921029,38sleep30-21swapper/311:49:573
7455210111,42sleep30-21swapper/309:04:403
24907210120,66sleep024988-21grep08:15:110
24907210120,66sleep024988-21grep08:15:110
12871210019,64sleep112929-21basename07:39:381
12871210019,64sleep112929-21basename07:39:381
3143129816,67sleep031471-21ps12:00:080
3143129816,67sleep031471-21ps12:00:080
27352989,76sleep32757-21kernelversion10:30:033
232052987,14sleep30-21swapper/308:10:053
22532987,11sleep00-21swapper/007:05:060
818729710,53sleep20-21swapper/212:29:582
107062977,17sleep310830-21timerwakeupswit07:30:123
281802966,77sleep327693-21apt-get10:09:523
211022968,71sleep30-21swapper/311:29:383
211022968,71sleep30-21swapper/311:29:383
147342969,13sleep30-21swapper/307:44:533
965529515,66sleep29664-21needreboot09:10:042
3157129523,44sleep031773-21latency_hist10:19:420
1994529524,22sleep30-21swapper/309:44:403
597229417,56sleep05578-21/usr/sbin/munin10:40:050
2462529414,67sleep224720-21latency_hist09:59:412
2462529414,67sleep224720-21latency_hist09:59:412
172982938,59sleep00-21swapper/009:35:000
1406629317,57sleep014158-21fschecks_time09:24:590
8942927,38sleep30-21swapper/310:24:503
1696629222,44sleep017114-21grep07:50:110
1090529023,43sleep110930-21systemctl12:39:381
849628937,35sleep30-21swapper/310:49:413
697528910,61sleep00-21swapper/007:19:550
104328921,18sleep30-21swapper/308:44:413
1940828828,34sleep10-21swapper/107:59:411
457228717,51sleep24080-21apt-get08:54:502
116042878,64sleep311546-21latency_hist10:59:423
32928616,15sleep20-21swapper/208:40:122
1868928524,35sleep20-21swapper/207:55:592
294662849,58sleep00-21swapper/011:54:550
2821328418,52sleep128254-21grep10:09:581
2769728419,30sleep20-21swapper/208:25:022
498328335,31sleep15016-21ntp_kernel_err08:55:131
3126828323,42sleep131334-21http10:19:391
3126828323,42sleep131334-21http10:19:391
255632836,27sleep30-21swapper/308:19:383
255632836,27sleep30-21swapper/308:19:383
304952826,30sleep30-21swapper/308:34:443
223952825,29sleep30-21swapper/311:31:203
223952825,29sleep30-21swapper/311:31:203
954728111,16sleep20-21swapper/210:50:122
528228126,37sleep25302-21apt10:39:382
51782815,29sleep30-21swapper/310:38:183
869728021,32sleep10-21swapper/110:49:551
741428015,41sleep07433-21kernelversion10:45:020
559028015,51sleep25660-21cat12:20:092
559028015,51sleep25660-21cat12:20:092
118212805,28sleep30-21swapper/309:16:063
118212805,28sleep30-21swapper/309:16:063
90872795,28sleep30-21swapper/312:31:083
697627916,43sleep17030-21cpuspeed07:19:561
2400227813,50sleep124143-21unixbench_singl09:55:121
2400227813,50sleep124143-21unixbench_singl09:55:121
2027827812,51sleep020289-21awk09:44:580
712227724,30sleep10-21swapper/110:44:501
464527712,50sleep14764-21timerwakeupswit10:35:091
380627715,24sleep00-21swapper/007:10:060
179532779,55sleep017955-21sshd11:19:180
450727612,49sleep14508-21sshd12:17:171
450727612,49sleep14508-21sshd12:17:171
193627629,24sleep20-21swapper/207:04:592
2767727520,37sleep227693-21apt-get10:09:382
2570127513,47sleep225849-21modprobe08:19:412
2096527522,28sleep00-21swapper/008:04:410
1507427515,39sleep114901-21/usr/sbin/munin11:10:021
2091227417,41sleep220928-21idleruntime-cro08:04:392
1549127417,43sleep215497-21cat09:29:502
76302735,12sleep00-21swapper/012:27:170
76302735,12sleep00-21swapper/012:27:170
292027320,35sleep23006-21cp12:14:392
281822737,29sleep00-21swapper/010:09:530
2001727318,19sleep10-21swapper/111:25:001
2001727318,19sleep10-21swapper/111:25:001
1996527326,29sleep020119-21cat09:44:410
162327310,47sleep30-21swapper/312:09:523
134192737,55sleep00-21swapper/009:21:160
134192737,55sleep00-21swapper/009:21:160
1120727312,47sleep011225-21netstat09:15:020
1120727312,47sleep011225-21netstat09:15:020
2889127224,26sleep20-21swapper/210:10:272
2783527215,35sleep127876-21ntpq08:25:051
2777427223,30sleep00-21swapper/010:09:400
991427120,30sleep00-21swapper/012:35:000
2182327110,48sleep221890-21fschecks_time09:49:562
1877227114,20sleep20-21swapper/209:40:032
1622827117,33sleep216254-21munin-run11:14:392
1269027123,27sleep20-21swapper/209:20:012
1269027123,27sleep20-21swapper/209:20:012
2264827017,34sleep222668-21munin-run11:34:382
2264827017,34sleep222668-21munin-run11:34:382
2179327017,27sleep00-21swapper/011:30:040
2179327017,27sleep00-21swapper/011:30:040
1560927011,42sleep015756-21unixbench_multi11:10:130
1132327010,17sleep10-21swapper/110:57:181
318926914,38sleep13240-21awk08:50:051
275926816,33sleep02837-21/usr/sbin/munin10:30:040
141452688,11sleep00-21swapper/007:40:470
141452688,11sleep00-21swapper/007:40:470
3002626714,28sleep00-21swapper/010:15:140
1598026717,26sleep00-21swapper/009:30:080
156362676,12sleep10-21swapper/109:30:011
684426622,25sleep20-21swapper/209:00:112
1928426619,23sleep10-21swapper/109:40:141
1416126616,24sleep10-21swapper/109:25:001
1108626616,34sleep211108-21irqstats09:15:002
1108626616,34sleep211108-21irqstats09:15:002
832926520,27sleep00-21swapper/009:05:090
8002650,7sleep20-21swapper/212:05:102
566126516,21sleep00-21swapper/007:15:020
2535026515,31sleep125436-21grep10:00:091
427426417,22sleep20-21swapper/208:54:422
294822649,35sleep20-21swapper/211:54:572
1886626413,37sleep118949-21sed11:20:071
162626417,24sleep10-21swapper/112:09:541
1523326417,32sleep215316-21systemctl07:45:092
120362645,46sleep10-21swapper/109:18:471
120362645,46sleep10-21swapper/109:18:471
1081826416,28sleep20-21swapper/210:55:092
2512126319,27sleep20-21swapper/210:00:012
249752638,12sleep10-21swapper/111:40:061
1313926316,25sleep00-21swapper/011:04:400
1313926316,25sleep00-21swapper/011:04:400
1209026317,25sleep10-21swapper/111:00:011
1209026317,25sleep10-21swapper/111:00:011
718126214,23sleep20-21swapper/207:20:012
592926219,26sleep00-21swapper/008:59:400
341126215,31sleep23413-21/usr/sbin/munin07:09:512
2617526222,24sleep00-21swapper/010:04:400
2199526214,20sleep10-21swapper/111:30:091
2199526214,20sleep10-21swapper/111:30:091
2029026212,30sleep10-21swapper/109:44:591
18452625,45sleep00-21swapper/010:25:190
1687026216,21sleep10-21swapper/111:15:271
6162618,13sleep00-21swapper/012:05:090
2840926120,20sleep10-21swapper/111:50:121
2840926120,20sleep10-21swapper/111:50:121
2161326119,23sleep10-21swapper/108:05:071
1715326115,22sleep20-21swapper/209:34:582
1628826114,31sleep116351-21/usr/sbin/munin07:49:561
1395926116,23sleep20-21swapper/207:40:122
1395926116,23sleep20-21swapper/207:40:122
990326019,20sleep10-21swapper/109:10:111
807526015,27sleep10-21swapper/112:29:491
3186926012,33sleep131871-21users12:00:131
3186926012,33sleep131871-21users12:00:131
28692608,12sleep20-21swapper/210:30:062
2167326011,27sleep00-21swapper/008:05:100
978025915,24sleep00-21swapper/007:29:400
536325916,27sleep05446-21latency_hist10:39:410
320972595,42sleep20-21swapper/212:02:532
320972595,42sleep20-21swapper/212:02:532
3124325925,18sleep20-21swapper/208:35:122
3023525916,28sleep230241-21tune2fs10:15:482
3023525916,28sleep230241-21tune2fs10:15:482
2357825913,18sleep00-21swapper/011:35:090
188925914,19sleep10-21swapper/108:45:121
1395025911,19sleep10-21swapper/111:05:091
983025812,24sleep10-21swapper/107:29:411
869625815,25sleep08690-21apt-get10:49:540
290692585,12sleep00-21swapper/008:29:570
2719525810,22sleep10-21swapper/108:24:411
246125820,19sleep10-21swapper/107:05:111
2298825818,19sleep20-21swapper/208:10:002
2165925817,25sleep221658-21ntp_states08:05:082
2115725814,28sleep221306-21cat11:29:412
2115725814,28sleep221306-21cat11:29:412
1346225813,21sleep10-21swapper/107:40:031
1346225813,21sleep10-21swapper/107:40:031
118825817,18sleep20-21swapper/210:25:012
1182225814,27sleep211816-21apt-get10:59:532
1182225814,27sleep211816-21apt-get10:59:532
845425714,20sleep20-21swapper/210:49:402
560225715,24sleep25596-21apt-get10:39:552
393625716,25sleep23942-21ntp_states12:15:492
393625716,25sleep23942-21ntp_states12:15:492
2830425713,26sleep228310-21ntp_states11:50:092
2830425713,26sleep228310-21ntp_states11:50:092
1241925716,25sleep112414-21apt-get09:19:521
1241925716,25sleep112414-21apt-get09:19:521
1062125718,19sleep20-21swapper/207:30:112
2903025614,26sleep229059-21df08:29:552
270652565,16sleep20-21swapper/211:46:212
2330225618,17sleep10-21swapper/111:35:011
1805725617,19sleep20-21swapper/211:19:402
891325519,19sleep20-21swapper/207:25:062
781725517,23sleep17853-21fschecks_count09:04:581
293625513,26sleep12974-21/usr/sbin/munin10:30:091
2606025513,25sleep026054-21apt-get11:44:540
2419625516,16sleep20-21swapper/211:39:392
1045625513,24sleep210547-21users12:35:112
458325417,20sleep10-21swapper/107:13:181
228882547,37sleep00-21swapper/009:53:130
1452625414,23sleep20-21swapper/209:25:102
114912545,38sleep00-21swapper/010:59:270
772525316,22sleep27721-21apt-get09:04:512
222725314,22sleep10-21swapper/110:29:401
2020125315,20sleep10-21swapper/108:00:101
1803725312,23sleep017556-21apt-get07:54:480
1346225313,19sleep20-21swapper/211:04:582
1006725313,24sleep110061-21apt-get07:29:531
941125210,21sleep00-21swapper/012:34:400
631525215,23sleep16286-21latency_hist12:24:401
631525215,23sleep16286-21latency_hist12:24:401
1706525213,22sleep117059-21apt-get09:34:531
1186125212,23sleep011884-21irqstats07:35:010
1186125212,23sleep011884-21irqstats07:35:010
704125115,16sleep10-21swapper/112:25:081
704125115,16sleep10-21swapper/112:25:081
317325119,17sleep00-21swapper/008:50:040
316125113,20sleep30-21swapper/307:09:403
285152515,37sleep00-21swapper/008:27:220
2451425113,22sleep024508-21apt-get11:39:530
2451125116,18sleep20-21swapper/211:39:512
2173625113,23sleep121730-21apt-get09:49:511
2058925116,21sleep220628-21ntp_kernel_pll_09:45:042
1482025115,16sleep00-21swapper/007:44:590
629925011,21sleep06314-21munin-run12:24:390
629925011,21sleep06314-21munin-run12:24:390
67182497,10sleep10-21swapper/110:42:481
2684724912,22sleep126871-21ntp_states10:05:051
680324812,22sleep26826-21irqstats12:25:012
680324812,22sleep26826-21irqstats12:25:012
3053124814,18sleep00-21swapper/008:34:530
675124716,18sleep10-21swapper/109:00:101
2429924711,21sleep224296-21apt-get08:14:532
2429924711,21sleep224296-21apt-get08:14:532
1829224710,22sleep018288-21apt-get11:19:500
147372437,13sleep10-21swapper/107:44:551
22402427,14sleep00-21swapper/012:10:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional