You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-23 - 15:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Aug 23, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
118322700013,6973sleep330483-21kworker/3:212:29:573
20883217388,1714sleep120905-21users11:14:531
6782213409,1306sleep16847-21/usr/sbin/munin12:14:491
20488283126,791sleep120533-21ntpq07:49:431
19528267716,644sleep019530-21sshd11:11:570
3215245721,419sleep33304-21perl12:04:373
28317239817,249sleep128368-21sendmail09:59:171
24173233710,310sleep224230-21/usr/sbin/munin09:44:402
150562327115,195sleep115243-21latency_hist07:34:201
7433231115,279sleep07519-21fw_packets10:34:380
2101228723,87sleep22131-21modprobe10:19:172
95872285101,129sleep19698-21timerandwakeup07:14:501
157732284245,25sleep315775-21sshd09:17:563
157732284245,25sleep315775-21sshd09:17:563
5992242108,23sleep10-21swapper/110:14:191
3727223817,205sleep33836-21sed10:24:193
6214223531,181sleep16253-21wc10:29:461
179872235124,51sleep118093-21apt-config11:09:191
27978222712,99sleep128036-21fschecks_count08:14:381
2684022269,202sleep126952-21cat11:34:471
586522190,68sleep16781-21cyclictest08:44:501
3025221810,134sleep13132-21timerandwakeup10:19:491
10196221714,149sleep110320-21latency_hist10:44:201
10196221714,149sleep110320-21latency_hist10:44:201
16701221518,150sleep116722-21modprobe07:39:171
3267219715,155sleep03413-21latency_hist08:39:200
30457219013,163sleep330511-21sed10:04:403
19180218825,128sleep119292-21sed07:44:501
1671218620,58sleep11758-21fschecks_time11:59:391
11242218531,92sleep111383-21sed07:19:511
11242218531,92sleep111383-21sed07:19:511
3126921717,103sleep131308-21ntpq11:49:411
3126921717,103sleep131308-21ntpq11:49:411
26315217017,138sleep226388-21fschecks_time11:34:362
29151216912,135sleep129296-21latency_hist11:44:201
23807215417,120sleep323931-21sed11:24:483
7990215319,111sleep38135-21sed10:34:493
11470215356,63sleep110988-21apt-get09:04:251
2168321519,128sleep121777-21fschecks_time11:19:371
13630214811,42sleep00-21swapper/007:29:310
10520214810,95sleep110587-21fschecks_time10:44:361
10520214810,95sleep110587-21fschecks_time10:44:361
9859214728,104sleep19336-21/usr/sbin/munin12:24:461
11283214379,18sleep10-21swapper/112:29:431
4192214210,23sleep00-21swapper/008:39:490
13536214116,104sleep10-21swapper/110:54:331
13529214162,55sleep10-21swapper/109:09:461
3677213910,113sleep13751-21switchtime12:04:481
19075213918,19sleep10-21swapper/109:29:211
16005213926,77sleep116157-21cut09:19:191
23341213729,49sleep123393-21if_eth011:24:381
29678213630,47sleep129721-21if_err_eth008:19:381
16873213616,100sleep10-21swapper/109:19:481
93421347,116sleep1367-21snmpd11:55:101
93421347,116sleep1367-21snmpd11:55:101
789421339,63sleep10-21swapper/108:54:201
1911213213,101sleep10-21swapper/108:34:331
5220213127,42sleep10-21swapper/112:09:481
26821213127,45sleep126962-21sed09:54:201
26319212713,63sleep10-21swapper/108:09:291
12663212711,97sleep212716-21/usr/sbin/munin12:34:372
417521259,74sleep14194-21apt-get10:24:291
417521259,74sleep14194-21apt-get10:24:291
712721228,78sleep17213-21fschecks_time07:09:371
17917212221,81sleep317932-21modprobe11:09:173
6698212136,63sleep16721-21apt07:09:171
911621208,72sleep19144-21latency_hist12:24:181
8647211738,20sleep10-21swapper/110:39:191
29374211740,31sleep129520-21unixbench_singl09:59:501
9467211519,77sleep3231-21systemd-journal08:59:183
9467211519,77sleep3231-21systemd-journal08:59:183
30659211221,78sleep330684-21latency_hist11:49:193
8107211021,68sleep28108-21/usr/sbin/munin08:54:292
8107211021,68sleep28108-21/usr/sbin/munin08:54:292
5409211018,73sleep35549-21unixbench_singl12:09:503
2577321105,30sleep10-21swapper/108:05:421
32522210921,64sleep232528-21modprobe08:29:172
26105210813,82sleep026144-21cut08:09:200
29158210722,67sleep029195-21basename08:19:170
2587421077,64sleep1367-21snmpd11:33:301
1657210712,79sleep01670-21fschecks_count11:59:380
1210421078,57sleep10-21swapper/107:24:311
22098210633,50sleep022137-21modprobe09:39:170
8972210515,74sleep29008-21fschecks_count10:39:362
8972210515,74sleep29008-21fschecks_count10:39:362
28260210517,73sleep228294-21ntp_kernel_err08:14:432
16371210518,71sleep316396-21awk11:04:163
11523210517,71sleep311494-21apt-get09:04:483
7908210411,79sleep08010-21grep12:19:360
22392210428,17sleep122506-21awk07:54:511
1844921048,62sleep118443-21apt-get11:09:311
28410210324,28sleep20-21swapper/211:39:492
10831210313,76sleep010881-21sed07:19:410
10831210313,76sleep010881-21sed07:19:410
347621027,78sleep10-21swapper/108:39:291
1591210135,50sleep10-21swapper/108:33:561
2446521006,62sleep1367-21snmpd08:03:481
1091229816,66sleep010913-21/usr/sbin/munin12:29:340
3055829726,18sleep10-21swapper/110:04:421
2627829614,67sleep326407-21/usr/sbin/munin09:49:503
2477129616,14sleep20-21swapper/208:04:292
1827629632,51sleep118278-21cron07:44:181
1827629632,51sleep118278-21cron07:44:181
2789229513,65sleep327946-21grep11:39:383
132402958,29sleep10-21swapper/110:54:171
132402958,29sleep10-21swapper/110:54:171
2331429412,59sleep022388-21/usr/sbin/munin09:39:500
1929329316,63sleep219431-21awk07:44:512
1488529114,63sleep315026-21latency_hist10:59:203
67429019,50sleep2815-21sed11:54:472
67429019,50sleep2815-21sed11:54:472
1941529026,42sleep10-21swapper/109:29:371
12002905,11sleep30-21swapper/310:14:423
952128913,60sleep09586-21swap07:14:490
1282828914,56sleep212956-21latency_hist09:09:192
3073628825,49sleep030787-21systemd08:24:180
2136928825,42sleep321389-21modprobe07:54:173
914528711,61sleep29283-21latency_hist12:24:192
1958528716,50sleep019269-21/usr/sbin/munin09:29:400
58328623,46sleep0596-21latency_hist10:14:180
3169428513,54sleep10-21swapper/108:25:081
131952859,59sleep1367-21snmpd12:34:481
313528417,35sleep20-21swapper/210:19:502
2241228317,48sleep10-21swapper/109:39:301
2084228324,15sleep10-21swapper/109:34:341
991528217,29sleep30-21swapper/308:59:393
666128215,53sleep26711-21grep08:49:362
251342828,55sleep20-21swapper/211:29:432
615128016,48sleep26193-21awk10:29:432
2102428012,54sleep221087-21if_eth009:34:392
170082800,6sleep00-21swapper/007:39:270
170082800,6sleep00-21swapper/007:39:270
710927917,37sleep07126-21fschecks_count07:09:360
44427921,37sleep30-21swapper/308:29:383
2700927921,39sleep027118-21cat08:09:480
1381227925,23sleep00-21swapper/012:39:190
124512799,57sleep312505-21processes10:49:473
124512799,57sleep312505-21processes10:49:473
2638927813,48sleep326447-21idleruntime11:34:373
2553627715,41sleep325679-21grep08:04:513
2455727726,29sleep00-21swapper/008:04:190
1401827716,29sleep20-21swapper/210:54:462
99812769,30sleep10-21swapper/108:59:411
2060227621,31sleep30-21swapper/309:34:193
1469627612,44sleep014703-21/usr/sbin/munin09:14:380
1469627612,44sleep014703-21/usr/sbin/munin09:14:380
1070327622,23sleep00-21swapper/010:44:390
1070327622,23sleep00-21swapper/010:44:390
2992727519,37sleep230002-21grep11:44:502
1970827515,40sleep219746-21modprobe11:14:172
701727422,37sleep37019-21cron10:34:163
53662746,12sleep00-21swapper/010:28:030
53662746,12sleep00-21swapper/010:28:030
35027413,39sleep332415-21/usr/sbin/munin11:54:403
35027413,39sleep332415-21/usr/sbin/munin11:54:403
1098527414,46sleep210992-21sh12:29:362
813227318,14sleep10-21swapper/108:54:321
813227318,14sleep10-21swapper/108:54:321
672727322,35sleep26767-21munin-run07:09:182
3116927315,44sleep331235-21latency11:49:393
3116927315,44sleep331235-21latency11:49:393
250142735,11sleep10-21swapper/109:46:411
2288827319,25sleep20-21swapper/211:24:182
1811627315,33sleep20-21swapper/211:09:202
1541227331,25sleep214928-21apt-get07:34:292
1456927310,17sleep30-21swapper/307:29:543
3098727221,29sleep00-21swapper/011:49:360
3098727221,29sleep00-21swapper/011:49:360
294182727,30sleep10-21swapper/111:44:361
1678127222,21sleep00-21swapper/007:39:200
157362725,11sleep10-21swapper/109:17:261
946027116,35sleep09336-21/usr/sbin/munin12:24:370
9112716,29sleep10-21swapper/110:14:361
2541627115,41sleep325560-21awk11:29:493
1801827115,38sleep218112-21memory09:24:412
384127022,29sleep00-21swapper/010:24:200
943826914,24sleep20-21swapper/207:14:472
265842699,15sleep30-21swapper/308:09:393
598026821,22sleep00-21swapper/012:14:190
1642326819,27sleep20-21swapper/211:04:192
1620126819,21sleep20-21swapper/209:19:332
1558326810,38sleep10-21swapper/110:59:461
832326717,32sleep28333-21munin-run07:14:182
2718826717,25sleep20-21swapper/209:54:382
218126718,33sleep02267-21grep10:19:190
212432678,50sleep20-21swapper/207:52:422
1710026715,30sleep317118-21fschecks_count07:39:353
1710026715,30sleep317118-21fschecks_count07:39:353
1250326711,27sleep00-21swapper/007:24:420
660026616,22sleep00-21swapper/012:14:440
571126618,23sleep00-21swapper/008:44:500
540526618,26sleep00-21swapper/007:04:280
3131026619,20sleep20-21swapper/211:49:422
3131026619,20sleep20-21swapper/211:49:422
2871626611,39sleep328748-21fschecks_count09:59:353
2134526621,27sleep321434-21ls11:19:193
1824026613,36sleep018319-21/usr/sbin/munin09:24:480
1666726614,36sleep016734-21/usr/sbin/munin11:04:340
698626518,21sleep00-21swapper/008:49:420
320726511,38sleep23214-21/usr/sbin/munin12:04:362
3019026515,37sleep030221-21grep10:04:330
226012656,47sleep00-21swapper/011:21:090
1003426523,23sleep30-21swapper/312:24:483
2450426418,30sleep224533-21basename08:04:172
235062646,16sleep30-21swapper/309:42:113
759726321,24sleep30-21swapper/307:09:473
676826319,23sleep30-21swapper/307:09:193
592826315,34sleep25977-21unin-run12:14:172
3069126314,26sleep00-21swapper/011:49:200
2910726320,28sleep029149-21latency_hist11:44:190
274242634,11sleep00-21swapper/011:38:300
2676726317,27sleep226790-21munin-run09:54:172
1181526325,22sleep30-21swapper/307:24:173
1181526325,22sleep30-21swapper/307:24:173
572826213,28sleep00-21swapper/010:29:290
2732626214,22sleep00-21swapper/009:54:400
2238626213,31sleep222388-21/usr/sbin/munin09:39:272
895726113,33sleep08963-21df_inode10:39:340
895726113,33sleep08963-21df_inode10:39:340
3243826116,26sleep032434-21apt-get11:54:290
3243826116,26sleep032434-21apt-get11:54:290
1943226120,23sleep319470-21users07:44:523
170026119,19sleep00-21swapper/008:34:200
3125526019,25sleep331251-21apt-get08:24:293
305842605,34sleep30-21swapper/308:22:233
2550726016,25sleep025502-21apt-get09:49:310
2443926015,21sleep20-21swapper/211:29:192
2404926018,19sleep00-21swapper/009:44:380
210442607,39sleep00-21swapper/011:16:250
1378726017,20sleep30-21swapper/312:39:183
3171125917,24sleep331730-21apt-get10:09:293
2209125911,32sleep322114-21ntp_states07:54:453
1899225917,27sleep319032-21/usr/sbin/munin11:10:433
1462025919,18sleep20-21swapper/209:14:362
1462025919,18sleep20-21swapper/209:14:362
1374225918,19sleep00-21swapper/009:09:490
1278525922,17sleep20-21swapper/207:24:482
125152595,11sleep00-21swapper/009:06:060
80852586,11sleep30-21swapper/312:19:403
2166425816,24sleep321671-21diskstats11:19:353
104762585,11sleep20-21swapper/212:27:262
897425714,26sleep38971-21apt-get07:14:323
503025718,22sleep35026-21apt-get08:44:323
488125719,17sleep20-21swapper/210:24:582
488125719,17sleep20-21swapper/210:24:582
2166325715,22sleep00-21swapper/007:54:340
1751125715,20sleep00-21swapper/009:24:190
1511125714,26sleep015105-21apt-get10:59:330
1124025717,24sleep011444-21date09:04:200
859625618,23sleep00-21swapper/008:54:470
859625618,23sleep00-21swapper/008:54:470
324525618,18sleep30-21swapper/308:39:193
3087925615,24sleep20-21swapper/208:24:202
2369725614,28sleep323721-21/usr/sbin/munin07:59:463
2008125619,22sleep220087-21diskstats11:14:372
196125616,24sleep21985-21df_inode08:34:342
1811325622,17sleep30-21swapper/309:24:423
1334225611,30sleep013484-21cat10:54:200
935325521,17sleep30-21swapper/310:39:453
935325521,17sleep30-21swapper/310:39:453
523925513,20sleep20-21swapper/208:44:392
3218925515,26sleep232195-21ntp_states10:09:442
2863125514,25sleep228627-21apt-get09:59:302
2839525516,19sleep30-21swapper/309:59:193
1174225512,15sleep30-21swapper/310:49:183
1174225512,15sleep30-21swapper/310:49:183
116322555,10sleep20-21swapper/210:48:332
116322555,10sleep20-21swapper/210:48:332
690825420,18sleep30-21swapper/312:14:513
3039925415,22sleep230456-21irqstats10:04:392
2863425416,21sleep00-21swapper/009:59:320
2789025413,26sleep327887-21apt-get08:14:323
1838625413,16sleep00-21swapper/007:44:210
1744425412,26sleep217468-21ntp_states07:39:422
1744425412,26sleep217468-21ntp_states07:39:422
133642549,20sleep30-21swapper/307:29:173
1306725414,24sleep313063-21apt-get09:09:323
657725313,17sleep30-21swapper/308:49:333
369325314,24sleep23750-21irqstats08:39:392
348225315,22sleep33478-21apt-get08:39:313
2127025311,17sleep30-21swapper/309:34:443
164925316,22sleep31655-21diskstats11:59:373
1619725316,22sleep316194-21apt-get09:19:303
1138425314,23sleep211462-21users07:19:522
1138425314,23sleep211462-21users07:19:522
1415725211,20sleep00-21swapper/010:54:480
1266625211,17sleep20-21swapper/210:49:502
1266625211,17sleep20-21swapper/210:49:502
622025114,21sleep26213-21apt-get12:14:342
3257425113,18sleep00-21swapper/008:29:190
2173525112,22sleep221741-21df_inode07:54:362
1465425112,26sleep2367-21snmpd07:30:582
551425018,17sleep30-21swapper/310:29:203
551425018,17sleep30-21swapper/310:29:203
2440925014,20sleep30-21swapper/309:44:453
231082509,16sleep00-21swapper/011:24:290
2011325014,21sleep020180-21df07:49:330
43924910,15sleep20-21swapper/208:29:372
318824912,14sleep00-21swapper/012:04:340
77572488,16sleep20-21swapper/210:34:442
2505624812,21sleep025097-21/usr/sbin/munin11:29:410
1950224811,22sleep219558-21/usr/sbin/munin09:29:382
1220424811,20sleep012253-21irqstats10:49:400
1101624712,21sleep211104-21gpgv09:04:182
1437724612,17sleep20-21swapper/209:14:202
263892457,14sleep20-21swapper/208:09:342
1609024412,18sleep30-21swapper/307:35:033
15672438,17sleep20-21swapper/211:59:312
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional