You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 08:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Aug 03, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
187872796125,7925sleep317219-21kworker/3:021:00:263
62072279112,2766sleep018363-1kworker/0:1H22:02:500
138882169319,1659sleep326563-1kworker/3:0H00:07:583
126432137511,1352sleep018363-1kworker/0:1H22:22:450
7229296717,933sleep07231-21ntp_states20:23:000
7229296717,933sleep07231-21ntp_states20:23:000
7567269410,672sleep37613-21ntpq23:48:033
16155262821,529sleep316294-21grep00:13:073
19457255928,514sleep219514-21/usr/sbin/munin21:02:562
9762245210,426sleep19768-21ntpq22:13:001
13377228614,258sleep113412-21ntpq20:43:001
13088226822,187sleep313102-21fschecks_count20:42:543
2312022380,7sleep20-21swapper/221:14:212
1869322247,203sleep318765-21sh19:17:343
1311222118,190sleep21422-21sed23:28:042
1311222118,190sleep21422-21sed23:28:042
14306221822,177sleep314337-21fschecks_count22:27:523
19515221112,149sleep319551-21latency21:02:573
22650220933,138sleep322797-21grep00:33:103
22650220933,138sleep322797-21grep00:33:103
21765220944,114sleep30-21swapper/322:52:353
27098220454,137sleep327221-21latency_hist21:27:363
18544220321,60sleep30-21swapper/300:22:443
13384220313,65sleep313522-21unixbench_multi22:23:283
8118220151,84sleep30-21swapper/320:27:363
8118220151,84sleep30-21swapper/320:27:363
31110219835,144sleep331144-21ps21:38:063
32112219729,43sleep30-21swapper/319:58:083
24846219559,66sleep324850-21modprobe23:02:333
881521918,46sleep38888-21tr20:28:043
11096216962,40sleep30-21swapper/322:17:553
17855216240,103sleep317933-21mailstats22:38:023
17855216240,103sleep317933-21mailstats22:38:023
12709216210,138sleep312739-21awk20:42:343
1185215927,45sleep30-21swapper/320:03:053
2896721578,67sleep328449-21/usr/sbin/munin23:13:033
7462215412,129sleep37496-21idleruntime-cro22:07:333
23879215330,64sleep323936-21unin-run21:17:343
699215231,58sleep30-21swapper/323:27:483
699215231,58sleep30-21swapper/323:27:483
5015215072,34sleep34384-21/usr/sbin/munin21:58:043
1948121508,82sleep319563-21switchtime22:43:023
4657214711,122sleep00-21swapper/020:14:240
27819214554,24sleep30-21swapper/321:28:043
1744521458,78sleep317552-21latency_hist20:57:363
8276214112,73sleep38352-21cut22:08:023
11212214026,56sleep311247-21cat23:58:053
26425213917,96sleep026578-21grep21:23:090
14308213917,109sleep014346-21cat20:47:350
25454213725,74sleep30-21swapper/323:02:573
2531521379,115sleep325389-21fschecks_time19:37:533
603721350,7sleep10-21swapper/123:43:041
21252213518,10sleep30-21swapper/300:30:563
12633213513,106sleep212157-21apt-get22:22:382
891721306,59sleep30-21swapper/323:52:443
31369213072,19sleep30-21swapper/323:22:353
31369213072,19sleep30-21swapper/323:22:353
7837212815,95sleep17857-21grep22:07:521
27381212310,75sleep30-21swapper/323:08:023
27381212310,75sleep30-21swapper/323:08:023
6737212254,50sleep36885-21sed23:47:353
15867211926,78sleep315935-21grep22:32:533
981221188,74sleep39903-21mailstats22:13:033
13867211712,89sleep213362-21apt-get00:07:452
13469211725,73sleep113614-21cat00:07:351
3317211410,39sleep30-21swapper/321:53:043
2727211415,85sleep22791-21find23:33:042
2727211415,85sleep22791-21find23:33:042
2096211418,22sleep10-21swapper/120:07:471
8791211313,83sleep28814-21/usr/sbin/munin20:28:032
7128211318,75sleep06644-21apt-get23:47:440
25070211216,76sleep224152-21/usr/sbin/munin21:18:062
2239211249,49sleep32241-21apt-get23:32:453
2239211249,49sleep32241-21apt-get23:32:453
10209211219,74sleep010215-21modprobe23:57:330
995321118,37sleep30-21swapper/320:32:523
209821079,37sleep30-21swapper/320:07:493
3640210533,15sleep30-21swapper/320:12:463
30080210548,44sleep330115-21munin-run21:37:333
17937210513,72sleep018031-21/usr/sbin/munin22:38:030
17937210513,72sleep018031-21/usr/sbin/munin22:38:030
25498210220,63sleep225546-21latency19:37:562
22411210211,74sleep022460-21/usr/sbin/munin22:53:010
22411210211,74sleep022460-21/usr/sbin/munin22:53:010
27363210119,67sleep127429-21fschecks_time21:27:541
2092929918,20sleep30-21swapper/319:22:593
511929821,48sleep05266-21grep21:58:050
2218629713,67sleep00-21swapper/019:27:530
2678029612,55sleep315125-21cyclictest19:42:503
512729535,15sleep30-21swapper/323:42:343
2187029524,21sleep00-21swapper/019:27:370
1702129322,55sleep30-21swapper/300:17:473
1702129322,55sleep30-21swapper/300:17:473
2335929228,49sleep323504-21latency_hist22:57:363
2335929228,49sleep323504-21latency_hist22:57:363
1024629213,61sleep20-21swapper/220:33:002
2246029113,58sleep121885-21/usr/sbin/munin00:33:081
2246029113,58sleep121885-21/usr/sbin/munin00:33:081
2116929111,59sleep30-21swapper/321:07:573
2116929111,59sleep30-21swapper/321:07:573
1580929116,54sleep015411-21/usr/sbin/munin00:13:000
2317429011,61sleep223243-21/usr/sbin/munin00:37:332
2317429011,61sleep223243-21/usr/sbin/munin00:37:332
2261528921,26sleep10-21swapper/122:53:061
2261528921,26sleep10-21swapper/122:53:061
841328825,42sleep18446-21grep20:27:541
58292888,66sleep35906-21grep20:18:033
540028814,60sleep15403-21sshd21:59:031
354828724,47sleep23693-21sed21:53:072
344328719,53sleep13543-21timerandwakeup21:53:061
3003128718,46sleep130050-21apt-get23:17:451
3003128718,46sleep130050-21apt-get23:17:451
254228749,21sleep10-21swapper/121:52:331
2031028630,17sleep320319-21munin-run19:22:353
1056928628,33sleep20-21swapper/222:17:342
3168828521,49sleep30-21swapper/323:22:523
3168828521,49sleep30-21swapper/323:22:523
3066028554,16sleep30-21swapper/319:53:083
2987528521,46sleep229869-21apt-get19:52:502
2433428521,43sleep224474-21sed19:33:042
1574328524,17sleep30-21swapper/319:08:423
237528419,10sleep30-21swapper/321:50:253
37728313,53sleep1500-21timerwakeupswit21:43:051
2486328319,11sleep30-21swapper/319:36:563
2327728316,43sleep00-21swapper/000:37:350
2327728316,43sleep00-21swapper/000:37:350
775128213,53sleep00-21swapper/022:07:470
2492428221,44sleep125036-21apt-key23:02:351
1495728225,27sleep10-21swapper/119:07:351
1475728218,11sleep30-21swapper/320:47:543
84428119,11sleep30-21swapper/321:45:373
44728118,18sleep10-21swapper/123:27:341
44728118,18sleep10-21swapper/123:27:341
2543028117,11sleep30-21swapper/321:22:313
230052817,60sleep323018-21awk19:28:083
1828128121,41sleep018297-21ls00:22:330
1828128121,41sleep018297-21ls00:22:330
2715528012,53sleep027196-21/usr/sbin/munin19:42:590
239202800,7sleep10-21swapper/119:32:541
2387728018,46sleep023949-21/usr/sbin/munin22:57:570
2171328021,36sleep121730-21munin-plugin-st22:52:331
1316928019,10sleep30-21swapper/300:05:153
1097928014,50sleep210989-21ntpq23:58:002
297627920,38sleep13117-21unixbench_singl23:33:071
297627920,38sleep13117-21unixbench_singl23:33:071
296252797,14sleep30-21swapper/319:52:353
2559927914,44sleep025635-21ntpq19:37:580
1605027924,39sleep216088-21wc22:32:562
1577927910,48sleep10-21swapper/122:32:471
3060627818,39sleep130101-21apt-get21:37:431
2318727812,50sleep123249-21/usr/sbin/munin21:14:261
142512780,6sleep30-21swapper/319:03:043
34872776,10sleep00-21swapper/023:37:180
34872776,10sleep00-21swapper/023:37:180
2948127718,38sleep10-21swapper/121:33:071
1969227723,27sleep219840-21unixbench_multi19:18:062
1624927711,51sleep016313-21ps22:33:020
2158327623,27sleep10-21swapper/100:32:351
2056027610,41sleep20-21swapper/219:22:502
3189927520,40sleep231917-21sh21:42:332
106927526,26sleep00-21swapper/023:27:570
106927526,26sleep00-21swapper/023:27:570
949427425,24sleep10-21swapper/123:53:391
500927416,35sleep15155-21latency_hist20:17:361
2580327415,43sleep225810-21df_inode21:22:552
1679127421,36sleep216804-21munin-node19:12:362
1678227418,27sleep20-21swapper/220:53:062
1523427419,23sleep10-21swapper/120:48:051
1803227314,37sleep10-21swapper/122:38:041
1803227314,37sleep10-21swapper/122:38:041
1338627317,25sleep00-21swapper/000:07:340
104627317,42sleep01183-21latency_hist21:47:360
103527319,27sleep10-21swapper/120:03:031
2333427222,30sleep20-21swapper/222:57:352
2333427222,30sleep20-21swapper/222:57:352
2273727224,28sleep022746-21diskstats21:13:440
1650727217,22sleep00-21swapper/020:53:000
364427114,44sleep23704-21cpuspeed20:12:492
292592710,7sleep20-21swapper/223:13:052
2386327110,15sleep00-21swapper/019:32:540
189227113,24sleep00-21swapper/021:48:040
17592717,12sleep20-21swapper/221:48:022
1759027114,43sleep017648-21sort19:12:550
1470827116,40sleep114788-21grep22:28:031
677827012,44sleep16779-21http20:22:491
677827012,44sleep16779-21http20:22:491
574027019,25sleep20-21swapper/223:42:582
3190027013,41sleep231909-21ntpq19:58:032
3185627012,43sleep131899-21cat19:58:021
2829227020,32sleep128199-21latency_hist23:12:361
2110527013,29sleep10-21swapper/119:23:051
2076827023,21sleep20-21swapper/200:28:082
154362709,50sleep00-21swapper/019:07:420
69232696,13sleep20-21swapper/220:22:542
69232696,13sleep20-21swapper/220:22:542
3037126912,39sleep030449-21/usr/sbin/munin19:53:050
2788426916,33sleep027991-21timerandwakeup21:28:050
202192690,7sleep00-21swapper/022:47:350
617326822,27sleep20-21swapper/222:02:442
3169426716,22sleep00-21swapper/023:22:530
3169426716,22sleep00-21swapper/023:22:530
2741326719,30sleep20-21swapper/219:43:052
1933126719,25sleep00-21swapper/000:23:050
1900226718,30sleep119108-21tune2fs22:42:521
1490426722,23sleep00-21swapper/019:07:340
763126619,30sleep27669-21/usr/sbin/munin23:48:062
2899926615,22sleep10-21swapper/119:48:051
248926615,24sleep00-21swapper/020:07:580
379626515,22sleep10-21swapper/123:37:441
2227126520,21sleep20-21swapper/221:12:372
2227126520,21sleep20-21swapper/221:12:372
2210926513,23sleep20-21swapper/219:27:512
1941426516,31sleep019461-21ntpq19:18:000
1850226512,39sleep20-21swapper/219:15:072
111026514,34sleep1364-21snmpd23:27:581
111026514,34sleep1364-21snmpd23:27:581
531326413,35sleep05345-21tune2fs20:17:520
2274226419,23sleep10-21swapper/119:28:061
1912426416,22sleep20-21swapper/200:23:032
1736826418,29sleep117428-21sendmail-msp20:57:341
54626321,22sleep00-21swapper/020:02:500
311062636,44sleep00-21swapper/023:19:500
311062636,44sleep00-21swapper/023:19:500
2811226322,19sleep10-21swapper/119:47:361
1670326315,29sleep116730-21munin-run00:17:331
1570826323,19sleep20-21swapper/200:12:582
295852628,14sleep20-21swapper/219:52:342
2613526219,22sleep20-21swapper/223:03:122
218226211,34sleep22073-21/usr/sbin/munin20:07:522
2096126210,40sleep221041-21tr22:48:022
1729026225,22sleep20-21swapper/200:17:542
1729026225,22sleep20-21swapper/200:17:542
1546226219,19sleep10-21swapper/119:07:471
1521626210,36sleep00-21swapper/000:12:350
1505626215,19sleep20-21swapper/219:07:362
516626117,25sleep00-21swapper/023:42:350
460326112,32sleep24722-21/usr/sbin/munin23:38:052
2698626117,29sleep11-21systemd21:27:341
2534426113,33sleep124863-21apt-get23:02:431
2503726114,22sleep20-21swapper/223:02:362
2218326115,28sleep122269-21munin-run21:12:361
2218326115,28sleep122269-21munin-run21:12:361
217922617,11sleep00-21swapper/021:09:040
217922617,11sleep00-21swapper/021:09:040
1890226120,24sleep218899-21apt-get22:42:452
186026120,20sleep20-21swapper/220:07:362
1805426117,23sleep10-21swapper/120:57:581
1784126114,30sleep117878-21ntp_states19:13:001
1450026112,35sleep114574-21grep00:08:161
3032226018,25sleep130329-21ntp_states19:53:021
1856826016,26sleep118565-21apt-get00:22:461
1789426022,17sleep20-21swapper/220:57:552
1182226024,21sleep00-21swapper/020:37:580
81202596,11sleep20-21swapper/222:07:572
63482596,38sleep00-21swapper/023:43:570
3085925912,23sleep00-21swapper/021:37:590
2050325916,25sleep020526-21munin-run21:07:340
2042925922,20sleep00-21swapper/022:47:440
569425813,33sleep15738-21processes20:18:011
290925814,29sleep02918-21/usr/sbin/munin21:52:530
1220225813,27sleep212285-21apt-config22:22:342
728425715,19sleep10-21swapper/123:47:571
3015425717,19sleep20-21swapper/223:17:542
3015425717,19sleep20-21swapper/223:17:542
1209025717,23sleep212087-21apt-get00:02:482
1125125715,24sleep011395-21unixbench_singl23:58:060
1020725714,28sleep110245-21memory20:32:591
648825615,27sleep26518-21munin-run20:22:342
2698925613,25sleep026997-21df23:07:510
2698925613,25sleep026997-21df23:07:510
1928125612,27sleep119312-21/usr/sbin/munin19:17:561
1422325614,25sleep014291-21df22:27:490
296662556,37sleep00-21swapper/023:16:350
2565225517,22sleep025655-21irqstats23:03:020
1513025515,17sleep00-21swapper/020:48:040
1014825517,20sleep00-21swapper/020:32:580
665725415,23sleep16668-21ntp_states22:03:071
2858325418,18sleep20-21swapper/221:32:362
2692325412,25sleep226917-21apt-get23:07:492
2692325412,25sleep226917-21apt-get23:07:492
1816025415,20sleep218256-21users22:38:052
1816025415,20sleep218256-21users22:38:052
1300225414,24sleep212998-21apt-get20:42:492
126452545,11sleep10-21swapper/122:22:461
1184925415,17sleep10-21swapper/100:02:351
1097725416,21sleep10-21swapper/122:17:531
930025312,24sleep09294-21apt-get22:12:470
3014325312,25sleep030231-21apt-key21:37:340
2491725312,18sleep10-21swapper/121:18:051
2463025315,23sleep024636-21ntp_states21:18:000
1814325315,20sleep00-21swapper/020:58:010
1491925311,28sleep214527-21/usr/sbin/munin20:47:582
1145225312,25sleep211445-21apt-get20:37:482
586225212,20sleep20-21swapper/222:02:362
2102425215,21sleep00-21swapper/000:28:100
1940525212,20sleep10-21swapper/121:02:551
1709425212,16sleep10-21swapper/100:17:491
1709425212,16sleep10-21swapper/100:17:491
1115625213,24sleep011194-21latency22:17:560
1558325118,16sleep20-21swapper/222:32:362
3032425015,18sleep20-21swapper/221:37:362
47624917,15sleep20-21swapper/223:27:352
47624917,15sleep20-21swapper/223:27:352
292932497,29sleep00-21swapper/019:48:490
1186824911,22sleep111905-21ntp_states20:37:591
3216424812,18sleep20-21swapper/221:42:452
2716024813,20sleep10-21swapper/123:07:561
2716024813,20sleep10-21swapper/123:07:561
2520124711,17sleep10-21swapper/119:37:431
944124612,20sleep29452-21ntp_states23:53:362
319202447,24sleep00-21swapper/019:58:050
83262418,15sleep00-21swapper/020:27:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional