You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-04 - 11:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Jan 04, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
244872118612,1158sleep010449-1kworker/0:2H23:58:590
13122210044,974sleep313047-1kworker/3:1H23:24:333
15744233310,305sleep315805-21latency_hist23:33:463
14253229794,143sleep30-21swapper/321:44:073
16952229011,265sleep216960-21ntpq20:09:132
7956227372,177sleep38018-21fschecks_time21:24:043
69712262123,23sleep30-21swapper/319:39:053
302022158,195sleep2197-21jbd2/mmcblk2p2-22:53:312
23973221543,122sleep30-21swapper/322:14:153
22303221513,178sleep222379-21mailstats22:09:132
9743220411,140sleep39795-21if_eth023:14:073
20361220121,125sleep320427-21fschecks_time22:04:063
20361220121,125sleep320427-21fschecks_time22:04:063
13830220050,132sleep313943-21latency_hist21:43:473
20033219558,74sleep30-21swapper/322:03:453
20033219558,74sleep30-21swapper/322:03:453
28303219015,111sleep30-21swapper/300:09:153
7181218712,57sleep07237-21irqstats19:39:100
1782321867,164sleep317911-21cat23:39:073
4225218136,129sleep34378-21latency_hist00:38:463
3036217516,82sleep30-21swapper/321:09:023
25765217113,47sleep30-21swapper/322:19:323
22410216943,109sleep3228-21systemd-journal20:28:443
2355021667,49sleep30-21swapper/320:29:163
10714216320,29sleep30-21swapper/321:33:453
22382216150,15sleep30-21swapper/322:09:153
17614216029,119sleep317624-21open_files21:54:123
17614216029,119sleep317624-21open_files21:54:123
26650215726,93sleep326759-21timerandwakeup00:04:143
26650215726,93sleep326759-21timerandwakeup00:04:143
1004521578,50sleep30-21swapper/319:48:583
1004521578,50sleep30-21swapper/319:48:583
32737215637,104sleep3381-21sed19:18:463
12697215612,132sleep212760-21cat21:39:052
17760215526,50sleep217864-21timerandwakeup21:54:152
17760215526,50sleep217864-21timerandwakeup21:54:152
15532215510,133sleep315621-21switchtime20:04:213
24001215215,76sleep324084-21apt-key23:58:453
20847215217,107sleep120363-21apt-get23:48:541
6716214826,71sleep30-21swapper/323:04:063
6138214811,115sleep30-21swapper/319:34:213
4658214818,109sleep04743-21cat22:58:460
4658214818,109sleep04743-21cat22:58:460
31416214846,54sleep30-21swapper/322:39:073
27853214713,121sleep227921-21ls00:09:062
10663214620,23sleep10-21swapper/121:33:431
32693214533,56sleep30-21swapper/322:43:593
30914214515,116sleep330959-21latency_hist22:38:463
15883214524,17sleep30-21swapper/321:49:083
26090214412,119sleep326132-21sort20:39:103
12841214460,30sleep312905-21http19:58:443
10305214310,115sleep210434-21awk23:14:192
2743421426,127sleep00-21swapper/022:24:460
435521377,95sleep3369-21snmpd22:55:503
435521377,95sleep3369-21snmpd22:55:503
3647213521,87sleep13811-21unixbench_multi21:09:161
25020213518,90sleep30-21swapper/320:34:213
7229213311,104sleep30-21swapper/321:20:123
6211213167,43sleep30-21swapper/323:03:463
17796212913,56sleep30-21swapper/320:13:463
27805212617,72sleep30-21swapper/320:44:103
19737212545,26sleep30-21swapper/323:44:163
19093212434,51sleep10-21swapper/123:43:571
28892212328,73sleep328957-21unin-run19:08:453
19097212314,89sleep019142-21/usr/sbin/munin23:44:010
24087212217,85sleep024250-21latency_hist23:58:460
4284211945,58sleep30-21swapper/319:29:163
2916321198,99sleep3369-21snmpd22:32:043
14204211945,59sleep314311-21cat20:00:583
29762211713,80sleep229553-21/usr/sbin/munin22:34:052
2153211616,79sleep02194-21fschecks_count19:24:040
12888211613,20sleep10-21swapper/121:39:081
694321158,95sleep26962-21apt-get19:39:002
26580211554,23sleep326675-21fschecks_time22:24:043
29675211421,70sleep129553-21/usr/sbin/munin22:34:041
21230211321,41sleep30-21swapper/323:50:093
1723021125,96sleep00-21swapper/023:38:390
22260211118,68sleep022263-21apt-key23:53:460
14318211118,77sleep00-21swapper/020:00:590
8078211046,19sleep30-21swapper/323:09:073
6681211016,69sleep26298-21/usr/sbin/munin21:20:032
11849210912,82sleep211861-21/usr/sbin/munin19:54:062
3809210712,80sleep03333-21apt-get19:28:480
31483210612,77sleep225770-21kworker/u8:219:14:052
28428210628,60sleep128462-21/usr/sbin/munin22:29:111
30863210521,67sleep231019-21latency_hist19:13:482
29578210529,67sleep010449-1kworker/0:2H22:33:590
862521039,20sleep30-21swapper/319:44:063
12284210145,21sleep312286-21munin-run21:38:463
1057210014,22sleep30-21swapper/319:19:143
3228729913,71sleep132337-21cat20:59:061
3051329928,19sleep330543-21munin-run19:13:453
2472029914,69sleep224868-21latency_hist22:18:462
3207929817,62sleep132157-21find19:15:211
1185329828,56sleep312001-21unixbench_singl23:19:163
307562968,45sleep30-21swapper/320:54:083
2445729616,59sleep123937-21apt-get23:58:561
1985529620,16sleep30-21swapper/320:19:073
1862629612,21sleep30-21swapper/320:14:143
1801429615,65sleep017526-21/usr/sbin/munin23:39:120
2733529416,63sleep127342-21systemctl00:08:441
2733529416,63sleep127342-21systemctl00:08:441
3006629229,51sleep30-21swapper/320:52:263
2950029217,21sleep30-21swapper/300:14:073
11032927,44sleep30-21swapper/300:28:453
3234529117,51sleep132362-21fschecks_count00:24:021
2771929029,27sleep20-21swapper/219:04:042
182629012,62sleep21891-21latency_hist19:23:462
301628943,35sleep30-21swapper/322:53:283
2391428925,46sleep223981-40mandb23:58:442
155428819,53sleep30-21swapper/321:04:583
172872870,8sleep00-21swapper/021:54:040
172872870,8sleep00-21swapper/021:54:040
799628616,55sleep08004-21cpu23:09:040
51928630,29sleep00-21swapper/019:18:590
3014228519,26sleep20-21swapper/220:53:262
254972845,12sleep30-21swapper/300:00:273
929228323,40sleep29344-21latency_hist23:13:462
398228321,36sleep14103-21ls22:54:151
398228321,36sleep14103-21ls22:54:151
174328218,23sleep00-21swapper/000:29:100
1606728211,51sleep30-21swapper/323:34:023
96128122,27sleep00-21swapper/021:03:460
813928116,49sleep28205-21wc21:24:072
261352810,5sleep00-21swapper/020:39:100
45742800,8sleep20-21swapper/221:14:002
214728030,33sleep30-21swapper/319:24:033
1070628015,15sleep10-21swapper/119:49:171
1070628015,15sleep10-21swapper/119:49:171
981227921,43sleep09857-21grep21:29:100
2803327923,26sleep00-21swapper/022:29:020
3060327815,37sleep030624-21munin-node19:13:460
2970727812,51sleep129783-21wc00:14:131
2733427818,41sleep00-21swapper/020:43:550
308042778,46sleep30-21swapper/300:19:043
557327629,12sleep30-21swapper/321:16:133
332527618,17sleep20-21swapper/222:53:592
332527618,17sleep20-21swapper/222:53:592
319462769,55sleep0369-21snmpd00:23:250
1853927621,27sleep20-21swapper/220:14:132
2707027524,35sleep127084-21apt20:43:441
86127429,12sleep30-21swapper/321:03:293
2190627417,42sleep121994-21tune2fs22:09:041
167127410,20sleep10-21swapper/121:05:331
13542748,14sleep30-21swapper/300:29:013
356927320,22sleep00-21swapper/021:09:150
2748227319,24sleep10-21swapper/120:44:041
119922738,55sleep00-21swapper/021:35:560
684527211,15sleep10-21swapper/121:20:061
594227221,33sleep10-21swapper/119:34:191
2091827225,21sleep20-21swapper/220:23:462
18062729,15sleep20-21swapper/200:29:122
138127215,25sleep00-21swapper/021:03:540
1200727219,24sleep00-21swapper/023:19:170
2854527122,25sleep00-21swapper/019:04:180
1927027119,27sleep10-21swapper/121:59:151
1927027119,27sleep10-21swapper/121:59:151
1849627112,45sleep118536-21ps20:14:121
1314927119,34sleep20-21swapper/219:58:472
1198727120,22sleep10-21swapper/119:54:091
228727022,33sleep12363-21awk22:49:141
1495927011,30sleep10-21swapper/120:04:071
871326916,23sleep00-21swapper/019:44:070
664326914,38sleep26706-21modprobe19:38:442
396926915,28sleep10-21swapper/119:29:101
336726922,22sleep10-21swapper/119:28:451
1934626916,21sleep20-21swapper/221:59:162
1934626916,21sleep20-21swapper/221:59:162
1452526915,39sleep214612-21fschecks_time23:29:042
1414926923,26sleep10-21swapper/123:28:431
3089826813,38sleep230935-21ntpq20:54:112
2812026816,30sleep10-21swapper/100:09:131
2452326815,39sleep224615-21load20:34:122
2410226819,32sleep024250-21unixbench_singl22:14:160
2330726817,31sleep123393-21ls20:29:141
2155026827,23sleep20-21swapper/223:50:162
1315826822,25sleep00-21swapper/021:39:150
587926713,22sleep00-21swapper/019:34:180
326226711,24sleep20-21swapper/200:34:092
298026715,34sleep13046-21fschecks_time00:34:041
1423426718,27sleep00-21swapper/023:28:460
116226711,43sleep11164-21sshd22:44:261
3182526621,21sleep00-21swapper/022:39:160
459326522,23sleep00-21swapper/021:14:040
583426418,30sleep15865-21munin-run21:18:451
557326411,20sleep20-21swapper/222:59:162
3272126413,35sleep232736-21/usr/sbin/munin00:24:122
3245526420,30sleep232417-21latency_hist22:43:462
31572646,16sleep20-21swapper/219:26:362
2760926410,11sleep20-21swapper/222:27:022
2503526415,34sleep225044-21awk00:00:142
123326419,28sleep11372-21/usr/sbin/munin19:19:161
71226322,21sleep00-21swapper/022:44:120
426826313,22sleep20-21swapper/221:13:452
307972639,13sleep20-21swapper/200:19:032
2931826319,25sleep00-21swapper/000:14:040
2929126316,30sleep21-21systemd22:33:442
2157926316,23sleep20-21swapper/222:08:452
2157926316,23sleep20-21swapper/222:08:452
192072636,13sleep00-21swapper/021:59:140
192072636,13sleep00-21swapper/021:59:140
1730326315,27sleep10-21swapper/123:38:451
1537326315,32sleep215489-21latency_hist21:48:462
510826213,30sleep05165-21/usr/sbin/munin22:59:060
2611326211,30sleep20-21swapper/200:04:012
2611326211,30sleep20-21swapper/200:04:012
1374726216,25sleep10-21swapper/120:00:481
1138226223,19sleep20-21swapper/223:19:072
858326112,32sleep18480-21/usr/sbin/munin19:44:041
83512616,12sleep00-21swapper/021:24:130
2075926120,24sleep00-21swapper/022:04:150
2075926120,24sleep00-21swapper/022:04:150
1689726113,25sleep00-21swapper/020:09:100
1470826123,17sleep10-21swapper/123:29:081
1013226113,22sleep00-21swapper/019:49:040
1013226113,22sleep00-21swapper/019:49:040
3250726017,27sleep232531-21ntp_states20:59:122
2246326016,23sleep10-21swapper/120:28:461
1871326011,20sleep00-21swapper/020:14:150
156252600,8sleep00-21swapper/020:04:210
644825915,20sleep00-21swapper/023:03:590
548025915,26sleep15572-21/usr/sbin/munin22:59:151
3069825914,26sleep130484-21apt-get19:13:471
268722595,45sleep20-21swapper/220:41:102
2546025914,27sleep025539-21/usr/sbin/munin22:19:290
2504725913,29sleep20-21swapper/222:19:192
2136625916,28sleep021422-21/usr/sbin/munin20:24:040
1773625915,25sleep10-21swapper/123:39:051
3114125815,20sleep00-21swapper/020:54:170
3087525817,21sleep10-21swapper/120:54:101
251962587,35sleep10-21swapper/120:34:481
1736425818,25sleep10-21swapper/121:54:061
1736425818,25sleep10-21swapper/121:54:061
1568125815,24sleep10-21swapper/121:49:011
1220825814,21sleep20-21swapper/221:38:452
918025714,20sleep20-21swapper/219:44:172
7842578,36sleep00-21swapper/000:25:110
665925713,27sleep26715-21irqstats23:04:052
2115625713,26sleep221150-21apt-get20:23:582
27322566,39sleep00-21swapper/022:49:450
2624125616,21sleep10-21swapper/122:23:461
2425525616,21sleep10-21swapper/122:14:171
159532568,38sleep20-21swapper/223:33:532
150682566,40sleep10-21swapper/121:45:261
1451325611,25sleep00-21swapper/023:29:020
107922567,38sleep00-21swapper/023:18:410
999025515,24sleep19996-21ntp_states23:14:141
949725516,26sleep20-21swapper/221:29:042
917425514,25sleep19307-21sed21:28:461
384725514,24sleep23840-21apt-get19:28:592
316325511,18sleep00-21swapper/000:34:070
290012558,34sleep127124-21ntpd20:49:051
2890725515,24sleep028902-21apt-get20:49:000
2790025512,23sleep10-21swapper/119:04:071
2758025512,24sleep027576-21apt-get00:08:560
1796925514,25sleep218005-21ntp_states23:39:102
1024925517,16sleep20-21swapper/219:49:062
1024925517,16sleep20-21swapper/219:49:062
837225412,19sleep10-21swapper/123:09:131
324142546,38sleep00-21swapper/019:15:400
2865825416,17sleep20-21swapper/220:48:462
1649325413,24sleep116487-21apt-get20:08:591
141302545,41sleep20-21swapper/223:28:282
692825318,19sleep10-21swapper/123:04:121
3269225312,24sleep232686-21apt-get22:43:582
2612925315,20sleep00-21swapper/000:04:030
2612925315,20sleep00-21swapper/000:04:030
2274125315,23sleep122745-21df_inode23:54:031
1641325311,17sleep00-21swapper/021:49:170
705025212,25sleep17054-21diskstats19:39:071
266625215,23sleep21-21systemd21:08:452
23472526,34sleep10-21swapper/100:30:241
2181925212,24sleep021813-21apt-get22:08:590
212362526,34sleep10-21swapper/122:04:561
212362526,34sleep10-21swapper/122:04:561
1620325213,16sleep20-21swapper/220:08:442
625225110,23sleep05789-21apt-get21:18:480
317902517,33sleep10-21swapper/100:21:241
3126925112,21sleep131359-21perl22:39:041
1917125116,18sleep20-21swapper/223:44:032
2705825014,22sleep127157-21cat22:24:151
2424925012,23sleep024245-21apt-get20:34:020
2323125011,22sleep023306-21sensors20:29:130
196152509,15sleep10-21swapper/120:19:011
1239825012,21sleep012489-21users19:54:160
227032499,17sleep20-21swapper/220:28:592
1558324914,19sleep20-21swapper/221:48:542
1180524910,19sleep10-21swapper/121:34:201
113322499,15sleep10-21swapper/123:19:061
2086724510,19sleep020899-21munin-run20:23:440
144812427,19sleep014523-21ntp_states21:44:120
291342397,16sleep20-21swapper/200:13:582
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional