You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 22:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Sun May 10, 2026 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4287251658,5097sleep029981-21kworker/u8:010:14:560
248142102512,1000sleep317706-1kworker/3:2H11:19:553
2181229837,964sleep021848-21ntpq11:10:030
224429808,957sleep1194-21jbd2/mmcblk2p2-10:09:401
19733283519,798sleep119743-21/usr/sbin/munin07:39:581
19733283519,798sleep119743-21/usr/sbin/munin07:39:581
17448239439,325sleep00-21swapper/010:55:140
10302318129,170sleep01048-21fschecks_count10:04:590
2029222878,267sleep3363-21snmpd09:23:113
2029222878,267sleep3363-21snmpd09:23:113
2323022596,241sleep00-21swapper/009:31:080
2323022596,241sleep00-21swapper/009:31:080
894822516,230sleep00-21swapper/008:46:060
11453224777,157sleep011500-21ntp_kernel_pll_08:55:060
16303223238,170sleep016453-21latency_hist07:29:430
8738223117,197sleep0363-21snmpd12:10:070
26834223155,45sleep026870-21ps11:25:090
14424222934,88sleep00-21swapper/009:04:510
12203222172,22sleep00-21swapper/010:40:000
12203222172,22sleep00-21swapper/010:40:000
2182622180,8sleep10-21swapper/107:45:101
13236221414,186sleep113314-21ls09:00:121
430722089,184sleep24273-21apt-get10:15:112
117942197101,36sleep011819-21apt10:39:390
4356219311,165sleep24388-21cat08:34:412
2589219320,94sleep02682-21fschecks_time10:10:000
17595219110,17sleep10-21swapper/110:55:151
9555218616,154sleep09631-21switchtime10:30:100
13924218013,151sleep013950-21awk09:04:390
2628621798,156sleep026400-21wc09:40:160
7263217118,26sleep00-21swapper/012:05:100
7263217118,26sleep00-21swapper/012:05:100
5236216915,18sleep00-21swapper/008:35:120
703921588,14sleep00-21swapper/008:40:130
521421549,133sleep05279-21sort11:59:570
521421549,133sleep05279-21sort11:59:570
2732321498,130sleep1363-21snmpd11:25:431
26558214915,114sleep00-21swapper/008:00:160
15483214818,109sleep00-21swapper/010:50:010
1261214710,122sleep01374-21latency_hist08:24:420
11638214520,102sleep211653-21fschecks_count12:19:582
2508121439,117sleep00-21swapper/011:20:570
2469321418,118sleep224732-21ntpq07:55:052
2469321418,118sleep224732-21ntpq07:55:052
24262214116,66sleep024326-21http11:19:400
24262214116,66sleep024326-21http11:19:400
16719214120,103sleep116777-21if_eth007:30:001
15554213913,70sleep00-21swapper/012:30:130
9958213519,95sleep29974-21fschecks_count07:09:592
3022213338,27sleep00-21swapper/008:29:510
13694213313,62sleep013767-21sed07:20:020
1006421339,52sleep00-21swapper/012:14:590
1006421339,52sleep00-21swapper/012:14:590
3253213125,76sleep20-21swapper/211:54:422
3253213125,76sleep20-21swapper/211:54:422
19867213012,105sleep019877-21find11:04:500
28366212912,102sleep128378-21ntpq11:30:071
2832021298,109sleep028364-21ntpq11:30:060
25582212615,93sleep325075-21apt-get09:39:513
23797212311,48sleep00-21swapper/009:34:550
1310221238,50sleep00-21swapper/012:24:520
26460212117,82sleep326519-21fschecks_time11:25:003
30843212022,78sleep130934-21switchtime09:55:111
30843212022,78sleep130934-21switchtime09:55:111
702221188,99sleep0363-21snmpd10:23:120
15399211817,83sleep315550-21unixbench_singl12:30:123
2829121177,47sleep00-21swapper/008:05:120
1807421179,46sleep00-21swapper/007:34:560
1008021178,54sleep010129-21cat07:10:010
4313211635,50sleep14273-21apt-get10:15:141
8901211514,86sleep29043-21timerwakeupswit07:05:112
1496321159,44sleep00-21swapper/010:49:400
1496321159,44sleep00-21swapper/010:49:400
2974521148,52sleep022643-21kworker/0:209:51:430
2974521148,52sleep022643-21kworker/0:209:51:430
2275321147,45sleep00-21swapper/007:49:580
2161521147,45sleep00-21swapper/007:45:060
6182211316,53sleep30-21swapper/308:39:553
21196211315,81sleep221211-21latency_hist11:09:412
2193821107,51sleep021022-21kworker/0:109:29:260
2193821107,51sleep021022-21kworker/0:109:29:260
19227211014,33sleep119234-21diskstats09:19:581
19227211014,33sleep119234-21diskstats09:19:581
442021096,52sleep00-21swapper/011:55:290
442021096,52sleep00-21swapper/011:55:290
2974021097,44sleep00-21swapper/011:35:020
1732521097,42sleep00-21swapper/007:30:120
16180210744,48sleep00-21swapper/012:34:500
1121210712,81sleep01248-21timerwakeupswit11:46:130
48121066,48sleep022643-21kworker/0:210:02:290
48121066,48sleep022643-21kworker/0:210:02:290
14926210625,61sleep114991-21latency12:30:021
6852210514,73sleep26886-21/usr/sbin/munin12:05:002
6852210514,73sleep26886-21/usr/sbin/munin12:05:002
248021035,42sleep00-21swapper/008:26:080
21443210310,78sleep321444-21http11:09:543
292421025,42sleep00-21swapper/011:51:130
292421025,42sleep00-21swapper/011:51:130
10051210213,72sleep310061-21/usr/sbin/munin12:14:583
10051210213,72sleep310061-21/usr/sbin/munin12:14:583
1952121009,57sleep019557-21ntpq09:20:060
1952121009,57sleep019557-21ntpq09:20:060
18511210034,56sleep00-21swapper/009:15:420
18511210034,56sleep00-21swapper/009:15:420
1646021009,79sleep016471-21ntpq09:10:040
2734329912,75sleep327392-21basename09:45:003
97352986,11sleep20-21swapper/208:50:002
313532985,41sleep00-21swapper/009:57:280
313532985,41sleep00-21swapper/009:57:280
709229513,57sleep16592-21/usr/sbin/munin12:05:071
709229513,57sleep16592-21/usr/sbin/munin12:05:071
2118329519,59sleep20-21swapper/209:25:082
2118329519,59sleep20-21swapper/209:25:082
1880229417,61sleep118846-21wc11:00:071
37029231,37sleep3364-21apt-get11:44:543
1598629215,58sleep10-21swapper/109:09:491
2702029119,52sleep11-21systemd08:03:071
1647129121,49sleep216490-21modprobe10:54:392
1273829129,30sleep00-21swapper/012:23:390
512829011,63sleep15137-21df_inode11:59:541
512829011,63sleep15137-21df_inode11:59:541
3174029029,10sleep00-21swapper/008:18:200
2524829029,9sleep00-21swapper/007:55:430
2524829029,9sleep00-21swapper/007:55:430
2209729015,58sleep322110-21awk07:45:133
218729015,56sleep12201-21irqstats11:50:011
218729015,56sleep12201-21irqstats11:50:011
2084929029,28sleep00-21swapper/007:43:560
2084929029,28sleep00-21swapper/007:43:560
2969728912,21sleep029824-21cat08:10:130
835328834,43sleep00-21swapper/010:25:220
835328834,43sleep00-21swapper/010:25:220
3172828852,21sleep031866-21awk11:40:110
305128815,57sleep23127-21switchtime10:10:102
2835528715,13sleep00-21swapper/009:48:480
1050828715,19sleep10-21swapper/110:34:521
1054028629,12sleep00-21swapper/008:51:310
1934328529,30sleep00-21swapper/011:01:150
515828421,49sleep35229-21ls08:35:113
2599128420,26sleep10-21swapper/109:40:091
1829928417,43sleep218449-21unixbench_singl09:15:112
1829928417,43sleep218449-21unixbench_singl09:15:112
1300128429,28sleep00-21swapper/007:18:210
111642845,11sleep20-21swapper/208:55:002
906428325,40sleep19120-21fschecks_count10:29:591
77042830,7sleep20-21swapper/210:25:032
77042830,7sleep20-21swapper/210:25:032
394028313,53sleep23980-21grep11:55:072
394028313,53sleep23980-21grep11:55:072
1074828312,52sleep310805-21/usr/sbin/munin10:35:023
874828221,29sleep30-21swapper/307:05:083
2307928219,41sleep323172-21cut11:14:593
2307928219,41sleep323172-21cut11:14:593
2103428224,32sleep20-21swapper/207:44:432
2103428224,32sleep20-21swapper/207:44:432
1118528224,40sleep111197-21munin-node07:14:421
3035028114,28sleep20-21swapper/209:55:002
3035028114,28sleep20-21swapper/209:55:002
1942128115,52sleep219475-21sed09:20:042
1942128115,52sleep219475-21sed09:20:042
1746328120,45sleep117464-21cron12:39:391
3080727920,28sleep30-21swapper/311:39:423
275727914,45sleep12758-21cron08:29:401
1274927913,48sleep212758-21fschecks_count09:00:012
1199427916,46sleep111052-21/usr/sbin/munin08:55:161
97642788,16sleep20-21swapper/210:30:122
3184527817,37sleep231866-21modprobe08:19:392
2155227812,52sleep221567-21needreboot07:45:032
876527716,46sleep08832-21sendmail_mailst07:05:090
718727722,35sleep10-21swapper/110:24:421
2125027716,44sleep321323-21switchtime09:25:093
2125027716,44sleep321323-21switchtime09:25:093
1039927614,48sleep110435-21ntp_states12:15:171
1039927614,48sleep110435-21ntp_states12:15:171
3033827516,26sleep10-21swapper/108:14:421
1521227515,25sleep30-21swapper/309:05:193
1219527514,44sleep212204-21/usr/sbin/munin07:15:042
922827416,43sleep29234-21modprobe08:49:392
637527423,21sleep30-21swapper/312:04:413
637527423,21sleep30-21swapper/312:04:413
632727412,21sleep20-21swapper/212:04:402
632727412,21sleep20-21swapper/212:04:402
280827420,27sleep30-21swapper/308:29:423
2189827421,34sleep221904-21ntp_states11:10:052
1346127423,25sleep20-21swapper/210:44:422
2282227314,44sleep322893-21switchtime09:30:103
2282227314,44sleep322893-21switchtime09:30:103
2275427327,25sleep10-21swapper/107:49:591
1232027318,21sleep30-21swapper/307:15:093
766327216,37sleep27866-21latency_hist08:44:432
26862729,15sleep10-21swapper/110:10:011
1465727220,28sleep20-21swapper/212:29:552
1403927230,23sleep10-21swapper/107:20:111
1802027113,40sleep218088-21modprobe10:59:402
1783127129,25sleep10-21swapper/109:15:021
1783127129,25sleep10-21swapper/109:15:021
1623427118,34sleep116258-21munin-run07:29:401
959927025,20sleep10-21swapper/107:09:421
2995527014,39sleep230037-21mailstats11:35:082
1497427014,33sleep10-21swapper/110:49:411
1497427014,33sleep10-21swapper/110:49:411
955426915,36sleep29571-21munin-run12:14:412
930926911,44sleep1225-21systemd-journal12:13:071
3206126916,32sleep132086-21apt-key08:19:431
2440126919,35sleep324475-21find09:35:113
1997026913,41sleep319977-21diskstats11:04:583
1859826918,29sleep30-21swapper/311:00:013
1576726921,24sleep10-21swapper/109:09:411
58126813,28sleep10-21swapper/111:46:011
2663826815,16sleep20-21swapper/211:25:032
2026026814,38sleep120289-21/usr/sbin/munin11:05:041
1654226821,24sleep10-21swapper/110:54:421
312826725,15sleep30-21swapper/310:10:113
2879126721,26sleep228755-21latency_hist08:09:422
2762826714,32sleep20-21swapper/209:45:072
2723526725,22sleep20-21swapper/208:04:422
247752676,10sleep10-21swapper/109:36:051
241902676,50sleep20-21swapper/211:18:452
241902676,50sleep20-21swapper/211:18:452
2243626716,39sleep222498-21if_eth009:30:012
2243626716,39sleep222498-21if_eth009:30:012
1415626722,23sleep20-21swapper/207:20:122
1315426714,39sleep313179-21df_inode12:24:553
1240826715,28sleep30-21swapper/308:59:423
638226618,24sleep30-21swapper/310:20:083
594126619,27sleep20-21swapper/208:39:422
3126126630,19sleep30-21swapper/308:15:123
1507926616,27sleep20-21swapper/209:05:182
1412726611,13sleep30-21swapper/310:45:073
1412726611,13sleep30-21swapper/310:45:073
3155226519,29sleep231644-21mailstats11:40:092
1736626511,39sleep317444-21find10:55:133
1374926522,21sleep20-21swapper/212:25:112
103352659,37sleep30-21swapper/307:10:083
49682640,6sleep10-21swapper/108:35:051
3063226415,27sleep30-21swapper/309:55:053
3063226415,27sleep30-21swapper/309:55:053
2838826415,35sleep328424-21ps11:30:093
2434226420,22sleep20-21swapper/209:35:102
196226414,38sleep21997-21grep08:25:092
1502426418,29sleep214948-21/usr/sbin/munin07:24:572
1131626423,20sleep20-21swapper/212:19:422
1131626423,20sleep20-21swapper/212:19:422
1074526413,39sleep1363-21snmpd08:54:111
978726317,22sleep30-21swapper/308:50:013
3240626317,30sleep232399-21apt-get08:19:562
2860026318,29sleep20-21swapper/211:30:122
171832636,47sleep30-21swapper/312:36:023
1274926316,26sleep30-21swapper/310:40:113
1274926316,26sleep30-21swapper/310:40:113
2563526213,36sleep325648-21latency_hist07:59:423
2563526213,36sleep325648-21latency_hist07:59:423
1750126212,29sleep20-21swapper/207:31:072
688726112,35sleep36931-21sed12:05:013
688726112,35sleep36931-21sed12:05:013
671226114,19sleep20-21swapper/210:20:112
36826118,26sleep232320-21apt-get11:44:532
33126117,22sleep30-21swapper/308:20:213
236782619,17sleep30-21swapper/307:50:253
1029526117,24sleep30-21swapper/310:34:423
54426018,24sleep1632-21sed08:20:271
3184426019,21sleep10-21swapper/109:59:551
3184426019,21sleep10-21swapper/109:59:551
2960726022,25sleep329676-21switchtime08:10:123
2555626021,23sleep125600-21awk07:59:401
2555626021,23sleep125600-21awk07:59:401
69225917,20sleep20-21swapper/210:04:412
69225917,20sleep20-21swapper/210:04:412
318825920,21sleep30-21swapper/311:54:393
318825920,21sleep30-21swapper/311:54:393
1895825921,18sleep20-21swapper/207:35:142
964625811,30sleep39826-21cat12:14:423
398225815,27sleep34055-21/usr/sbin/munin11:55:083
398225815,27sleep34055-21/usr/sbin/munin11:55:083
228825817,26sleep32444-21latency_hist10:09:423
20032587,38sleep20-21swapper/210:07:072
172232587,14sleep20-21swapper/209:14:062
1652125816,25sleep316613-21grep09:10:073
1641325814,19sleep20-21swapper/212:35:002
163202587,12sleep30-21swapper/310:52:423
1289025816,23sleep10-21swapper/112:24:411
1276125818,17sleep30-21swapper/309:00:023
1267225818,24sleep212748-21sed10:40:102
3002625719,19sleep10-21swapper/109:54:421
3002625719,19sleep10-21swapper/109:54:421
2946925714,27sleep129471-21/usr/sbin/munin11:34:551
2748125715,25sleep327476-21apt-get08:04:553
2692725715,26sleep227000-21cat09:44:422
2615025714,19sleep30-21swapper/308:00:063
2514025714,26sleep125168-21irqstats11:20:581
2324025719,19sleep20-21swapper/207:50:122
1880825718,23sleep10-21swapper/107:35:131
873325618,22sleep30-21swapper/310:29:413
873325618,22sleep30-21swapper/310:29:413
818225615,19sleep10-21swapper/108:45:061
3238925622,19sleep30-21swapper/310:00:103
3238925622,19sleep30-21swapper/310:00:103
3131125618,18sleep30-21swapper/311:40:023
2876425614,25sleep228789-21df_inode09:49:552
2876425614,25sleep228789-21df_inode09:49:552
2481325612,29sleep224805-21apt-get11:19:542
2105925614,18sleep10-21swapper/109:25:031
2105925614,18sleep10-21swapper/109:25:031
1383925615,23sleep30-21swapper/307:20:053
1183425612,27sleep111828-21apt-get07:14:551
311725510,28sleep13128-21df_inode08:30:001
2234325519,16sleep10-21swapper/109:30:001
2234325519,16sleep10-21swapper/109:30:001
1497025512,26sleep314966-21apt-get07:24:543
757525416,21sleep10-21swapper/108:44:421
393325413,16sleep10-21swapper/111:55:061
393325413,16sleep10-21swapper/111:55:061
2354225420,16sleep10-21swapper/111:15:091
2354225420,16sleep10-21swapper/111:15:091
195472547,10sleep20-21swapper/211:03:562
1861925416,19sleep30-21swapper/307:35:113
1374525415,21sleep20-21swapper/210:44:582
1374525415,21sleep20-21swapper/210:44:582
1067225416,19sleep20-21swapper/210:35:012
865125316,22sleep37937-21/usr/sbin/munin08:45:163
787025318,21sleep17882-21sshd10:25:071
787025318,21sleep17882-21sshd10:25:071
281472539,24sleep10-21swapper/109:46:071
1755525313,21sleep317553-21apt-get09:14:523
1755525313,21sleep317553-21apt-get09:14:523
1155425313,23sleep311622-21df12:19:553
1248725220,19sleep10-21swapper/110:40:051
1248725220,19sleep10-21swapper/110:40:051
2910525013,21sleep229114-21diskstats08:09:592
2015525012,24sleep320243-21grep07:40:083
2015525012,24sleep320243-21grep07:40:083
481724912,21sleep34882-21grep10:15:483
332424912,18sleep30-21swapper/308:30:063
1353524912,23sleep113558-21ntp_states12:25:061
2788124818,14sleep10-21swapper/108:05:051
219832479,18sleep10-21swapper/111:10:081
3104224611,21sleep131039-21apt-get11:39:521
2915424312,16sleep30-21swapper/309:50:043
2915424312,16sleep30-21swapper/309:50:043
165212407,16sleep30-21swapper/307:29:543
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional