You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-06 - 17:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Freescale i.MX6 Quad @996 MHz, Linux 5.15.26-xenomai (Profile)

Latency plot of shadow in rack #b, slot #7
Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s (updated Wed Jul 06, 2022 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24127257815,545sleep132715-21kworker/u8:209:14:441
9200247511,420sleep09214-21fschecks_count11:54:480
23510247214,401sleep023649-21tail10:54:590
19724239425,309sleep019727-21cron12:29:290
19724239425,309sleep019727-21cron12:29:290
26048234827,306sleep026140-21tune2fs11:04:490
32734230441,124sleep0428-21grep09:40:000
17003229750,73sleep00-21swapper/012:19:480
154192275133,128sleep015446-21fschecks_count12:14:510
9569227323,191sleep09705-21unixbench_singl10:10:000
8254225814,227sleep18147-21cron08:24:321
3240922510,11sleep231972-21/usr/sbin/munin09:39:532
18281224345,128sleep00-21swapper/010:39:490
17596223020,114sleep00-21swapper/010:35:530
17596223020,114sleep00-21swapper/010:35:530
1903221610,146sleep02058-21latency_hist08:04:320
30498221563,75sleep00-21swapper/009:34:470
10773221410,143sleep010819-21fschecks_count11:59:490
2042122130,7sleep00-21swapper/010:45:010
2399221010,184sleep02435-21latency08:04:530
2399221010,184sleep02435-21latency08:04:530
2040220419,127sleep02188-21unixbench_multi11:30:010
973922029,131sleep09846-21latency_hist08:29:320
2724220058,84sleep00-21swapper/009:49:490
2724220058,84sleep00-21swapper/009:49:490
18382219515,16sleep30-21swapper/307:14:323
13895219314,164sleep213937-21ntp_kernel_pll_10:24:532
1843121929,117sleep018552-21date07:14:330
127762190123,26sleep012813-21cron08:39:300
11232219015,159sleep111243-21cron08:34:301
22589218968,57sleep00-21swapper/009:09:510
27415218816,158sleep327489-21find07:40:003
27415218816,158sleep327489-21find07:40:003
24491218759,76sleep024559-21fschecks_time10:59:480
2336921865,169sleep00-21swapper/007:29:020
18557218510,134sleep018696-21awk08:55:010
20287218447,59sleep020314-21cat07:19:290
8245218325,140sleep08388-21grep11:50:000
25930217538,75sleep026083-21unixbench_multi07:35:000
25930217538,75sleep026083-21unixbench_multi07:35:000
14393217436,75sleep00-21swapper/012:09:590
416421679,142sleep00-21swapper/009:54:460
416421679,142sleep00-21swapper/009:54:460
6525216511,141sleep06584-21dpkg08:19:300
25004216513,139sleep024974-21latency_hist07:34:320
7956216110,17sleep00-21swapper/008:22:090
4164215617,79sleep00-21swapper/011:39:300
32124215629,57sleep00-21swapper/007:54:590
29995215529,89sleep030054-21fschecks_count07:49:500
15581215345,80sleep015595-21grep07:04:480
28073214911,61sleep028083-21modprobe07:44:290
28073214911,61sleep028083-21modprobe07:44:290
22117214910,124sleep00-21swapper/012:34:580
15275214719,111sleep015041-21/usr/sbin/munin10:29:510
29102214630,46sleep00-21swapper/011:14:380
19785214414,76sleep00-21swapper/008:59:530
2917221428,74sleep029216-21ntpq09:29:540
1922421410,11sleep10-21swapper/107:15:011
24374213919,96sleep224510-21unixbench_multi07:30:012
29619213817,84sleep01-21systemd07:49:300
29619213817,84sleep01-21systemd07:49:300
15945213524,66sleep00-21swapper/008:49:310
7391213358,18sleep00-21swapper/010:04:470
26308213228,60sleep00-21swapper/009:20:170
21514213216,21sleep10-21swapper/109:04:591
22149213127,87sleep222163-21cpu07:24:442
210521317,36sleep00-21swapper/009:46:220
10855212811,75sleep010895-21proc_pri10:14:570
1615521275,111sleep00-21swapper/008:49:390
1397121276,33sleep00-21swapper/010:24:550
24702212511,66sleep024776-21switchtime09:15:040
487821227,79sleep04924-21ntpq11:39:540
24674212222,81sleep324706-21latency10:59:513
2047621229,76sleep020517-21cut12:29:580
2047621229,76sleep020517-21cut12:29:580
20663211914,23sleep10-21swapper/109:04:311
22302211515,17sleep00-21swapper/010:51:240
22302211515,17sleep00-21swapper/010:51:240
31962211418,67sleep20-21swapper/207:54:572
24251211417,76sleep224307-21grep09:14:542
30051211124,67sleep330084-21munin-run09:34:293
22814211153,42sleep222870-21sendmail-msp12:39:292
676621105,13sleep00-21swapper/010:01:160
28445211021,72sleep328462-21fschecks_count07:44:483
28445211021,72sleep328462-21fschecks_count07:44:483
1238121100,17sleep112-21rcu_sched08:35:021
22200210931,65sleep022237-21munin-run09:09:300
713921085,12sleep00-21swapper/011:47:420
994421075,12sleep00-21swapper/008:29:360
853210714,76sleep3897-21latency_hist09:44:313
3248321077,14sleep20-21swapper/211:24:502
3248321077,14sleep20-21swapper/211:24:502
28006210730,64sleep00-21swapper/011:09:580
4582210618,69sleep14165-21/usr/sbin/munin09:54:591
4582210618,69sleep14165-21/usr/sbin/munin09:54:591
16754210637,44sleep216758-21cron07:09:302
16595210514,62sleep216658-21lockfile-remove12:19:302
13793210516,63sleep113495-21/usr/sbin/munin10:24:521
27253210422,69sleep027293-21cpu09:24:450
16759210440,41sleep31-21systemd07:09:313
32351210324,65sleep032446-21perl11:24:480
32351210324,65sleep032446-21perl11:24:480
23103210016,63sleep322479-21/usr/sbin/munin09:10:023
289729911,71sleep13026-21awk08:05:021
289729911,71sleep13026-21awk08:05:021
2817829822,60sleep228288-21cat11:10:002
2057429719,56sleep119714-21/usr/sbin/munin10:45:031
1712629716,68sleep217165-21cat12:19:502
911729621,55sleep10-21swapper/111:54:451
1328329615,66sleep113253-21latency_hist10:24:311
1503329422,51sleep314611-21/usr/sbin/munin08:44:543
1475629413,68sleep214789-21fschecks_time08:44:492
1422429413,20sleep30-21swapper/308:42:563
911329219,44sleep29110-21apt-get11:54:422
2345929025,33sleep10-21swapper/107:29:321
413228918,13sleep00-21swapper/008:09:570
2194228617,42sleep222089-21latency_hist07:24:322
120012867,16sleep00-21swapper/010:19:460
648528530,36sleep36513-21munin-run08:19:293
1321228518,12sleep00-21swapper/012:05:510
596428419,49sleep26005-21if_eth009:59:532
1221528414,55sleep212236-21apt-get12:04:402
3109528311,49sleep131215-21cat09:35:001
2104828317,51sleep321141-21fschecks_time09:04:493
2061128326,33sleep20-21swapper/212:30:002
2061128326,33sleep20-21swapper/212:30:002
860628216,53sleep38630-21latency_hist10:09:313
3024328216,36sleep20-21swapper/209:34:322
396428123,38sleep34110-21wc09:54:323
396428123,38sleep34110-21wc09:54:323
307202819,15sleep30-21swapper/311:19:463
307202819,15sleep30-21swapper/311:19:463
2655228117,44sleep226635-21/usr/sbin/munin11:04:592
1559128113,51sleep315604-21irqstats12:14:543
303928019,38sleep32105-21/usr/sbin/munin08:05:043
303928019,38sleep32105-21/usr/sbin/munin08:05:043
183662797,11sleep20-21swapper/208:54:592
716127811,49sleep17256-21/usr/sbin/munin08:19:561
393727818,45sleep23962-21cat09:54:312
393727818,45sleep23962-21cat09:54:312
312562786,12sleep00-21swapper/011:19:590
312562786,12sleep00-21swapper/011:19:590
68652770,6sleep20-21swapper/208:19:402
475327723,41sleep24815-21cat09:55:042
475327723,41sleep24815-21cat09:55:042
2705127714,45sleep227085-21/usr/sbin/munin07:39:512
2705127714,45sleep227085-21/usr/sbin/munin07:39:512
1669727744,19sleep10-21swapper/110:34:491
1669727744,19sleep10-21swapper/110:34:491
317927613,45sleep23268-21/usr/sbin/munin11:34:512
2218927618,43sleep122305-21timerwakeupswit12:35:001
88927415,46sleep3370-21sshd11:27:093
88927415,46sleep3370-21sshd11:27:093
1632227410,14sleep20-21swapper/210:34:302
1285527417,40sleep212796-21latency_hist08:39:322
412427313,44sleep34130-21ntp_states08:09:563
1873827324,26sleep318836-21grep12:24:523
1873827324,26sleep318836-21grep12:24:523
176827319,33sleep10-21swapper/111:29:541
1351827322,36sleep00-21swapper/008:39:540
1317727226,25sleep20-21swapper/208:39:482
1197627215,16sleep30-21swapper/310:19:453
498527119,27sleep20-21swapper/208:14:322
2883327123,27sleep128783-21apt-get11:14:311
2502627120,21sleep30-21swapper/309:15:073
1117027119,27sleep10-21swapper/111:59:581
188127017,39sleep31899-21wc08:04:313
1716627018,37sleep117191-21fschecks_count07:09:491
2001226918,31sleep319714-21/usr/sbin/munin10:44:533
117626920,26sleep30-21swapper/309:44:473
80972687,11sleep20-21swapper/211:49:582
2138126819,21sleep30-21swapper/307:20:013
1807826819,26sleep30-21swapper/308:54:513
1530926816,22sleep10-21swapper/108:45:001
971626727,25sleep30-21swapper/311:55:003
814426721,29sleep30-21swapper/308:24:303
569526717,19sleep10-21swapper/108:14:591
445726726,25sleep30-21swapper/309:54:573
445726726,25sleep30-21swapper/309:54:573
3236926711,23sleep10-21swapper/109:39:531
1750426716,32sleep317584-21grep08:54:313
171926723,16sleep20-21swapper/209:44:582
326926620,19sleep30-21swapper/311:34:523
2941026621,23sleep30-21swapper/311:14:533
236722669,36sleep30-21swapper/307:29:423
1954126616,36sleep219617-21mailstats07:15:112
163322667,11sleep10-21swapper/112:16:141
756826514,35sleep17617-21/usr/sbin/munin11:49:451
649626512,36sleep26553-21/usr/sbin/munin11:44:572
2429726519,22sleep10-21swapper/107:30:001
2306626520,26sleep30-21swapper/307:25:103
2283326516,33sleep122851-21cpu10:54:441
225432659,39sleep222545-21cron10:54:292
225432659,39sleep222545-21cron10:54:292
1674226521,19sleep20-21swapper/210:34:502
1674226521,19sleep20-21swapper/210:34:502
1432426523,23sleep30-21swapper/312:09:583
116102656,50sleep30-21swapper/310:18:523
578626422,19sleep10-21swapper/111:44:321
456226421,22sleep20-21swapper/211:39:482
2910926416,26sleep20-21swapper/211:14:402
2489826418,31sleep124974-21grep10:59:571
2190526412,37sleep121941-21sed07:24:311
144526420,28sleep31452-21/usr/sbin/munin11:29:473
891226323,24sleep20-21swapper/208:24:582
494126321,18sleep30-21swapper/311:39:573
3142626317,27sleep10-21swapper/107:54:401
3102726326,22sleep1226-21systemd-journal07:52:461
3074926315,33sleep230819-21/usr/sbin/munin09:34:532
2855726320,22sleep20-21swapper/209:29:322
219926319,21sleep20-21swapper/208:04:462
219926319,21sleep20-21swapper/208:04:462
2070726323,23sleep20-21swapper/209:04:322
1744826323,25sleep21-21systemd08:54:302
1682426318,20sleep30-21swapper/310:34:513
1682426318,20sleep30-21swapper/310:34:513
38352626,16sleep10-21swapper/111:35:151
281526219,21sleep10-21swapper/109:49:501
281526219,21sleep10-21swapper/109:49:501
2797026211,37sleep328002-21ps11:09:573
2711226214,26sleep10-21swapper/111:08:091
2620326221,16sleep20-21swapper/209:20:152
105126217,29sleep21108-21/usr/sbin/munin07:59:582
3275626118,21sleep10-21swapper/111:24:571
3275626118,21sleep10-21swapper/111:24:571
262512619,16sleep20-21swapper/207:36:192
262512619,16sleep20-21swapper/207:36:192
1939026113,23sleep30-21swapper/308:59:443
1405726121,23sleep114155-21grep12:09:521
1206026120,25sleep20-21swapper/210:19:482
296702609,37sleep129741-21sensors_temp11:14:591
2584426022,24sleep325923-21cut09:20:043
2390226018,19sleep10-21swapper/109:14:321
220522608,17sleep10-21swapper/110:50:001
220522608,17sleep10-21swapper/110:50:001
1822926020,19sleep30-21swapper/312:24:323
164126015,33sleep11702-21switchtime09:44:571
905825918,19sleep20-21swapper/210:09:502
635625921,25sleep36433-21grep10:00:023
2786025920,21sleep30-21swapper/309:25:003
2658225916,20sleep10-21swapper/107:39:331
2658225916,20sleep10-21swapper/107:39:331
215552595,43sleep00-21swapper/007:20:340
201852595,11sleep30-21swapper/307:18:113
1876525918,26sleep218834-21/usr/sbin/munin10:39:592
621925819,19sleep30-21swapper/311:44:503
3012125816,24sleep130064-21apt-get09:34:311
2639625816,19sleep10-21swapper/109:20:181
219372585,11sleep20-21swapper/209:06:092
1641225815,25sleep116547-21sed10:34:331
770325711,21sleep20-21swapper/210:04:532
670925718,16sleep20-21swapper/208:19:332
365025713,23sleep13669-21apt-get08:09:421
3033425715,25sleep230358-21ntp_states07:49:562
1372725713,17sleep30-21swapper/310:24:493
1354725717,20sleep10-21swapper/112:09:311
1131425721,20sleep30-21swapper/312:00:003
1015825718,17sleep30-21swapper/310:14:323
864125617,23sleep38698-21irqstats08:24:513
602325613,24sleep16017-21apt-get11:44:441
2860825616,23sleep228631-21irqstats07:44:522
2860825616,23sleep228631-21irqstats07:44:522
2569125615,24sleep325697-21ntp_states07:34:553
2569125615,24sleep325697-21ntp_states07:34:553
1883525618,17sleep30-21swapper/310:40:003
1869725618,23sleep118717-21users08:55:021
182912566,39sleep20-21swapper/207:13:542
1505825616,22sleep315060-21apt-get10:29:423
895625516,22sleep10-21swapper/110:09:491
2461625518,22sleep224673-21/usr/sbin/munin10:59:502
2290525512,20sleep30-21swapper/310:54:463
2149425515,19sleep30-21swapper/310:49:493
2149425515,19sleep30-21swapper/310:49:493
1728525514,22sleep10-21swapper/112:19:531
142172559,15sleep10-21swapper/108:42:501
1178125517,18sleep30-21swapper/308:34:523
103525519,19sleep10-21swapper/107:59:571
574925412,24sleep15752-21apt-get09:59:441
3054025420,17sleep30-21swapper/307:50:013
2132325416,20sleep10-21swapper/112:34:311
2132325416,20sleep10-21swapper/112:34:311
85402535,36sleep30-21swapper/311:50:523
324562537,10sleep30-21swapper/307:56:153
2728025316,19sleep10-21swapper/107:39:581
2728025316,19sleep10-21swapper/107:39:581
1846525312,25sleep218461-21apt-get12:24:432
1846525312,25sleep218461-21apt-get12:24:432
1626725314,22sleep116362-21fschecks_time08:49:491
1227725313,24sleep312312-21df12:04:453
2303325216,17sleep20-21swapper/209:10:012
165922527,10sleep10-21swapper/107:07:271
3071825112,23sleep130713-21apt-get11:19:441
3071825112,23sleep130713-21apt-get11:19:441
3042625114,24sleep330496-21/usr/sbin/munin09:34:463
2966825115,19sleep10-21swapper/107:49:311
2966825115,19sleep10-21swapper/107:49:311
2245025115,20sleep30-21swapper/312:35:023
1998525113,22sleep220011-21irqstats10:44:522
196222517,33sleep10-21swapper/112:28:131
196222517,33sleep10-21swapper/112:28:131
2764525012,18sleep10-21swapper/109:24:541
117462506,32sleep20-21swapper/212:01:532
1039925014,22sleep110466-21df10:14:461
804624918,17sleep10-21swapper/110:05:011
1021124912,16sleep30-21swapper/308:29:513
1819024811,18sleep10-21swapper/110:39:421
22892475,32sleep20-21swapper/211:30:152
2046924710,15sleep30-21swapper/312:29:573
2046924710,15sleep30-21swapper/312:29:573
1083524711,21sleep210846-21ntp_states10:14:552
3069324618,15sleep20-21swapper/211:19:412
3069324618,15sleep20-21swapper/211:19:412
2285324410,20sleep222904-21df10:54:452
2596124310,17sleep30-21swapper/311:04:443
1037424211,11sleep20-21swapper/208:29:552
139062419,14sleep20-21swapper/212:09:492
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional