You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-08 - 17:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Freescale i.MX6 Quad @996 MHz, Linux 5.15.26-xenomai (Profile)

Latency plot of shadow in rack #b, slot #7
Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rackbslot7s (updated Mon Aug 08, 2022 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8552259411,567sleep38588-21ntpq11:08:013
9827235328,300sleep29858-21wc09:28:062
5148234023,292sleep35212-21fschecks_time10:57:563
5148234023,292sleep35212-21fschecks_time10:57:563
5831232513,295sleep15837-21/usr/sbin/munin07:32:561
31536230810,282sleep031578-21ntpq10:38:020
31136230114,77sleep20-21swapper/210:37:522
6706229726,244sleep36769-21fschecks_time11:02:573
7871228288,174sleep27874-21cron11:07:322
8868227466,189sleep20-21swapper/207:42:512
25530226269,34sleep229776-21cyclictest12:02:592
25530226269,34sleep229776-21cyclictest12:02:592
647422578,233sleep26566-21sed07:33:222
12668223968,84sleep20-21swapper/211:22:342
30130223341,27sleep20-21swapper/208:48:092
20503223227,191sleep2226-21systemd-journal11:47:332
5948222833,65sleep25987-21run-parts09:17:322
17260222773,116sleep217263-21cron11:37:322
13373222773,90sleep213412-21proc_pri11:23:022
22256221934,49sleep20-21swapper/208:23:092
17911221812,192sleep117947-21unin-run08:12:341
397221512,187sleep0402-21diskstats10:43:030
397221512,187sleep0402-21diskstats10:43:030
21544221149,62sleep20-21swapper/210:07:352
18833219813,120sleep218871-21modprobe11:42:322
19184219417,81sleep019203-21apt11:42:470
3632218311,157sleep23744-21latency_hist12:37:352
29308218314,157sleep129388-21date10:32:341
29308218314,157sleep129388-21date10:32:341
13208218328,83sleep20-21swapper/207:57:332
23632217411,87sleep223746-21latency_hist11:57:342
1529521729,129sleep215342-21if_eth008:03:022
29316217125,128sleep029004-21apt-get08:47:420
18373217122,129sleep118442-21fschecks_time08:12:541
3176216610,44sleep20-21swapper/210:52:342
15326216419,124sleep015457-21latency_hist09:47:350
502216265,82sleep2571-21latency_hist12:27:342
722821585,141sleep00-21swapper/009:18:480
550215815,128sleep1622-21grep08:58:051
100421578,89sleep21156-21awk10:43:172
100421578,89sleep21156-21awk10:43:172
5458214918,116sleep00-21swapper/010:58:010
5458214918,116sleep00-21swapper/010:58:010
2054214928,91sleep20-21swapper/212:32:342
2418214820,110sleep22505-21latency_hist07:22:352
28356214010,116sleep228480-21timerwakeupswit08:43:082
28356214010,116sleep228480-21timerwakeupswit08:43:082
31366213927,100sleep231377-21open_files08:53:012
8575213518,96sleep08577-21cron07:42:320
1099021349,113sleep211054-21sed07:48:122
1141321330,12sleep30-21swapper/309:33:073
1141321330,12sleep30-21swapper/309:33:073
8626213210,108sleep28697-21latency_hist07:42:342
5331213070,18sleep20-21swapper/210:57:592
5331213070,18sleep20-21swapper/210:57:592
24884212930,39sleep20-21swapper/210:17:452
11373212941,67sleep20-21swapper/209:33:072
11373212941,67sleep20-21swapper/209:33:072
627121278,45sleep20-21swapper/209:17:532
32056212612,32sleep20-21swapper/207:12:552
23146212422,84sleep323200-21grep08:27:543
1353221229,48sleep20-21swapper/207:57:532
15937212115,81sleep20-21swapper/209:48:052
15937212115,81sleep20-21swapper/209:48:052
8375211913,85sleep08431-21interrupts11:07:580
28478211929,47sleep20-21swapper/210:28:112
28478211929,47sleep20-21swapper/210:28:112
17571211919,36sleep20-21swapper/208:08:352
1765521189,84sleep00-21swapper/011:37:540
30122211716,20sleep00-21swapper/007:07:560
30122211716,20sleep00-21swapper/007:07:560
29609211715,41sleep20-21swapper/210:32:592
29609211715,41sleep20-21swapper/210:32:592
14293211710,74sleep214367-21cat09:43:002
2895211614,89sleep32972-21threads12:33:083
8090211313,25sleep38184-21/usr/sbin/munin09:23:033
22060211215,72sleep20-21swapper/211:52:342
22060211215,72sleep20-21swapper/211:52:342
21094211115,12sleep20-21swapper/208:21:562
12878211115,78sleep012555-21apt-get11:22:400
9362211015,78sleep19363-21/usr/sbin/munin09:27:531
2467121100,7sleep00-21swapper/010:17:340
2467121100,7sleep00-21swapper/010:17:340
22157211016,75sleep322195-21ntpq10:08:053
4725210912,15sleep00-21swapper/010:57:200
14909210812,84sleep314918-21uname11:28:043
14909210812,84sleep314918-21uname11:28:043
8166210771,17sleep20-21swapper/211:07:522
4508210718,11sleep20-21swapper/210:54:322
311521077,87sleep2369-21snmpd07:23:082
14441210722,60sleep014460-21apt-get11:27:490
14441210722,60sleep014460-21apt-get11:27:490
26783210626,39sleep20-21swapper/208:38:072
24592210516,42sleep224606-21modprobe10:17:332
24592210516,42sleep224606-21modprobe10:17:332
281821047,79sleep00-21swapper/007:22:580
1565621039,81sleep00-21swapper/009:47:590
1565621039,81sleep00-21swapper/009:47:590
14839210214,74sleep014880-21cat08:02:350
4202210125,42sleep24200-21apt-get07:27:482
25187210011,75sleep025255-21latency10:18:030
781629921,38sleep20-21swapper/209:22:532
3001829922,16sleep20-21swapper/207:07:502
3001829922,16sleep20-21swapper/207:07:502
2333429920,63sleep323388-21cpuspeed10:12:533
2333429920,63sleep323388-21cpuspeed10:12:533
1919429924,39sleep20-21swapper/208:13:322
154122995,12sleep20-21swapper/211:28:402
154122995,12sleep20-21swapper/211:28:402
126862996,13sleep10-21swapper/109:37:591
3263329818,65sleep032725-21cat07:13:170
311929823,36sleep00-21swapper/009:07:490
311929823,36sleep00-21swapper/009:07:490
2147929818,62sleep321156-21apt-get08:22:413
2772029611,23sleep00-21swapper/008:42:490
2772029611,23sleep00-21swapper/008:42:490
2461229621,29sleep20-21swapper/208:32:542
2461229621,29sleep20-21swapper/208:32:542
257529430,28sleep20-21swapper/209:04:122
257529430,28sleep20-21swapper/209:04:122
1711429414,63sleep00-21swapper/009:52:520
314229321,34sleep20-21swapper/209:07:512
314229321,34sleep20-21swapper/209:07:512
2545929317,59sleep324891-21/usr/sbin/munin10:18:103
2307929318,56sleep023098-21latency_hist10:12:340
214782939,68sleep021609-21unixbench_singl11:48:080
214782939,68sleep021609-21unixbench_singl11:48:080
2507529128,28sleep20-21swapper/212:01:462
1016729115,56sleep110135-21latency_hist07:47:341
498929014,56sleep35113-21unixbench_singl07:28:223
4732909,40sleep20-21swapper/208:58:022
2953529017,30sleep10-21swapper/110:32:541
2953529017,30sleep10-21swapper/110:32:541
92728916,54sleep11003-21/usr/sbin/munin10:43:161
92728916,54sleep11003-21/usr/sbin/munin10:43:161
2525628919,20sleep00-21swapper/008:33:120
2525628919,20sleep00-21swapper/008:33:120
1273828916,32sleep20-21swapper/209:38:002
2686828817,52sleep327015-21grep08:38:083
199972888,39sleep20-21swapper/210:02:352
1736428814,35sleep20-21swapper/209:53:022
1733928821,46sleep017367-21latency_hist11:37:340
2697928614,52sleep026953-21/usr/sbin/munin12:07:540
2697928614,52sleep026953-21/usr/sbin/munin12:07:540
264622869,35sleep20-21swapper/210:22:542
264622869,35sleep20-21swapper/210:22:542
260272867,61sleep00-21swapper/012:03:090
260272867,61sleep00-21swapper/012:03:090
212992868,65sleep00-21swapper/010:05:070
1962528614,52sleep30-21swapper/309:58:323
970528520,33sleep20-21swapper/211:12:492
970528520,33sleep20-21swapper/211:12:492
314042858,14sleep20-21swapper/212:22:332
314042858,14sleep20-21swapper/212:22:332
232882856,12sleep00-21swapper/011:54:090
2305628523,38sleep023052-21apt-get08:27:470
1822828522,31sleep00-21swapper/008:12:450
1638628517,53sleep316401-21latency_hist08:07:353
1564528516,53sleep315654-21diskstats09:47:583
1564528516,53sleep315654-21diskstats09:47:583
345528418,47sleep13137-21apt-get10:52:411
285528417,49sleep12899-21unin-run09:07:341
285528417,49sleep12899-21unin-run09:07:341
2851328418,51sleep20-21swapper/212:12:482
210012845,11sleep00-21swapper/008:20:450
136682849,63sleep00-21swapper/009:41:450
3010328318,45sleep330118-21fschecks_count07:07:553
3010328318,45sleep330118-21fschecks_count07:07:553
1872428313,17sleep00-21swapper/009:58:010
115182836,66sleep00-21swapper/007:50:400
10542836,15sleep00-21swapper/008:59:330
2842728116,34sleep20-21swapper/207:02:542
275328128,27sleep30-21swapper/307:22:573
2163728116,49sleep021714-21cut08:22:550
1772228122,27sleep30-21swapper/309:53:113
109042816,66sleep00-21swapper/011:16:420
109042816,66sleep00-21swapper/011:16:420
67452806,64sleep00-21swapper/007:34:170
242412806,11sleep20-21swapper/208:31:372
3061627912,17sleep30-21swapper/308:52:343
2873927912,46sleep328788-21irqstats12:12:573
2070127911,54sleep120707-21ntp_states10:03:101
133927919,30sleep30-21swapper/309:02:353
119527923,36sleep11235-21sort07:17:581
3138027820,27sleep30-21swapper/307:12:353
282252788,34sleep20-21swapper/212:12:322
282252788,34sleep20-21swapper/212:12:322
267327823,28sleep10-21swapper/112:33:021
2224827824,42sleep222256-21open_files10:08:082
173242788,12sleep10-21swapper/109:53:011
1267327812,46sleep012801-21/usr/sbin/munin07:53:060
1090727819,37sleep110901-21apt-get09:32:541
1090727819,37sleep110901-21apt-get09:32:541
881427710,53sleep00-21swapper/009:24:210
719027720,40sleep17269-21switchtime11:03:071
2373427613,47sleep123775-21ntpq10:13:041
2373427613,47sleep123775-21ntpq10:13:041
2238927622,34sleep10-21swapper/110:08:111
102902767,12sleep00-21swapper/009:28:250
83227520,25sleep30-21swapper/312:27:553
51642756,19sleep00-21swapper/007:28:470
175692757,13sleep00-21swapper/008:08:330
3020527411,16sleep00-21swapper/012:17:550
3020527411,16sleep00-21swapper/012:17:550
2734927412,55sleep00-21swapper/008:41:280
340727317,40sleep13554-21awk07:23:151
2562627313,38sleep325670-21if_err_eth012:03:003
2562627313,38sleep325670-21if_err_eth012:03:003
2380127320,31sleep10-21swapper/108:28:091
215527315,42sleep32215-21sed09:03:083
215527315,42sleep32215-21sed09:03:083
1690127325,25sleep30-21swapper/308:08:003
907527214,40sleep09079-21/usr/sbin/munin07:42:570
479427214,16sleep00-21swapper/009:12:580
297702725,59sleep00-21swapper/012:17:220
580327119,29sleep30-21swapper/307:32:543
3172727114,42sleep331734-21df_inode12:22:503
2862927120,33sleep028682-21/usr/sbin/munin07:03:040
2561427131,20sleep125761-21grep10:18:121
166927114,41sleep01670-21diskstats09:02:560
166927114,41sleep01670-21diskstats09:02:560
86162709,42sleep10-21swapper/111:08:031
489127014,43sleep14934-21if_err_eth009:12:591
429627012,41sleep14298-21/usr/sbin/munin07:28:001
383527019,29sleep13967-21sed09:08:111
383527019,29sleep13967-21sed09:08:111
368027013,38sleep33750-21/usr/sbin/munin09:08:093
368027013,38sleep33750-21/usr/sbin/munin09:08:093
35052704,57sleep00-21swapper/012:36:560
3030427021,23sleep10-21swapper/107:08:011
3030427021,23sleep10-21swapper/107:08:011
253927012,23sleep00-21swapper/010:48:120
2129127014,39sleep121414-21latency_hist08:22:351
1849427014,42sleep318523-21/usr/sbin/munin08:12:563
129612706,15sleep30-21swapper/311:22:513
1203827016,33sleep311932-21/usr/sbin/munin07:52:533
3037926912,38sleep330421-21sort12:17:583
3037926912,38sleep330421-21sort12:17:583
2925626921,31sleep329276-21modprobe10:32:323
2925626921,31sleep329276-21modprobe10:32:323
702326819,30sleep37153-21grep09:18:103
496226822,27sleep30-21swapper/309:13:013
3208026817,32sleep132162-21tune2fs07:12:581
1981026817,28sleep319886-21awk11:43:063
672326720,27sleep16722-21ntp_states09:18:041
2904726737,17sleep10-21swapper/107:03:131
231032677,13sleep30-21swapper/311:53:113
1420226715,15sleep10-21swapper/107:58:081
1402926717,21sleep30-21swapper/307:58:063
1299926711,43sleep113001-21entropy11:22:531
3143926611,20sleep00-21swapper/008:53:030
121102665,52sleep00-21swapper/009:36:560
121102665,52sleep00-21swapper/009:36:560
1835626519,21sleep30-21swapper/311:38:093
1522626512,34sleep10-21swapper/111:28:101
1522626512,34sleep10-21swapper/111:28:101
1243626514,25sleep10-21swapper/107:53:031
3149026417,31sleep331533-21nfs_client10:38:013
2718426418,27sleep126953-21/usr/sbin/munin12:07:591
2718426418,27sleep126953-21/usr/sbin/munin12:07:591
2062926410,28sleep30-21swapper/308:18:053
1256326415,30sleep312604-21uniq09:37:563
1254126416,24sleep00-21swapper/011:22:320
112882645,44sleep30-21swapper/311:17:503
793526312,38sleep38002-21sed07:38:153
44626313,22sleep00-21swapper/012:27:320
321572634,10sleep10-21swapper/108:57:281
778226220,23sleep10-21swapper/107:38:091
71162627,13sleep00-21swapper/011:03:060
2911926215,23sleep10-21swapper/108:47:351
2485926215,32sleep124880-21irqstats08:33:011
2485926215,32sleep124880-21irqstats08:33:011
2472326219,26sleep124776-21users11:58:101
1592526212,21sleep10-21swapper/108:03:131
1038526210,39sleep110398-21basename07:47:461
2990826116,20sleep10-21swapper/112:17:361
1555226117,25sleep115566-21apt-get09:47:521
1555226117,25sleep115566-21apt-get09:47:521
1257426115,18sleep10-21swapper/111:22:331
1918826015,28sleep119266-21/usr/sbin/munin09:58:141
2983725918,17sleep00-21swapper/010:33:050
2983725918,17sleep00-21swapper/010:33:050
2727025912,30sleep027363-21users10:23:120
2727025912,30sleep027363-21users10:23:120
195782595,12sleep10-21swapper/111:43:001
1577625917,17sleep00-21swapper/008:03:120
111425917,18sleep00-21swapper/012:28:000
915025823,19sleep30-21swapper/307:43:003
311525813,33sleep13175-21/usr/sbin/munin10:52:331
2776425814,20sleep30-21swapper/310:27:353
2776425814,20sleep30-21swapper/310:27:353
2764925816,26sleep327773-21cat12:08:093
2764925816,26sleep327773-21cat12:08:093
2151225814,29sleep119522-21kworker/u8:308:22:471
979525716,26sleep19876-21grep11:12:561
979525716,26sleep19876-21grep11:12:561
3071625717,18sleep10-21swapper/108:52:361
3013925616,20sleep30-21swapper/310:33:133
3013925616,20sleep30-21swapper/310:33:133
2460825614,25sleep324605-21apt-get08:32:513
2460825614,25sleep324605-21apt-get08:32:513
2448125618,21sleep30-21swapper/311:58:083
1605725615,23sleep116084-21df11:32:541
1071525614,21sleep30-21swapper/309:32:353
1013525616,24sleep310158-21ntp_states11:13:023
1013525616,24sleep310158-21ntp_states11:13:023
3223025521,19sleep30-21swapper/308:57:343
3113525513,26sleep131130-21apt-get10:37:511
2897725515,17sleep30-21swapper/308:47:333
2897725515,17sleep30-21swapper/308:47:333
2136525518,17sleep30-21swapper/311:48:073
2136525518,17sleep30-21swapper/311:48:073
2024625514,24sleep320241-21apt-get10:02:563
115725515,21sleep31222-21users10:43:183
115725515,21sleep31222-21users10:43:183
397725420,20sleep30-21swapper/310:53:073
2748425417,15sleep10-21swapper/108:42:341
2543425414,24sleep125428-21apt-get12:02:541
2543425414,24sleep125428-21apt-get12:02:541
87742535,10sleep10-21swapper/109:23:491
563425316,16sleep10-21swapper/110:58:061
563425316,16sleep10-21swapper/110:58:061
3096025317,20sleep131010-21users12:18:081
3096025317,20sleep131010-21users12:18:081
2385225311,25sleep00-21swapper/011:57:480
2254725315,23sleep122576-21irqstats11:53:011
325882526,35sleep10-21swapper/112:23:251
2113125211,21sleep10-21swapper/111:48:001
2113125211,21sleep10-21swapper/111:48:001
152362524,37sleep10-21swapper/109:47:031
2854025113,21sleep328627-21perl07:03:033
2646125113,22sleep126455-21apt-get10:22:531
2646125113,22sleep126455-21apt-get10:22:531
234925111,25sleep02350-21sshd07:22:210
1448125115,22sleep314496-21open_inodes09:43:053
8182509,16sleep10-21swapper/112:27:531
2772525010,23sleep127722-21apt-get08:42:501
1815425011,17sleep10-21swapper/111:38:071
1626825010,14sleep30-21swapper/311:33:013
19832468,16sleep30-21swapper/310:47:583
15592468,15sleep30-21swapper/307:18:083
2801424513,17sleep00-21swapper/010:27:570
2801424513,17sleep00-21swapper/010:27:570
1087524410,15sleep30-21swapper/307:48:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional