You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-01-31 - 05:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Freescale i.MX6 Quad @996 MHz, Linux 5.15.26-xenomai (Profile)

Latency plot of shadow in rack #b, slot #7
Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s (updated Tue Jan 31, 2023 00:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6102279647,426sleep06250-21unixbench_singl21:12:430
834426137,594sleep08380-21ntpq19:37:400
2650241512,387sleep02693-21ntpq21:02:380
4915232612,219sleep05005-21grep19:27:310
6762228515,256sleep36798-21ntpq19:32:383
9876223512,210sleep09767-21latency_hist21:27:080
1243223128,30sleep20-21swapper/220:57:402
27821221232,165sleep027884-21mailstats22:22:430
1883222108,185sleep018907-21mailstats23:37:470
1883222108,185sleep018907-21mailstats23:37:470
1430122107,190sleep014314-21basename19:57:220
3041722085,190sleep00-21swapper/020:47:370
2500921978,176sleep024610-21/usr/sbin/munin23:57:410
784217814,143sleep0886-21fschecks_time20:57:290
3097621785,24sleep00-21swapper/022:32:360
3097621785,24sleep00-21swapper/022:32:360
12291217811,147sleep012318-21awk19:52:050
27450217411,139sleep00-21swapper/000:07:060
475217210,140sleep00-21swapper/000:22:360
475217210,140sleep00-21swapper/000:22:360
18290217112,145sleep318339-21grep21:52:353
13531217114,134sleep00-21swapper/021:37:420
3415217014,18sleep00-21swapper/022:47:420
13917216815,140sleep113974-21latency23:22:341
13917216815,140sleep113974-21latency23:22:341
4858216414,52sleep00-21swapper/022:52:420
2525421645,14sleep030236-21cyclictest22:14:280
18755216217,22sleep00-21swapper/020:12:080
840321605,14sleep00-21swapper/023:03:440
27082216013,129sleep027131-21latency_hist22:22:060
18157215414,18sleep00-21swapper/021:52:320
1643821538,10sleep00-21swapper/023:31:580
1643821538,10sleep00-21swapper/023:31:580
5107215117,89sleep00-21swapper/021:12:070
3044221505,27sleep00-21swapper/000:14:370
3044221505,27sleep00-21swapper/000:14:370
172621497,19sleep01720-21apt-get19:17:250
2043421445,18sleep00-21swapper/021:58:210
22465214314,118sleep00-21swapper/023:49:300
796421415,124sleep00-21swapper/021:19:050
22461214110,119sleep022496-21latency20:22:340
22461214110,119sleep022496-21latency20:22:340
1597821405,28sleep00-21swapper/021:46:100
1459214010,115sleep11538-21cat22:42:341
1459214010,115sleep11538-21cat22:42:341
24379213919,100sleep324253-40mandb23:57:083
24379213919,100sleep324253-40mandb23:57:083
2210521397,120sleep00-21swapper/022:03:500
242821386,120sleep00-21swapper/022:46:570
242821386,120sleep00-21swapper/022:46:570
754221375,11sleep00-21swapper/019:36:070
1468621375,121sleep00-21swapper/023:24:230
1468621375,121sleep00-21swapper/023:24:230
122521375,120sleep00-21swapper/019:15:520
30844213412,19sleep00-21swapper/019:07:320
10264213330,30sleep30-21swapper/323:12:073
10264213330,30sleep30-21swapper/323:12:073
1174621325,116sleep00-21swapper/023:16:440
2883521316,38sleep00-21swapper/000:09:080
1383221316,39sleep00-21swapper/019:55:570
2807521306,112sleep00-21swapper/020:40:540
2807521306,112sleep00-21swapper/020:40:540
2425021308,112sleep024284-21ps20:27:380
2425021308,112sleep024284-21ps20:27:380
2395421305,113sleep00-21swapper/023:53:460
2395421305,113sleep00-21swapper/023:53:460
1321721305,35sleep00-21swapper/023:20:230
12287212710,94sleep00-21swapper/021:32:420
20730212634,29sleep00-21swapper/020:17:450
20730212634,29sleep00-21swapper/020:17:450
31711212421,84sleep331853-21grep00:17:513
31711212421,84sleep331853-21grep00:17:513
3002221246,106sleep00-21swapper/022:29:370
11087212416,91sleep111161-21wc23:12:411
9461212113,96sleep00-21swapper/019:42:200
708921216,104sleep00-21swapper/023:01:420
327221208,16sleep00-21swapper/019:22:260
2306021178,15sleep023092-21ntpq22:07:410
2306021178,15sleep023092-21ntpq22:07:410
3209621169,42sleep00-21swapper/022:37:280
3209621169,42sleep00-21swapper/022:37:280
25457211518,82sleep325507-21sendmail22:17:053
3190621148,16sleep0369-21snmpd00:18:120
3190621148,16sleep0369-21snmpd00:18:120
28966211438,51sleep128981-21cut19:02:291
28966211438,51sleep128981-21cut19:02:291
15953211413,68sleep00-21swapper/020:02:270
2864321135,21sleep00-21swapper/020:42:320
2864321135,21sleep00-21swapper/020:42:320
2036321136,30sleep00-21swapper/023:42:480
2036321136,30sleep00-21swapper/023:42:480
771211218,75sleep3910-21unixbench_singl19:12:443
1694721128,90sleep017022-21awk21:47:390
2564921100,6sleep20-21swapper/222:17:072
21064211018,73sleep120741-21apt-get22:02:131
19957211083,13sleep00-21swapper/020:13:330
3201421096,12sleep00-21swapper/020:52:370
1674521088,35sleep00-21swapper/023:32:260
1674521088,35sleep00-21swapper/023:32:260
892321077,13sleep00-21swapper/023:07:110
892321077,13sleep00-21swapper/023:07:110
420821068,11sleep00-21swapper/000:34:310
2637021066,11sleep00-21swapper/020:34:160
2637021066,11sleep00-21swapper/020:34:160
181121068,11sleep00-21swapper/000:27:330
181121068,11sleep00-21swapper/000:27:330
10594210679,16sleep010658-21awk21:27:410
2735210417,76sleep32736-21sshd00:30:343
2735210417,76sleep32736-21sshd00:30:343
15662210437,55sleep315171-21/usr/sbin/munin23:27:413
15662210437,55sleep315171-21/usr/sbin/munin23:27:413
12392210423,67sleep212504-21latency_hist19:52:072
1175210015,70sleep11242-21systemctl20:57:391
3092929914,67sleep330951-21/usr/sbin/munin19:07:353
1455229917,66sleep214614-21sed21:42:082
41129822,55sleep2434-21netstat00:22:342
41129822,55sleep2434-21netstat00:22:342
965829718,62sleep29815-21unixbench_multi23:07:462
965829718,62sleep29815-21unixbench_multi23:07:462
563929712,70sleep25792-21sed22:57:082
128329719,63sleep21379-21fschecks_time22:42:312
128329719,63sleep21379-21fschecks_time22:42:312
3214829615,63sleep232232-21fschecks_count19:12:102
3013629621,54sleep230152-21modprobe19:07:062
3013629621,54sleep230152-21modprobe19:07:062
2042029510,72sleep2369-21snmpd21:58:112
2807729425,42sleep328226-21unixbench_singl22:22:463
1880329418,57sleep318829-21proc_pri23:37:463
1880329418,57sleep318829-21proc_pri23:37:463
281629312,65sleep32821-21entropy22:47:283
1650529314,53sleep216560-21idleruntime21:47:292
602929210,69sleep36031-21sshd19:31:333
1501129216,58sleep115134-21unixbench_multi19:57:441
310212918,18sleep10-21swapper/119:07:371
3246629015,55sleep332073-21/usr/sbin/munin22:37:403
3246629015,55sleep332073-21/usr/sbin/munin22:37:403
298302905,14sleep30-21swapper/300:12:423
298302905,14sleep30-21swapper/300:12:423
320228818,54sleep13265-21ps22:47:391
317112889,13sleep20-21swapper/220:52:312
2470628813,31sleep324712-21df_inode23:57:323
2349228826,42sleep323511-21munin-run20:27:053
2349228826,42sleep323511-21munin-run20:27:053
1232128824,38sleep112359-21munin-run19:52:061
2963028740,30sleep129754-21timerwakeupswit22:27:451
2024028721,47sleep120301-21modprobe20:17:061
1981628716,53sleep319882-21/usr/sbin/munin20:12:443
1176828724,35sleep10-21swapper/121:32:301
298502868,65sleep00-21swapper/019:03:250
298502868,65sleep00-21swapper/019:03:250
1799228624,41sleep218018-21munin-run23:37:042
1799228624,41sleep218018-21munin-run23:37:042
1262928620,48sleep112307-21apt-get19:52:141
3176828525,37sleep231792-21munin-run19:12:062
978628327,38sleep2232-21systemd-journal21:27:062
3063128323,44sleep1232-21systemd-journal00:17:041
3063128323,44sleep1232-21systemd-journal00:17:041
3066328218,16sleep20-21swapper/220:47:432
1932428214,45sleep219348-21memory20:12:342
1209828217,49sleep212165-21df23:17:272
2896428114,52sleep129040-21sendmail_mailtr20:42:411
2896428114,52sleep129040-21sendmail_mailtr20:42:411
2116927920,26sleep30-21swapper/323:47:073
2116927920,26sleep30-21swapper/323:47:073
1808727916,40sleep117409-21/usr/sbin/munin20:07:451
2187827824,36sleep221832-21apt-get20:22:072
2187827824,36sleep221832-21apt-get20:22:072
315127720,37sleep23147-21apt-get00:32:272
2908827617,30sleep30-21swapper/319:02:313
2908827617,30sleep30-21swapper/319:02:313
645527520,27sleep10-21swapper/119:32:321
2976427532,21sleep30-21swapper/320:47:073
2976427532,21sleep30-21swapper/320:47:073
2197327520,27sleep30-21swapper/323:47:433
1642827522,23sleep10-21swapper/121:47:281
1526627519,35sleep115302-21grep21:42:401
2599427415,42sleep226049-21/usr/sbin/munin22:17:322
756927319,33sleep27683-21sed21:17:502
3209827325,24sleep20-21swapper/222:37:302
3209827325,24sleep20-21swapper/222:37:302
2196427319,22sleep30-21swapper/320:22:083
2196427319,22sleep30-21swapper/320:22:083
1733327319,27sleep30-21swapper/323:32:463
1733327319,27sleep30-21swapper/323:32:463
3026127221,37sleep330300-21latency_hist22:32:073
2072627218,33sleep120761-21http22:02:051
1805727215,39sleep218100-21ls21:52:302
1489727223,32sleep30-21swapper/323:27:043
1489727223,32sleep30-21swapper/323:27:043
1485627214,43sleep214857-21sshd23:26:352
1485627214,43sleep214857-21sshd23:26:352
1473027217,33sleep314778-21grep19:57:373
2639527116,33sleep30-21swapper/300:02:333
1701727120,33sleep117076-21sed23:32:351
1701727120,33sleep117076-21sed23:32:351
1615427125,26sleep20-21swapper/220:02:332
2421327021,30sleep124305-21systemd-run23:57:061
2421327021,30sleep124305-21systemd-run23:57:061
203727021,23sleep30-21swapper/322:42:443
203727021,23sleep30-21swapper/322:42:443
1185227015,39sleep311812-21latency_hist23:17:073
856526916,31sleep28690-21/usr/sbin/munin19:37:462
356726921,18sleep20-21swapper/221:07:082
1059526920,31sleep20-21swapper/223:12:302
1039626914,21sleep20-21swapper/221:27:352
1020726915,36sleep110098-21/usr/sbin/munin21:27:301
322226824,20sleep10-21swapper/100:32:301
3206826821,27sleep131740-21apt-get19:12:091
3104526816,29sleep10-21swapper/100:17:321
3104526816,29sleep10-21swapper/100:17:321
2594126819,30sleep226063-21latency_hist00:02:072
1520226816,36sleep215249-21grep21:42:372
1797426722,24sleep10-21swapper/121:52:291
3165026616,25sleep10-21swapper/120:52:301
3028626616,35sleep130310-21sshd20:47:331
292026615,22sleep20-21swapper/200:32:072
292026615,22sleep20-21swapper/200:32:072
2373426615,30sleep223866-21/usr/sbin/munin23:52:472
2373426615,30sleep223866-21/usr/sbin/munin23:52:472
170226610,22sleep10-21swapper/119:17:221
1687626617,20sleep30-21swapper/321:47:383
824526520,23sleep30-21swapper/321:22:073
2777426511,38sleep327451-21apt-get00:07:133
2583726518,29sleep125913-21/usr/sbin/munin20:32:401
2583726518,29sleep125913-21/usr/sbin/munin20:32:401
2559726519,24sleep20-21swapper/220:32:332
2559726519,24sleep20-21swapper/220:32:332
225326513,37sleep32320-21/usr/sbin/munin21:02:293
182072650,8sleep20-21swapper/220:07:452
1271626519,24sleep30-21swapper/323:17:443
866426419,20sleep20-21swapper/223:07:052
2274626421,18sleep20-21swapper/222:07:352
2274626421,18sleep20-21swapper/222:07:352
2219326410,31sleep10-21swapper/120:22:261
2219326410,31sleep10-21swapper/120:22:261
661926320,26sleep26667-21idleruntime-cro21:17:052
43832636,37sleep10-21swapper/121:07:441
2512726319,20sleep10-21swapper/120:32:071
2512726319,20sleep10-21swapper/120:32:071
2292826320,31sleep122952-21kernelversion22:07:381
2292826320,31sleep122952-21kernelversion22:07:381
1221826315,25sleep30-21swapper/321:32:413
58242626,10sleep10-21swapper/119:28:531
457626213,18sleep10-21swapper/119:27:071
3123326216,30sleep231236-21/usr/sbin/munin00:17:382
3123326216,30sleep231236-21/usr/sbin/munin00:17:382
2475426220,24sleep10-21swapper/122:12:431
2216126226,21sleep10-21swapper/123:47:451
1058526210,38sleep310098-21/usr/sbin/munin21:27:403
67382618,39sleep26761-21netstat19:32:372
3064526124,22sleep20-21swapper/200:17:052
3064526124,22sleep20-21swapper/200:17:052
264002615,18sleep30-21swapper/320:34:403
264002615,18sleep30-21swapper/320:34:403
238802619,40sleep3369-21snmpd22:11:483
238802619,40sleep3369-21snmpd22:11:483
234762616,16sleep20-21swapper/220:26:522
234762616,16sleep20-21swapper/220:26:522
1678126120,23sleep10-21swapper/120:02:441
975826014,25sleep20-21swapper/219:42:352
2141526013,30sleep221422-21wc23:47:262
199992609,13sleep10-21swapper/123:42:361
199992609,13sleep10-21swapper/123:42:361
180826015,29sleep21813-21/usr/sbin/munin19:17:312
1407826016,27sleep214220-21latency_hist19:57:082
1129726015,20sleep30-21swapper/319:47:323
3209725913,27sleep132091-21apt-get22:37:291
3209725913,27sleep132091-21apt-get22:37:291
295125914,29sleep23000-21irqstats22:47:322
289325918,19sleep10-21swapper/100:32:061
289325918,19sleep10-21swapper/100:32:061
2699325916,27sleep227001-21/usr/sbin/munin20:37:332
2699325916,27sleep227001-21/usr/sbin/munin20:37:332
490025814,20sleep20-21swapper/219:27:292
408625817,25sleep24180-21memory21:07:372
2750625823,19sleep10-21swapper/100:07:071
2733425816,25sleep127329-21apt-get22:22:281
2082325814,20sleep30-21swapper/322:02:073
131225818,20sleep30-21swapper/320:57:413
80572578,11sleep30-21swapper/323:02:423
477025720,22sleep34856-21mailstats22:52:413
3207225714,21sleep30-21swapper/320:52:403
2854425721,16sleep20-21swapper/220:42:302
2854425721,16sleep20-21swapper/220:42:302
2710225714,17sleep20-21swapper/200:02:482
2405525718,25sleep224063-21uname20:27:322
2405525718,25sleep224063-21uname20:27:322
849325615,25sleep38560-21df21:22:273
79962567,39sleep10-21swapper/121:19:301
428425614,26sleep14278-21apt-get22:52:271
3038825615,22sleep30-21swapper/320:47:353
2903325622,17sleep20-21swapper/222:27:292
2240025618,16sleep20-21swapper/222:07:072
1764225620,16sleep30-21swapper/320:07:353
1565325613,23sleep20-21swapper/220:02:092
999725516,21sleep110089-21mailstats19:42:421
452425517,22sleep34531-21modprobe19:27:053
269525518,22sleep12706-21ntp_states21:02:391
1079325514,18sleep30-21swapper/319:47:083
90202546,15sleep30-21swapper/319:39:493
77025416,16sleep30-21swapper/300:22:433
77025416,16sleep30-21swapper/300:22:433
627925414,16sleep20-21swapper/222:57:372
594125421,19sleep30-21swapper/322:57:303
535925414,23sleep35426-21df21:12:263
381425416,20sleep33881-21df21:07:303
323625416,16sleep30-21swapper/300:32:323
3051225413,25sleep230508-21apt-get22:32:222
3051225413,25sleep230508-21apt-get22:32:222
2846725414,20sleep10-21swapper/100:07:481
2691525412,25sleep126912-21apt-get20:37:281
2691525412,25sleep126912-21apt-get20:37:281
2487625415,22sleep124908-21irqstats23:57:381
2147725412,23sleep30-21swapper/322:02:363
1829825413,25sleep218316-21apt-get23:37:252
1829825413,25sleep218316-21apt-get23:37:252
1335525418,20sleep10-21swapper/121:37:391
131392546,12sleep30-21swapper/321:37:093
1254825415,22sleep212576-21users21:32:442
290032534,15sleep20-21swapper/200:11:162
850325210,18sleep10-21swapper/119:37:451
585525213,23sleep15852-21apt-get22:57:241
33172527,19sleep20-21swapper/219:22:282
1987125112,24sleep219867-21apt-get23:42:292
1987125112,24sleep219867-21apt-get23:42:292
1890925121,16sleep10-21swapper/123:37:481
1890925121,16sleep10-21swapper/123:37:481
974525015,22sleep19785-21http21:27:051
748525012,22sleep17492-21df_inode23:02:281
2181625016,19sleep121877-21apt-config20:22:061
2181625016,19sleep121877-21apt-config20:22:061
1610025015,19sleep10-21swapper/123:27:471
1610025015,19sleep10-21swapper/123:27:471
1588425012,23sleep315951-21df20:02:263
1295125012,22sleep113003-21users23:17:461
2578324913,20sleep325778-21apt-get22:17:253
200324810,13sleep30-21swapper/319:17:363
1506624817,14sleep30-21swapper/321:42:333
625424712,20sleep16262-21users21:12:441
2593624711,17sleep10-21swapper/122:17:311
957324611,22sleep19657-21expr23:07:451
957324611,22sleep19657-21expr23:07:451
3088624612,17sleep10-21swapper/122:32:331
3088624612,17sleep10-21swapper/122:32:331
3051324510,17sleep30-21swapper/322:32:233
3051324510,17sleep30-21swapper/322:32:233
3249924411,17sleep10-21swapper/100:22:211
3249924411,17sleep10-21swapper/100:22:211
2858024410,19sleep228606-21munin-run22:27:052
250412448,16sleep30-21swapper/320:32:053
250412448,16sleep30-21swapper/320:32:053
1555824412,16sleep20-21swapper/223:27:362
1555824412,16sleep20-21swapper/223:27:362
125052446,14sleep30-21swapper/319:52:083
2821124110,14sleep30-21swapper/320:42:063
2821124110,14sleep30-21swapper/320:42:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional