You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-04 - 23:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Pentium N4200 @1100 MHz, Linux 5.9.1-rt18 (Profile)

Latency plot of shadow in rack #c, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot4s.osadl.org (updated Sun Dec 04, 2022 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2271199412,19cyclictest30600-21unattended-upgr12:22:593
22711994014,13cyclictest7479-21if_enp2s009:35:223
22711993510,13cyclictest31179-21sshd07:30:333
2271199350,17cyclictest22047-21latency_hist12:00:003
2270799349,8cyclictest24021-21/usr/sbin/munin07:10:180
2270799338,8cyclictest28196-21apt-esm-hook07:25:150
2271199320,29cyclictest8641-21head08:00:263
2270799325,15cyclictest0-21swapper/012:20:140
2270799324,17cyclictest0-21swapper/009:25:200
2270799324,16cyclictest0-21swapper/011:35:150
2270799324,10cyclictest0-21swapper/012:15:190
2270799316,8cyclictest30604-21ntpq07:30:250
2270799315,16cyclictest0-21swapper/009:05:050
2270799314,17cyclictest0-21swapper/010:20:260
2270799314,17cyclictest0-21swapper/009:55:260
2270799314,16cyclictest0-21swapper/008:25:130
2270799314,16cyclictest0-21swapper/007:45:180
2270799313,16cyclictest0-21swapper/007:50:140
2271199300,15cyclictest0-21swapper/307:40:003
2270799305,9cyclictest0-21swapper/009:45:220
2270799305,15cyclictest0-21swapper/011:45:150
2270799297,8cyclictest16293-21wc10:00:340
2270799297,7cyclictest23543-21latency_hist08:50:000
2270799295,9cyclictest0-21swapper/009:20:300
2270799294,9cyclictest0-21swapper/007:40:130
2270799293,9cyclictest0-21swapper/011:20:150
2270799293,9cyclictest0-21swapper/009:35:070
22707992922,4cyclictest28339-21https09:05:000
22711992812,14cyclictest12602-21cat11:30:203
22711992811,15cyclictest24089-21ntpq10:25:243
22708992816,6cyclictest11920-21latency_hist11:30:001
2270799287,6cyclictest3996-21apt-get11:05:140
2271199278,16cyclictest14065-21tune2fs11:35:173
22711992713,12cyclictest1747-21sh12:35:153
22711992712,13cyclictest27944-21ls09:00:263
22711992711,14cyclictest13789-21cut09:55:183
2270899272,17cyclictest28896-21apt-get09:05:101
2270799274,8cyclictest0-21swapper/008:05:120
2270799273,8cyclictest0-21swapper/012:05:120
2271199263,16cyclictest28305-21apt-get09:05:113
2270799267,5cyclictest2049-21perl12:35:180
2270799265,5cyclictest11905-21apt-get09:50:140
2270799265,5cyclictest11905-21apt-get09:50:140
2270799264,9cyclictest0-21swapper/011:40:240
2270799264,8cyclictest0-21swapper/008:10:210
2271199259,14cyclictest5083-21wc11:05:303
2271199251,21cyclictest3186-21awk09:25:003
2271199251,21cyclictest2247-21latency_hist07:45:003
2271199251,21cyclictest18907-21latency_hist11:50:003
22711992512,11cyclictest22293-21/usr/sbin/munin12:00:173
2271199250,1cyclictest0-21swapper/310:30:343
2270999251,3cyclictest22301-21apt-esm-hook12:00:142
2270799258,14cyclictest26260-21grep08:55:220
2270799253,7cyclictest0-21swapper/010:40:130
22707992521,2cyclictest24408-21ntp_states08:50:200
22707992517,4cyclictest24912-21apt-get07:15:130
2271199249,13cyclictest6710-21sed09:34:593
2271199249,13cyclictest12858-21tr09:50:243
2271199249,13cyclictest12858-21tr09:50:243
2271199248,13cyclictest27589-21/usr/sbin/munin10:35:253
2271199248,13cyclictest1188-21timerandwakeup12:30:343
2271199243,16cyclictest18735-21apt-get08:35:153
2271199242,17cyclictest19023-21apt-get11:50:183
2271199241,21cyclictest15260-21latency_hist08:25:003
2271199241,20cyclictest31277-21ntpq09:10:193
2271199241,20cyclictest21518-21latency_hist10:20:013
2271199241,20cyclictest18227-21latency_hist10:10:003
2271199241,18cyclictest0-21swapper/310:40:143
22711992411,11cyclictest11036-21idleruntime08:10:173
22711992410,12cyclictest11842-21sed09:50:003
22711992410,11cyclictest4717-21ls07:50:203
2271199240,21cyclictest31895-21sed10:50:223
2271199240,21cyclictest2153-21perf11:00:013
2271199240,20cyclictest10292-21munin-run11:25:003
2271199240,1cyclictest0-21swapper/311:25:173
2270799245,5cyclictest8114-21ps11:15:270
2270799243,8cyclictest0-21swapper/010:55:240
22707992413,8cyclictest10077-21latency_hist09:45:000
2271199239,11cyclictest0-21swapper/308:40:223
2271199238,13cyclictest9706-21cat09:40:263
2271199232,17cyclictest25554-21apt-get12:10:123
2271199232,16cyclictest25265-21apt-get08:55:143
2271199231,19cyclictest31239-21cron10:50:013
2271199231,19cyclictest15606-21hostname10:00:223
2271199231,17cyclictest28186-21apt-get07:25:143
2271199231,17cyclictest0-21swapper/310:10:213
22711992311,11cyclictest15613-21apt-config11:40:013
2271199230,21cyclictest3446-21snmp_o5powerctl07:45:233
2271199230,20cyclictest3501-21cstates09:25:143
2271199230,20cyclictest2770-21head11:00:223
2271199230,20cyclictest15584-21tr08:25:153
2271199230,19cyclictest24022-21cpuspeed_turbos12:05:153
2270999232,18cyclictest13553-21lsb_release11:35:002
22709992315,6cyclictest29644-21lsb_release10:45:012
2270999231,19cyclictest24348-21grep12:05:172
22707992315,4cyclictest21659-21snmp_o5powerctl11:55:320
22707992313,6cyclictest12415-21diskstats08:15:150
22707992312,8cyclictest24737-21awk10:29:590
2271199223,15cyclictest23642-21apt-get08:50:143
2271199222,14cyclictest17115-21apt-esm-hook08:30:153
2271199221,18cyclictest7034-21vmstat07:55:283
2271199221,18cyclictest7034-21vmstat07:55:283
2271199221,18cyclictest28438-21unixbench_multi12:15:323
2271199221,18cyclictest25186-21sort07:15:163
2271199221,18cyclictest1211-21https09:20:013
2271199221,17cyclictest5930-21grep11:10:183
2271199221,17cyclictest23682-21gltestperf07:10:163
2271199221,16cyclictest0-21swapper/308:05:143
2271199220,1cyclictest0-21swapper/312:25:133
2270999221,18cyclictest15774-21apt-get11:40:162
2270999220,5cyclictest23401-21cron08:50:002
2270899222,14cyclictest21705-21apt-esm-hook10:20:141
22708992213,6cyclictest15287-21grep08:25:011
22708992213,6cyclictest1217-21sort09:20:011
2270799229,10cyclictest2427-21cpu11:00:180
2270799229,10cyclictest14023-21expr08:20:160
2270799227,10cyclictest0-21swapper/010:35:130
2270799222,14cyclictest5620-21apt-get11:10:140
22707992215,3cyclictest5018-21apt-get09:30:170
22707992213,5cyclictest843-21thermald08:00:130
22707992212,7cyclictest7099-21munin-run08:00:000
22707992212,7cyclictest7099-21munin-run08:00:000
22707992212,7cyclictest23715-21awk12:05:000
22707992211,8cyclictest20435-21gltestperf10:15:170
22707992211,8cyclictest10739-21/usr/sbin/munin11:25:150
2271199212,13cyclictest22017-21apt-esm-hook08:45:143
2271199211,16cyclictest26561-21apt-get07:20:153
2271199211,15cyclictest0-21swapper/308:15:123
2271199210,18cyclictest7605-21head11:15:223
2271199210,17cyclictest0-21swapper/310:20:153
2270999211,4cyclictest10283-21sh11:25:002
22709992114,5cyclictest31882-21snmp_o5powerctl12:25:252
22709992114,5cyclictest25969-21snmp_o5powerctl10:30:372
22709992113,6cyclictest5821-21df_inode11:10:162
2270999211,17cyclictest28896-21apt-get09:05:162
2270999210,4cyclictest288-21systemd-journal10:55:012
2270899212,13cyclictest15786-21apt-esm-hook11:40:181
22708992114,5cyclictest25568-21memory10:30:271
22708992113,6cyclictest6435-21lsb_release09:35:001
2270899211,14cyclictest8968-21apt-esm-hook08:05:141
2270899211,14cyclictest4006-21apt-esm-hook11:05:161
2270799219,10cyclictest27180-21wc07:20:210
2270799219,10cyclictest19804-21cat08:35:260
22707992114,3cyclictest29852-21apt-get10:45:130
22707992113,4cyclictest31757-21diskstats07:35:140
22707992113,4cyclictest24957-21apt-get10:30:210
22707992111,8cyclictest21345-21grep08:40:230
22707992111,7cyclictest13107-21/usr/sbin/munin11:30:260
22707992111,6cyclictest32509-21apt-esm-hook12:30:200
22709992015,3cyclictest19136-21snmpd12:20:202
22709992015,3cyclictest19136-21snmpd09:49:222
22709992013,5cyclictest11918-21users08:10:272
2270999201,2cyclictest0-21swapper/208:30:152
2270999201,15cyclictest28343-21http09:04:592
2270999200,6cyclictest14932-21latency_hist10:00:012
2270999200,4cyclictest13321-21turbostat09:55:002
2270999200,4cyclictest13321-21turbostat09:55:002
2270999200,18cyclictest1952-21cstates09:20:162
2270899207,8cyclictest5692-21apt-get07:55:151
2270899207,8cyclictest5692-21apt-get07:55:141
2270899202,12cyclictest25559-21apt-esm-hook12:10:131
2270899202,12cyclictest15486-21apt-get08:25:141
22708992013,5cyclictest1733-21sed12:35:001
22708992012,6cyclictest32636-21sed10:50:311
2270899201,14cyclictest13542-21apt-esm-hook09:55:161
2270899201,13cyclictest24917-21apt-esm-hook07:15:131
2270899201,13cyclictest20663-21apt-esm-hook11:55:221
2270799207,8cyclictest0-21swapper/012:10:130
2270799206,9cyclictest0-21swapper/010:10:190
22707992012,6cyclictest32669-21cat10:50:310
22707992011,6cyclictest31813-21sendmail_mailst12:25:230
22707992011,6cyclictest17044-21tune2fs10:05:190
22707992010,8cyclictest20033-21snmp_o5powerctl11:50:310
22707992010,8cyclictest1140-21sendmail09:20:010
22707992010,6cyclictest17120-21apt-esm-hook08:30:150
2270999192,15cyclictest22125-21cat12:00:002
2270999192,15cyclictest21447-21latency_hist10:20:012
22709991915,2cyclictest19136-21snmpd11:53:582
22709991915,2cyclictest17713-21processes10:05:292
22709991915,2cyclictest1503-21sh12:34:592
22709991913,4cyclictest18503-21ls10:10:212
22709991912,5cyclictest4209-21entropy11:05:182
22709991912,5cyclictest3401-21mailstats07:45:222
22709991912,5cyclictest25394-21expr08:55:142
22709991912,5cyclictest25032-21seq07:15:132
22709991912,5cyclictest23352-21grep10:25:162
22709991912,5cyclictest2329-21cat07:45:002
22709991912,5cyclictest22895-21ls07:10:032
22709991911,6cyclictest25624-21sort12:10:142
22709991911,6cyclictest17819-21expr11:45:182
2270999191,14cyclictest6020-21snmp_o5powerctl09:30:272
22709991910,6cyclictest487-21cron07:39:592
2270899192,12cyclictest28836-21apt-esm-hook12:20:151
2270899192,12cyclictest26923-21apt-esm-hook09:00:151
2270899192,11cyclictest29831-21apt-get07:30:161
2270899192,11cyclictest23936-21apt-esm-hook12:05:141
22708991912,5cyclictest31082-21diskstats12:25:171
22708991912,5cyclictest24925-21grep08:50:261
22708991912,5cyclictest10839-21cat09:45:201
22708991912,4cyclictest4791-21idleruntime-cro09:30:001
22708991912,4cyclictest30310-21basename10:45:171
22708991911,6cyclictest19061-21tune2fs08:35:171
2270899191,13cyclictest2375-21apt-esm-hook11:00:181
2270899191,13cyclictest23293-21apt-esm-hook07:10:121
2270899191,12cyclictest7348-21apt-esm-hook08:00:141
2270899191,12cyclictest7007-21apt-esm-hook09:35:191
2270899191,12cyclictest30536-21apt-esm-hook09:10:141
2270899191,12cyclictest28238-21apt-esm-hook10:40:131
2270899191,12cyclictest20387-21apt-esm-hook08:40:141
2270899191,12cyclictest16813-21apt-esm-hook10:05:161
22708991910,6cyclictest1589-21sort09:20:021
2270799195,9cyclictest0-21swapper/009:10:130
2270999189,6cyclictest288-21systemd-journal07:29:592
2270999182,11cyclictest26571-21apt-esm-hook07:20:152
22709991815,2cyclictest19136-21snmpd10:22:082
22709991814,2cyclictest30529-21sh09:10:132
22709991812,4cyclictest23769-21expr08:50:142
22709991812,4cyclictest2137-21munin-run11:00:012
22709991812,4cyclictest15316-21ls10:00:202
22709991811,5cyclictest5752-21awk07:55:142
22709991811,5cyclictest5752-21awk07:55:142
2270999181,15cyclictest4793-21latency_hist09:29:592
2270999181,15cyclictest3760-21latency_hist11:05:002
22709991811,5cyclictest32152-21dpkg09:15:112
22709991811,5cyclictest29963-21seq10:45:152
2270999181,12cyclictest8973-21apt-esm-hook08:05:142
2270999181,12cyclictest8645-21apt-get09:40:132
2270999181,12cyclictest27178-21apt-get12:15:192
2270999181,12cyclictest12210-21apt-get08:15:112
2270999181,12cyclictest10518-21apt-get11:25:132
22709991810,6cyclictest8361-21grep11:15:322
2270899182,11cyclictest26587-21apt-esm-hook10:35:131
22708991812,4cyclictest23461-21expr10:25:161
22708991811,5cyclictest26799-21ls09:00:011
2270899181,13cyclictest5620-21apt-get11:10:151
2270899181,13cyclictest2514-21ntp_states12:35:231
2270899181,12cyclictest8650-21apt-esm-hook09:40:131
2270899181,12cyclictest785-21apt-esm-hook07:40:141
2270899181,12cyclictest7249-21apt-esm-hook11:15:181
2270899181,12cyclictest27183-21apt-esm-hook12:15:191
2270899181,12cyclictest22306-21apt-esm-hook12:00:161
2270899181,12cyclictest22007-21apt-get08:45:131
2270899181,12cyclictest17398-21apt-get11:45:141
22708991810,6cyclictest39-21kauditd10:15:011
22708991810,6cyclictest15246-21sort10:00:201
22709991713,2cyclictest21318-21sendmail_mailst08:40:232
22709991713,2cyclictest19136-21snmpd11:36:252
22709991713,2cyclictest19136-21snmpd07:30:022
2270999171,12cyclictest14859-21snmp_o5powerctl08:20:252
22709991710,5cyclictest7913-21wc08:00:182
22709991710,5cyclictest18858-21expr08:35:152
22709991710,5cyclictest1806-21ls12:35:162
22709991710,5cyclictest15661-21expr08:25:152
2270999170,2cyclictest449-1kworker/2:2H07:50:102
2270999170,15cyclictest0-21swapper/209:35:162
2270899179,6cyclictest13010-21pluginstate08:15:201
22708991711,4cyclictest26833-21grep07:20:171
22708991711,4cyclictest12294-21sed09:50:181
22708991711,4cyclictest12294-21sed09:50:181
2270899171,11cyclictest8893-21apt-get11:20:161
2270899171,11cyclictest4067-21apt-get07:50:151
2270899171,11cyclictest28191-21apt-esm-hook07:25:141
2270899171,11cyclictest2451-21apt-esm-hook07:45:151
2270899171,11cyclictest19028-21apt-esm-hook11:50:201
2270899171,11cyclictest13766-21apt-get11:35:151
22709991610,4cyclictest288-21systemd-journal10:40:002
2270899169,5cyclictest12148-21dpkg11:30:171
2270899161,11cyclictest31561-21apt-esm-hook07:35:131
2270899161,11cyclictest2143-21lsb_release11:00:001
22708991610,4cyclictest11697-21cat08:10:261
2270899159,3cyclictest0-21swapper/108:30:241
22708991511,2cyclictest21288-21unixbench_multi10:15:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional