You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-14 - 19:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Pentium N4200 @1100 MHz, Linux 5.9.1-rt18 (Profile)

Latency plot of shadow in rack #c, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rackcslot4s.osadl.org (updated Sun Aug 14, 2022 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
75532293245,35sleep00-21swapper/007:09:550
73582258221,25sleep20-21swapper/207:08:122
74212246195,36sleep30-21swapper/307:08:453
74052244196,35sleep10-21swapper/107:08:361
88321020,4sleep3783599cyclictest08:05:193
262782480,4sleep3783599cyclictest11:30:273
114332460,3sleep20-21swapper/212:10:222
23622360,1sleep20-21swapper/209:20:242
783099352,25cyclictest10625-21memory08:25:212
783099303,21cyclictest16865-21apt-get08:40:142
783099291,23cyclictest18290-40apt-check12:25:062
783099287,19cyclictest24591-21cron12:40:012
783099283,7cyclictest6285-21apt-get09:30:212
7830992816,8cyclictest14814-21idleruntime-cro09:50:002
783099281,24cyclictest18494-21perf11:15:002
783099273,19cyclictest4836-21apt-get10:40:212
783099273,19cyclictest23388-21apt-get08:55:142
783099262,19cyclictest4110-21apt-get09:25:192
783099261,19cyclictest7429-21apt-get08:20:152
7820992619,4cyclictest14589-21mailstats12:15:280
783099252,4cyclictest28137-21apt-get10:20:152
783099252,4cyclictest28137-21apt-get10:20:152
783099252,18cyclictest21210-21apt-get08:50:122
7820992511,8cyclictest4506-21apt-get11:55:150
783099245,17cyclictest0-21swapper/211:00:202
783099242,17cyclictest30315-21apt-get10:25:172
783099242,14cyclictest15069-21taskset09:50:182
783099241,3cyclictest27568-21munin-run11:35:002
7830992413,5cyclictest277612sleep209:05:162
783099240,11cyclictest258472sort09:00:192
782099245,16cyclictest16070-21grep12:20:170
7835992314,5cyclictest0-21swapper/307:50:113
783099232,4cyclictest27888-21sendmail10:20:012
7830992316,4cyclictest10429-21idleruntime-cro09:40:012
783099231,19cyclictest1124-21runrttasks08:00:012
783099230,6cyclictest22444-21awk12:35:002
783099230,6cyclictest22444-21awk12:35:002
783099230,6cyclictest16315-21munin-run11:10:002
7820992314,5cyclictest20285-21latency_hist12:29:590
7835992215,5cyclictest16020-21df12:20:173
783099227,3cyclictest25-21ksoftirqd/210:10:012
783099223,10cyclictest111rcu_preempt08:20:002
783099222,14cyclictest19829-21apt-get07:35:142
783099221,7cyclictest16921-21sh07:30:002
7830992217,3cyclictest14100-21lsb_release08:34:592
7830992216,3cyclictest21359-21lsb_release10:05:002
7830992214,6cyclictest27893-21sed07:50:252
7830992213,6cyclictest19170-21http07:35:002
783099221,18cyclictest2340-21apt-get11:50:162
7826992214,5cyclictest24783-21diskstats07:45:181
7826992210,8cyclictest23106-21apt-get11:25:181
782699220,5cyclictest0-21swapper/109:38:011
782699220,5cyclictest0-21swapper/109:20:381
7820992213,6cyclictest19174-21http07:35:000
782099221,14cyclictest23141-21taskset11:25:190
53182220,2chrt783099cyclictest11:55:212
7835992114,5cyclictest31125-21ls11:40:303
7835992114,5cyclictest22888-21cat07:40:193
7835992113,6cyclictest5488-21sed10:40:243
7835992112,6cyclictest28634-21awk11:35:263
7835992112,6cyclictest13834-21df_inode12:15:193
7835992110,7cyclictest12157-21apt-get08:30:193
783099219,8cyclictest12834-21apt-get07:20:172
783099211,5cyclictest26464-21perf07:50:002
783099211,4cyclictest20724-1kworker/2:0H11:30:002
783099211,4cyclictest10831-21munin-run12:10:002
7830992114,5cyclictest15536-21munin-run12:20:002
783099211,3cyclictest8655-21awk12:05:002
783099211,17cyclictest17891-21cp12:25:002
783099211,15cyclictest0-21swapper/209:40:182
783099210,3cyclictest1649-21munin-run09:20:002
783099210,18cyclictest4419-21apt-key10:40:012
7826992113,6cyclictest749-21snmp_o5powerctl09:15:331
782699211,17cyclictest5900-21snmp_o5powerctl10:40:291
782699210,5cyclictest0-21swapper/108:22:061
782699210,4cyclictest0-21swapper/110:27:341
782699210,4cyclictest0-21swapper/108:19:201
782099213,15cyclictest26372-21gltestperf10:15:220
782099212,12cyclictest16560-21apt-get11:10:200
7820992117,2cyclictest16604-21turbostat08:40:000
7820992115,4cyclictest25885-21df11:30:230
782099211,14cyclictest12214-21apt-get11:00:200
783599209,10cyclictest6538-21date12:00:003
7835992013,5cyclictest3866-21ntpq08:10:203
7835992013,5cyclictest21002-21basename12:30:223
7835992013,5cyclictest21002-21basename12:30:223
7835992013,5cyclictest15852-21ntpq09:50:263
7835992012,6cyclictest18258-21snmp_o5powerctl07:30:263
7835992012,5cyclictest31090-21apt-get08:00:163
7835992011,7cyclictest32207-21ls09:15:193
783099202,12cyclictest18732-21apt-get11:15:182
7830992013,5cyclictest16732-21ls08:40:002
783099201,17cyclictest1719-21perf11:50:012
783099201,16cyclictest12555-21sendmail07:20:002
783099201,13cyclictest3056-21apt-get08:10:142
783099200,3cyclictest19174-21lsb_release10:00:002
783099200,3cyclictest10390-21munin-run07:15:012
7826992014,4cyclictest11434-21fschecks_time12:10:221
7826992013,5cyclictest6468-21expr09:30:221
7826992013,4cyclictest18655-21tune2fs12:25:211
7826992010,6cyclictest27814-21apt-get11:35:181
782699200,5cyclictest0-21swapper/112:19:251
782699200,5cyclictest0-21swapper/109:04:521
782699200,4cyclictest0-21swapper/110:01:511
782699200,4cyclictest0-21swapper/108:26:291
782099203,15cyclictest29055-21ps10:20:230
782099203,15cyclictest29055-21ps10:20:230
782099202,14cyclictest25585-21apt-get09:00:150
782099202,12cyclictest2286-21apt-get10:35:210
7820992013,5cyclictest22900-21sed11:25:000
7820992011,7cyclictest5764-21cat08:15:190
782099201,13cyclictest15048-21apt-get09:50:160
782099201,13cyclictest14420-21kworker/u8:007:35:160
782099201,13cyclictest10665-21apt-esm-hook07:15:150
80022190,2sleep30-21swapper/310:45:333
783599197,8cyclictest16560-21apt-get11:10:203
783599195,9cyclictest17228-21apt-get09:55:183
7835991915,2cyclictest1887-21snmpd09:40:213
7835991913,4cyclictest17395-21kworker/u8:212:40:003
7835991912,5cyclictest8561-21ls07:10:143
7835991912,5cyclictest10460-21cat10:55:183
7835991911,5cyclictest73550irq/130-enp2s011:05:193
7835991911,5cyclictest10039-21wc12:05:323
783099192,12cyclictest7033-21apt-esm-hook10:45:172
7830991914,3cyclictest23781-21apt-esm-json-ho10:10:192
783099191,2cyclictest31935-21perf11:45:002
783099191,16cyclictest24303-21munin-run07:45:002
783099191,16cyclictest11939-21latency_hist11:00:002
783099191,12cyclictest27848-21apt-esm-hook11:35:182
783099191,12cyclictest21001-21latency_hist08:50:002
7826991914,3cyclictest7227-21cat12:00:221
7826991914,3cyclictest29152-21sendmail_mailtr10:20:241
7826991914,3cyclictest29152-21sendmail_mailtr10:20:241
7826991914,3cyclictest18157-21unixbench_multi08:40:271
7826991914,3cyclictest16319-21unixbench-2d09:50:341
7826991913,4cyclictest24753-21awk08:55:281
7826991913,4cyclictest1526-21load08:05:231
7826991912,5cyclictest5652-21ls11:55:261
7826991912,5cyclictest5263-21cat09:25:321
7826991912,5cyclictest22989-21sed11:25:001
7826991912,5cyclictest21931-21cat07:40:001
7826991912,5cyclictest13605-21users11:00:351
7826991912,4cyclictest16838-21tune2fs11:10:221
782699191,13cyclictest32494-21apt-get10:30:211
782699191,13cyclictest31090-21apt-get08:00:161
782699191,13cyclictest19046-21apt-get08:45:181
782699191,13cyclictest12835-21apt-get09:45:201
782699191,12cyclictest15037-21apt-esm-hook07:25:151
782699190,4cyclictest0-21swapper/112:37:531
782699190,4cyclictest0-21swapper/109:41:111
782699190,4cyclictest0-21swapper/109:12:011
782099199,7cyclictest10802-21cron12:09:590
782099198,8cyclictest17228-21apt-get09:55:170
782099195,9cyclictest19426-21apt-get10:00:200
782099194,2cyclictest111rcu_preempt08:06:410
782099192,12cyclictest30315-21apt-get10:25:160
782099192,12cyclictest21599-21apt-get10:05:190
7820991914,3cyclictest13416-21latency09:45:240
7820991912,5cyclictest2710-21awk09:20:260
7820991912,5cyclictest16402-21date11:10:000
7820991912,5cyclictest1021-21grep09:15:340
7820991912,4cyclictest20098-21awk11:15:310
782099191,13cyclictest9899-21apt-get08:25:150
782099191,13cyclictest23949-21runrttasks07:42:060
782099191,13cyclictest12851-21apt-esm-hook07:20:170
782099191,13cyclictest10009-21apt-esm-hook10:55:140
782099191,12cyclictest4806-40apt-check10:40:030
782099191,12cyclictest15018-21apt-esm-hook07:25:150
783599182,14cyclictest20973-21ls08:50:013
7835991811,5cyclictest7305-21which09:30:303
7835991811,5cyclictest6939-21df12:00:193
7835991811,5cyclictest31-21ksoftirqd/311:00:213
7835991811,5cyclictest29194-21snmp_o5powerctl10:20:273
7835991811,5cyclictest29194-21snmp_o5powerctl10:20:263
7835991811,5cyclictest21294-21tune2fs11:20:203
7835991811,5cyclictest20890-21cut11:20:003
7835991811,5cyclictest13304-21cat09:45:233
783599181,13cyclictest23775-21apt-get10:10:193
783599181,12cyclictest28889-21apt-get07:55:153
783599181,12cyclictest2297-40apt-check11:50:063
783599181,12cyclictest21248-21apt-esm-hook08:50:133
783599181,12cyclictest19130-21apt-get07:35:113
783599181,12cyclictest17923-21snmp_o5powerctl08:40:253
7835991810,6cyclictest15773-21ls08:35:373
7835991810,5cyclictest4981-21pmu-power09:25:263
783599180,9cyclictest11691-21snmp_o5powerctl07:15:293
783099187,7cyclictest856-21apt-get08:05:192
783099185,11cyclictest0-21swapper/208:05:012
7830991814,3cyclictest22868-21basename11:25:002
7830991810,6cyclictest29933-21apt-get09:10:152
7826991814,2cyclictest26829-21pmu-power10:15:311
7826991812,4cyclictest29540-21memory07:55:201
7826991811,5cyclictest30831-21open_files11:40:251
7826991811,5cyclictest26014-21perl11:30:241
782699181,12cyclictest8899-21apt-get12:05:211
782699181,12cyclictest15810-21apt-esm-hook12:20:161
782699181,12cyclictest10628-21apt-get07:15:151
7820991814,2cyclictest30162-21sh09:10:170
7820991814,2cyclictest12527-21gltestperf08:30:220
7820991813,3cyclictest8656-21cat07:10:140
7820991811,5cyclictest28144-21expr09:05:190
7820991811,5cyclictest21664-21expr08:50:160
782099181,12cyclictest7466-21apt-esm-hook08:20:150
782099181,12cyclictest30008-21apt-get11:40:170
782099181,12cyclictest28906-21apt-esm-hook07:55:150
782099181,12cyclictest26749-21apt-esm-hook07:50:130
782099181,12cyclictest23789-21apt-esm-hook10:10:190
782099181,12cyclictest2355-21apt-esm-hook11:50:180
782099181,12cyclictest23421-21apt-esm-hook08:55:140
782099181,12cyclictest11072-21apt-get12:10:180
782099181,12cyclictest10673-21apt-esm-hook09:40:170
782099181,11cyclictest3070-21apt-esm-hook08:10:130
782099181,11cyclictest27814-21apt-get11:35:180
783599177,8cyclictest73550irq/130-enp2s009:05:223
783599175,8cyclictest15004-21apt-get07:25:143
783599172,9cyclictest23388-21apt-get08:55:133
7835991713,2cyclictest26635-21runrttasks09:00:253
7835991713,2cyclictest23335-21sh11:25:193
7835991713,2cyclictest21612-21sh10:05:193
7835991713,2cyclictest1051-21processes10:30:313
7835991711,5cyclictest10250-21runrttasks09:38:563
7835991711,4cyclictest6376-21timerandwakeup08:15:273
7835991711,4cyclictest11374-21tune2fs12:10:223
783599171,11cyclictest30315-21apt-get10:25:183
783599171,11cyclictest29950-21apt-esm-hook09:10:153
783599171,11cyclictest24542-21apt-get07:45:163
783599171,11cyclictest19426-21apt-get10:00:203
7835991710,5cyclictest8147-21cat08:20:223
7835991710,5cyclictest32543-21fschecks_time11:45:163
7835991710,5cyclictest10539-21sed08:25:213
783099171,12cyclictest1154-21snmp_o5powerctl10:30:332
782699179,5cyclictest20521-21apt-esm-hook12:30:191
782699179,5cyclictest20521-21apt-esm-hook12:30:191
7826991713,2cyclictest18177-21sendmail_mailqu07:30:241
7826991712,3cyclictest1720-21munin-plugin-st11:50:011
7826991712,3cyclictest15200-21ntp_states11:05:251
782699171,13cyclictest24846-21snmp_o5powerctl10:10:361
782699171,12cyclictest7016-21apt-get10:45:181
782699171,12cyclictest22262-21snmp_o5powerctl08:50:221
782699171,12cyclictest11903-21apt-esm-hook10:59:551
782699171,11cyclictest2297-40apt-check11:50:101
782699171,11cyclictest18746-21apt-esm-hook11:15:171
7826991710,5cyclictest3198-21cat08:10:131
7826991710,5cyclictest22404-21dpkg07:40:151
7826991710,5cyclictest22168-21cut10:05:241
7826991710,5cyclictest12324-21expr08:30:201
7820991711,4cyclictest9716-21sed09:35:320
7820991711,4cyclictest4548-21sed09:25:210
7820991711,4cyclictest1144-21awk11:45:270
782099171,12cyclictest24579-21apt-esm-hook07:45:160
782099171,11cyclictest31104-21apt-esm-hook08:00:160
7820991710,5cyclictest7517-21basename10:45:260
7820991710,5cyclictest24809-21wc12:40:000
783599168,5cyclictest3944-21grep10:37:483
7835991611,3cyclictest28109-21cut10:20:003
783599161,11cyclictest1966-21apt-esm-hook09:20:193
783599160,5cyclictest0-21swapper/307:20:173
783099169,4cyclictest9761-21ls10:55:002
782699169,5cyclictest28035-21tune2fs09:05:191
7826991614,1cyclictest0-21swapper/107:51:381
7826991613,2cyclictest2592-21fschecks_count10:35:241
782699161,11cyclictest9508-21snmp_o5powerctl07:10:251
782699161,11cyclictest15744-21snmp_o5powerctl08:35:351
7826991610,4cyclictest19-21ksoftirqd/110:55:001
7826991610,4cyclictest17416-21expr09:55:201
7826991610,4cyclictest13612-21ntp_kernel_err07:20:231
782099169,5cyclictest17049-21expr08:40:150
782099162,8cyclictest20507-21apt-get12:30:160
782099162,8cyclictest20507-21apt-get12:30:160
7820991614,1cyclictest0-21swapper/010:50:470
782099161,11cyclictest6321-21apt-esm-hook09:30:200
782099161,11cyclictest32471-21runrttasks10:30:070
782099160,11cyclictest19046-21apt-get08:45:170
7835991510,3cyclictest19241-21ps12:25:263
7820991513,1cyclictest0-21swapper/012:00:240
7835991412,1cyclictest0-21swapper/310:54:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional