You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-11-29 - 09:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Kontron

x86 Intel Core i7 7828EQ @3000 MHz, Linux 4.19.37-rt20 (Profile)

Latency plot of shadow in rack #9, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot3s.osadl.org (updated Tue Nov 29, 2022 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
134291690,49ptp4l0-21swapper/019:08:250
134291560,36ptp4l0-21swapper/119:08:271
147425045,2sleep30-21swapper/319:05:263
3227924941,2sleep20-21swapper/219:05:012
87382480,0sleep20-21swapper/219:20:142
134291220,0ptp4l0-21swapper/023:10:250
134291220,0ptp4l0-21swapper/019:15:270
2639992012,6cyclictest25-21ksoftirqd/121:40:191
134291180,0ptp4l0-21swapper/120:25:201
134291180,0ptp4l0-21swapper/019:30:200
134291180,0ptp4l0-21swapper/019:25:190
134291180,0ptp4l0-21swapper/019:20:180
134291170,0ptp4l0-21swapper/022:35:250
134291170,0ptp4l0-21swapper/000:15:240
134291160,1ptp4l27495-21apt-get00:10:160
134291160,1ptp4l19781-21apt-get23:55:191
134291160,0ptp4l0-21swapper/019:41:550
134291160,0ptp4l0-21swapper/000:35:300
134291150,1ptp4l6924-21apt-get23:30:150
134291150,1ptp4l5374-21apt-get22:25:021
134291150,1ptp4l33-21ksoftirqd/220:15:242
134291150,1ptp4l27829-21snmp_o5powerctl23:05:310
134291150,1ptp4l16438-21ls21:40:200
134291150,1ptp4l14002-21ntp_states21:35:271
134291150,1ptp4l13084-21df21:35:202
134291150,0ptp4l0-21swapper/123:35:161
134291150,0ptp4l0-21swapper/122:30:141
134291150,0ptp4l0-21swapper/121:25:171
134291150,0ptp4l0-21swapper/120:15:141
134291150,0ptp4l0-21swapper/119:20:131
134291150,0ptp4l0-21swapper/021:35:190
134291150,0ptp4l0-21swapper/020:30:200
134291150,0ptp4l0-21swapper/019:10:140
134291140,1ptp4l9206-21sed20:25:010
134291140,1ptp4l9022-21ls00:35:231
134291140,1ptp4l8381-21snmp_o5powerctl20:20:391
134291140,1ptp4l7226-21apt-get21:25:020
134291140,1ptp4l6001-21cat22:25:140
134291140,1ptp4l5692-21expr00:30:130
134291140,1ptp4l4470-21latency_hist21:20:010
134291140,1ptp4l32734-21/usr/sbin/munin23:15:270
134291140,1ptp4l32734-21/usr/sbin/munin23:15:270
134291140,1ptp4l31842-21perf21:10:010
134291140,1ptp4l27034-21apt-get21:00:160
134291140,1ptp4l26748-21ls22:00:210
134291140,1ptp4l2601-21fschecks_time21:15:151
134291140,1ptp4l22590-21grep19:45:220
134291140,1ptp4l22229-21sed00:00:010
134291140,1ptp4l21818-21if_enp0s31f622:55:231
134291140,1ptp4l19088-21unixbench_multi23:50:320
134291140,1ptp4l18652-21/usr/sbin/munin22:50:171
134291140,1ptp4l17632-21cat19:35:270
134291140,1ptp4l15926-21/usr/sbin/munin22:45:260
134291140,1ptp4l13615-21grep23:40:221
134291140,1ptp4l12440-21smartctl22:35:321
134291140,0ptp4l4801-21ls22:20:271
134291140,0ptp4l1868-21apt-get23:20:150
134291140,0ptp4l15605-21ntpq20:35:210
134291140,0ptp4l0-21swapper/123:30:271
134291140,0ptp4l0-21swapper/123:20:021
134291140,0ptp4l0-21swapper/123:10:151
134291140,0ptp4l0-21swapper/122:10:161
134291140,0ptp4l0-21swapper/121:45:161
134291140,0ptp4l0-21swapper/121:30:131
134291140,0ptp4l0-21swapper/121:10:231
134291140,0ptp4l0-21swapper/121:05:211
134291140,0ptp4l0-21swapper/121:00:151
134291140,0ptp4l0-21swapper/120:40:231
134291140,0ptp4l0-21swapper/120:35:141
134291140,0ptp4l0-21swapper/120:11:311
134291140,0ptp4l0-21swapper/120:08:011
134291140,0ptp4l0-21swapper/119:45:231
134291140,0ptp4l0-21swapper/119:35:261
134291140,0ptp4l0-21swapper/119:30:131
134291140,0ptp4l0-21swapper/119:25:131
134291140,0ptp4l0-21swapper/119:13:211
134291140,0ptp4l0-21swapper/023:45:170
134291140,0ptp4l0-21swapper/023:35:250
134291140,0ptp4l0-21swapper/022:55:330
134291140,0ptp4l0-21swapper/021:30:230
134291140,0ptp4l0-21swapper/021:13:230
134291140,0ptp4l0-21swapper/020:05:520
134291130,0ptp4l22844-21apt-get21:55:130
134291130,0ptp4l0-21swapper/222:55:292
134291130,0ptp4l0-21swapper/222:40:192
134291130,0ptp4l0-21swapper/221:20:232
134291130,0ptp4l0-21swapper/220:40:172
134291130,0ptp4l0-21swapper/220:10:262
134291130,0ptp4l0-21swapper/220:00:192
134291130,0ptp4l0-21swapper/220:00:012
134291130,0ptp4l0-21swapper/219:50:012
134291130,0ptp4l0-21swapper/200:35:192
134291130,0ptp4l0-21swapper/123:50:221
134291130,0ptp4l0-21swapper/123:45:011
134291130,0ptp4l0-21swapper/123:25:221
134291130,0ptp4l0-21swapper/123:15:211
134291130,0ptp4l0-21swapper/123:15:211
134291130,0ptp4l0-21swapper/123:05:191
134291130,0ptp4l0-21swapper/123:00:211
134291130,0ptp4l0-21swapper/122:45:181
134291130,0ptp4l0-21swapper/122:40:211
134291130,0ptp4l0-21swapper/122:15:231
134291130,0ptp4l0-21swapper/122:05:231
134291130,0ptp4l0-21swapper/122:00:181
134291130,0ptp4l0-21swapper/121:55:131
134291130,0ptp4l0-21swapper/121:50:251
134291130,0ptp4l0-21swapper/121:20:141
134291130,0ptp4l0-21swapper/120:55:191
134291130,0ptp4l0-21swapper/120:50:141
134291130,0ptp4l0-21swapper/120:45:171
134291130,0ptp4l0-21swapper/120:30:211
134291130,0ptp4l0-21swapper/120:00:171
134291130,0ptp4l0-21swapper/119:55:011
134291130,0ptp4l0-21swapper/119:50:011
134291130,0ptp4l0-21swapper/119:40:141
134291130,0ptp4l0-21swapper/119:15:211
134291130,0ptp4l0-21swapper/100:30:141
134291130,0ptp4l0-21swapper/100:25:241
134291130,0ptp4l0-21swapper/100:20:161
134291130,0ptp4l0-21swapper/100:15:151
134291130,0ptp4l0-21swapper/100:10:171
134291130,0ptp4l0-21swapper/100:05:241
134291130,0ptp4l0-21swapper/100:00:231
134291130,0ptp4l0-21swapper/023:40:010
134291130,0ptp4l0-21swapper/023:25:260
134291130,0ptp4l0-21swapper/023:00:170
134291130,0ptp4l0-21swapper/022:50:150
134291130,0ptp4l0-21swapper/022:40:150
134291130,0ptp4l0-21swapper/022:30:160
134291130,0ptp4l0-21swapper/022:20:210
134291130,0ptp4l0-21swapper/022:15:010
134291130,0ptp4l0-21swapper/022:10:170
134291130,0ptp4l0-21swapper/022:05:210
134291130,0ptp4l0-21swapper/021:50:200
134291130,0ptp4l0-21swapper/021:45:170
134291130,0ptp4l0-21swapper/021:15:190
134291130,0ptp4l0-21swapper/020:55:180
134291130,0ptp4l0-21swapper/020:50:150
134291130,0ptp4l0-21swapper/020:45:150
134291130,0ptp4l0-21swapper/020:40:130
134291130,0ptp4l0-21swapper/020:25:140
134291130,0ptp4l0-21swapper/020:15:210
134291130,0ptp4l0-21swapper/020:10:150
134291130,0ptp4l0-21swapper/020:00:160
134291130,0ptp4l0-21swapper/019:55:250
134291130,0ptp4l0-21swapper/019:50:180
134291130,0ptp4l0-21swapper/000:25:190
134291130,0ptp4l0-21swapper/000:20:260
134291130,0ptp4l0-21swapper/000:05:290
134291130,0ptp4l0-21swapper/000:00:210
264199120,11cyclictest4409-21sshd21:19:223
134291120,0ptp4l0-21swapper/221:50:152
134291120,0ptp4l0-21swapper/200:30:372
134291110,2ptp4l0-21swapper/220:56:412
134291110,0ptp4l0-21swapper/220:35:282
134291110,0ptp4l0-21swapper/219:45:172
134291110,0ptp4l0-21swapper/200:20:452
134291100,1ptp4l33-21ksoftirqd/220:54:112
134291100,1ptp4l2636-21cyclictest21:10:042
134291100,1ptp4l0-21swapper/223:07:582
134291100,0ptp4l0-21swapper/223:30:192
134291100,0ptp4l0-21swapper/223:25:222
134291100,0ptp4l0-21swapper/222:24:222
26419997,1cyclictest8338-21processes23:30:233
26419997,1cyclictest30733-21if_err_enp3s000:15:183
26419997,1cyclictest27813-21expr00:10:173
26419997,1cyclictest27362-21grep23:05:253
26419997,1cyclictest23849-21head20:50:243
26419997,1cyclictest18951-21cat21:45:203
26419997,1cyclictest18432-21dpkg22:50:013
26419997,1cyclictest15534-21taskset22:42:583
26419997,1cyclictest15466-21chrt19:31:043
26419997,1cyclictest14854-21chrt21:38:073
26419997,1cyclictest11027-21cut21:30:183
26419996,2cyclictest17582sleep321:12:533
26409997,1cyclictest3526-21tr19:10:182
26409997,1cyclictest11482-21chrt20:28:282
26409990,1cyclictest0-21swapper/222:00:132
13429190,0ptp4l33-21ksoftirqd/222:10:352
13429190,0ptp4l0-21swapper/223:57:552
13429190,0ptp4l0-21swapper/223:50:122
13429190,0ptp4l0-21swapper/223:48:442
13429190,0ptp4l0-21swapper/223:44:172
13429190,0ptp4l0-21swapper/223:35:012
13429190,0ptp4l0-21swapper/223:21:082
13429190,0ptp4l0-21swapper/223:15:372
13429190,0ptp4l0-21swapper/223:15:372
13429190,0ptp4l0-21swapper/223:01:542
13429190,0ptp4l0-21swapper/222:50:282
13429190,0ptp4l0-21swapper/222:49:252
13429190,0ptp4l0-21swapper/222:38:272
13429190,0ptp4l0-21swapper/222:31:512
13429190,0ptp4l0-21swapper/222:25:042
13429190,0ptp4l0-21swapper/222:16:532
13429190,0ptp4l0-21swapper/222:06:552
13429190,0ptp4l0-21swapper/221:57:172
13429190,0ptp4l0-21swapper/221:46:252
13429190,0ptp4l0-21swapper/221:42:092
13429190,0ptp4l0-21swapper/221:30:292
13429190,0ptp4l0-21swapper/221:17:252
13429190,0ptp4l0-21swapper/221:05:412
13429190,0ptp4l0-21swapper/221:01:212
13429190,0ptp4l0-21swapper/220:45:032
13429190,0ptp4l0-21swapper/220:24:432
13429190,0ptp4l0-21swapper/220:07:232
13429190,0ptp4l0-21swapper/219:42:532
13429190,0ptp4l0-21swapper/219:36:032
13429190,0ptp4l0-21swapper/219:16:122
13429190,0ptp4l0-21swapper/200:27:362
13429190,0ptp4l0-21swapper/200:14:302
13429190,0ptp4l0-21swapper/200:06:152
13429190,0ptp4l0-21swapper/200:00:392
26419987,1cyclictest20766-21fschecks_time21:50:163
26419986,1cyclictest9495-21unixbench_multi21:25:313
26419986,1cyclictest9264-21date20:25:013
26419986,1cyclictest8575-21cat00:35:213
26419986,1cyclictest68702sleep321:20:333
26419986,1cyclictest5892-21grep00:30:153
26419986,1cyclictest5620-21cat22:25:013
26419986,1cyclictest465-21taskset22:15:063
26419986,1cyclictest4573-21cat20:15:143
26419986,1cyclictest432-21cut23:15:293
26419986,1cyclictest432-21cut23:15:293
26419986,1cyclictest3474-21sort22:20:173
26419986,1cyclictest32533-21chrt00:20:133
26419986,1cyclictest30804-21awk22:10:163
26419986,1cyclictest2967-21expr00:25:193
26419986,1cyclictest29553-21cat20:00:183
26419986,1cyclictest295232sleep321:05:123
26419986,1cyclictest29372-21cat23:10:173
26419986,1cyclictest28176-21expr22:05:213
26419986,1cyclictest27379-21df21:00:173
26419986,1cyclictest26747-21uniq19:55:163
26419986,1cyclictest26472sleep319:10:063
26419986,1cyclictest26164-21cat23:05:013
26419986,1cyclictest25578-21expr22:00:143
26419986,1cyclictest25217-21tr19:50:253
26419986,1cyclictest24949-21expr00:05:233
26419986,1cyclictest24643-21expr20:55:173
26419986,1cyclictest24090-21sed00:00:373
26419986,1cyclictest23948-21expr21:55:183
26419986,1cyclictest23109-21taskset19:46:483
26419986,1cyclictest2222-21tail20:10:163
26419986,1cyclictest214352sleep320:46:303
26419986,1cyclictest21287-21wc22:55:203
26419986,1cyclictest19886-21cut23:55:203
26419986,1cyclictest18815-21expr19:40:143
26419986,1cyclictest17765-21grep23:50:213
26419986,1cyclictest17173-21expr20:40:143
26419986,1cyclictest16162-21expr19:35:173
26419986,1cyclictest152432sleep321:40:073
26419986,1cyclictest14858-21grep23:45:183
26419986,1cyclictest14561-21expr20:35:143
26419986,1cyclictest13367-21ls23:40:193
26419986,1cyclictest13209-21cut20:30:273
26419986,1cyclictest130312sleep322:38:283
26419986,1cyclictest11892-21date23:40:013
26419986,1cyclictest10375-21taskset19:20:303
26419986,1cyclictest10321-21taskset22:31:103
26419985,2cyclictest6669-21cron23:30:013
26419982,3cyclictest0-21swapper/319:15:183
26409986,1cyclictest7583-21mktemp21:25:012
26409986,1cyclictest30197-21expr00:15:142
26409986,1cyclictest28912-21tr23:10:152
26409986,1cyclictest12983-21cat20:30:252
13429180,0ptp4l0-21swapper/219:30:292
13429180,0ptp4l0-21swapper/219:25:302
26419975,1cyclictest9597-21expr20:25:123
26419975,1cyclictest32291-21mii-tool20:05:193
26419975,1cyclictest3105-21ntp_kernel_err23:20:213
26419975,1cyclictest15918-21chrt22:45:133
26419975,1cyclictest11651-21if_enp4s019:25:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional