You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-06 - 09:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Kontron

x86 Intel Core i7 7828EQ @3000 MHz, Linux 4.19.37-rt20 (Profile)

Latency plot of shadow in rack #9, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot3s.osadl.org (updated Wed Jul 06, 2022 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
134291650,41ptp4l0-21swapper/019:05:200
134291620,43ptp4l0-21swapper/119:09:591
2472625954,2sleep20-21swapper/219:05:212
2349025048,1sleep30-21swapper/319:05:133
134291320,0ptp4l0-21swapper/123:50:221
134291320,0ptp4l0-21swapper/123:50:221
134291200,7ptp4l13749-21unixbench-2d00:00:250
134291200,0ptp4l0-21swapper/023:35:190
134291190,0ptp4l0-21swapper/121:10:301
134291190,0ptp4l0-21swapper/100:35:201
134291190,0ptp4l0-21swapper/022:30:310
134291180,3ptp4l0-21swapper/021:35:180
134291180,0ptp4l0-21swapper/121:00:171
134291170,0ptp4l0-21swapper/120:25:181
134291170,0ptp4l0-21swapper/019:55:150
134291160,2ptp4l10529-21snmp_o5powerctl22:50:211
134291160,2ptp4l0-21swapper/022:48:500
134291160,1ptp4l8112-21apt-get21:45:121
134291160,1ptp4l8013-21snmp_o5powerctl22:45:201
134291160,1ptp4l27496-21apt-get00:30:130
134291160,0ptp4l0-21swapper/123:35:121
134291160,0ptp4l0-21swapper/023:50:250
134291160,0ptp4l0-21swapper/023:50:250
134291160,0ptp4l0-21swapper/023:45:340
173012150,0sleep10-21swapper/100:10:011
134291150,1ptp4l8994-21snmp_o5powerctl20:40:200
134291150,1ptp4l3917-21/usr/sbin/munin22:40:130
134291150,1ptp4l3415-21snmp_o5powerctl23:40:221
134291150,1ptp4l3402-21diskstats21:35:121
134291150,1ptp4l30975-21hddtemp_smartct21:25:140
134291150,1ptp4l29137-21snmp_o5powerctl19:15:030
134291150,1ptp4l29041-21snmp_o5powerctl00:30:221
134291150,1ptp4l27958-21ntp_states23:25:190
134291150,1ptp4l27794-21snmp_o5powerctl22:20:210
134291150,1ptp4l27601-21snmp_o5powerctl19:10:211
134291150,1ptp4l26228-21ntp_states00:25:180
134291150,1ptp4l23528-21grep00:20:161
134291150,1ptp4l22879-21ntp_states23:15:181
134291150,1ptp4l2073-21snmp_o5powerctl21:30:230
134291150,1ptp4l20457-21hddtemp_smartct00:15:161
134291150,1ptp4l17679-21snmp_o5powerctl22:00:223
134291150,1ptp4l16595-21perl22:00:141
134291150,1ptp4l14921-21diskstats19:50:131
134291150,1ptp4l14919-21snmp_o5powerctl21:55:200
134291150,1ptp4l12267-21ls21:50:240
134291150,1ptp4l12186-21snmp_o5powerctl21:50:221
134291150,0ptp4l0-21swapper/123:25:221
134291150,0ptp4l0-21swapper/123:05:211
134291150,0ptp4l0-21swapper/122:05:131
134291150,0ptp4l0-21swapper/121:15:121
134291150,0ptp4l0-21swapper/120:30:201
134291150,0ptp4l0-21swapper/120:00:101
134291150,0ptp4l0-21swapper/119:25:121
134291150,0ptp4l0-21swapper/023:44:560
134291150,0ptp4l0-21swapper/023:10:220
134291150,0ptp4l0-21swapper/022:10:130
134291150,0ptp4l0-21swapper/022:00:120
134291150,0ptp4l0-21swapper/021:45:200
134291150,0ptp4l0-21swapper/021:05:120
134291150,0ptp4l0-21swapper/020:35:130
134291150,0ptp4l0-21swapper/020:25:120
134291150,0ptp4l0-21swapper/019:45:140
134291150,0ptp4l0-21swapper/000:20:130
134291150,0ptp4l0-21swapper/000:10:120
134291140,1ptp4l864-21which19:20:202
134291140,1ptp4l592-21irqbalance22:40:141
134291140,1ptp4l31827-21apt19:20:111
134291140,1ptp4l29501-21sendmail20:20:010
134291140,1ptp4l28926-21cut22:25:141
134291140,1ptp4l27147-21memory19:10:160
134291140,1ptp4l24861-21expr20:10:120
134291140,1ptp4l24377-21snmp_o5powerctl21:10:280
134291140,1ptp4l19822-21if_err_enp3s023:10:161
134291140,1ptp4l17723-21df_inode00:10:131
134291140,1ptp4l1362-21/usr/sbin/munin22:35:120
134291140,1ptp4l11818-21/usr/sbin/munin20:45:211
134291140,1ptp4l11365-21unixbench_singl19:40:220
134291140,1ptp4l10901-21wc23:55:200
134291140,1ptp4l10695-21timerwakeupswit22:50:230
134291140,0ptp4l30984-21ls23:30:221
134291140,0ptp4l0-21swapper/221:30:232
134291140,0ptp4l0-21swapper/123:55:241
134291140,0ptp4l0-21swapper/123:45:141
134291140,0ptp4l0-21swapper/123:20:211
134291140,0ptp4l0-21swapper/123:00:221
134291140,0ptp4l0-21swapper/122:15:211
134291140,0ptp4l0-21swapper/122:10:221
134291140,0ptp4l0-21swapper/121:40:121
134291140,0ptp4l0-21swapper/121:30:121
134291140,0ptp4l0-21swapper/121:25:211
134291140,0ptp4l0-21swapper/121:05:171
134291140,0ptp4l0-21swapper/120:50:201
134291140,0ptp4l0-21swapper/120:20:201
134291140,0ptp4l0-21swapper/120:05:181
134291140,0ptp4l0-21swapper/119:55:201
134291140,0ptp4l0-21swapper/119:45:261
134291140,0ptp4l0-21swapper/119:40:201
134291140,0ptp4l0-21swapper/119:35:111
134291140,0ptp4l0-21swapper/119:15:021
134291140,0ptp4l0-21swapper/023:30:130
134291140,0ptp4l0-21swapper/023:20:210
134291140,0ptp4l0-21swapper/023:15:210
134291140,0ptp4l0-21swapper/022:55:120
134291140,0ptp4l0-21swapper/021:40:170
134291140,0ptp4l0-21swapper/020:55:210
134291140,0ptp4l0-21swapper/020:45:010
134291140,0ptp4l0-21swapper/020:30:020
134291140,0ptp4l0-21swapper/019:54:440
134291140,0ptp4l0-21swapper/000:35:120
134291130,1ptp4l17742-21/usr/sbin/munin21:00:120
134291130,0ptp4l0-21swapper/323:20:183
134291130,0ptp4l0-21swapper/323:00:233
134291130,0ptp4l0-21swapper/319:10:173
134291130,0ptp4l0-21swapper/223:30:202
134291130,0ptp4l0-21swapper/223:05:142
134291130,0ptp4l0-21swapper/223:00:152
134291130,0ptp4l0-21swapper/222:05:142
134291130,0ptp4l0-21swapper/221:40:212
134291130,0ptp4l0-21swapper/220:55:122
134291130,0ptp4l0-21swapper/122:55:141
134291130,0ptp4l0-21swapper/122:35:131
134291130,0ptp4l0-21swapper/122:30:221
134291130,0ptp4l0-21swapper/122:20:141
134291130,0ptp4l0-21swapper/121:55:181
134291130,0ptp4l0-21swapper/121:20:121
134291130,0ptp4l0-21swapper/120:55:141
134291130,0ptp4l0-21swapper/120:40:131
134291130,0ptp4l0-21swapper/120:35:131
134291130,0ptp4l0-21swapper/120:15:151
134291130,0ptp4l0-21swapper/120:10:181
134291130,0ptp4l0-21swapper/119:30:021
134291130,0ptp4l0-21swapper/100:25:161
134291130,0ptp4l0-21swapper/100:00:141
134291130,0ptp4l0-21swapper/023:05:130
134291130,0ptp4l0-21swapper/023:00:130
134291130,0ptp4l0-21swapper/022:25:140
134291130,0ptp4l0-21swapper/022:15:120
134291130,0ptp4l0-21swapper/022:05:010
134291130,0ptp4l0-21swapper/021:20:130
134291130,0ptp4l0-21swapper/021:15:130
134291130,0ptp4l0-21swapper/020:50:130
134291130,0ptp4l0-21swapper/020:20:140
134291130,0ptp4l0-21swapper/020:05:200
134291130,0ptp4l0-21swapper/020:00:110
134291130,0ptp4l0-21swapper/019:35:130
134291130,0ptp4l0-21swapper/019:30:120
134291130,0ptp4l0-21swapper/019:25:120
134291130,0ptp4l0-21swapper/019:20:120
134291130,0ptp4l0-21swapper/000:15:140
134291130,0ptp4l0-21swapper/000:05:120
231702120,0sleep30-21swapper/300:20:163
134291120,0ptp4l0-21swapper/221:20:172
134291110,2ptp4l0-21swapper/300:19:443
134291110,2ptp4l0-21swapper/219:40:582
134291110,0ptp4l0-21swapper/222:27:282
134291110,0ptp4l0-21swapper/200:30:192
134291100,1ptp4l9329-21df22:50:132
134291100,1ptp4l41-21ksoftirqd/319:33:103
134291100,1ptp4l0-21swapper/321:42:273
134291100,1ptp4l0-21swapper/300:29:513
134291100,0ptp4l0-21swapper/320:47:273
134291100,0ptp4l0-21swapper/222:31:202
134291100,0ptp4l0-21swapper/221:19:282
134291100,0ptp4l0-21swapper/220:39:172
134291100,0ptp4l0-21swapper/219:59:112
134291100,0ptp4l0-21swapper/219:18:282
260069997,1cyclictest4005-21taskset23:43:173
260069997,1cyclictest318962sleep320:23:233
260069997,1cyclictest26462-21expr22:20:133
260069997,1cyclictest21102-21grep20:00:183
260069997,1cyclictest12129-21expr00:00:143
259999998,0cyclictest33-21ksoftirqd/200:15:172
259999997,1cyclictest8512-21grep23:50:262
259999997,1cyclictest8512-21grep23:50:252
259999997,1cyclictest7523-21expr19:35:142
259999997,1cyclictest4039-21chrt23:43:432
259999997,1cyclictest3938-21snmp_o5powerctl20:30:212
259999997,1cyclictest321702sleep200:38:322
259999997,1cyclictest23497-21cat22:15:002
259999997,1cyclictest18552-21tail00:10:182
259999997,1cyclictest16339-21sed00:05:222
259999997,1cyclictest13960-21chrt00:01:212
259999997,1cyclictest10717-21tr21:50:122
259999990,8cyclictest12184-21tr19:45:152
13429190,0ptp4l0-21swapper/323:50:113
13429190,0ptp4l0-21swapper/323:50:113
13429190,0ptp4l0-21swapper/323:45:363
13429190,0ptp4l0-21swapper/323:37:093
13429190,0ptp4l0-21swapper/323:32:243
13429190,0ptp4l0-21swapper/323:27:283
13429190,0ptp4l0-21swapper/323:16:183
13429190,0ptp4l0-21swapper/323:11:233
13429190,0ptp4l0-21swapper/322:59:183
13429190,0ptp4l0-21swapper/322:47:523
13429190,0ptp4l0-21swapper/322:43:173
13429190,0ptp4l0-21swapper/322:30:183
13429190,0ptp4l0-21swapper/322:19:083
13429190,0ptp4l0-21swapper/322:13:283
13429190,0ptp4l0-21swapper/321:51:573
13429190,0ptp4l0-21swapper/321:46:083
13429190,0ptp4l0-21swapper/321:35:403
13429190,0ptp4l0-21swapper/321:33:043
13429190,0ptp4l0-21swapper/321:27:203
13429190,0ptp4l0-21swapper/321:15:103
13429190,0ptp4l0-21swapper/321:10:143
13429190,0ptp4l0-21swapper/320:34:223
13429190,0ptp4l0-21swapper/320:26:043
13429190,0ptp4l0-21swapper/320:16:433
13429190,0ptp4l0-21swapper/320:13:443
13429190,0ptp4l0-21swapper/320:07:363
13429190,0ptp4l0-21swapper/319:50:523
13429190,0ptp4l0-21swapper/319:40:413
13429190,0ptp4l0-21swapper/319:16:563
13429190,0ptp4l0-21swapper/300:36:293
13429190,0ptp4l0-21swapper/300:10:343
13429190,0ptp4l0-21swapper/223:47:512
13429190,0ptp4l0-21swapper/222:59:432
13429190,0ptp4l0-21swapper/222:45:362
13429190,0ptp4l0-21swapper/222:00:512
13429190,0ptp4l0-21swapper/221:58:492
13429190,0ptp4l0-21swapper/221:45:022
13429190,0ptp4l0-21swapper/221:37:372
13429190,0ptp4l0-21swapper/221:07:162
13429190,0ptp4l0-21swapper/221:01:162
13429190,0ptp4l0-21swapper/220:53:352
13429190,0ptp4l0-21swapper/220:10:142
13429190,0ptp4l0-21swapper/220:09:332
13429190,0ptp4l0-21swapper/219:52:202
13429190,0ptp4l0-21swapper/219:31:422
13429190,0ptp4l0-21swapper/219:10:022
13429190,0ptp4l0-21swapper/200:27:312
13429190,0ptp4l0-21swapper/200:20:042
260069987,1cyclictest771-21awk19:20:203
260069986,1cyclictest8536-21load20:40:163
260069986,1cyclictest7617-21if_enp3s019:35:163
260069986,1cyclictest28919-21tr22:25:133
260069986,1cyclictest28867-21cut21:20:163
260069986,1cyclictest27633-21tr00:30:133
260069986,1cyclictest2389-21cut19:25:143
260069986,1cyclictest20304-21egrep21:05:123
260069986,1cyclictest19508-21cut22:05:173
260069986,1cyclictest18636-21snmp_o5powerctl19:55:213
260069986,1cyclictest185332sleep323:06:013
260069986,1cyclictest18365-21ls21:00:143
260069986,1cyclictest17142-21taskset20:55:283
260069986,1cyclictest1620-21cat22:35:123
260069986,1cyclictest15900-21cut00:05:173
260069986,1cyclictest15171-21unixbench_multi21:55:223
260069986,1cyclictest12862-21sed19:45:223
260069986,1cyclictest12690-21basename20:50:113
260069986,1cyclictest10317-21tr23:55:163
260069986,1cyclictest10209-21ntpq22:50:183
260069981,3cyclictest0-21swapper/320:35:133
259999986,1cyclictest8909-21grep20:40:202
259999986,1cyclictest758-21gzip23:35:222
259999986,1cyclictest3998-21ls22:40:112
259999986,1cyclictest32619-21df20:25:132
259999986,1cyclictest30550-21cut20:20:172
259999986,1cyclictest29572sleep219:25:182
259999986,1cyclictest28159-21chrt22:20:252
259999986,1cyclictest27474-21cut23:25:162
259999986,1cyclictest24618-21sed22:15:172
259999986,1cyclictest23011-21cat21:10:182
259999986,1cyclictest215792sleep223:15:022
259999986,1cyclictest20819-21cut20:00:162
259999986,1cyclictest2000-21awk22:35:142
259999986,1cyclictest10155-21cat20:45:102
13429180,0ptp4l0-21swapper/223:58:312
13429180,0ptp4l0-21swapper/223:20:042
13429180,0ptp4l0-21swapper/223:11:282
13429180,0ptp4l0-21swapper/221:26:002
13429180,0ptp4l0-21swapper/220:15:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional