You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-09-17 - 15:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Kontron

Intel(R) Core(TM) i7-7820EQ CPU @ 3.00GHz, Linux 4.19.37-rt20 (Profile)

Latency plot of shadow in rack #9, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack9slot3s.osadl.org (updated Fri Sep 17, 2021 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
111591720,50ptp4l0-21swapper/007:09:550
111591620,44ptp4l0-21swapper/107:06:171
1230915345,6phc2sys0-21swapper/207:09:532
1140425146,2sleep30-21swapper/307:05:153
18422470,0sleep10-21swapper/112:00:261
314132400,0sleep00-21swapper/011:55:240
111591220,0ptp4l0-21swapper/009:05:170
111591200,0ptp4l0-21swapper/011:00:230
111591190,0ptp4l0-21swapper/110:35:231
111591180,0ptp4l0-21swapper/108:15:261
111591170,2ptp4l13921-21apt-get07:10:141
111591170,0ptp4l0-21swapper/111:45:231
111591170,0ptp4l0-21swapper/110:50:231
111591170,0ptp4l0-21swapper/110:25:261
111591170,0ptp4l0-21swapper/010:50:020
111591160,2ptp4l17581-21ntp_states09:20:280
111591160,1ptp4l7283-21apt-get11:10:201
111591160,1ptp4l5204-21snmp_o5powerctl08:55:260
111591160,1ptp4l2683-21snmp_o5powerctl08:50:261
111591160,1ptp4l26491-21snmp_o5powerctl07:30:270
111591160,1ptp4l15466-21snmp_o5powerctl07:10:240
111591160,1ptp4l11681-21apt-get10:15:150
111591160,0ptp4l0-21swapper/110:55:161
111591160,0ptp4l0-21swapper/110:15:151
111591160,0ptp4l0-21swapper/109:25:151
111591160,0ptp4l0-21swapper/107:50:161
111591160,0ptp4l0-21swapper/012:20:190
111591160,0ptp4l0-21swapper/009:10:130
1382999153,3cyclictest0-21swapper/007:35:170
1382999153,3cyclictest0-21swapper/007:35:170
111591150,1ptp4l9242-21ls08:00:191
111591150,1ptp4l7894-21ntpq10:05:220
111591150,1ptp4l6783-21ntpq07:55:220
111591150,1ptp4l25422-21python09:35:250
111591150,1ptp4l25337-21irqstats10:40:290
111591150,1ptp4l22906-21snmp_o5powerctl09:30:261
111591150,1ptp4l19294-21ntp_states12:35:231
111591150,1ptp4l18587-21/usr/sbin/munin11:30:200
111591150,1ptp4l1841-21cat07:45:331
111591150,1ptp4l18301-21/usr/sbin/munin10:25:250
111591150,1ptp4l17407-21taskset12:30:291
111591150,1ptp4l1690-21snmp_o5powerctl12:00:230
111591150,1ptp4l16421-21snmp_o5powerctl11:25:261
111591150,1ptp4l14546-21smartctl12:25:361
111591150,1ptp4l14055-21ls11:20:321
111591150,1ptp4l10445-21latency_hist08:05:010
111591150,0ptp4l0-21swapper/112:10:281
111591150,0ptp4l0-21swapper/111:50:201
111591150,0ptp4l0-21swapper/111:18:591
111591150,0ptp4l0-21swapper/111:00:161
111591150,0ptp4l0-21swapper/110:20:151
111591150,0ptp4l0-21swapper/109:44:341
111591150,0ptp4l0-21swapper/109:35:161
111591150,0ptp4l0-21swapper/108:25:291
111591150,0ptp4l0-21swapper/012:15:260
111591150,0ptp4l0-21swapper/011:54:150
111591150,0ptp4l0-21swapper/010:10:170
111591150,0ptp4l0-21swapper/008:50:160
111591140,2ptp4l0-21swapper/211:40:232
111591140,1ptp4l5164-21fschecks_count11:05:120
111591140,0ptp4l31571-21snmp_o5powerctl11:55:271
111591140,0ptp4l0-21swapper/209:15:282
111591140,0ptp4l0-21swapper/112:20:071
111591140,0ptp4l0-21swapper/112:15:241
111591140,0ptp4l0-21swapper/111:35:161
111591140,0ptp4l0-21swapper/111:30:211
111591140,0ptp4l0-21swapper/111:05:111
111591140,0ptp4l0-21swapper/110:45:221
111591140,0ptp4l0-21swapper/110:40:271
111591140,0ptp4l0-21swapper/110:30:281
111591140,0ptp4l0-21swapper/110:10:251
111591140,0ptp4l0-21swapper/110:05:211
111591140,0ptp4l0-21swapper/110:00:201
111591140,0ptp4l0-21swapper/109:55:221
111591140,0ptp4l0-21swapper/109:50:191
111591140,0ptp4l0-21swapper/109:45:211
111591140,0ptp4l0-21swapper/109:20:241
111591140,0ptp4l0-21swapper/109:15:161
111591140,0ptp4l0-21swapper/109:10:141
111591140,0ptp4l0-21swapper/109:05:321
111591140,0ptp4l0-21swapper/109:00:181
111591140,0ptp4l0-21swapper/108:45:281
111591140,0ptp4l0-21swapper/108:35:201
111591140,0ptp4l0-21swapper/108:30:241
111591140,0ptp4l0-21swapper/108:20:221
111591140,0ptp4l0-21swapper/108:10:571
111591140,0ptp4l0-21swapper/108:05:201
111591140,0ptp4l0-21swapper/107:55:041
111591140,0ptp4l0-21swapper/107:40:221
111591140,0ptp4l0-21swapper/107:30:271
111591140,0ptp4l0-21swapper/107:25:171
111591140,0ptp4l0-21swapper/107:20:251
111591140,0ptp4l0-21swapper/107:15:191
111591140,0ptp4l0-21swapper/012:31:100
111591140,0ptp4l0-21swapper/012:10:280
111591140,0ptp4l0-21swapper/012:05:330
111591140,0ptp4l0-21swapper/011:45:230
111591140,0ptp4l0-21swapper/011:35:170
111591140,0ptp4l0-21swapper/011:25:160
111591140,0ptp4l0-21swapper/011:20:300
111591140,0ptp4l0-21swapper/011:10:260
111591140,0ptp4l0-21swapper/010:55:260
111591140,0ptp4l0-21swapper/010:46:190
111591140,0ptp4l0-21swapper/010:35:160
111591140,0ptp4l0-21swapper/010:20:170
111591140,0ptp4l0-21swapper/010:00:180
111591140,0ptp4l0-21swapper/009:50:180
111591140,0ptp4l0-21swapper/009:45:280
111591140,0ptp4l0-21swapper/009:30:210
111591140,0ptp4l0-21swapper/009:25:150
111591140,0ptp4l0-21swapper/009:15:260
111591140,0ptp4l0-21swapper/008:45:170
111591140,0ptp4l0-21swapper/008:35:170
111591140,0ptp4l0-21swapper/008:30:040
111591140,0ptp4l0-21swapper/008:25:260
111591140,0ptp4l0-21swapper/008:20:150
111591140,0ptp4l0-21swapper/008:05:240
111591140,0ptp4l0-21swapper/007:50:160
111591140,0ptp4l0-21swapper/007:45:310
111591140,0ptp4l0-21swapper/007:25:250
111591140,0ptp4l0-21swapper/007:20:220
111591140,0ptp4l0-21swapper/007:15:310
111591130,0ptp4l9975-21cstates11:15:152
111591130,0ptp4l0-21swapper/212:15:222
111591130,0ptp4l0-21swapper/212:05:312
111591130,0ptp4l0-21swapper/211:55:242
111591130,0ptp4l0-21swapper/211:50:222
111591130,0ptp4l0-21swapper/211:10:222
111591130,0ptp4l0-21swapper/208:30:232
111591130,0ptp4l0-21swapper/207:35:222
111591130,0ptp4l0-21swapper/207:35:212
111591130,0ptp4l0-21swapper/112:05:221
111591130,0ptp4l0-21swapper/111:40:201
111591130,0ptp4l0-21swapper/108:55:001
111591130,0ptp4l0-21swapper/108:40:201
111591130,0ptp4l0-21swapper/107:35:021
111591130,0ptp4l0-21swapper/107:35:011
111591130,0ptp4l0-21swapper/012:35:180
111591130,0ptp4l0-21swapper/012:25:250
111591130,0ptp4l0-21swapper/011:40:180
111591130,0ptp4l0-21swapper/011:15:180
111591130,0ptp4l0-21swapper/010:30:160
111591130,0ptp4l0-21swapper/009:55:140
111591130,0ptp4l0-21swapper/009:40:200
111591130,0ptp4l0-21swapper/009:00:130
111591130,0ptp4l0-21swapper/008:40:170
111591130,0ptp4l0-21swapper/008:15:190
111591130,0ptp4l0-21swapper/008:10:160
111591130,0ptp4l0-21swapper/007:40:240
111591120,0ptp4l0-21swapper/209:55:282
111591120,0ptp4l0-21swapper/209:30:252
111591120,0ptp4l0-21swapper/207:20:332
1385099111,8cyclictest0-21swapper/307:35:173
1385099111,8cyclictest0-21swapper/307:35:173
1384299105,1cyclictest33-21ksoftirqd/212:40:012
111591100,2ptp4l0-21swapper/209:51:322
111591100,0ptp4l0-21swapper/210:36:572
111591100,0ptp4l0-21swapper/210:33:572
111591100,0ptp4l0-21swapper/207:25:292
138509996,2cyclictest17368-21basename11:30:133
138429990,8cyclictest17856-21cat12:35:012
11159190,0ptp4l0-21swapper/212:25:072
11159190,0ptp4l0-21swapper/212:14:292
11159190,0ptp4l0-21swapper/212:00:412
11159190,0ptp4l0-21swapper/211:35:092
11159190,0ptp4l0-21swapper/211:31:502
11159190,0ptp4l0-21swapper/211:27:482
11159190,0ptp4l0-21swapper/211:20:012
11159190,0ptp4l0-21swapper/211:08:542
11159190,0ptp4l0-21swapper/211:01:152
11159190,0ptp4l0-21swapper/210:50:072
11159190,0ptp4l0-21swapper/210:44:072
11159190,0ptp4l0-21swapper/210:29:172
11159190,0ptp4l0-21swapper/210:20:092
11159190,0ptp4l0-21swapper/210:17:382
11159190,0ptp4l0-21swapper/209:41:552
11159190,0ptp4l0-21swapper/209:23:122
11159190,0ptp4l0-21swapper/209:05:132
11159190,0ptp4l0-21swapper/209:03:482
11159190,0ptp4l0-21swapper/208:45:592
11159190,0ptp4l0-21swapper/208:40:332
11159190,0ptp4l0-21swapper/208:38:022
11159190,0ptp4l0-21swapper/208:20:362
11159190,0ptp4l0-21swapper/208:18:222
11159190,0ptp4l0-21swapper/208:11:502
11159190,0ptp4l0-21swapper/208:03:332
11159190,0ptp4l0-21swapper/207:59:092
11159190,0ptp4l0-21swapper/207:49:162
11159190,0ptp4l0-21swapper/207:31:582
138509986,1cyclictest9716-21tail10:10:203
138509986,1cyclictest9289-21chrt09:05:203
138509986,1cyclictest8164-21tr08:00:153
138509986,1cyclictest8025-21if_err_enp4s011:10:253
138509986,1cyclictest7971-21expr12:15:143
138509986,1cyclictest6753-21tr10:05:163
138509986,1cyclictest6736-21tail09:00:173
138509986,1cyclictest5510-21grep08:55:283
138509986,1cyclictest5419-21sed07:55:013
138509986,1cyclictest5317-21sh11:05:143
138509986,1cyclictest5169-21seq12:10:183
138509986,1cyclictest3540-21cut12:05:243
138509986,1cyclictest32440-21expr10:55:173
138509986,1cyclictest32125-21if_enp3s009:50:183
138509986,1cyclictest31210-21cut11:55:213
138509986,1cyclictest30259-21if_enp4s010:50:173
138509986,1cyclictest30255-21expr07:40:203
138509986,1cyclictest3012-21cat07:50:163
138509986,1cyclictest29429-21chrt11:50:473
138509986,1cyclictest29044-21tr09:45:203
138509986,1cyclictest28397-21chrt09:41:203
138509986,1cyclictest2818-21cat08:50:273
138509986,1cyclictest27959-21chrt08:39:063
138509986,1cyclictest26845-21cat10:45:203
138509986,1cyclictest25939-21sed10:40:373
138509986,1cyclictest25632-21awk11:45:173
138509986,1cyclictest25479-21sed07:30:193
138509986,1cyclictest245432sleep311:42:453
138509986,1cyclictest243762sleep307:26:083
138509986,1cyclictest24168-21cat09:35:163
138509986,1cyclictest24038-21taskset10:39:463
138509986,1cyclictest23257-21egrep08:30:173
138509986,1cyclictest22041-21tail08:25:253
138509986,1cyclictest21851-21tail09:30:183
138509986,1cyclictest2110-21chrt07:46:373
138509986,1cyclictest2042-21taskset09:55:163
138509986,1cyclictest20351-21mailstats09:25:253
138509986,1cyclictest20144-21cat11:35:153
138509986,1cyclictest20118-21cat07:20:153
138509986,1cyclictest192332sleep310:30:043
138509986,1cyclictest18544-21expr12:35:193
138509986,1cyclictest18300-21tr08:20:123
138509986,1cyclictest17800-21grep09:20:313
138509986,1cyclictest17694-21run-parts08:17:003
138509986,1cyclictest17692-21sed07:15:213
138509986,1cyclictest16880-21cut10:25:163
138509986,1cyclictest15797-21expr12:30:153
138509986,1cyclictest14843-21seq11:25:143
138509986,1cyclictest14744-21tail10:20:173
138509986,1cyclictest14503-21cut12:25:353
138509986,1cyclictest14212-21expr07:10:153
138509986,1cyclictest13950-21expr09:15:163
138509986,1cyclictest13308-21seq08:10:173
138509986,1cyclictest1329-21uname12:00:203
138509986,1cyclictest12798-21sed10:15:243
138509986,1cyclictest12739-21cut09:10:233
138509986,1cyclictest12508-21cat11:20:173
138509986,1cyclictest11851-21taskset11:17:173
138509986,1cyclictest11222-21sed12:20:173
138429986,1cyclictest9287-21cut10:10:182
138429986,1cyclictest7274-21cat10:05:182
138429986,1cyclictest5592-21taskset10:00:242
138429986,1cyclictest5183-21cut08:55:252
138429986,1cyclictest30207-21cat07:40:202
138429986,1cyclictest2941-21expr07:50:162
138429986,1cyclictest29253-21expr09:45:202
138429986,1cyclictest25321-21sh11:45:142
138429986,1cyclictest20940-21ls08:25:142
138429986,1cyclictest19767-21tr09:25:212
138429986,1cyclictest1666-21taskset10:56:202
138429986,1cyclictest16535-21grep07:15:012
138429986,1cyclictest14693-21ps07:10:182
138429986,1cyclictest1216-21tr08:50:162
138429986,1cyclictest11602-21uname08:05:192
11159180,0ptp4l0-21swapper/212:21:142
11159180,0ptp4l0-21swapper/210:46:042
11159180,0ptp4l0-21swapper/209:35:392
11159180,0ptp4l0-21swapper/209:12:442
138509975,1cyclictest61642sleep310:02:173
138509975,1cyclictest3889-21expr11:00:383
138509975,1cyclictest31508-21expr08:45:183
138509975,1cyclictest11029-21uniq08:05:163
138509942,1cyclictest28644-21expr08:40:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional