You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-04 - 22:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Kontron

x86 Intel Core i7 7828EQ @3000 MHz, Linux 4.19.37-rt20 (Profile)

Latency plot of shadow in rack #9, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot3s.osadl.org (updated Sun Dec 04, 2022 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
134291640,44ptp4l0-21swapper/007:05:190
134291570,35ptp4l0-21swapper/107:05:181
2247424941,6sleep20-21swapper/207:05:572
2048424945,2sleep30-21swapper/307:05:023
134291280,0ptp4l0-21swapper/112:05:181
134291200,2ptp4l228-21systemd-journal11:30:010
134291200,1ptp4l9081-21if_err_enp3s009:50:191
134291190,0ptp4l0-21swapper/107:50:211
134291190,0ptp4l0-21swapper/008:45:220
134291180,0ptp4l0-21swapper/111:00:211
134291180,0ptp4l0-21swapper/110:05:301
134291180,0ptp4l0-21swapper/009:15:130
134291180,0ptp4l0-21swapper/008:05:210
134291170,1ptp4l228-21systemd-journal11:50:010
134291170,0ptp4l0-21swapper/011:00:190
134291170,0ptp4l0-21swapper/010:55:300
134291170,0ptp4l0-21swapper/009:35:200
134291170,0ptp4l0-21swapper/009:35:200
134291160,1ptp4l24428-21apt-get11:25:141
134291160,1ptp4l21760-21apt-get08:10:140
134291160,1ptp4l20912-21snmp_o5powerctl11:15:291
134291160,1ptp4l20082-21apt-get09:10:011
134291160,1ptp4l14974-21apt-get12:10:121
134291160,1ptp4l12280-21apt-get08:55:010
134291160,1ptp4l12242-21apt-get12:05:130
134291160,0ptp4l0-21swapper/110:31:571
134291160,0ptp4l0-21swapper/110:31:571
319092150,0sleep30-21swapper/308:30:013
134291150,1ptp4l3784-21munin-run07:35:020
134291150,1ptp4l368-21/usr/sbin/munin07:25:221
134291150,1ptp4l26880-21hddtemp_smartct07:15:150
134291150,1ptp4l23458-21snmp_o5powerctl11:20:270
134291150,1ptp4l20038-21snmp_o5powerctl10:10:391
134291150,1ptp4l17772-21/usr/sbin/munin09:05:221
134291150,1ptp4l13298-21hddtemp_smartct08:55:151
134291150,0ptp4l25917-21ls08:15:230
134291150,0ptp4l12772-21ls07:50:220
134291150,0ptp4l0-21swapper/112:20:011
134291150,0ptp4l0-21swapper/111:58:371
134291150,0ptp4l0-21swapper/110:20:011
134291150,0ptp4l0-21swapper/109:55:281
134291150,0ptp4l0-21swapper/109:40:171
134291150,0ptp4l0-21swapper/108:53:141
134291150,0ptp4l0-21swapper/108:25:141
134291150,0ptp4l0-21swapper/108:15:131
134291150,0ptp4l0-21swapper/108:00:151
134291150,0ptp4l0-21swapper/107:45:231
134291150,0ptp4l0-21swapper/107:40:131
134291150,0ptp4l0-21swapper/012:30:200
134291150,0ptp4l0-21swapper/012:25:010
134291150,0ptp4l0-21swapper/011:15:290
134291150,0ptp4l0-21swapper/010:00:200
134291150,0ptp4l0-21swapper/009:20:270
134291150,0ptp4l0-21swapper/009:10:170
134291150,0ptp4l0-21swapper/007:45:130
134291140,1ptp4l8104-21pmu-power10:50:271
134291140,1ptp4l31513-21/usr/sbin/munin10:35:220
134291140,1ptp4l3111-21irqstats11:45:191
134291140,1ptp4l29207-21cstates07:20:151
134291140,1ptp4l23129-21df12:25:161
134291140,1ptp4l16265-21irqstats09:00:211
134291140,0ptp4l22158-21ls10:15:221
134291140,0ptp4l0-21swapper/211:40:242
134291140,0ptp4l0-21swapper/112:35:151
134291140,0ptp4l0-21swapper/112:30:201
134291140,0ptp4l0-21swapper/112:15:311
134291140,0ptp4l0-21swapper/112:00:151
134291140,0ptp4l0-21swapper/111:52:431
134291140,0ptp4l0-21swapper/111:40:241
134291140,0ptp4l0-21swapper/111:30:171
134291140,0ptp4l0-21swapper/111:20:171
134291140,0ptp4l0-21swapper/111:10:241
134291140,0ptp4l0-21swapper/110:55:291
134291140,0ptp4l0-21swapper/110:45:261
134291140,0ptp4l0-21swapper/110:40:251
134291140,0ptp4l0-21swapper/110:00:271
134291140,0ptp4l0-21swapper/109:45:211
134291140,0ptp4l0-21swapper/109:35:261
134291140,0ptp4l0-21swapper/109:35:251
134291140,0ptp4l0-21swapper/109:25:131
134291140,0ptp4l0-21swapper/109:15:221
134291140,0ptp4l0-21swapper/108:45:241
134291140,0ptp4l0-21swapper/108:35:231
134291140,0ptp4l0-21swapper/108:30:251
134291140,0ptp4l0-21swapper/108:10:141
134291140,0ptp4l0-21swapper/108:05:241
134291140,0ptp4l0-21swapper/107:55:181
134291140,0ptp4l0-21swapper/107:35:141
134291140,0ptp4l0-21swapper/107:15:011
134291140,0ptp4l0-21swapper/107:10:181
134291140,0ptp4l0-21swapper/012:20:250
134291140,0ptp4l0-21swapper/012:15:250
134291140,0ptp4l0-21swapper/012:10:020
134291140,0ptp4l0-21swapper/012:00:240
134291140,0ptp4l0-21swapper/011:55:230
134291140,0ptp4l0-21swapper/011:30:260
134291140,0ptp4l0-21swapper/011:10:260
134291140,0ptp4l0-21swapper/010:50:240
134291140,0ptp4l0-21swapper/010:45:230
134291140,0ptp4l0-21swapper/010:40:250
134291140,0ptp4l0-21swapper/010:30:240
134291140,0ptp4l0-21swapper/010:30:230
134291140,0ptp4l0-21swapper/010:25:200
134291140,0ptp4l0-21swapper/010:20:150
134291140,0ptp4l0-21swapper/010:10:300
134291140,0ptp4l0-21swapper/010:05:210
134291140,0ptp4l0-21swapper/009:50:160
134291140,0ptp4l0-21swapper/009:45:250
134291140,0ptp4l0-21swapper/009:40:210
134291140,0ptp4l0-21swapper/009:05:120
134291140,0ptp4l0-21swapper/009:00:210
134291140,0ptp4l0-21swapper/008:50:150
134291140,0ptp4l0-21swapper/008:40:250
134291140,0ptp4l0-21swapper/008:35:230
134291140,0ptp4l0-21swapper/008:30:270
134291140,0ptp4l0-21swapper/008:00:190
134291140,0ptp4l0-21swapper/007:55:170
134291140,0ptp4l0-21swapper/007:35:170
134291140,0ptp4l0-21swapper/007:30:010
134291130,1ptp4l29384-21snmp_o5powerctl12:35:262
134291130,1ptp4l1666-21ls07:30:171
134291130,0ptp4l0-21swapper/211:35:262
134291130,0ptp4l0-21swapper/210:50:182
134291130,0ptp4l0-21swapper/208:00:242
134291130,0ptp4l0-21swapper/207:35:192
134291130,0ptp4l0-21swapper/111:35:011
134291130,0ptp4l0-21swapper/111:05:161
134291130,0ptp4l0-21swapper/110:35:011
134291130,0ptp4l0-21swapper/110:25:171
134291130,0ptp4l0-21swapper/109:30:161
134291130,0ptp4l0-21swapper/109:20:021
134291130,0ptp4l0-21swapper/108:40:141
134291130,0ptp4l0-21swapper/108:20:011
134291130,0ptp4l0-21swapper/012:35:180
134291130,0ptp4l0-21swapper/011:45:130
134291130,0ptp4l0-21swapper/011:40:130
134291130,0ptp4l0-21swapper/011:35:170
134291130,0ptp4l0-21swapper/011:05:170
134291130,0ptp4l0-21swapper/010:15:010
134291130,0ptp4l0-21swapper/009:55:200
134291130,0ptp4l0-21swapper/009:30:190
134291130,0ptp4l0-21swapper/009:25:180
134291130,0ptp4l0-21swapper/008:25:020
134291130,0ptp4l0-21swapper/008:20:170
134291130,0ptp4l0-21swapper/007:40:150
134291130,0ptp4l0-21swapper/007:20:170
134291130,0ptp4l0-21swapper/007:10:180
134291120,1ptp4l15932-21/usr/sbin/munin10:05:152
134291120,0ptp4l0-21swapper/209:10:152
134291120,0ptp4l0-21swapper/208:25:152
134291110,2ptp4l41-21ksoftirqd/308:30:423
134291110,2ptp4l0-21swapper/210:31:032
134291110,2ptp4l0-21swapper/210:31:032
134291110,2ptp4l0-21swapper/209:52:322
134291110,0ptp4l0-21swapper/210:46:042
134291110,0ptp4l0-21swapper/209:41:112
134291110,0ptp4l0-21swapper/208:20:242
134291110,0ptp4l0-21swapper/208:10:172
134291110,0ptp4l0-21swapper/207:25:172
134291100,1ptp4l0-21swapper/209:04:202
134291100,0ptp4l12774-21kworker/2:111:05:502
134291100,0ptp4l0-21swapper/212:34:392
134291100,0ptp4l0-21swapper/211:15:312
134291100,0ptp4l0-21swapper/211:00:352
134291100,0ptp4l0-21swapper/210:25:402
134291100,0ptp4l0-21swapper/207:20:022
230499997,1cyclictest9296-21taskset08:47:033
230499997,1cyclictest6845-21cut10:50:193
230499997,1cyclictest31707-21chrt11:39:383
230499997,1cyclictest30127-21tr12:40:013
230499997,1cyclictest25131-21taskset09:17:203
230499997,1cyclictest23871-21taskset11:21:093
230499997,1cyclictest22508-21chrt12:24:493
230499997,1cyclictest17997-21cut11:10:213
230499997,1cyclictest142132sleep312:06:163
230499997,1cyclictest13877-21cron07:55:023
230499996,2cyclictest10382-21cat10:55:283
230499990,8cyclictest29708-21sed10:30:283
230499990,8cyclictest29708-21sed10:30:283
230499990,8cyclictest13146-21taskset09:59:583
230449997,1cyclictest6206-21snmp_o5powerctl11:50:312
230449997,1cyclictest24776-21df11:25:142
230449997,1cyclictest11042-21chrt12:00:232
230449991,4cyclictest0-21swapper/210:40:152
13429190,0ptp4l0-21swapper/312:11:083
13429190,0ptp4l0-21swapper/312:04:553
13429190,0ptp4l0-21swapper/212:26:012
13429190,0ptp4l0-21swapper/212:20:562
13429190,0ptp4l0-21swapper/212:16:462
13429190,0ptp4l0-21swapper/212:10:272
13429190,0ptp4l0-21swapper/212:05:332
13429190,0ptp4l0-21swapper/211:55:382
13429190,0ptp4l0-21swapper/211:46:002
13429190,0ptp4l0-21swapper/211:30:092
13429190,0ptp4l0-21swapper/211:20:142
13429190,0ptp4l0-21swapper/211:12:282
13429190,0ptp4l0-21swapper/210:57:592
13429190,0ptp4l0-21swapper/210:39:312
13429190,0ptp4l0-21swapper/210:20:302
13429190,0ptp4l0-21swapper/210:15:112
13429190,0ptp4l0-21swapper/210:10:432
13429190,0ptp4l0-21swapper/210:00:052
13429190,0ptp4l0-21swapper/209:55:392
13429190,0ptp4l0-21swapper/209:45:132
13429190,0ptp4l0-21swapper/209:35:392
13429190,0ptp4l0-21swapper/209:35:382
13429190,0ptp4l0-21swapper/209:30:402
13429190,0ptp4l0-21swapper/209:25:392
13429190,0ptp4l0-21swapper/209:20:022
13429190,0ptp4l0-21swapper/209:15:562
13429190,0ptp4l0-21swapper/209:07:232
13429190,0ptp4l0-21swapper/208:56:552
13429190,0ptp4l0-21swapper/208:50:052
13429190,0ptp4l0-21swapper/208:46:312
13429190,0ptp4l0-21swapper/208:43:352
13429190,0ptp4l0-21swapper/208:35:522
13429190,0ptp4l0-21swapper/208:31:172
13429190,0ptp4l0-21swapper/208:16:462
13429190,0ptp4l0-21swapper/208:05:082
13429190,0ptp4l0-21swapper/207:55:282
13429190,0ptp4l0-21swapper/207:52:202
13429190,0ptp4l0-21swapper/207:45:072
13429190,0ptp4l0-21swapper/207:42:162
13429190,0ptp4l0-21swapper/207:30:492
13429190,0ptp4l0-21swapper/207:15:412
13429190,0ptp4l0-21swapper/207:12:072
230499986,1cyclictest9358-21expr07:45:133
230499986,1cyclictest8404-21grep09:50:153
230499986,1cyclictest7381-21tr07:40:173
230499986,1cyclictest7358-21expr11:55:213
230499986,1cyclictest6001-21cat09:45:143
230499986,1cyclictest5317-21if_enp4s011:50:243
230499986,1cyclictest5249-21ntpq07:35:183
230499986,1cyclictest4989-21expr08:40:153
230499986,1cyclictest4337-21cut10:45:143
230499986,1cyclictest3952-21cat09:40:183
230499986,1cyclictest32165-21grep11:40:133
230499986,1cyclictest32130-21cat07:25:173
230499986,1cyclictest31459-21cut09:30:223
230499986,1cyclictest31414-21sed10:35:023
230499986,1cyclictest30154-21taskset09:26:483
230499986,1cyclictest29519-21cat07:20:183
230499986,1cyclictest2795-21chrt09:36:553
230499986,1cyclictest2795-21chrt09:36:553
230499986,1cyclictest27373-21cat10:25:253
230499986,1cyclictest27256-21expr11:30:183
230499986,1cyclictest26012-21fschecks_count09:20:163
230499986,1cyclictest25851-21cut07:15:023
230499986,1cyclictest25690-21grep12:30:233
230499986,1cyclictest2545-21df08:35:163
230499986,1cyclictest24693-21cut10:20:203
230499986,1cyclictest24581-21cut11:25:133
230499986,1cyclictest23065-21acpi07:10:123
230499986,1cyclictest2301-21cat11:45:143
230499986,1cyclictest22900-21cut12:25:153
230499986,1cyclictest22318-21hddtemp_smartct08:10:173
230499986,1cyclictest20484-21sed09:10:023
230499986,1cyclictest19688-21expr11:15:203
230499986,1cyclictest19321-21grep08:05:143
230499986,1cyclictest19140-21sed09:05:203
230499986,1cyclictest19035-21grep10:10:263
230499986,1cyclictest1816-21tr10:40:153
230499986,1cyclictest17863-21cat12:15:203
230499986,1cyclictest17774-21wc08:00:203
230499986,1cyclictest17505-21snmp_o5powerctl10:05:293
230499986,1cyclictest15539-21ntp_kernel_pll_11:05:233
230499986,1cyclictest1450-21chrt07:30:093
230499986,1cyclictest14133-21dpkg07:55:143
230499986,1cyclictest13692-21expr10:00:213
230499986,1cyclictest13152-21grep08:55:153
230499986,1cyclictest11743-21taskset11:00:063
230499976,1cyclictest10337-21df08:50:153
230499975,1cyclictest25313-21cat08:15:183
230499975,1cyclictest21233-21cat10:15:143
230499975,1cyclictest15637-21entropy09:00:173
230499964,1cyclictest27146-21expr08:20:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional