You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-06-13 - 01:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Kontron

Intel(R) Core(TM) i7-7820EQ CPU @ 3.00GHz, Linux 4.19.37-rt20 (Profile)

Latency plot of shadow in rack #9, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack9slot3s.osadl.org (updated Sat Jun 12, 2021 12:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
134391680,48ptp4l0-21swapper/007:06:500
134391600,40ptp4l0-21swapper/107:09:211
1713825346,5sleep20-21swapper/207:06:192
1731025046,2sleep30-21swapper/307:08:333
134391320,0ptp4l0-21swapper/110:35:541
134391310,0ptp4l0-21swapper/008:10:300
134391300,0ptp4l0-21swapper/110:06:051
134391200,0ptp4l0-21swapper/007:10:390
134391190,0ptp4l0-21swapper/108:55:351
134391190,0ptp4l0-21swapper/012:15:330
134391190,0ptp4l0-21swapper/007:55:340
134391180,0ptp4l0-21swapper/011:20:510
134391170,0ptp4l0-21swapper/012:00:320
134391160,1ptp4l6778-21apt-get08:55:260
134391160,1ptp4l5422-21apt-get11:00:261
134391160,1ptp4l29705-21apt-get09:40:321
134391160,1ptp4l29107-21apt-get08:35:270
134391160,0ptp4l0-21swapper/107:35:401
134391160,0ptp4l0-21swapper/011:50:350
134391160,0ptp4l0-21swapper/007:20:260
134391150,1ptp4l7252-21ntp_states12:05:331
134391150,1ptp4l5807-21snmp_o5powerctl08:50:400
134391150,1ptp4l3287-21snmp_o5powerctl08:45:360
134391150,1ptp4l31923-21cstates08:40:260
134391150,1ptp4l31815-21snmp_o5powerctl10:45:401
134391150,1ptp4l30651-21snmp_o5powerctl08:35:371
134391150,1ptp4l28280-21perf07:30:000
134391150,1ptp4l23667-21snmp_o5powerctl09:25:351
134391150,1ptp4l14731-21snmp_o5powerctl11:15:451
134391150,0ptp4l13572-21snmp_o5powerctl09:05:370
134391150,0ptp4l0-21swapper/211:50:342
134391150,0ptp4l0-21swapper/209:35:312
134391150,0ptp4l0-21swapper/111:05:111
134391150,0ptp4l0-21swapper/110:30:031
134391150,0ptp4l0-21swapper/109:50:271
134391150,0ptp4l0-21swapper/109:15:261
134391150,0ptp4l0-21swapper/011:10:250
134391150,0ptp4l0-21swapper/009:20:410
134391150,0ptp4l0-21swapper/008:02:480
1762699145,3cyclictest0-21swapper/210:35:332
134391140,1ptp4l6558-21df07:50:301
134391140,1ptp4l2862-21/usr/sbin/munin10:55:380
134391140,1ptp4l2628-21munin-run10:55:000
134391140,1ptp4l23680-21swap09:25:360
134391140,1ptp4l22859-21users07:15:421
134391140,1ptp4l22251-21missed_timers12:35:311
134391140,1ptp4l2099-21diskstats08:45:281
134391140,1ptp4l19047-21snmp_o5powerctl10:21:010
134391140,1ptp4l16654-21hddtemp_smartct12:25:300
134391140,1ptp4l1583-21if_enp3s011:55:280
134391140,1ptp4l15413-21processes08:05:392
134391140,1ptp4l1378-21snmp_o5powerctl09:46:071
134391140,1ptp4l13695-21cpuspeed_turbos12:20:250
134391140,1ptp4l0-21swapper/211:15:282
134391140,1ptp4l0-21swapper/107:55:381
134391140,0ptp4l8520-21apt-get12:10:300
134391140,0ptp4l29302-21python10:40:481
134391140,0ptp4l0-21swapper/208:30:272
134391140,0ptp4l0-21swapper/208:10:372
134391140,0ptp4l0-21swapper/207:40:372
134391140,0ptp4l0-21swapper/112:30:351
134391140,0ptp4l0-21swapper/112:15:351
134391140,0ptp4l0-21swapper/112:00:351
134391140,0ptp4l0-21swapper/111:40:391
134391140,0ptp4l0-21swapper/110:50:411
134391140,0ptp4l0-21swapper/110:21:001
134391140,0ptp4l0-21swapper/110:15:541
134391140,0ptp4l0-21swapper/110:00:251
134391140,0ptp4l0-21swapper/109:55:591
134391140,0ptp4l0-21swapper/109:35:231
134391140,0ptp4l0-21swapper/109:24:101
134391140,0ptp4l0-21swapper/109:00:011
134391140,0ptp4l0-21swapper/108:40:291
134391140,0ptp4l0-21swapper/108:30:331
134391140,0ptp4l0-21swapper/108:25:321
134391140,0ptp4l0-21swapper/107:30:291
134391140,0ptp4l0-21swapper/107:20:251
134391140,0ptp4l0-21swapper/107:10:311
134391140,0ptp4l0-21swapper/012:35:300
134391140,0ptp4l0-21swapper/011:40:330
134391140,0ptp4l0-21swapper/011:30:370
134391140,0ptp4l0-21swapper/011:15:470
134391140,0ptp4l0-21swapper/011:05:010
134391140,0ptp4l0-21swapper/010:35:430
134391140,0ptp4l0-21swapper/010:15:550
134391140,0ptp4l0-21swapper/010:05:340
134391140,0ptp4l0-21swapper/010:00:310
134391140,0ptp4l0-21swapper/009:40:350
134391140,0ptp4l0-21swapper/009:40:000
134391140,0ptp4l0-21swapper/007:50:290
134391140,0ptp4l0-21swapper/007:30:290
134391130,0ptp4l20213-21ntp_states07:15:010
134391130,0ptp4l12961-21irqrtprio09:05:321
134391130,0ptp4l0-21swapper/212:05:262
134391130,0ptp4l0-21swapper/211:55:272
134391130,0ptp4l0-21swapper/211:40:342
134391130,0ptp4l0-21swapper/211:00:292
134391130,0ptp4l0-21swapper/210:30:472
134391130,0ptp4l0-21swapper/210:25:392
134391130,0ptp4l0-21swapper/210:10:382
134391130,0ptp4l0-21swapper/210:10:382
134391130,0ptp4l0-21swapper/207:53:482
134391130,0ptp4l0-21swapper/207:10:402
134391130,0ptp4l0-21swapper/112:25:291
134391130,0ptp4l0-21swapper/112:20:251
134391130,0ptp4l0-21swapper/112:10:311
134391130,0ptp4l0-21swapper/111:55:291
134391130,0ptp4l0-21swapper/111:50:281
134391130,0ptp4l0-21swapper/111:45:271
134391130,0ptp4l0-21swapper/111:35:301
134391130,0ptp4l0-21swapper/111:30:271
134391130,0ptp4l0-21swapper/111:25:291
134391130,0ptp4l0-21swapper/111:20:311
134391130,0ptp4l0-21swapper/111:10:261
134391130,0ptp4l0-21swapper/110:55:501
134391130,0ptp4l0-21swapper/110:25:271
134391130,0ptp4l0-21swapper/110:10:341
134391130,0ptp4l0-21swapper/110:10:341
134391130,0ptp4l0-21swapper/109:30:311
134391130,0ptp4l0-21swapper/109:10:291
134391130,0ptp4l0-21swapper/108:50:271
134391130,0ptp4l0-21swapper/108:20:251
134391130,0ptp4l0-21swapper/108:15:321
134391130,0ptp4l0-21swapper/108:10:261
134391130,0ptp4l0-21swapper/108:05:281
134391130,0ptp4l0-21swapper/108:00:011
134391130,0ptp4l0-21swapper/107:45:301
134391130,0ptp4l0-21swapper/107:40:391
134391130,0ptp4l0-21swapper/107:25:311
134391130,0ptp4l0-21swapper/012:30:260
134391130,0ptp4l0-21swapper/012:05:280
134391130,0ptp4l0-21swapper/011:45:240
134391130,0ptp4l0-21swapper/011:35:270
134391130,0ptp4l0-21swapper/011:25:310
134391130,0ptp4l0-21swapper/011:00:270
134391130,0ptp4l0-21swapper/010:45:290
134391130,0ptp4l0-21swapper/010:40:290
134391130,0ptp4l0-21swapper/010:30:360
134391130,0ptp4l0-21swapper/010:25:270
134391130,0ptp4l0-21swapper/010:10:400
134391130,0ptp4l0-21swapper/010:10:400
134391130,0ptp4l0-21swapper/009:55:270
134391130,0ptp4l0-21swapper/009:50:290
134391130,0ptp4l0-21swapper/009:46:040
134391130,0ptp4l0-21swapper/009:30:330
134391130,0ptp4l0-21swapper/009:15:290
134391130,0ptp4l0-21swapper/009:10:270
134391130,0ptp4l0-21swapper/009:00:300
134391130,0ptp4l0-21swapper/008:30:000
134391130,0ptp4l0-21swapper/008:25:290
134391130,0ptp4l0-21swapper/008:20:270
134391130,0ptp4l0-21swapper/008:15:270
134391130,0ptp4l0-21swapper/008:05:320
134391130,0ptp4l0-21swapper/007:45:310
134391130,0ptp4l0-21swapper/007:40:300
134391130,0ptp4l0-21swapper/007:35:370
1763399120,1cyclictest16849-21diskstats08:10:273
134391120,1ptp4l0-21swapper/210:01:592
134391120,0ptp4l0-21swapper/211:30:322
134391120,0ptp4l0-21swapper/211:20:392
134391120,0ptp4l0-21swapper/209:25:002
134391110,2ptp4l0-21swapper/209:54:232
134391110,1ptp4l751-21chrt07:38:332
134391100,0ptp4l0-21swapper/211:25:392
134391100,0ptp4l0-21swapper/211:05:412
134391100,0ptp4l0-21swapper/210:06:012
134391100,0ptp4l0-21swapper/209:00:132
134391100,0ptp4l0-21swapper/207:48:512
176339990,8cyclictest15545-21cut09:10:313
13439190,0ptp4l0-21swapper/308:30:473
13439190,0ptp4l0-21swapper/212:35:572
13439190,0ptp4l0-21swapper/212:31:392
13439190,0ptp4l0-21swapper/212:26:352
13439190,0ptp4l0-21swapper/212:19:562
13439190,0ptp4l0-21swapper/212:13:312
13439190,0ptp4l0-21swapper/212:01:312
13439190,0ptp4l0-21swapper/211:45:112
13439190,0ptp4l0-21swapper/211:39:232
13439190,0ptp4l0-21swapper/211:13:372
13439190,0ptp4l0-21swapper/210:55:192
13439190,0ptp4l0-21swapper/210:50:432
13439190,0ptp4l0-21swapper/210:40:082
13439190,0ptp4l0-21swapper/210:20:212
13439190,0ptp4l0-21swapper/210:15:062
13439190,0ptp4l0-21swapper/209:55:032
13439190,0ptp4l0-21swapper/209:45:252
13439190,0ptp4l0-21swapper/209:40:252
13439190,0ptp4l0-21swapper/209:31:412
13439190,0ptp4l0-21swapper/209:21:202
13439190,0ptp4l0-21swapper/209:17:332
13439190,0ptp4l0-21swapper/209:10:512
13439190,0ptp4l0-21swapper/209:06:102
13439190,0ptp4l0-21swapper/208:57:062
13439190,0ptp4l0-21swapper/208:51:032
13439190,0ptp4l0-21swapper/208:46:172
13439190,0ptp4l0-21swapper/208:41:432
13439190,0ptp4l0-21swapper/208:35:192
13439190,0ptp4l0-21swapper/208:25:152
13439190,0ptp4l0-21swapper/208:21:072
13439190,0ptp4l0-21swapper/208:15:212
13439190,0ptp4l0-21swapper/208:01:372
13439190,0ptp4l0-21swapper/207:58:292
13439190,0ptp4l0-21swapper/207:31:592
13439190,0ptp4l0-21swapper/207:23:092
13439190,0ptp4l0-21swapper/207:16:072
176339986,1cyclictest998-21ls11:55:253
176339986,1cyclictest9221-21grep07:55:313
176339986,1cyclictest8988-21fw_packets12:10:333
176339986,1cyclictest894-21date08:40:363
176339986,1cyclictest8264-21tr11:05:263
176339986,1cyclictest8009-21if_enp2s010:00:293
176339986,1cyclictest7138-21cat12:05:323
176339986,1cyclictest7074-21expr08:55:273
176339986,1cyclictest6356-21cut07:50:303
176339986,1cyclictest593-21seq07:36:353
176339986,1cyclictest5425-21taskset11:00:263
176339986,1cyclictest5209-21fschecks_count09:55:313
176339986,1cyclictest5206-21which07:45:393
176339986,1cyclictest4619-21date09:55:003
176339986,1cyclictest3921-21cut12:00:283
176339986,1cyclictest3455-21sh10:55:353
176339986,1cyclictest32231-21egrep09:45:363
176339986,1cyclictest31685-21chrt09:41:453
176339986,1cyclictest31217-21cut11:50:273
176339986,1cyclictest30236-21chrt11:45:473
176339986,1cyclictest29234-21cut08:35:283
176339986,1cyclictest28528-21bc07:30:293
176339986,1cyclictest28519-21cut09:35:403
176339986,1cyclictest27981-21expr10:40:303
176339986,1cyclictest2731-21snmp_o5powerctl07:40:423
176339986,1cyclictest27244-21taskset10:37:163
176339986,1cyclictest26006-21expr11:40:293
176339986,1cyclictest26000-21cut09:30:433
176339986,1cyclictest24126-21grep08:25:283
176339986,1cyclictest23850-21df_inode07:20:263
176339986,1cyclictest23824-21sed11:35:293
176339986,1cyclictest23176-21tune2fs10:30:383
176339986,1cyclictest22466-21cut08:20:293
176339986,1cyclictest22195-21cat09:25:253
176339986,1cyclictest21414-21if_enp4s011:30:283
176339986,1cyclictest21057-21tr07:15:273
176339986,1cyclictest20543-21cut09:20:303
176339986,1cyclictest20337-21df_inode10:25:273
176339986,1cyclictest2012-21expr08:45:273
176339986,1cyclictest19795-21taskset07:12:313
176339986,1cyclictest19308-21expr08:15:273
176339986,1cyclictest19230-21cat12:30:293
176339986,1cyclictest19038-21taskset10:21:003
176339986,1cyclictest18574-21mailstats09:15:403
176339986,1cyclictest18189-21taskset11:25:083
176339986,1cyclictest17644-21snmp_o5powerctl12:25:383
176339986,1cyclictest16911-21cut11:20:413
176339986,1cyclictest15652-21grep08:05:423
176339986,1cyclictest14720-21smart_sda11:15:433
176339986,1cyclictest14615-21sed12:20:303
176339986,1cyclictest13339-21phc2sys-jitter09:05:353
176339986,1cyclictest13106-21sed08:00:373
176339986,1cyclictest12645-21expr10:10:343
176339986,1cyclictest12645-21expr10:10:343
176339986,1cyclictest11689-21if_enp3s012:15:293
176339986,1cyclictest11505-21chrt09:01:493
176339986,1cyclictest1143-21cut10:50:353
176339986,1cyclictest10941-21expr11:10:263
176339981,3cyclictest0-21swapper/310:45:263
13439180,0ptp4l0-21swapper/212:21:152
13439180,0ptp4l0-21swapper/210:45:142
13439180,0ptp4l0-21swapper/207:25:272
176339975,1cyclictest4650-21grep08:50:293
176339975,1cyclictest26756-21taskset07:25:353
176339975,1cyclictest22061-21cut12:35:303
176339975,1cyclictest15071-21tr10:15:293
176339975,1cyclictest10739-21cut10:05:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional