You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-16 - 13:51

Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz, Linux 5.4.17-rt9 (Profile)

Latency plot of system in rack #1, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack1slot3.osadl.org (updated Tue May 26, 2020 00:45:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
149122550,2sleep09-21ksoftirqd/022:55:020
329525344,6sleep40-21swapper/419:06:074
353624938,7sleep10-21swapper/119:09:331
350324838,7sleep60-21swapper/619:09:066
348224837,7sleep30-21swapper/319:08:473
325024744,2sleep50-21swapper/519:05:285
355124536,6sleep70-21swapper/719:09:477
336024534,7sleep20-21swapper/219:07:022
347924334,6sleep00-21swapper/019:08:440
167542430,0sleep50-21swapper/500:05:075
211652400,0sleep10-21swapper/122:00:031
377999170,16cyclictest0-21swapper/522:30:095
377499160,16cyclictest0-21swapper/423:20:014
377999140,13cyclictest0-21swapper/500:15:025
376899130,12cyclictest0-21swapper/300:05:023
3763991311,1cyclictest0-21swapper/200:15:072
376399130,0cyclictest0-21swapper/221:10:022
3758991312,1cyclictest0-21swapper/119:35:031
378999120,0cyclictest0-21swapper/719:55:017
3774991212,0cyclictest0-21swapper/419:25:014
376399120,0cyclictest0-21swapper/200:00:022
375399120,0cyclictest0-21swapper/000:00:080
3768991111,0cyclictest0-21swapper/320:00:063
3763991111,0cyclictest0-21swapper/223:10:092
109662110,0sleep0101rcuc/019:25:000
3784991010,0cyclictest0-21swapper/619:25:056
3774991010,0cyclictest0-21swapper/422:00:014
3774991010,0cyclictest0-21swapper/421:45:024
3753991010,0cyclictest0-21swapper/000:20:000
37899998,1cyclictest0-21swapper/720:45:097
37849999,0cyclictest0-21swapper/600:10:086
37849998,1cyclictest56-21ksoftirqd/623:28:126
37689994,4cyclictest7753-21tr22:40:023
37689990,6cyclictest205371latency_hist22:00:003
37689990,0cyclictest0-21swapper/323:08:313
37689990,0cyclictest0-21swapper/322:17:503
37689990,0cyclictest0-21swapper/321:50:573
37689990,0cyclictest0-21swapper/300:34:563
37689990,0cyclictest0-21swapper/300:11:443
37589999,0cyclictest0-21swapper/120:50:041
37539999,0cyclictest0-21swapper/023:19:550
37539999,0cyclictest0-21swapper/021:14:160
37539990,9cyclictest0-21swapper/020:24:550
37899988,0cyclictest0-21swapper/722:39:187
37899988,0cyclictest0-21swapper/719:10:057
37899980,8cyclictest0-21swapper/722:58:047
37899980,8cyclictest0-21swapper/720:35:327
37899980,8cyclictest0-21swapper/720:35:327
37899980,0cyclictest0-21swapper/721:49:197
37849987,1cyclictest0-21swapper/623:55:026
37799988,0cyclictest0-21swapper/522:25:095
37799980,0cyclictest0-21swapper/519:45:175
37799980,0cyclictest0-21swapper/519:45:175
37749988,0cyclictest0-21swapper/422:35:114
37689988,0cyclictest0-21swapper/323:10:153
37689988,0cyclictest0-21swapper/320:25:103
37689984,3cyclictest28599-21cron23:25:003
37689983,4cyclictest1265-21lldpd20:40:023
37689980,8cyclictest0-21swapper/320:58:553
37689980,7cyclictest9943-21cpuspeed_turbos20:30:023
37689980,7cyclictest31025-21grep00:35:043
37689980,0cyclictest0-21swapper/322:45:023
37689980,0cyclictest0-21swapper/322:34:543
37689980,0cyclictest0-21swapper/319:40:063
37689980,0cyclictest0-21swapper/300:15:023
37689980,0cyclictest0-21swapper/300:01:063
37639980,8cyclictest0-21swapper/221:06:522
37639980,8cyclictest0-21swapper/220:08:502
37639980,7cyclictest0-21swapper/219:10:032
37639980,0cyclictest0-21swapper/221:50:132
37639980,0cyclictest0-21swapper/221:40:172
37589980,8cyclictest0-21swapper/122:29:051
37589980,8cyclictest0-21swapper/120:55:351
37539988,0cyclictest0-21swapper/022:29:570
37539988,0cyclictest0-21swapper/021:45:380
37539988,0cyclictest0-21swapper/021:37:170
37539988,0cyclictest0-21swapper/000:06:170
37539987,1cyclictest0-21swapper/019:29:060
37539980,8cyclictest0-21swapper/023:53:590
37539980,8cyclictest0-21swapper/023:21:210
37539980,8cyclictest0-21swapper/021:27:120
37539980,8cyclictest0-21swapper/021:27:120
37539980,0cyclictest0-21swapper/023:02:250
37539980,0cyclictest0-21swapper/020:06:280
37899977,0cyclictest0-21swapper/721:42:427
37899977,0cyclictest0-21swapper/721:00:087
37899977,0cyclictest0-21swapper/720:02:077
37899977,0cyclictest0-21swapper/719:37:497
37899977,0cyclictest0-21swapper/719:30:067
37899977,0cyclictest0-21swapper/719:28:327
37899977,0cyclictest0-21swapper/700:15:107
37899972,4cyclictest1569-21cat23:35:017
37899970,7cyclictest0-21swapper/723:50:447
37899970,7cyclictest0-21swapper/723:34:107
37899970,7cyclictest0-21swapper/721:59:257
37899970,7cyclictest0-21swapper/721:05:397
37899970,7cyclictest0-21swapper/720:58:147
37899970,6cyclictest337-21systemd-journal21:17:007
37899970,5cyclictest0-21swapper/723:47:527
37899970,0cyclictest0-21swapper/723:03:547
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional