You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-06-27 - 05:33

x86 Intel N455 @1660 MHz, Linux 5.4.34-rt21 (Profile)

Latency plot of system in rack #3, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6 (updated Mon Jun 27, 2022 00:45:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2400021580,7sleep1127499cyclictest21:50:161
2419421050,14sleep1127499cyclictest23:25:221
982210125,16sleep00-21swapper/019:08:170
248312980,15sleep0126399cyclictest20:15:480
248312980,15sleep0126399cyclictest20:15:470
1263998858,20cyclictest9-21ksoftirqd/022:00:180
79728523,49sleep10-21swapper/119:06:151
228342840,4sleep1201rcuc/121:45:301
1263998354,14cyclictest0-21swapper/022:55:250
1263998352,16cyclictest9-21ksoftirqd/022:50:110
14522820,6sleep00-21swapper/019:10:150
1263998052,18cyclictest9-21ksoftirqd/021:15:120
1263997952,18cyclictest9-21ksoftirqd/020:30:120
1263997947,18cyclictest0-21swapper/020:00:140
1263997848,17cyclictest0-21swapper/000:10:130
1263997748,17cyclictest0-21swapper/021:05:150
1263997646,16cyclictest0-21swapper/019:55:140
1263997447,15cyclictest9-21ksoftirqd/020:40:110
1263997346,15cyclictest0-21swapper/019:50:130
1263997345,17cyclictest0-21swapper/000:25:150
1263997345,15cyclictest0-21swapper/019:40:100
127499728,53cyclictest25756-21apt-get21:55:271
1263997244,16cyclictest0-21swapper/022:15:240
1263997243,17cyclictest0-21swapper/023:10:120
1263997150,10cyclictest0-21swapper/021:20:120
1263997142,18cyclictest0-21swapper/022:10:240
1263997040,16cyclictest9-21ksoftirqd/000:35:120
1263997040,15cyclictest0-21swapper/019:30:140
1263997036,26cyclictest0-21swapper/000:15:110
1274996939,13cyclictest21-21ksoftirqd/121:35:011
1274996937,10cyclictest21-21ksoftirqd/119:40:001
1263996941,16cyclictest0-21swapper/000:00:140
1274996838,13cyclictest21-21ksoftirqd/100:00:001
1263996841,15cyclictest0-21swapper/023:05:160
1263996839,13cyclictest0-21swapper/022:30:150
1263996739,19cyclictest0-21swapper/021:55:250
1263996714,40cyclictest0-21swapper/023:35:110
127499663,10cyclictest20340-21munin-run21:40:001
126399665,10cyclictest9-21ksoftirqd/000:25:000
1263996638,15cyclictest0-21swapper/000:05:140
1274996536,12cyclictest21-21ksoftirqd/119:35:011
1274996535,13cyclictest21-21ksoftirqd/123:40:001
1263996537,16cyclictest0-21swapper/019:25:130
127499644,10cyclictest18961-21munin-run23:10:001
1263996439,13cyclictest0-21swapper/023:20:140
1263996436,16cyclictest0-21swapper/020:45:170
1263996435,18cyclictest414-21dbus-daemon22:30:000
1263996435,17cyclictest0-21swapper/023:40:150
1263996420,34cyclictest0-21swapper/023:25:200
127499635,10cyclictest22008-21munin-run21:45:001
127499632,10cyclictest10152-21grep00:20:001
1263996340,12cyclictest0-21swapper/021:10:110
1263996337,18cyclictest1253-21cyclictest19:35:140
1263996337,17cyclictest3464-21tr00:00:010
1263996337,14cyclictest0-21swapper/021:30:130
1263996334,22cyclictest1253-21cyclictest23:15:120
296572620,15chrt127499cyclictest20:30:341
127499625,10cyclictest13468-21munin-run00:30:001
1274996236,8cyclictest21-21ksoftirqd/121:17:121
127499623,43cyclictest21-21ksoftirqd/119:45:121
1274996228,10cyclictest21-21ksoftirqd/123:00:181
127499622,11cyclictest14093-21runrttasks22:55:051
126399629,10cyclictest9-21ksoftirqd/019:20:000
1263996230,23cyclictest9-21ksoftirqd/022:05:190
1263996226,26cyclictest276-21systemd-journal20:56:180
126399622,13cyclictest9-21ksoftirqd/022:45:050
127499613,13cyclictest2854-21munin-run19:15:001
127499613,11cyclictest5145-21munin-run00:05:001
127499612,54cyclictest12791-21kworker/1:322:00:111
1274996125,21cyclictest21-21ksoftirqd/122:35:221
1274996111,17cyclictest21-21ksoftirqd/120:40:121
1263996140,14cyclictest9-21ksoftirqd/021:40:000
1263996133,19cyclictest1253-21cyclictest21:00:140
1263996132,20cyclictest1253-21cyclictest21:45:120
1263996117,17cyclictest0-21swapper/022:40:140
127499603,13cyclictest3123-21munin-run20:50:011
127499603,13cyclictest1760-21munin-run22:20:011
127499603,11cyclictest13696-21idleruntime-cro22:55:001
127499603,10cyclictest17005-21munin-run21:30:001
127499602,9cyclictest32204-21munin-run20:40:001
1274996014,7cyclictest111rcu_preempt22:45:051
126399608,10cyclictest9-21ksoftirqd/020:40:010
1263996038,13cyclictest0-21swapper/021:25:190
1263996032,21cyclictest1253-21cyclictest23:30:140
127499593,9cyclictest31005-21appstreamcli22:10:061
1274995931,11cyclictest21-21ksoftirqd/119:20:011
1274995918,16cyclictest21-21ksoftirqd/123:45:201
1274995910,16cyclictest111rcu_preempt22:10:001
1263995934,17cyclictest1253-21cyclictest00:30:110
1263995934,17cyclictest1253-21cyclictest00:30:110
1263995933,17cyclictest276-21systemd-journal22:21:360
1263995930,15cyclictest0-21swapper/021:40:120
1263995924,25cyclictest540-21runrttasks23:00:130
1263995914,13cyclictest276-21systemd-journal19:21:170
1263995913,13cyclictest14787-21runrttasks19:47:290
127499588,8cyclictest21-21ksoftirqd/121:15:001
127499584,14cyclictest111rcu_preempt22:45:001
127499583,12cyclictest22284-21munin-run23:20:011
127499583,12cyclictest18351-21munin-run20:00:011
127499582,10cyclictest20626-21idleruntime-cro23:15:011
1274995814,13cyclictest111rcu_preempt22:25:251
1263995833,14cyclictest0-21swapper/021:50:120
1263995814,14cyclictest32521-21runrttasks23:49:510
1263995813,33cyclictest0-21swapper/020:10:130
127499574,12cyclictest11810-21munin-run00:25:011
1274995724,27cyclictest0-21swapper/121:00:331
1274995715,15cyclictest111rcu_preempt20:15:061
1274995715,15cyclictest111rcu_preempt20:15:051
1274995712,8cyclictest111rcu_preempt20:05:001
1263995723,26cyclictest1922-21latency_hist23:55:020
1263995723,19cyclictest0-21swapper/022:35:110
1263995713,31cyclictest0-21swapper/020:05:120
127499563,11cyclictest8461-21ls00:15:001
127499563,11cyclictest15120-21munin-run00:35:011
127499563,11cyclictest15120-21munin-run00:35:001
1274995621,13cyclictest21-21ksoftirqd/120:10:011
1274995616,15cyclictest21-21ksoftirqd/119:30:011
1274995615,16cyclictest111rcu_preempt23:53:181
1274995614,12cyclictest4788-21turbostat.cron20:55:011
1263995613,15cyclictest25450-21runrttasks20:20:040
127499554,15cyclictest111rcu_preempt20:30:011
127499553,12cyclictest25471-21apt-get20:20:171
1274995531,10cyclictest21-21ksoftirqd/119:25:001
1274995513,16cyclictest21-21ksoftirqd/121:22:181
126399554,37cyclictest0-21swapper/020:50:110
127499549,16cyclictest0-21swapper/121:05:351
127499543,12cyclictest15374-21df_inode19:50:171
1274995423,26cyclictest0-21swapper/120:10:521
1274995412,7cyclictest111rcu_preempt21:00:011
1263995413,16cyclictest28332-21runrttasks20:26:590
127499538,15cyclictest111rcu_preempt23:20:371
1274995312,13cyclictest111rcu_preempt19:40:181
127499523,13cyclictest26275-21memory23:30:251
127499523,12cyclictest3435-21munin-run22:25:011
1274995226,10cyclictest21-21ksoftirqd/122:35:001
1274995220,18cyclictest21-21ksoftirqd/100:36:171
1274995215,14cyclictest111rcu_preempt23:45:001
1274995110,15cyclictest111rcu_preempt00:10:031
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:363
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
none15:02:362
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional