You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2022-01-23 - 19:46

Intel(R) Atom(TM) CPU N455 @ 1.66GHz, Linux 5.4.34-rt21 (Profile)

Latency plot of system in rack #3, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6 (updated Sun Jan 23, 2022 12:45:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1807722000,17sleep121-21ksoftirqd/107:08:321
1081621520,17sleep11834499cyclictest08:20:531
17978210322,44sleep00-21swapper/007:07:280
258802930,4sleep10-21swapper/109:05:341
178902910,17sleep11834499cyclictest11:55:271
185402760,4sleep1201rcuc/108:45:021
210442730,5sleep121047-21sed08:50:441
18344997343,18cyclictest15650irq/30-ahci[00010:20:201
18344997343,18cyclictest15650irq/30-ahci[00010:20:191
18336997135,12cyclictest9-21ksoftirqd/009:00:480
54892700,9chrt488-21dbus-daemon09:40:570
24142700,11sleep10-21swapper/107:55:441
18344996936,20cyclictest488-21dbus-daemon07:40:141
18344996838,16cyclictest15650irq/30-ahci[00010:00:171
18344996638,17cyclictest504-21in:imuxsock12:00:221
227162650,9sleep118326-21cyclictest08:55:411
17052650,19sleep11834499cyclictest09:30:341
18344996441,14cyclictest15650irq/30-ahci[00011:20:201
18336996427,13cyclictest9-21ksoftirqd/010:25:010
18336996427,13cyclictest9-21ksoftirqd/010:25:000
216242630,3sleep0101rcuc/010:30:250
18344996334,16cyclictest488-21dbus-daemon07:20:211
18344996333,19cyclictest488-21dbus-daemon11:00:191
18336996334,16cyclictest9-21ksoftirqd/012:22:120
18336996327,15cyclictest9-21ksoftirqd/008:20:410
18344996237,16cyclictest520-21gdbus11:40:191
18336996228,13cyclictest9-21ksoftirqd/007:40:550
18336996227,13cyclictest9-21ksoftirqd/011:40:300
18344996131,24cyclictest488-21dbus-daemon08:40:211
18336996128,11cyclictest9-21ksoftirqd/011:00:300
18336996127,10cyclictest9-21ksoftirqd/010:05:050
18344996033,18cyclictest520-21gdbus10:40:211
18344996029,26cyclictest488-21dbus-daemon12:21:021
18344996026,22cyclictest488-21dbus-daemon09:20:221
1833699604,10cyclictest9-21ksoftirqd/011:15:190
1834499597,9cyclictest111rcu_preempt10:57:531
18344995928,26cyclictest488-21dbus-daemon09:40:391
18336995926,11cyclictest9-21ksoftirqd/010:00:410
18336995922,12cyclictest9-21ksoftirqd/012:25:100
59022585,45sleep00-21swapper/011:20:060
1834499587,23cyclictest23571-21apt-get09:00:211
18344995827,25cyclictest488-21dbus-daemon08:00:201
1833699585,10cyclictest9-21ksoftirqd/010:41:350
18336995823,13cyclictest9-21ksoftirqd/008:01:250
18336995822,13cyclictest9-21ksoftirqd/010:55:210
18344995729,15cyclictest0-21swapper/112:10:181
18344995724,28cyclictest488-21dbus-daemon12:40:021
18344995713,14cyclictest0-21swapper/110:45:181
18344995712,14cyclictest0-21swapper/109:10:171
18336995725,14cyclictest488-21dbus-daemon07:20:150
18336995724,29cyclictest0-21swapper/009:58:560
18336995723,12cyclictest9-21ksoftirqd/012:00:290
18336995722,13cyclictest9-21ksoftirqd/011:31:340
239552560,4sleep10-21swapper/107:25:201
18344995632,12cyclictest0-21swapper/110:35:171
1833699569,11cyclictest9-21ksoftirqd/008:14:340
18336995616,13cyclictest0-21swapper/008:40:570
18344995526,12cyclictest11492-21chrt11:35:331
18344995516,25cyclictest31907-21chrt09:25:181
1833699559,11cyclictest9-21ksoftirqd/012:15:020
1833699556,21cyclictest16400-21apt-get10:15:190
18336995525,25cyclictest0-21swapper/009:30:120
18336995521,13cyclictest9-21ksoftirqd/008:16:010
18336995521,13cyclictest9-21ksoftirqd/008:16:000
18336995520,13cyclictest9-21ksoftirqd/009:52:480
18336995520,13cyclictest9-21ksoftirqd/008:06:130
18344995436,9cyclictest21-21ksoftirqd/108:30:211
18344995436,9cyclictest21-21ksoftirqd/108:30:201
18344995436,14cyclictest488-21dbus-daemon09:50:001
18336995427,21cyclictest520-21gdbus09:20:410
18336995424,26cyclictest0-21swapper/011:38:060
18336995423,27cyclictest0-21swapper/011:14:270
18336995423,13cyclictest9-21ksoftirqd/010:53:520
247182535,11sleep10-21swapper/112:15:271
18344995336,11cyclictest488-21dbus-daemon09:20:031
18344995330,13cyclictest0-21swapper/110:50:181
18344995327,13cyclictest21-21ksoftirqd/111:30:191
18344995225,15cyclictest21-21ksoftirqd/110:05:201
18344995220,26cyclictest16610-21latency_hist08:40:041
18344995220,26cyclictest16610-21latency_hist08:40:031
18344995216,24cyclictest20088-21munin-node07:15:221
18336995231,14cyclictest488-21dbus-daemon12:00:010
18336995222,26cyclictest0-21swapper/012:07:230
18336995222,26cyclictest0-21swapper/009:07:030
18336995218,12cyclictest9-21ksoftirqd/009:47:130
18336995216,17cyclictest20630-21/usr/sbin/munin08:50:330
18336995216,14cyclictest9-21ksoftirqd/012:17:470
18344995134,12cyclictest28866-21run-parts07:40:001
18344995129,13cyclictest19931-21turbostat07:15:011
18344995124,14cyclictest0-21swapper/108:05:211
18344995123,20cyclictest0-21swapper/111:20:031
1833699519,9cyclictest111rcu_preempt11:47:320
1833699519,9cyclictest111rcu_preempt08:55:170
18336995121,25cyclictest0-21swapper/007:14:410
18336995116,20cyclictest0-21swapper/009:12:580
18336995111,8cyclictest111rcu_preempt07:50:180
18336995110,8cyclictest111rcu_preempt10:26:370
18344995022,7cyclictest0-21swapper/107:30:441
18336995021,25cyclictest0-21swapper/012:37:580
18336995021,25cyclictest0-21swapper/012:31:100
18336995020,13cyclictest9-21ksoftirqd/008:35:270
18336995020,13cyclictest9-21ksoftirqd/008:35:260
18336995015,11cyclictest9-21ksoftirqd/009:15:280
18336995010,9cyclictest111rcu_preempt08:33:220
18336995010,9cyclictest111rcu_preempt08:33:220
1834499494,40cyclictest31422-21wc07:45:341
18344994934,9cyclictest1419-21cron07:55:001
18344994934,10cyclictest21-21ksoftirqd/111:05:061
18344994927,14cyclictest488-21dbus-daemon11:29:591
18344994927,13cyclictest1-21systemd10:00:001
18344994923,15cyclictest17406-21cron11:55:001
18344994917,12cyclictest13113-21idleruntime-cro08:30:001
18336994922,22cyclictest0-21swapper/011:25:290
18336994921,24cyclictest0-21swapper/007:37:520
18336994918,13cyclictest9-21ksoftirqd/009:28:470
18336994917,12cyclictest0-21swapper/010:35:170
18336994911,32cyclictest0-21swapper/008:45:290
18336994910,8cyclictest111rcu_preempt11:51:040
18336994910,8cyclictest111rcu_preempt09:35:330
18344994826,13cyclictest9278-21cron09:55:001
18344994820,24cyclictest9913-21latency_hist08:20:031
18344994820,24cyclictest9913-21latency_hist08:20:031
18344994819,15cyclictest0-21swapper/109:35:191
18344994815,17cyclictest0-21swapper/110:15:191
1833699489,8cyclictest111rcu_preempt10:46:210
1833699489,8cyclictest111rcu_preempt08:26:310
1833699487,9cyclictest111rcu_preempt07:34:050
1833699485,38cyclictest0-21swapper/007:18:310
18336994830,13cyclictest31217-21cat07:45:280
18336994820,24cyclictest0-21swapper/010:13:480
18336994819,24cyclictest1829-21gltestperf07:55:290
18336994816,11cyclictest9-21ksoftirqd/007:25:320
1834499474,22cyclictest21235-21latency_hist10:30:021
18344994733,9cyclictest21791-21threads12:05:431
18344994730,10cyclictest28219-21sort12:25:221
18336994730,11cyclictest488-21dbus-daemon11:05:050
1834499467,23cyclictest6609-21apt-get08:10:201
1834499464,17cyclictest4061-21turbostat.cron11:14:591
18344994623,15cyclictest15339-21seq11:45:451
18344994617,17cyclictest0-21swapper/110:30:181
18344994617,15cyclictest0-21swapper/110:10:201
18344994515,16cyclictest0-21swapper/112:30:141
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:123
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
none17:13:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional