You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-06-30 - 09:42

x86 Intel Celeron N3350 @1100 MHz, Linux 5.10.90-rt60 (Profile)

Latency plot of system in rack #c, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot4.osadl.org (updated Thu Jun 30, 2022 00:44:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19278982318276,27sleep00-21swapper/019:09:420
19278252309267,27sleep10-21swapper/119:08:561
21056342570,3sleep013-21ksoftirqd/023:55:440
21327972550,3sleep00-21swapper/000:33:500
172099540,4rtkit-daemon1719-21rtkit-daemon22:42:481
172099540,4rtkit-daemon1719-21rtkit-daemon22:37:221
172099520,4rtkit-daemon1719-21rtkit-daemon19:31:151
172099510,4rtkit-daemon1719-21rtkit-daemon22:49:241
172099510,4rtkit-daemon1719-21rtkit-daemon19:37:171
172099500,4rtkit-daemon1719-21rtkit-daemon20:32:510
172099500,4rtkit-daemon1719-21rtkit-daemon19:40:481
172099490,3rtkit-daemon1719-21rtkit-daemon21:49:251
19834122410,1sleep10-21swapper/121:16:221
172099410,7rtkit-daemon1719-21rtkit-daemon23:30:120
172099410,4rtkit-daemon1719-21rtkit-daemon22:01:521
172099380,6rtkit-daemon2024187-21/usr/sbin/munin22:10:130
172099380,6rtkit-daemon2024187-21/usr/sbin/munin22:10:120
172099380,4rtkit-daemon1719-21rtkit-daemon21:35:100
172099380,3rtkit-daemon0-21swapper/000:31:521
172099370,3rtkit-daemon0-21swapper/020:15:440
172099360,4rtkit-daemon1719-21rtkit-daemon23:22:191
172099360,3rtkit-daemon0-21swapper/122:54:051
172099360,3rtkit-daemon0-21swapper/121:15:520
172099350,4rtkit-daemon2090482-21ssh23:35:341
172099350,4rtkit-daemon1719-21rtkit-daemon21:21:271
172099350,3rtkit-daemon1719-21rtkit-daemon21:13:451
172099350,3rtkit-daemon0-21swapper/123:55:441
172099350,3rtkit-daemon0-21swapper/121:23:450
172099350,3rtkit-daemon0-21swapper/120:57:401
172099340,4rtkit-daemon1719-21rtkit-daemon22:23:181
172099340,4rtkit-daemon1719-21rtkit-daemon20:47:421
172099340,3rtkit-daemon0-21swapper/121:51:381
172099340,3rtkit-daemon0-21swapper/120:10:200
172099340,3rtkit-daemon0-21swapper/119:45:361
172099340,3rtkit-daemon0-21swapper/119:29:121
172099340,3rtkit-daemon0-21swapper/022:25:381
172099340,3rtkit-daemon0-21swapper/021:04:560
172099330,4rtkit-daemon2039033-21ssh22:30:040
172099330,4rtkit-daemon1719-21rtkit-daemon22:05:230
172099330,4rtkit-daemon1719-21rtkit-daemon00:16:360
172099330,3rtkit-daemon1719-21rtkit-daemon19:13:181
172099330,3rtkit-daemon0-21swapper/123:44:580
172099320,4rtkit-daemon1719-21rtkit-daemon00:11:550
172099320,3rtkit-daemon1719-21rtkit-daemon00:09:210
172099320,3rtkit-daemon0-21swapper/121:12:210
172099320,3rtkit-daemon0-21swapper/120:50:060
172099320,3rtkit-daemon0-21swapper/119:57:280
172099310,4rtkit-daemon1719-21rtkit-daemon20:54:071
172099310,4rtkit-daemon0-21swapper/121:28:450
172099310,3rtkit-daemon0-21swapper/123:53:050
172099310,3rtkit-daemon0-21swapper/123:46:170
172099310,3rtkit-daemon0-21swapper/121:42:540
172099310,3rtkit-daemon0-21swapper/120:43:180
172099310,3rtkit-daemon0-21swapper/020:30:591
172099310,3rtkit-daemon0-21swapper/020:23:071
172099310,18rtkit-daemon20419012sleep122:33:211
19930612300,2sleep00-21swapper/021:30:080
172099300,4rtkit-daemon1719-21rtkit-daemon21:55:570
172099300,3rtkit-daemon62050irq/130-enp2s022:49:030
172099300,3rtkit-daemon1719-21rtkit-daemon23:03:500
172099300,3rtkit-daemon1719-21rtkit-daemon00:01:241
172099300,3rtkit-daemon0-21swapper/122:58:040
172099300,3rtkit-daemon0-21swapper/120:49:010
172099300,3rtkit-daemon0-21swapper/119:17:400
172099300,3rtkit-daemon0-21swapper/021:07:261
172099300,3rtkit-daemon0-21swapper/021:02:401
1928053992916,3cyclictest26-21ksoftirqd/122:09:591
172099290,4rtkit-daemon1719-21rtkit-daemon22:16:310
172099290,4rtkit-daemon1719-21rtkit-daemon20:04:560
172099290,4rtkit-daemon1719-21rtkit-daemon19:46:490
172099290,4rtkit-daemon1719-21rtkit-daemon00:04:190
172099290,3rtkit-daemon0-21swapper/122:39:000
172099280,4rtkit-daemon1719-21rtkit-daemon21:40:081
172099280,3rtkit-daemon0-21swapper/123:21:280
172099280,3rtkit-daemon0-21swapper/123:18:520
172099280,3rtkit-daemon0-21swapper/119:13:250
172099280,1rtkit-daemon1719-21rtkit-daemon23:09:010
192805399270,6cyclictest0-21swapper/121:36:161
192805399270,6cyclictest0-21swapper/120:37:411
192805399270,6cyclictest0-21swapper/120:06:271
192805399270,6cyclictest0-21swapper/119:54:321
172099270,3rtkit-daemon0-21swapper/120:09:310
172099270,3rtkit-daemon0-21swapper/119:23:440
172099270,3rtkit-daemon0-21swapper/119:23:430
172099270,1rtkit-daemon1719-21rtkit-daemon22:16:441
192805399260,6cyclictest0-21swapper/123:42:441
192805399260,6cyclictest0-21swapper/121:34:471
192805399260,6cyclictest0-21swapper/120:17:301
192805399260,21cyclictest2115828-21apt-get00:10:141
172099260,3rtkit-daemon0-21swapper/119:26:110
172099260,18rtkit-daemon808-21snmpd23:35:200
1928053992515,6cyclictest26-21ksoftirqd/100:20:121
192804999252,14cyclictest1975143-21apt-get21:05:110
1928049992520,3cyclictest13-21ksoftirqd/022:45:010
1928049992511,3cyclictest141rcu_preempt00:25:000
172099250,3rtkit-daemon0-21swapper/121:53:480
172099250,1rtkit-daemon1719-21rtkit-daemon23:54:261
172099250,1rtkit-daemon1719-21rtkit-daemon23:47:311
172099250,1rtkit-daemon1719-21rtkit-daemon21:25:331
172099250,1rtkit-daemon1719-21rtkit-daemon00:28:050
192805399249,5cyclictest0-21swapper/100:06:161
192805399242,12cyclictest2133587-21apt-get00:35:121
1928053992411,7cyclictest62050irq/130-enp2s023:05:121
172099240,1rtkit-daemon1719-21rtkit-daemon23:13:270
172099240,1rtkit-daemon1719-21rtkit-daemon20:23:140
172099240,1rtkit-daemon1719-21rtkit-daemon19:21:571
172099240,1rtkit-daemon1719-21rtkit-daemon19:21:571
172099240,19rtkit-daemon1928040-21cyclictest20:28:211
172099240,15rtkit-daemon0-21swapper/022:04:420
1928053992315,6cyclictest26-21ksoftirqd/123:35:001
1928053992312,6cyclictest62050irq/130-enp2s023:10:161
1928053992311,6cyclictest2058642-21apt-get22:55:111
192805399230,7cyclictest62050irq/130-enp2s023:19:541
192804999233,13cyclictest1971158-21apt-get20:55:150
192805399224,10cyclictest2081407-21apt-get23:25:111
192805399223,17cyclictest2129649-21latency_hist00:30:001
192805399223,11cyclictest2118872-21apt-get00:15:111
192805399222,13cyclictest1964955-21apt-esm-hook20:40:141
192805399222,13cyclictest1946683-21apt-esm-hook19:55:131
1928053992212,5cyclictest2024055-21apt-get22:10:111
1928053992212,5cyclictest2024055-21apt-get22:10:111
1928053992210,2cyclictest141rcu_preempt23:01:551
192804999226,4cyclictest1936892-21taskset19:30:150
192804999222,7cyclictest787-21runrttasks00:39:000
192804999222,14cyclictest2054572-21apt-esm-hook22:50:130
192804999222,13cyclictest1940721-21apt-get19:40:180
192804999222,13cyclictest1938706-21apt-get19:35:120
172099220,14rtkit-daemon0-21swapper/121:58:211
192805399212,14cyclictest1948706-21apt-esm-hook20:00:131
192805399212,13cyclictest1952642-21apt-get20:10:131
192805399210,19cyclictest1932549-21perf19:20:001
192804999217,8cyclictest13-21ksoftirqd/021:45:020
192804999212,13cyclictest2081455-21apt-esm-hook23:25:130
192804999212,13cyclictest2035266-21apt-get22:25:180
192804999212,13cyclictest2031638-21apt-esm-hook22:20:120
192804999212,11cyclictest1944688-21apt-get19:50:130
192804999202,11cyclictest1962935-21apt-get20:35:110
192804999202,10cyclictest1958997-21apt-esm-hook20:25:140
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:393
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
none08:51:392
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional