You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-17 - 06:08

Intel(R) Celeron(R) CPU G3900 @ 2.80GHz, Linux 5.10.35-rt39 (Profile)

Latency plot of system in rack #c, slot #5
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rackcslot5.osadl.org (updated Fri Oct 15, 2021 12:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
252872580,1sleep02321-21runrttasks12:24:100
3169399290,28cyclictest9265-21meminfo08:55:191
3169299260,25cyclictest29397-21meminfo11:10:200
3169399250,24cyclictest22296-21meminfo12:15:181
3169399230,22cyclictest31339-21meminfo11:15:211
3169299230,23cyclictest20357-21meminfo12:10:210
3169299230,23cyclictest20283-21meminfo08:00:210
3169399220,22cyclictest14516-21meminfo11:55:161
3169299220,21cyclictest18432-21meminfo12:05:270
3169399210,21cyclictest18147-21meminfo07:55:201
3169399210,20cyclictest28732-21meminfo09:45:261
3169399200,20cyclictest871-21meminfo11:20:171
3169399200,19cyclictest32379-21meminfo07:10:251
31692992018,1cyclictest11-21ksoftirqd/009:10:220
31693991917,1cyclictest8464-21awk08:55:001
3169399190,18cyclictest29396-21meminfo11:10:201
3169399190,18cyclictest15803-21meminfo10:35:301
3169299190,18cyclictest20948-21meminfo09:25:220
3169399180,17cyclictest26093-21meminfo08:15:191
31693991715,1cyclictest2253-21tail07:15:211
3169399170,17cyclictest9970-21meminfo10:20:231
3169399170,16cyclictest30077-21meminfo12:35:221
31693991613,2cyclictest392-21/usr/sbin/munin09:55:341
31693991611,4cyclictest21493-21latency_hist08:05:001
3169399160,4cyclictest14282-21diskmemload10:27:201
3169399160,1cyclictest0-21swapper/110:58:301
3169399160,16cyclictest17740-21meminfo10:40:181
3169299160,16cyclictest19680-21meminfo10:45:160
3169299160,15cyclictest4130-21meminfo10:05:240
3169299160,15cyclictest28132-21meminfo12:30:160
3169299160,15cyclictest11236-21meminfo09:00:280
3169399158,5cyclictest14061-21awk07:45:201
3169399156,8cyclictest0-21swapper/109:03:531
3169399154,6cyclictest0-21swapper/109:50:121
3169399153,9cyclictest0-21swapper/112:20:271
31693991514,1cyclictest0-21swapper/110:50:351
31693991514,0cyclictest7498-21seq07:29:231
31693991514,0cyclictest7498-21seq07:29:221
3169399150,14cyclictest25520-21meminfo11:00:191
3169299153,5cyclictest0-21swapper/011:45:150
3169299152,5cyclictest23504-21apt-get08:10:170
3169299151,1cyclictest17730-21ls10:40:170
3169299150,15cyclictest12575-21meminfo11:50:160
3169299150,14cyclictest4815-21meminfo11:30:250
3169299150,14cyclictest21641-21meminfo10:50:300
3169399148,5cyclictest9481-21runrttasks11:41:291
3169399147,4cyclictest30453-21sed08:25:281
3169399145,5cyclictest21-21ksoftirqd/109:14:491
3169399143,10cyclictest4658-21/usr/sbin/munin07:20:251
3169399142,2cyclictest0-21swapper/110:17:111
31693991413,0cyclictest2321-21runrttasks10:01:181
31693991413,0cyclictest14282-21diskmemload12:01:341
31693991413,0cyclictest0-21swapper/110:08:351
3169399140,5cyclictest14282-21diskmemload12:10:011
3169399140,3cyclictest0-21swapper/111:54:301
3169399140,14cyclictest2876-21meminfo11:25:251
3169399140,14cyclictest24158-21meminfo08:10:221
3169399140,14cyclictest17051-21meminfo09:15:211
3169399140,13cyclictest26796-21meminfo09:40:281
3169299140,13cyclictest5397-21meminfo08:45:240
299892147,4sleep030157-21idleruntime07:05:240
3169399132,5cyclictest0-21swapper/111:30:171
3169399132,10cyclictest0-21swapper/111:05:141
31693991311,1cyclictest20456-21sh09:25:171
31693991310,2cyclictest11561-21grep07:40:001
3169399130,12cyclictest24823-21meminfo09:35:181
3169299136,4cyclictest2014-21timerwakeupswit08:35:240
3169299134,6cyclictest1521-21apt-get10:00:130
3169299131,1cyclictest0-21swapper/011:27:190
3169299130,13cyclictest30658-21meminfo09:50:170
3169399122,6cyclictest12899-21ls07:40:431
31693991211,1cyclictest17427-21latency_hist07:54:591
31693991210,2cyclictest0-21swapper/108:30:141
31693991210,1cyclictest2321-21runrttasks08:08:531
3169399120,12cyclictest3452-21meminfo08:40:211
3169399120,11cyclictest8396-21meminfo07:30:211
3169299121,1cyclictest0-21swapper/009:45:300
3169299120,12cyclictest32624-21meminfo09:55:310
3169299120,12cyclictest22886-21meminfo09:30:180
3169399117,1cyclictest13996-21chrt09:07:051
3169399110,11cyclictest6752-21meminfo11:35:251
3169299114,3cyclictest26906-21/usr/sbin/munin09:40:290
3169299112,6cyclictest0-21swapper/012:35:240
3169299111,4cyclictest0-21swapper/008:40:200
3169299110,6cyclictest18349-21apt-get09:20:120
297632111,7sleep129813-21cpu07:05:211
71862100,2sleep17177-21cron10:15:001
3169399109,1cyclictest21-21ksoftirqd/109:20:171
3169299106,1cyclictest0-21swapper/012:15:120
3169299103,3cyclictest31935-21/usr/sbin/munin08:30:200
3169299102,4cyclictest0-21swapper/012:00:180
3169299102,3cyclictest13620-21apt-get07:45:160
3169299101,6cyclictest10377-21/usr/sbin/munin07:35:210
3169299101,5cyclictest0-21swapper/010:35:210
3169299100,9cyclictest26196-21meminfo12:25:190
3169299100,7cyclictest540-21tail11:20:140
3169299100,4cyclictest5804-21apt-get07:25:200
3169299100,4cyclictest5804-21apt-get07:25:200
3169299100,10cyclictest6089-21meminfo10:10:350
316939998,1cyclictest0-21swapper/108:35:181
316939998,0cyclictest14282-21diskmemload12:30:121
316939996,2cyclictest18384-21ls12:05:271
316939992,4cyclictest0-21swapper/111:47:071
316939991,6cyclictest23990-21runrttasks09:34:561
316939990,4cyclictest0-21swapper/112:25:141
316929993,2cyclictest2321-21runrttasks11:08:100
316929992,6cyclictest19035-31update-apt-xapi07:56:570
316929992,1cyclictest0-21swapper/007:15:190
316929991,6cyclictest0-21swapper/008:25:150
316929991,5cyclictest0-21swapper/011:35:280
316929991,4cyclictest11684-21apt-get07:40:190
316929991,0cyclictest0-21swapper/009:05:140
316929990,9cyclictest22219-21meminfo08:05:210
316929990,9cyclictest13844-21meminfo10:30:190
316929990,8cyclictest9971-21meminfo10:20:230
316939986,2cyclictest4752-21dpkg08:45:181
316939983,3cyclictest0-21swapper/110:30:131
316939982,4cyclictest27395-21/usr/sbin/munin08:20:171
316929986,1cyclictest26012-21cut08:15:190
316929984,1cyclictest0-21swapper/008:50:230
316929983,3cyclictest7743-21apt-get07:30:160
316929983,1cyclictest8611-21apt-get08:55:140
316929982,4cyclictest2321-21runrttasks07:12:100
316929982,4cyclictest0-21swapper/010:25:200
316929982,3cyclictest5131-21/usr/sbin/munin07:20:290
316929982,2cyclictest22701-21kworker/u4:311:00:200
316929980,7cyclictest25877-21chrt09:39:250
316939974,2cyclictest19026-21apt-get10:45:111
316929976,1cyclictest8587-21awk10:15:240
316929976,1cyclictest28501-21grep08:20:240
316929976,1cyclictest26010-21kworker/0:111:42:130
316929976,1cyclictest18055-21chrt09:18:540
316929976,1cyclictest15812-21grep07:50:200
316929976,1cyclictest14093-21fschecks_time11:55:130
316929975,1cyclictest30805-21cut11:15:160
316929971,3cyclictest0-21swapper/010:55:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional