You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-07-29 - 18:37

Intel(R) Celeron(R) CPU G3900 @ 2.80GHz, Linux 5.10.35-rt39 (Profile)

Latency plot of system in rack #c, slot #5
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot5.osadl.org (updated Thu Jul 29, 2021 12:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
88492330,0sleep10-21swapper/110:40:221
2332999270,26cyclictest17834-21meminfo12:25:291
2332999250,24cyclictest13917-21meminfo12:15:181
2332899240,23cyclictest22959-21meminfo11:15:170
2332999230,22cyclictest28126-21meminfo10:05:161
2332999230,22cyclictest26561-21meminfo07:15:201
2332999230,22cyclictest16460-21meminfo09:35:241
2332899230,22cyclictest11737-21meminfo08:00:200
2332999220,22cyclictest11317-21meminfo10:45:251
2332999220,21cyclictest19087-21meminfo11:05:201
2332999210,20cyclictest30756-21meminfo11:35:251
2332899210,21cyclictest13257-21meminfo10:50:270
2332899210,20cyclictest1575-21meminfo10:20:160
2332899210,20cyclictest10051-21meminfo12:05:260
2332999202,17cyclictest12600-21apt-get10:50:201
2332999200,19cyclictest30075-21meminfo10:10:241
2332999200,19cyclictest15788-21meminfo08:10:241
2332899200,20cyclictest21697-21meminfo12:35:190
2332899190,19cyclictest28521-21meminfo07:20:190
2332899190,18cyclictest3550-21meminfo10:25:350
2332899190,18cyclictest2848-21meminfo09:00:170
2332899190,18cyclictest27462-21meminfo08:40:230
2332899180,18cyclictest8105-21meminfo12:00:210
2332999171,8cyclictest30077-21runrttasks08:45:241
2332999170,17cyclictest8669-21meminfo09:15:171
2332999170,17cyclictest20343-21meminfo09:45:171
2332999170,16cyclictest2262-21meminfo11:45:191
2332899170,16cyclictest870-21meminfo08:55:360
2332999163,2cyclictest0-21swapper/109:21:121
2332999160,15cyclictest24235-21meminfo09:55:211
2332999160,10cyclictest0-21swapper/107:55:181
2332899161,10cyclictest15801-21apt-get09:35:150
2332999153,10cyclictest26805-21apt-get08:40:151
2332999150,15cyclictest7407-21meminfo10:35:191
2332899150,15cyclictest2004-21meminfo07:35:210
2332999144,4cyclictest26176-21latency10:00:201
2332999141,5cyclictest11075-21kworker/u4:109:42:501
2332999140,14cyclictest7834-21meminfo07:50:171
2332899140,14cyclictest12589-21meminfo09:25:300
2332899140,13cyclictest24942-21meminfo11:20:290
2332999133,6cyclictest28144-21/usr/sbin/munin11:30:151
2332999133,5cyclictest14332-21/usr/sbin/munin09:30:241
2332999132,7cyclictest5910-21diskmemload11:04:241
2332999132,10cyclictest17434-21head08:15:151
2332999131,2cyclictest0-21swapper/112:05:391
2332899135,5cyclictest2321-21runrttasks10:07:220
2332899134,4cyclictest10666-21apt-get10:45:210
2332899132,6cyclictest13273-21apt-esm-hook12:15:140
2332899132,6cyclictest0-21swapper/011:25:100
2332899130,6cyclictest0-21swapper/011:00:140
2332999123,7cyclictest17027-21cat12:25:011
2332999120,4cyclictest0-21swapper/108:50:111
2332999120,12cyclictest24011-21meminfo07:10:231
2332999120,11cyclictest21021-21meminfo11:10:191
2332899127,3cyclictest0-21swapper/009:20:180
2332899124,7cyclictest17494-21if_ecat008:15:160
2332899122,9cyclictest24230-21users08:30:250
2332899122,6cyclictest15136-21apt-get08:10:190
2332899122,5cyclictest2321-21runrttasks07:10:240
2332899120,8cyclictest26024-21kworker/u4:411:44:190
2332899120,6cyclictest0-21swapper/012:10:140
2332999112,7cyclictest13505-21runrttasks09:28:051
2332999111,7cyclictest0-21swapper/112:00:161
2332899115,3cyclictest2321-21runrttasks09:52:200
2332899113,4cyclictest6332-21grep09:10:170
2332899112,5cyclictest24054-21/usr/sbin/munin09:55:200
2332899111,9cyclictest29921-21cron11:35:000
2332899111,7cyclictest19753-21/usr/sbin/munin11:05:250
2332899111,4cyclictest0-21swapper/012:20:160
229992112,2sleep10-21swapper/107:08:301
227732112,1sleep00-21swapper/007:06:020
2332999105,3cyclictest0-21swapper/107:20:131
2332999105,2cyclictest0-21swapper/110:55:111
2332999102,6cyclictest2896-21apt-get10:25:301
2332999101,7cyclictest697-21sendmail10:20:001
2332999101,7cyclictest2192-21apt-get09:00:111
2332999101,5cyclictest2901-21taskset07:37:431
2332999100,9cyclictest4800-21meminfo09:05:251
2332999100,9cyclictest386-21ntp_states11:40:211
2332999100,5cyclictest0-21swapper/108:35:221
2332999100,10cyclictest4213-21meminfo11:50:251
2332999100,10cyclictest11978-21meminfo12:10:191
2332899104,2cyclictest8969-21gunzip09:15:200
2332899103,4cyclictest31352-21apt-get10:15:130
2332899102,5cyclictest20226-21sed08:20:240
2332899101,5cyclictest31997-21uptime08:50:240
2332899101,2cyclictest0-21swapper/007:55:200
233299996,2cyclictest6244-21seq10:31:261
233299993,4cyclictest26210-21apt-get11:25:101
233299992,6cyclictest22304-21/usr/sbin/munin11:15:181
233299992,4cyclictest0-21swapper/112:30:501
233299992,3cyclictest3312-21/usr/sbin/munin07:40:241
233299990,9cyclictest6144-21meminfo11:55:201
233299990,9cyclictest21633-21meminfo08:25:291
233299990,6cyclictest25498-21sed11:20:341
233299990,4cyclictest11084-21apt-get08:00:141
233289996,3cyclictest1662-21snmpd10:42:470
233289993,5cyclictest5910-21diskmemload11:58:510
233289993,5cyclictest1792-21/usr/sbin/munin11:45:160
233289993,2cyclictest2321-21runrttasks10:13:280
233289992,4cyclictest29831-21ls08:45:210
233289990,9cyclictest19762-21meminfo12:30:220
233289990,4cyclictest19485-21kworker/u4:409:48:110
233289990,3cyclictest0-21swapper/012:25:340
233289990,3cyclictest0-21swapper/008:25:240
233289990,1cyclictest7932-21lxcfs08:05:140
233299986,2cyclictest15072-21cut08:10:001
233299986,1cyclictest21383-21tail12:35:171
233299986,1cyclictest1263-21tail10:20:141
233299982,5cyclictest0-21swapper/107:45:141
233299981,6cyclictest6057-21seq09:10:031
233299980,8cyclictest30458-21meminfo07:25:191
233299980,8cyclictest22296-21meminfo09:50:261
233299980,4cyclictest0-21swapper/108:58:361
233289986,1cyclictest82422sleep010:37:030
233289986,1cyclictest67522sleep007:47:160
233289986,1cyclictest4548-21basename09:05:230
233289986,1cyclictest25569-21cut08:35:210
233289986,1cyclictest19177-21taskset09:41:220
233289986,1cyclictest13837-21chrt09:30:010
233289985,2cyclictest30403-21latency07:25:190
233289983,2cyclictest26796-21sed10:00:250
233289981,5cyclictest0-21swapper/010:30:030
233289981,4cyclictest0-21swapper/007:50:220
233289981,3cyclictest0-21swapper/007:30:160
233289980,3cyclictest0-21swapper/007:40:190
233299976,1cyclictest32114-21tail07:30:191
233299976,1cyclictest23025-21cat08:30:151
233299976,1cyclictest20443-21seq08:21:181
233289975,1cyclictest30387-21cut11:35:240
233289975,1cyclictest161242sleep010:58:070
233289973,0cyclictest2321-21runrttasks11:52:130
233289971,5cyclictest20372-21apt-esm-hook11:10:120
233289971,2cyclictest0-21swapper/007:15:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional