You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-05-12 - 14:23

Intel(R) Celeron(R) CPU N3350 @ 1.10GHz, Linux 5.9.1-rt18 (Profile)

Latency plot of system in rack #c, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot4.osadl.org (updated Wed May 12, 2021 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
251592263213,36sleep10-21swapper/119:08:341
249632255191,36sleep00-21swapper/019:07:190
2661321360,4sleep02559499cyclictest00:10:000
2559599492,31cyclictest11275-21apt-get21:05:121
2559599491,31cyclictest5328-21apt-get00:35:131
2559599451,25cyclictest3091-21apt-get23:15:091
53122430,6sleep1181rcuc/122:05:161
2559599438,20cyclictest1269-21sed19:25:271
2559599431,28cyclictest31114-21apt-get21:50:121
2559599427,20cyclictest19120-21awk21:20:261
2559599421,29cyclictest26713-21apt-get00:10:111
2559599420,29cyclictest15904-21apt-get22:30:141
2559599420,28cyclictest28542-21apt-get21:45:151
2559599420,28cyclictest15826-21apt-get21:15:191
2559599410,28cyclictest31280-21apt-get00:20:211
2559599410,28cyclictest13844-21apt-get23:40:111
2559599410,25cyclictest853-21apt-get21:55:161
2559599400,28cyclictest6656-21apt-get19:40:121
2559599400,28cyclictest11679-21apt-get23:35:121
2559599400,27cyclictest4839-21apt-get20:50:121
2559599400,27cyclictest20134-21apt-get21:25:151
2559599400,24cyclictest24253-21latency_hist21:35:001
2559599391,28cyclictest6984-21apt-get20:55:151
2559599391,27cyclictest9748-21perl22:15:181
2559599391,27cyclictest17639-40apt-check20:05:121
2559599390,28cyclictest26152-21apt-get19:10:121
2559599390,27cyclictest7403-21apt-get23:25:141
2559599390,27cyclictest20204-21apt-get22:40:121
2559599390,27cyclictest10934-21apt-get19:50:121
2559599390,26cyclictest9134-21apt-get21:00:111
2559599390,26cyclictest4495-21apt-get19:35:101
2559599390,25cyclictest3872-21awk20:45:261
2559599390,25cyclictest31428-21sed19:20:241
2559599390,24cyclictest14277-21awk19:55:261
2559599390,23cyclictest7423-21ls22:10:131
2559599390,22cyclictest12778-21awk22:20:271
2559599381,28cyclictest31145-21apt-config23:05:011
2559599381,27cyclictest20044-21perl20:10:141
2559599381,26cyclictest1720-21snmpd23:45:261
2559599381,25cyclictest26630-21apt-get22:55:131
2559599380,27cyclictest8796-21apt-get19:45:121
2559599380,26cyclictest489-21apt-get20:40:121
2559599380,26cyclictest3029-21apt-get22:00:091
2559599380,26cyclictest13768-21apt-get22:25:151
2559599380,25cyclictest24514-21ls21:35:141
2559599380,21cyclictest24240-21expr20:20:161
2559599380,21cyclictest22880-21cat00:00:181
2559599374,19cyclictest1966-21sed23:10:251
2559599372,28cyclictest28637-21apt-get20:30:121
2559599372,20cyclictest18744-21awk22:35:191
2559599370,27cyclictest24572-21apt-get00:05:131
2559599370,26cyclictest5267-21apt-get23:20:141
2559599370,26cyclictest22344-21apt-get22:45:111
2559599370,26cyclictest21939-21apt-get20:15:171
2559599370,26cyclictest18136-21apt-get23:50:151
2559599370,22cyclictest2349-21apt-get19:30:111
2559599370,21cyclictest1139-21ls00:25:181
2559599370,20cyclictest29830-21sed00:15:251
2559599362,26cyclictest13431-21apt-get21:10:191
2559599361,26cyclictest28316-21grep19:15:181
2559599360,26cyclictest15242-21apt-get20:00:121
2559599360,25cyclictest9805-21perl23:30:161
2559599360,20cyclictest26749-21cat20:25:171
2559599360,19cyclictest3179-21apt-get00:30:131
2559599356,16cyclictest24936-21cat22:50:161
2559599353,26cyclictest20275-21apt-get23:55:171
2559599350,25cyclictest28793-21apt-get23:00:101
2559599341,24cyclictest30779-21/usr/sbin/munin20:35:221
2559599340,19cyclictest27007-21basename21:40:241
8752290,3sleep00-21swapper/019:25:210
2559499240,6cyclictest17666-21pidof21:19:210
40632230,1chrt4062-21/usr/sbin/munin22:00:240
2559499233,14cyclictest13844-21apt-get23:40:130
2559499233,13cyclictest22344-21apt-get22:45:140
2559499232,14cyclictest28976-21apt-get21:45:140
2559499232,14cyclictest28599-40apt-check20:30:120
2559499232,14cyclictest11679-21apt-get23:35:140
2559499230,5cyclictest0-21swapper/022:24:560
2559499222,14cyclictest28855-21apt-get00:15:140
2559499222,14cyclictest10068-21snmp_o5powerctl21:00:220
2559499222,13cyclictest26630-21apt-get22:55:130
2559499212,13cyclictest925-21apt-get23:10:130
2559499212,13cyclictest18044-21apt-get22:35:130
2559499212,13cyclictest17608-40apt-check20:05:120
2559499212,13cyclictest15990-21apt-get23:45:150
2559499212,12cyclictest13075-21apt-get19:55:150
2559499211,14cyclictest5267-21apt-get23:20:150
2559499211,14cyclictest32054-21snmp_o5powerctl21:50:230
2559499211,14cyclictest31190-21apt-get23:05:120
2559499211,13cyclictest17989-21apt-get21:20:140
2559499210,4cyclictest0-21swapper/022:53:400
2559499202,13cyclictest7594-21snmp_o5powerctl19:40:230
2559499202,13cyclictest10489-21snmp_o5powerctl23:30:250
2559499202,12cyclictest4839-21apt-get20:50:140
2559499201,13cyclictest31280-21apt-get00:20:130
2559499192,13cyclictest27092-21snmp_o5powerctl19:10:230
2559499192,13cyclictest25038-21snmp_o5powerctl20:20:260
2559499192,12cyclictest31390-21snmp_o5powerctl19:20:220
2559499192,12cyclictest21140-21snmp_o5powerctl22:40:220
2559499191,15cyclictest3626-21snmp_o5powerctl20:45:230
2559499191,14cyclictest16841-21snmp_o5powerctl22:30:240
2559499191,13cyclictest27653-21snmp_o5powerctl00:10:240
2559499191,13cyclictest14376-21snmp_o5powerctl21:10:310
2559499186,3cyclictest111rcu_preempt00:00:000
2559499183,8cyclictest19782-21apt-get20:10:110
2559499182,12cyclictest27176-21snmp_o5powerctl20:25:230
25594991813,3cyclictest1764-21open_inodes00:25:230
2559499181,14cyclictest19075-21snmp_o5powerctl23:50:270
2559499181,13cyclictest9730-21snmp_o5powerctl19:45:230
2559499181,13cyclictest6115-21snmp_o5powerctl22:05:250
2559499181,13cyclictest3288-21snmp_o5powerctl19:30:240
2559499181,13cyclictest29271-21snmp_o5powerctl19:15:380
2559499181,13cyclictest23371-21snmp_o5powerctl00:00:240
2559499181,13cyclictest22875-21snmp_o5powerctl20:15:290
2559499181,13cyclictest1817-21snmp_o5powerctl21:55:290
2559499181,13cyclictest16188-21snmp_o5powerctl20:00:260
2559499181,13cyclictest12215-21snmp_o5powerctl21:05:220
2559499181,13cyclictest11866-21snmp_o5powerctl19:50:210
25594991810,6cyclictest15724-21pluginstate22:29:590
25594991713,2cyclictest31282-21latency20:35:200
25594991712,3cyclictest3960-21sendmail_mailst23:15:200
2559499171,13cyclictest6259-21snmp_o5powerctl00:35:220
2559499171,13cyclictest27528-21snmp_o5powerctl21:40:300
2559499171,12cyclictest5432-21snmp_o5powerctl19:35:230
2559499171,12cyclictest10404-21snmp_o5powerctl22:15:260
2559499170,15cyclictest7933-21snmp_o5powerctl20:55:320
2559499170,15cyclictest7463-21df22:10:140
2559499168,6cyclictest20808-21ntp_kernel_pll_21:25:220
2559499167,7cyclictest9351-21cron23:30:010
25594991611,3cyclictest22717-21if_err_enp2s021:30:190
2559499161,12cyclictest29725-21snmp_o5powerctl23:00:230
2559499160,1cyclictest0-21swapper/021:35:010
2559499157,6cyclictest1275-21pluginstate20:40:210
2559499152,6cyclictest4119-21snmp_o5powerctl00:30:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional