You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-07-29 - 05:12

Intel(R) Atom(TM) CPU N455 @ 1.66GHz, Linux 5.4.34-rt21 (Profile)

Latency plot of system in rack #3, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack3slot6 (updated Thu Jul 29, 2021 00:45:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2673621270,5sleep025902-21cyclictest22:20:320
1277321270,3sleep00-21swapper/020:05:180
2539529023,31sleep00-21swapper/019:05:470
2539529023,31sleep00-21swapper/019:05:470
130832890,5sleep113085-21ls21:40:231
105622890,4sleep00-21swapper/023:10:020
2563627838,24sleep10-21swapper/119:08:261
2563627838,24sleep10-21swapper/119:08:261
25919997442,17cyclictest0-21swapper/119:15:131
199572720,5sleep00-21swapper/022:00:280
25919997140,19cyclictest0-21swapper/123:55:151
248702710,17sleep09-21ksoftirqd/020:40:220
187222700,4sleep00-21swapper/021:55:420
25919996944,16cyclictest0-21swapper/120:50:161
25919996944,16cyclictest0-21swapper/120:50:151
25919996942,17cyclictest21-21ksoftirqd/123:35:131
25919996942,15cyclictest0-21swapper/100:30:151
25919996941,15cyclictest0-21swapper/119:55:151
25919996940,17cyclictest0-21swapper/100:05:221
25919996937,17cyclictest0-21swapper/120:00:181
25919996840,14cyclictest0-21swapper/121:35:111
25919996839,16cyclictest0-21swapper/100:10:111
25919996737,15cyclictest0-21swapper/120:35:151
25919996736,16cyclictest0-21swapper/100:20:111
25919996641,14cyclictest0-21swapper/123:00:131
25919996639,16cyclictest0-21swapper/119:25:141
25919996638,15cyclictest0-21swapper/122:30:151
25919996635,16cyclictest0-21swapper/100:35:151
25919996540,14cyclictest0-21swapper/121:30:181
25919996539,15cyclictest0-21swapper/119:45:221
25919996535,17cyclictest0-21swapper/120:10:161
25919996535,16cyclictest0-21swapper/122:40:151
25919996535,16cyclictest0-21swapper/122:40:141
25919996534,17cyclictest0-21swapper/120:55:161
25919996447,10cyclictest0-21swapper/122:50:371
25919996439,14cyclictest0-21swapper/122:35:211
25919996438,14cyclictest0-21swapper/121:50:121
25919996436,16cyclictest0-21swapper/123:20:131
25919996436,15cyclictest0-21swapper/119:20:251
25919996337,16cyclictest21-21ksoftirqd/122:55:141
25919996337,14cyclictest0-21swapper/122:15:261
25919996337,14cyclictest0-21swapper/122:15:251
25919996336,16cyclictest0-21swapper/121:15:221
25919996336,16cyclictest0-21swapper/121:15:211
25919996238,15cyclictest21-21ksoftirqd/123:30:241
25919996235,16cyclictest21-21ksoftirqd/122:45:171
25919996235,15cyclictest0-21swapper/122:25:141
25919996234,19cyclictest21-21ksoftirqd/121:20:141
25919996233,14cyclictest0-21swapper/120:20:121
25919996232,16cyclictest0-21swapper/123:50:211
25919996134,14cyclictest0-21swapper/100:00:161
2591999606,18cyclictest668-21munin-run21:05:011
25919995936,16cyclictest21-21ksoftirqd/121:55:191
25919995934,14cyclictest8557-21systemd-journal20:25:051
25919995933,17cyclictest12183-21snmpd20:07:221
25919995932,16cyclictest0-21swapper/123:40:151
2591199592,52cyclictest29059-21kworker/0:000:10:120
306992570,5sleep0101rcuc/022:30:390
29442572,47sleep12952-21df21:10:301
25919995734,13cyclictest0-21swapper/100:25:161
25919995733,14cyclictest24353-21wc22:15:011
2591199573,41cyclictest0-21swapper/020:17:490
25911995724,28cyclictest0-21swapper/000:29:010
25919995633,16cyclictest8557-21systemd-journal20:46:161
25919995631,15cyclictest27621-21latency_hist19:15:021
248022565,10sleep10-21swapper/120:40:211
25919995534,14cyclictest4860-21wc19:40:261
25919995534,13cyclictest0-21swapper/120:15:151
25919995531,16cyclictest488-21dbus-daemon00:20:011
25919995531,15cyclictest20959-21idleruntime-cro22:04:591
25911995527,11cyclictest488-21dbus-daemon22:40:000
25911995524,27cyclictest0-21swapper/022:14:390
25911995520,31cyclictest0-21swapper/023:20:520
25911995518,20cyclictest0-21swapper/000:00:230
25911995518,15cyclictest9-21ksoftirqd/023:34:520
25919995432,14cyclictest8557-21systemd-journal21:26:241
25919995432,13cyclictest8557-21systemd-journal23:46:521
25919995429,16cyclictest8557-21systemd-journal22:06:331
25919995429,16cyclictest8557-21systemd-journal22:06:331
25919995429,15cyclictest8557-21systemd-journal23:11:461
25919995426,16cyclictest12183-21snmpd21:05:401
2591199545,16cyclictest19483-21apt-get23:35:140
25911995427,11cyclictest13811-21sendmail-msp23:20:010
2591199542,38cyclictest9-21ksoftirqd/019:29:140
25911995423,27cyclictest0-21swapper/020:50:590
25911995423,27cyclictest0-21swapper/020:50:580
25911995422,26cyclictest17107-21sed21:50:380
25911995416,16cyclictest9-21ksoftirqd/023:26:220
25911995412,30cyclictest111rcu_preempt00:31:190
25919995319,21cyclictest26411-21seq22:20:221
25911995325,11cyclictest7137-21sendmail-msp23:00:010
25911995323,26cyclictest0-21swapper/022:19:210
25911995323,26cyclictest0-21swapper/022:19:200
25911995323,13cyclictest8557-21systemd-journal19:50:450
25911995322,13cyclictest8557-21systemd-journal00:05:580
25911995318,17cyclictest8557-21systemd-journal23:52:570
25911995316,16cyclictest9-21ksoftirqd/000:22:310
131192533,12sleep112286-21/usr/sbin/munin23:15:421
2591999526,13cyclictest0-21swapper/119:37:161
2591199523,43cyclictest488-21dbus-daemon22:45:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional