You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-20 - 23:29

x86 Intel N455 @1660 MHz, Linux 5.15.49-rt47 (Profile)

Latency plot of system in rack #3, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6 (updated Sat Mar 18, 2023 00:45:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
916399247227,14cyclictest25155-21apt-get00:30:261
91609911391,15cyclictest14-21ksoftirqd/021:35:160
91639910362,27cyclictest26-21ksoftirqd/119:50:161
9160999776,12cyclictest14-21ksoftirqd/023:20:240
9160999568,13cyclictest14-21ksoftirqd/021:45:150
9160998861,14cyclictest14-21ksoftirqd/022:10:130
9160998860,16cyclictest14-21ksoftirqd/000:20:150
772728824,16sleep10-21swapper/119:07:521
9160998764,11cyclictest14-21ksoftirqd/020:45:210
9160998762,14cyclictest14-21ksoftirqd/022:15:210
9160998664,11cyclictest14-21ksoftirqd/023:40:200
9160998657,15cyclictest14-21ksoftirqd/020:10:160
9160998657,15cyclictest14-21ksoftirqd/020:10:160
9160998561,15cyclictest14-21ksoftirqd/000:00:240
9160998458,15cyclictest14-21ksoftirqd/022:30:180
9160998454,18cyclictest14-21ksoftirqd/019:30:240
9160998452,17cyclictest14-21ksoftirqd/020:15:220
9160998452,17cyclictest14-21ksoftirqd/020:15:220
9160998451,20cyclictest14-21ksoftirqd/020:30:160
88042846,28sleep08823-21ps19:10:310
9160998351,19cyclictest14-21ksoftirqd/021:15:150
9160998264,10cyclictest14-21ksoftirqd/022:40:140
9160998258,14cyclictest14-21ksoftirqd/020:50:180
9160998152,19cyclictest14-21ksoftirqd/023:25:150
9160998151,18cyclictest14-21ksoftirqd/021:00:210
9160997953,18cyclictest14-21ksoftirqd/023:35:160
9160997951,18cyclictest14-21ksoftirqd/000:40:140
9160997950,15cyclictest14-21ksoftirqd/022:45:200
9160997854,14cyclictest14-21ksoftirqd/021:25:200
9160997853,20cyclictest14-21ksoftirqd/020:35:170
9160997852,17cyclictest14-21ksoftirqd/019:45:170
9160997851,22cyclictest14-21ksoftirqd/000:25:150
9160997851,21cyclictest14-21ksoftirqd/022:35:220
9160997851,18cyclictest14-21ksoftirqd/022:05:150
9163997710,8cyclictest151rcu_preempt21:10:361
9160997750,19cyclictest14-21ksoftirqd/000:05:220
9160997748,19cyclictest0-21swapper/021:30:200
9160997747,20cyclictest14-21ksoftirqd/022:50:160
9160997745,23cyclictest14-21ksoftirqd/023:45:160
9163997615,43cyclictest0-21swapper/122:20:151
9163997615,43cyclictest0-21swapper/122:20:151
9160997652,17cyclictest14-21ksoftirqd/000:10:170
9160997630,40cyclictest276-21systemd-journal21:06:020
9160997551,16cyclictest14-21ksoftirqd/023:50:160
9160997548,17cyclictest0-21swapper/023:15:150
9160997548,17cyclictest0-21swapper/023:15:150
9160997534,35cyclictest14-21ksoftirqd/019:10:350
9160997344,19cyclictest0-21swapper/020:55:170
9160997339,26cyclictest14-21ksoftirqd/022:20:220
9160997339,26cyclictest14-21ksoftirqd/022:20:220
9160997334,18cyclictest14-21ksoftirqd/020:15:360
120642730,8chrt0-21swapper/122:20:451
916099726,17cyclictest19992-21apt-get00:15:200
916099726,17cyclictest19992-21apt-get00:15:200
9160997240,23cyclictest14-21ksoftirqd/022:55:450
9160997142,20cyclictest14-21ksoftirqd/000:30:240
9160997139,18cyclictest0-21swapper/020:00:170
916099707,16cyclictest31166-21apt-get21:45:420
9160997042,17cyclictest0-21swapper/021:40:180
9160996941,17cyclictest14-21ksoftirqd/021:20:170
9160996941,17cyclictest14-21ksoftirqd/021:20:170
9160996940,23cyclictest14-21ksoftirqd/023:05:150
9160996939,21cyclictest14-21ksoftirqd/000:35:160
9160996925,38cyclictest276-21systemd-journal23:06:170
916399687,13cyclictest0-21swapper/123:45:131
9160996843,19cyclictest14-21ksoftirqd/020:25:150
222482680,5sleep10-21swapper/121:20:321
222482680,5sleep10-21swapper/121:20:321
9163996736,18cyclictest26-21ksoftirqd/120:41:131
9160996744,18cyclictest14-21ksoftirqd/021:55:220
9160996734,23cyclictest14-21ksoftirqd/019:45:380
9160996639,20cyclictest14-21ksoftirqd/023:55:210
9160996540,19cyclictest14-21ksoftirqd/019:20:160
9163996412,13cyclictest151rcu_preempt22:30:331
9160996436,17cyclictest14-21ksoftirqd/022:55:160
9160996436,17cyclictest14-21ksoftirqd/022:55:160
9160996428,20cyclictest14-21ksoftirqd/020:05:160
9163996320,23cyclictest26-21ksoftirqd/100:10:441
9163996320,23cyclictest26-21ksoftirqd/100:10:441
916099634,51cyclictest19552-21sendmail19:40:310
9160996341,12cyclictest14-21ksoftirqd/022:25:200
9160996339,18cyclictest9151-21cyclictest19:55:160
9160996337,19cyclictest14-21ksoftirqd/022:00:320
9160996336,21cyclictest14-21ksoftirqd/021:05:180
9160996335,21cyclictest14-21ksoftirqd/020:40:310
9160996335,21cyclictest14-21ksoftirqd/019:35:270
9160996319,34cyclictest490-21in:imuxsock19:22:560
9160996135,9cyclictest14-21ksoftirqd/023:30:300
69172610,6chrt6918-21/usr/sbin/munin23:35:481
916399603,15cyclictest8172-21latency_hist20:40:321
9163995936,8cyclictest26-21ksoftirqd/120:46:521
9163995936,7cyclictest26-21ksoftirqd/119:43:181
9163995927,14cyclictest26-21ksoftirqd/122:00:431
9163995839,12cyclictest26-21ksoftirqd/123:25:381
9163995834,8cyclictest26-21ksoftirqd/122:40:311
9163995816,13cyclictest26-21ksoftirqd/100:30:351
916399576,36cyclictest0-21swapper/123:25:161
9163995739,11cyclictest9456-21apt-get22:15:311
9163995739,11cyclictest26-21ksoftirqd/123:18:081
9163995739,10cyclictest26-21ksoftirqd/119:40:241
9163995734,8cyclictest26-21ksoftirqd/100:25:131
9163995729,20cyclictest2316-21apt-get20:25:141
9163995726,24cyclictest276-21systemd-journal20:10:311
9163995726,24cyclictest276-21systemd-journal20:10:311
916399572,17cyclictest14080-21apt-get19:25:481
916399568,35cyclictest0-21swapper/123:00:211
9163995641,10cyclictest26-21ksoftirqd/123:50:141
9163995640,10cyclictest26-21ksoftirqd/120:15:311
9163995640,10cyclictest26-21ksoftirqd/120:15:301
9163995639,12cyclictest26-21ksoftirqd/123:50:441
9163995639,10cyclictest26-21ksoftirqd/122:25:391
9163995638,11cyclictest26-21ksoftirqd/123:35:301
9163995638,11cyclictest0-21swapper/120:25:391
9163995636,13cyclictest4123-21latency_hist22:00:321
9163995634,8cyclictest26-21ksoftirqd/120:05:311
9163995634,17cyclictest26-21ksoftirqd/121:25:281
916399563,26cyclictest29317-21awk00:40:311
916399563,13cyclictest31853-21find21:45:451
916399562,13cyclictest21079-21grep22:45:491
916399557,41cyclictest0-21swapper/119:17:181
916399557,16cyclictest151rcu_preempt20:20:311
916399556,43cyclictest0-21swapper/121:52:461
916399554,44cyclictest26050-21ntp_states21:30:381
9163995540,10cyclictest26-21ksoftirqd/123:05:381
9163995538,11cyclictest0-21swapper/120:55:351
9163995538,10cyclictest26-21ksoftirqd/123:10:401
9163995538,10cyclictest26-21ksoftirqd/123:10:401
9163995538,10cyclictest26-21ksoftirqd/119:15:251
9163995537,11cyclictest26-21ksoftirqd/121:25:431
9163995536,12cyclictest26-21ksoftirqd/119:50:341
9163995536,12cyclictest22627-21find22:50:401
9163995536,12cyclictest22627-21find22:50:401
9163995533,8cyclictest26-21ksoftirqd/122:10:231
9163995533,8cyclictest26-21ksoftirqd/120:30:341
916399545,44cyclictest0-21swapper/120:54:141
916399545,43cyclictest0-21swapper/121:00:501
9163995438,9cyclictest26-21ksoftirqd/121:40:371
9163995438,9cyclictest26-21ksoftirqd/119:25:241
9163995438,10cyclictest26-21ksoftirqd/100:20:151
916399543,45cyclictest29394-21snmpd22:40:351
9163995433,15cyclictest26275-21ps23:00:501
9163995432,7cyclictest26-21ksoftirqd/100:05:291
916399543,11cyclictest27969-21grep21:35:421
9163995429,19cyclictest17899-21cron21:09:001
9163995418,23cyclictest26-21ksoftirqd/119:57:071
916399533,10cyclictest14615-21df_inode00:00:311
9163995237,9cyclictest26-21ksoftirqd/100:05:391
9163995231,16cyclictest26-21ksoftirqd/119:30:331
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:323
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
none18:41:322
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional