You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-16 - 07:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6.osadl.org (updated Mon Sep 16, 2024 00:45:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31178999780,11cyclictest26-21ksoftirqd/120:00:221
11432970,5chrt1142-21sshd23:42:440
301602968,71sleep030215-21/usr/sbin/munin19:05:260
31178999066,11cyclictest26-21ksoftirqd/123:55:151
31178998962,14cyclictest26-21ksoftirqd/122:35:251
31178998854,24cyclictest26-21ksoftirqd/121:20:111
31178998760,14cyclictest26-21ksoftirqd/121:35:241
31178998658,16cyclictest26-21ksoftirqd/121:00:141
31178998658,16cyclictest26-21ksoftirqd/100:05:141
31178998657,19cyclictest26-21ksoftirqd/100:30:131
31178998657,16cyclictest26-21ksoftirqd/121:45:111
31178998656,20cyclictest26-21ksoftirqd/100:20:101
3117199868,13cyclictest0-21swapper/022:50:110
95162850,6sleep0171rcuc/019:35:380
31178998556,16cyclictest26-21ksoftirqd/120:45:111
31178998555,19cyclictest26-21ksoftirqd/119:15:131
31178998552,18cyclictest26-21ksoftirqd/122:00:161
31178998461,14cyclictest26-21ksoftirqd/123:40:141
31178998458,16cyclictest26-21ksoftirqd/121:50:151
31178998456,18cyclictest26-21ksoftirqd/122:10:221
31178998355,18cyclictest26-21ksoftirqd/120:35:121
31178998355,18cyclictest26-21ksoftirqd/120:35:121
31178998354,20cyclictest26-21ksoftirqd/119:55:121
31178998353,20cyclictest26-21ksoftirqd/120:25:171
112332830,5chrt151rcu_preempt21:10:300
31178998257,16cyclictest26-21ksoftirqd/121:15:221
31178998153,16cyclictest26-21ksoftirqd/123:30:121
31178998153,16cyclictest26-21ksoftirqd/119:50:141
31178998151,19cyclictest26-21ksoftirqd/120:15:131
31178998149,18cyclictest26-21ksoftirqd/121:25:121
31178998054,18cyclictest26-21ksoftirqd/100:35:161
31178998053,18cyclictest26-21ksoftirqd/121:10:121
31178998047,22cyclictest26-21ksoftirqd/121:30:121
31178997954,20cyclictest26-21ksoftirqd/122:45:131
31178997849,17cyclictest26-21ksoftirqd/119:10:141
31178997753,14cyclictest26-21ksoftirqd/121:55:141
31178997752,16cyclictest26-21ksoftirqd/122:55:131
31178997748,15cyclictest0-21swapper/122:25:121
31178997747,19cyclictest26-21ksoftirqd/119:45:131
31178997649,16cyclictest26-21ksoftirqd/122:20:111
31171997614,43cyclictest0-21swapper/020:30:140
31171997526,16cyclictest14-21ksoftirqd/020:40:120
31178997449,18cyclictest26-21ksoftirqd/123:10:141
31178997447,17cyclictest0-21swapper/123:25:221
31171997436,22cyclictest0-21swapper/023:50:140
31171997349,14cyclictest17801-21apt-get21:30:140
3081827339,25sleep10-21swapper/119:08:091
31178997244,16cyclictest0-21swapper/122:05:141
31178997044,20cyclictest26-21ksoftirqd/122:40:141
31178996938,19cyclictest0-21swapper/122:50:121
31171996927,18cyclictest7088-21aten2.4-expect22:30:160
31178996848,16cyclictest26-21ksoftirqd/120:10:241
31178996839,18cyclictest0-21swapper/123:00:131
31178996838,17cyclictest0-21swapper/119:40:241
31178996838,17cyclictest0-21swapper/119:40:241
3117199685,44cyclictest151rcu_preempt23:35:020
31171996828,20cyclictest14-21ksoftirqd/021:55:130
31178996743,16cyclictest31164-21cyclictest23:05:121
31178996644,16cyclictest26-21ksoftirqd/120:30:131
31178996637,17cyclictest0-21swapper/121:05:121
31178996637,17cyclictest0-21swapper/121:05:121
31171996623,15cyclictest14-21ksoftirqd/022:00:130
31171996619,35cyclictest0-21swapper/022:10:240
31178996538,16cyclictest0-21swapper/120:05:121
31178996537,22cyclictest26-21ksoftirqd/119:25:141
31178996537,17cyclictest0-21swapper/120:50:131
31178996535,22cyclictest26-21ksoftirqd/123:20:141
31171996425,16cyclictest25195-21aten2.4-expect20:20:170
31171996422,15cyclictest12224-21seq19:45:110
31171996419,27cyclictest14-21ksoftirqd/023:37:180
241022640,19sleep126-21ksoftirqd/123:15:361
31178996338,15cyclictest0-21swapper/119:30:231
31171996339,18cyclictest14-21ksoftirqd/021:40:180
31171996319,32cyclictest14-21ksoftirqd/019:25:110
104972630,6chrt9993-21apt-get19:40:150
104972630,6chrt9993-21apt-get19:40:150
31178996238,16cyclictest31164-21cyclictest00:25:141
31178996232,19cyclictest0-21swapper/123:50:121
176582620,6sleep0171rcuc/021:30:010
31178996138,18cyclictest26-21ksoftirqd/119:20:131
31178996135,19cyclictest31164-21cyclictest20:55:231
838026023,9sleep126-21ksoftirqd/119:35:141
31178996014,38cyclictest0-21swapper/122:15:151
3117199609,45cyclictest0-21swapper/023:45:310
31178995937,14cyclictest231irq_work/123:45:141
31178995930,21cyclictest23218-21latency_hist21:44:591
31178995927,23cyclictest26659-21cron20:24:591
3117199598,45cyclictest0-21swapper/019:20:210
31171995936,8cyclictest14-21ksoftirqd/023:05:350
31171995935,9cyclictest14-21ksoftirqd/019:30:370
31171995919,18cyclictest17849-21aten2.4-expect20:00:260
31171995916,31cyclictest14-21ksoftirqd/022:25:140
31178995831,19cyclictest475-21dbus-daemon23:39:001
31178995826,25cyclictest26-21ksoftirqd/100:10:141
31171995841,10cyclictest14-21ksoftirqd/021:35:350
31171995835,8cyclictest14-21ksoftirqd/022:55:180
31171995835,8cyclictest14-21ksoftirqd/000:15:350
3117199583,48cyclictest4903-21latency_hist20:55:020
31178995734,9cyclictest26-21ksoftirqd/100:01:171
31178995733,9cyclictest26-21ksoftirqd/122:30:181
3117199579,42cyclictest0-21swapper/000:25:210
3117199579,36cyclictest0-21swapper/023:15:110
31171995743,7cyclictest14-21ksoftirqd/021:20:000
31171995740,10cyclictest14-21ksoftirqd/020:20:010
31171995735,8cyclictest14-21ksoftirqd/023:55:200
31171995735,8cyclictest14-21ksoftirqd/022:40:350
31171995735,17cyclictest14-21ksoftirqd/021:50:370
31171995734,9cyclictest14-21ksoftirqd/023:20:260
31171995734,8cyclictest14-21ksoftirqd/021:00:250
31171995734,18cyclictest14-21ksoftirqd/000:20:160
31171995734,17cyclictest14-21ksoftirqd/019:15:030
31178995629,22cyclictest26-21ksoftirqd/100:15:141
3117199569,42cyclictest0-21swapper/023:29:400
31171995640,10cyclictest14-21ksoftirqd/000:35:010
31171995639,11cyclictest14-21ksoftirqd/020:05:340
31171995637,12cyclictest14-21ksoftirqd/019:10:120
31171995633,8cyclictest14-21ksoftirqd/023:10:160
31171995633,8cyclictest14-21ksoftirqd/020:10:370
31171995632,8cyclictest14-21ksoftirqd/000:05:220
31171995630,19cyclictest14-21ksoftirqd/021:20:110
31171995623,27cyclictest0-21swapper/020:26:490
31171995539,10cyclictest14-21ksoftirqd/019:55:390
31171995539,10cyclictest14-21ksoftirqd/000:35:220
31171995534,8cyclictest14-21ksoftirqd/022:45:260
31171995534,16cyclictest14-21ksoftirqd/021:45:200
31171995534,16cyclictest14-21ksoftirqd/021:05:170
31171995534,16cyclictest14-21ksoftirqd/021:05:170
31171995533,8cyclictest14-21ksoftirqd/019:50:230
31171995532,8cyclictest14-21ksoftirqd/022:35:470
31171995532,8cyclictest14-21ksoftirqd/022:15:360
307422550,5sleep0171rcuc/022:05:240
3117199546,43cyclictest0-21swapper/020:45:210
31171995439,9cyclictest14-21ksoftirqd/000:15:010
31171995432,8cyclictest14-21ksoftirqd/022:20:180
31171995432,17cyclictest14-21ksoftirqd/020:55:290
31171995413,28cyclictest151rcu_preempt00:00:040
31171995331,8cyclictest14-21ksoftirqd/020:35:140
31171995331,8cyclictest14-21ksoftirqd/020:35:140
3117199532,19cyclictest18170-21expr23:00:210
31178995225,20cyclictest250-21systemd-journal20:41:171
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:453
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
none11:43:452
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional