You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-24 - 13:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot3.osadl.org (updated Wed Apr 24, 2024 12:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
193679899190,11cyclictest0-21swapper/311:30:013
193679899141,12cyclictest0-21swapper/310:50:163
1936798991312,1cyclictest0-21swapper/311:50:153
1936798991211,0cyclictest0-21swapper/311:30:093
193679899110,8cyclictest0-21swapper/310:05:143
193679899110,11cyclictest0-21swapper/312:14:143
193679899110,11cyclictest0-21swapper/311:00:013
193679899110,11cyclictest0-21swapper/309:25:133
193679899110,11cyclictest0-21swapper/309:20:003
193679899110,11cyclictest0-21swapper/307:45:133
193679899110,0cyclictest0-21swapper/312:00:143
193679899110,0cyclictest0-21swapper/308:45:253
193679899109,0cyclictest0-21swapper/308:10:143
1936798991010,0cyclictest0-21swapper/311:45:013
193679899100,10cyclictest0-21swapper/310:25:203
193679899100,10cyclictest0-21swapper/308:55:133
193679899100,10cyclictest0-21swapper/308:35:233
193679899100,10cyclictest0-21swapper/308:10:003
193679899100,0cyclictest0-21swapper/307:35:013
19367989999,0cyclictest0-21swapper/312:35:023
19367989999,0cyclictest0-21swapper/312:15:113
19367989999,0cyclictest0-21swapper/312:05:123
19367989990,9cyclictest0-21swapper/309:10:113
19367989990,8cyclictest0-21swapper/310:35:543
19367989990,8cyclictest0-21swapper/308:40:113
19367989990,0cyclictest0-21swapper/310:25:013
19367989988,0cyclictest0-21swapper/312:35:133
19367989988,0cyclictest0-21swapper/311:35:223
19367989988,0cyclictest0-21swapper/311:20:123
19367989988,0cyclictest0-21swapper/311:15:103
19367989988,0cyclictest0-21swapper/310:45:233
19367989988,0cyclictest0-21swapper/309:00:133
19367989986,1cyclictest0-21swapper/308:30:143
13129880,3rtkit-daemon0-21swapper/007:09:180
19367989977,0cyclictest0-21swapper/311:05:323
19367989977,0cyclictest0-21swapper/310:32:353
19367989977,0cyclictest0-21swapper/310:10:373
19367989977,0cyclictest0-21swapper/310:00:143
19367989977,0cyclictest0-21swapper/309:59:383
19367989977,0cyclictest0-21swapper/309:45:143
19367989977,0cyclictest0-21swapper/307:35:073
19367989977,0cyclictest0-21swapper/307:27:343
19367989977,0cyclictest0-21swapper/307:10:173
19367989976,1cyclictest0-21swapper/311:45:123
19367989970,7cyclictest0-21swapper/309:40:013
1935958271,4sleep30-21swapper/307:05:203
19367989966,0cyclictest0-21swapper/312:22:253
19367989966,0cyclictest0-21swapper/308:20:463
19367989966,0cyclictest0-21swapper/308:05:003
19367989966,0cyclictest0-21swapper/307:50:113
19367989960,4cyclictest0-21swapper/307:20:163
19367989960,0cyclictest0-21swapper/309:20:263
19367989955,0cyclictest0-21swapper/311:11:253
19367989950,5cyclictest0-21swapper/308:15:113
1934916250,3sleep10-21swapper/107:05:021
13129850,0rtkit-daemon0-21swapper/009:05:160
19367989944,0cyclictest0-21swapper/311:00:103
19367989944,0cyclictest0-21swapper/310:40:143
19367989944,0cyclictest0-21swapper/310:15:153
19367989944,0cyclictest0-21swapper/309:43:313
19367989944,0cyclictest0-21swapper/309:06:173
19367989944,0cyclictest0-21swapper/307:55:123
19367989940,3cyclictest19738702sleep308:50:333
19367989940,3cyclictest0-21swapper/311:57:123
19367989940,3cyclictest0-21swapper/309:31:473
19367989940,0cyclictest0-21swapper/307:20:013
1935477240,2sleep20-21swapper/207:05:152
13129840,2rtkit-daemon356-21systemd-journal11:14:300
13129840,1rtkit-daemon717-21lldpd10:29:560
13129840,1rtkit-daemon717-21lldpd07:50:370
13129840,0rtkit-daemon0-21swapper/012:24:160
2057986230,1sleep12058012-21apt-get12:15:111
1958010230,0sleep2151rcu_preempt08:10:232
1951309230,1sleep01951335-21apt-get07:55:120
1946571230,1sleep21946627-21munin-run07:40:012
19367989930,3cyclictest0-21swapper/307:40:033
19367989930,2cyclictest0-21swapper/312:25:113
19367989930,2cyclictest0-21swapper/309:50:123
19367989930,2cyclictest0-21swapper/308:25:133
13129830,2rtkit-daemon3191333-21snmpd09:35:200
13129830,1rtkit-daemon717-21lldpd12:15:280
13129830,1rtkit-daemon717-21lldpd12:14:280
13129830,1rtkit-daemon717-21lldpd11:15:440
13129830,1rtkit-daemon717-21lldpd10:50:120
13129830,1rtkit-daemon717-21lldpd10:45:420
13129830,1rtkit-daemon717-21lldpd10:22:080
13129830,1rtkit-daemon717-21lldpd10:16:260
13129830,1rtkit-daemon717-21lldpd10:11:080
13129830,1rtkit-daemon717-21lldpd09:45:080
13129830,1rtkit-daemon717-21lldpd09:22:180
13129830,1rtkit-daemon717-21lldpd09:18:030
13129830,1rtkit-daemon717-21lldpd08:58:210
13129830,1rtkit-daemon717-21lldpd08:41:510
13129830,1rtkit-daemon714-21ntpd10:35:010
13129830,1rtkit-daemon617-21gmain11:46:120
13129830,1rtkit-daemon617-21gmain11:25:520
13129830,1rtkit-daemon524-21avahi-daemon11:40:290
13129830,1rtkit-daemon356-21systemd-journal07:40:140
13129830,1rtkit-daemon3191333-21snmpd12:35:130
13129830,1rtkit-daemon3191333-21snmpd11:02:060
13129830,1rtkit-daemon3191333-21snmpd09:55:010
13129830,1rtkit-daemon3191333-21snmpd08:15:220
13129830,1rtkit-daemon3191333-21snmpd07:25:170
13129830,1rtkit-daemon2052415-21python312:00:230
13129830,1rtkit-daemon1980779-21irqstats09:10:150
13129830,1rtkit-daemon1977410-21python309:00:320
13129830,1rtkit-daemon1961695-21strings08:20:180
13129830,1rtkit-daemon1944236-21apt-get07:35:010
13129830,1rtkit-daemon1938792-21munin-node12:30:060
13129830,1rtkit-daemon1938792-21munin-node12:27:470
13129830,1rtkit-daemon1938792-21munin-node11:20:310
13129830,1rtkit-daemon1938792-21munin-node09:51:270
13129830,1rtkit-daemon1938792-21munin-node08:25:250
13129830,1rtkit-daemon1-21systemd12:06:130
13129830,1rtkit-daemon1-21systemd11:56:150
13129830,1rtkit-daemon1-21systemd11:06:000
13129830,1rtkit-daemon1-21systemd10:56:010
13129830,1rtkit-daemon1-21systemd10:41:050
13129830,1rtkit-daemon1-21systemd10:07:520
13129830,1rtkit-daemon1-21systemd10:00:240
13129830,1rtkit-daemon1-21systemd09:40:070
13129830,1rtkit-daemon1-21systemd07:31:130
13129830,1rtkit-daemon1-21systemd07:24:190
13129830,0rtkit-daemon0-21swapper/011:50:160
13129830,0rtkit-daemon0-21swapper/008:14:180
2063565220,1sleep20-21swapper/212:25:382
2060939220,1sleep10-21swapper/112:20:231
2056255220,1sleep12056323-21diskstats12:10:151
2039204220,1sleep22039280-21irqstats11:30:132
2036412220,1sleep22036510-21sed11:25:012
2023306220,0sleep10-21swapper/110:53:141
2014185220,1sleep20-21swapper/210:29:152
2012176220,1sleep12012258-21grep10:25:011
2005612220,1sleep20-21swapper/210:10:022
2002418220,1sleep22002460-21ntp_states10:00:202
1999562220,1sleep10-21swapper/109:55:111
1998123220,1sleep21998228-21ntp_states09:50:202
1984536220,1sleep10-21swapper/109:20:131
1980583220,1sleep20-21swapper/209:10:142
1978083220,1sleep11978195-21latency_hist09:05:001
1969504220,1sleep11969675-21python308:40:191
1953998220,1sleep21954041-21ntp_states08:00:182
1953707220,0sleep00-21swapper/008:00:160
13129820,1rtkit-daemon717-21lldpd07:10:360
13129820,1rtkit-daemon714-21ntpd07:45:320
13129820,1rtkit-daemon539-21wpa_supplicant09:30:560
13129820,1rtkit-daemon3191333-21snmpd11:35:210
13129820,1rtkit-daemon3191333-21snmpd08:45:020
13129820,1rtkit-daemon3191333-21snmpd08:32:130
13129820,1rtkit-daemon1938792-21munin-node11:30:160
13129820,1rtkit-daemon1938792-21munin-node10:30:090
13129820,1rtkit-daemon1-21systemd08:50:110
13129820,1rtkit-daemon1-21systemd08:35:160
13129820,1rtkit-daemon1-21systemd08:05:110
13129820,0rtkit-daemon0-21swapper/009:25:220
13129820,0rtkit-daemon0-21swapper/007:15:010
2066588210,0sleep10-21swapper/112:35:161
2066439210,0sleep20-21swapper/212:35:132
2064426210,0sleep20-21swapper/212:30:172
2064332210,0sleep10-21swapper/112:30:161
2062639210,0sleep10-21swapper/112:25:151
2061106210,0sleep20-21swapper/212:20:242
2058015210,0sleep20-21swapper/212:15:122
2056510210,0sleep20-21swapper/212:10:212
2053947210,0sleep20-21swapper/212:05:152
2053479210,0sleep10-21swapper/112:05:021
2051456210,0sleep20-21swapper/212:00:152
2051427210,0sleep10-21swapper/112:00:141
2049737210,0sleep20-21swapper/211:55:162
2049293210,0sleep10-21swapper/111:55:101
2047178210,0sleep20-21swapper/211:50:032
2046943210,0sleep10-21swapper/111:50:021
2045159210,0sleep20-21swapper/211:45:142
2045127210,0sleep10-21swapper/111:45:131
2042997210,0sleep10-21swapper/111:40:131
2042718210,0sleep20-21swapper/211:40:022
2040969210,0sleep10-21swapper/111:35:171
2040619210,0sleep20-21swapper/211:35:022
2039051210,0sleep10-21swapper/111:30:121
2036793210,0sleep20-21swapper/211:25:132
2036721210,0sleep10-21swapper/111:25:121
2035011210,0sleep10-21swapper/111:20:161
2032820210,0sleep20-21swapper/211:15:172
2032243210,0sleep10-21swapper/111:15:121
2030229210,0sleep10-21swapper/111:10:151
2030093210,0sleep20-21swapper/211:10:122
2028491210,0sleep20-21swapper/211:05:202
2028064210,0sleep10-21swapper/111:05:151
2025979210,0sleep20-21swapper/211:00:122
2025912210,0sleep10-21swapper/111:00:111
2024221210,0sleep20-21swapper/210:55:202
2023760210,0sleep10-21swapper/110:55:111
2022154210,0sleep20-21swapper/210:50:232
2019986210,0sleep10-21swapper/110:45:181
2019568210,0sleep20-21swapper/210:45:152
2017440210,0sleep20-21swapper/210:40:032
2017181210,0sleep10-21swapper/110:40:021
2015745210,0sleep20-21swapper/210:35:102
2015081210,0sleep10-21swapper/110:35:011
2014531210,0sleep20-21swapper/210:30:142
2014272210,0sleep10-21swapper/110:30:011
2012428210,0sleep10-21swapper/110:25:131
2010323210,0sleep20-21swapper/210:20:062
2008239210,0sleep10-21swapper/110:15:121
2007956210,0sleep20-21swapper/210:15:012
2006127210,0sleep12006155-21apt-get10:10:121
2004204210,0sleep10-21swapper/110:05:201
2003865210,0sleep20-21swapper/210:05:172
2002221210,0sleep10-21swapper/110:00:191
1999631210,0sleep20-21swapper/209:55:122
1998023210,0sleep10-21swapper/109:50:191
1995437210,0sleep20-21swapper/209:45:162
1995375210,0sleep10-21swapper/109:45:151
1993667210,0sleep20-21swapper/209:40:152
1992962210,0sleep10-21swapper/109:40:021
1991045210,0sleep10-21swapper/109:35:141
1990867210,0sleep20-21swapper/209:35:072
1988788210,0sleep20-21swapper/209:30:112
1988757210,0sleep10-21swapper/109:30:101
1986708210,0sleep20-21swapper/209:25:162
1986649210,0sleep10-21swapper/109:25:141
1984566210,0sleep20-21swapper/209:20:142
1982645210,0sleep20-21swapper/209:15:142
1982515210,0sleep10-21swapper/109:15:131
1980007210,0sleep10-21swapper/109:10:011
1979107210,0sleep20-21swapper/209:05:212
1978522210,0sleep10-21swapper/109:05:161
1976139210,0sleep20-21swapper/209:00:012
1975253210,0sleep10-21swapper/108:55:241
1974217210,0sleep20-21swapper/208:55:012
1972539210,0sleep10-21swapper/108:50:121
1972292210,0sleep20-21swapper/208:50:012
1970651210,0sleep10-21swapper/108:45:151
1970368210,0sleep20-21swapper/208:45:002
1968751210,0sleep20-21swapper/208:40:122
1966983210,0sleep10-21swapper/108:35:151
1966746210,0sleep20-21swapper/208:35:122
1965686210,0sleep20-21swapper/208:30:242
1964837210,0sleep10-21swapper/108:30:151
1964561210,0sleep20-21swapper/208:29:572
1962927210,0sleep10-21swapper/108:25:141
1961079210,0sleep20-21swapper/208:20:152
1960998210,0sleep10-21swapper/108:20:141
1959551210,0sleep10-21swapper/108:15:181
1959311210,0sleep20-21swapper/208:15:152
1957103210,0sleep10-21swapper/108:10:141
1955903210,0sleep20-21swapper/208:05:182
1955362210,0sleep10-21swapper/108:05:131
1953022210,0sleep10-21swapper/108:00:011
1951462210,0sleep20-21swapper/207:55:152
1951338210,0sleep10-21swapper/107:55:131
1950308210,0sleep10-21swapper/107:50:321
1949140210,0sleep20-21swapper/207:50:012
1947949210,0sleep10-21swapper/107:45:121
1947548210,0sleep20-21swapper/207:45:092
1947207210,0sleep10-21swapper/107:44:561
1944723210,0sleep10-21swapper/107:35:081
1944217210,0sleep20-21swapper/207:35:012
1943856210,0sleep10-21swapper/107:30:231
1941954210,0sleep20-21swapper/207:25:122
1941915210,0sleep10-21swapper/107:25:111
1940851210,0sleep10-21swapper/107:20:151
1939985210,0sleep20-21swapper/207:20:082
1939708210,0sleep10-21swapper/107:19:511
1937329210,0sleep10-21swapper/107:10:141
1936963210,0sleep20-21swapper/207:10:112
1946829200,0sleep20-21swapper/207:40:052
1939333200,0sleep20-21swapper/207:15:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional