You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 12:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot3.osadl.org (updated Thu Dec 04, 2025 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2023585991310,2cyclictest69450irq/121-eno122:40:123
202358599130,11cyclictest0-21swapper/321:40:013
202358599130,11cyclictest0-21swapper/319:35:133
202358599130,11cyclictest0-21swapper/319:10:123
202358599130,11cyclictest0-21swapper/300:10:153
202358599128,0cyclictest0-21swapper/323:00:133
2023585991210,1cyclictest0-21swapper/322:00:123
202358599120,8cyclictest0-21swapper/322:10:143
202358599120,1cyclictest0-21swapper/320:15:113
202358599120,1cyclictest0-21swapper/320:05:153
202358599120,1cyclictest0-21swapper/319:20:133
202358599120,11cyclictest0-21swapper/321:50:143
202358599120,11cyclictest0-21swapper/321:05:133
202358599120,11cyclictest0-21swapper/319:40:183
202358599119,1cyclictest0-21swapper/323:45:123
202358599119,1cyclictest0-21swapper/320:30:123
2023585991111,0cyclictest69450irq/121-eno122:29:213
2023585991111,0cyclictest0-21swapper/322:20:123
2023585991111,0cyclictest0-21swapper/322:20:123
2023585991110,0cyclictest69450irq/121-eno122:30:203
202358599110,1cyclictest20330582sleep319:30:163
202358599110,10cyclictest69450irq/121-eno123:20:133
202358599110,10cyclictest69450irq/121-eno123:05:283
202358599110,10cyclictest69450irq/121-eno122:35:163
202358599110,10cyclictest69450irq/121-eno120:40:143
202358599110,10cyclictest69450irq/121-eno120:26:233
202358599110,10cyclictest69450irq/121-eno120:00:183
202358599110,10cyclictest69450irq/121-eno100:05:183
202358599110,10cyclictest0-21swapper/323:10:193
202358599110,10cyclictest0-21swapper/321:30:173
202358599110,10cyclictest0-21swapper/300:30:013
202358599110,0cyclictest0-21swapper/323:15:133
202358599110,0cyclictest0-21swapper/321:00:133
202358599110,0cyclictest0-21swapper/320:55:143
202358599109,0cyclictest0-21swapper/322:05:163
202358599109,0cyclictest0-21swapper/320:10:113
2023585991010,0cyclictest69450irq/121-eno123:58:043
2023585991010,0cyclictest69450irq/121-eno122:58:223
2023585991010,0cyclictest69450irq/121-eno121:23:173
2023585991010,0cyclictest0-21swapper/320:45:113
2023585991010,0cyclictest0-21swapper/319:15:133
2023585991010,0cyclictest0-21swapper/300:15:163
202358599100,9cyclictest69450irq/121-eno121:36:333
202358599100,9cyclictest69450irq/121-eno121:28:393
202358599100,9cyclictest0-21swapper/323:35:203
202358599100,9cyclictest0-21swapper/321:20:013
20235859999,0cyclictest69450irq/121-eno122:45:043
20235859999,0cyclictest69450irq/121-eno121:49:353
20235859999,0cyclictest69450irq/121-eno120:20:173
20235859999,0cyclictest69450irq/121-eno100:35:293
20235859999,0cyclictest69450irq/121-eno100:31:133
20235859999,0cyclictest0-21swapper/322:50:133
20235859999,0cyclictest0-21swapper/300:05:013
20235859988,0cyclictest69450irq/121-eno123:30:013
20235859988,0cyclictest69450irq/121-eno119:26:493
20235859988,0cyclictest69450irq/121-eno100:21:333
20235859980,8cyclictest0-21swapper/322:20:013
20235859980,7cyclictest69450irq/121-eno119:51:023
20235859980,7cyclictest0-21swapper/320:50:183
20235859970,6cyclictest69450irq/121-eno123:50:223
20235859970,6cyclictest69450irq/121-eno123:26:453
20235859966,0cyclictest69450irq/121-eno121:55:143
20235859965,1cyclictest69450irq/121-eno120:38:393
20235859960,5cyclictest69450irq/121-eno119:45:153
20235859955,0cyclictest69450irq/121-eno123:40:233
20235859950,4cyclictest69450irq/121-eno121:10:133
2110215240,3sleep22110443-21sed22:45:022
20235859940,3cyclictest0-21swapper/319:55:133
2022409241,1sleep10-21swapper/119:05:181
2021543241,2sleep30-21swapper/319:05:013
2155342230,1sleep02155496-21ntp_states00:30:180
2104178230,1sleep22104204-21apt-get22:30:132
2095545230,2sleep02095659-21latency_hist22:10:010
2088280230,2sleep02088433-21sed21:50:200
2085492230,1sleep12085765-21df_inode21:45:161
2082707230,1sleep22082724-21apt-get21:40:012
2064902230,1sleep12064928-21apt-get20:55:141
2058711230,0sleep10-21swapper/120:37:451
2051484230,1sleep12051510-21apt-get20:20:151
2038662230,2sleep02038812-21/usr/sbin/munin19:45:160
2021787233,0sleep00-21swapper/019:05:100
2021784231,1sleep20-21swapper/219:05:072
2157236220,1sleep10-21swapper/100:35:191
2147764220,1sleep12147768-21ssh00:11:261
2146087220,1sleep02146230-21latency_hist00:10:010
2139283220,1sleep00-21swapper/023:50:360
2131052220,1sleep10-21swapper/123:31:511
2128274220,1sleep02128432-21mailstats23:25:180
2126177220,1sleep20-21swapper/223:20:202
2124034220,1sleep12124143-21ps23:15:171
2122929220,0sleep00-21swapper/023:14:090
2120572220,0sleep00-21swapper/023:06:450
2111439220,1sleep02111715-21/usr/sbin/munin22:45:200
2106735220,1sleep22106891-21memory22:35:172
2099352220,1sleep00-21swapper/022:16:460
2090852220,1sleep00-21swapper/021:55:400
2089776220,1sleep22089955-21fw_packets21:55:142
2085428220,1sleep00-21swapper/021:45:160
2070819220,1sleep12071087-21df21:10:141
2069478220,1sleep12069581-21grep21:05:171
2067018220,1sleep22067265-21diskstats21:00:152
2066566220,1sleep02066661-21cat21:00:010
2050036220,1sleep12050203-21cat20:15:151
2047364220,0sleep00-21swapper/020:09:460
2045721220,1sleep12045747-21apt-get20:05:151
2042489220,1sleep12042643-21ntp_states19:55:181
2041993220,1sleep22042289-21df19:55:152
2034664220,1sleep12034817-21memory19:35:171
2028820220,1sleep12028904-21irqstats19:20:161
2025467220,1sleep22026041-21ntp_offset19:15:002
2158482210,1sleep22158484-21ssh00:38:412
2156970210,0sleep00-21swapper/000:35:190
2156162210,0sleep10-21swapper/100:31:351
2155140210,0sleep20-21swapper/200:30:162
2153352210,0sleep20-21swapper/200:25:202
2153151210,0sleep10-21swapper/100:25:191
2152404210,0sleep00-21swapper/000:25:020
2150729210,0sleep20-21swapper/200:20:152
2150583210,0sleep10-21swapper/100:20:151
2150267210,0sleep00-21swapper/000:20:020
2149935210,0sleep00-21swapper/000:17:090
2148673210,0sleep10-21swapper/100:15:181
2147838210,0sleep00-21swapper/000:12:040
2146830210,0sleep20-21swapper/200:10:192
2144700210,0sleep20-21swapper/200:05:192
2144240210,0sleep10-21swapper/100:05:061
2143922210,0sleep10-21swapper/100:04:221
2143541210,0sleep00-21swapper/000:01:000
2140560210,0sleep10-21swapper/123:55:161
2140478210,0sleep00-21swapper/023:55:150
2140100210,0sleep20-21swapper/223:55:142
2139246210,0sleep20-21swapper/223:50:222
2138402210,0sleep10-21swapper/123:50:171
2137339210,1sleep00-21swapper/023:46:510
2136630210,0sleep22136687-21pmu-power23:45:172
2136480210,0sleep10-21swapper/123:45:161
2135102210,0sleep10-21swapper/123:40:311
2134828210,0sleep00-21swapper/023:40:230
2132152210,1sleep00-21swapper/023:35:190
2131696210,0sleep10-21swapper/123:35:161
2131658210,0sleep20-21swapper/223:35:092
2130290210,0sleep20-21swapper/223:30:162
2128961210,0sleep10-21swapper/123:26:511
2128594210,0sleep20-21swapper/223:25:202
2125434210,0sleep12125539-21aten_r1power_po23:20:151
2125405210,0sleep00-21swapper/023:20:140
2123542210,0sleep20-21swapper/223:15:152
2123288210,0sleep00-21swapper/023:15:130
2122790210,0sleep20-21swapper/223:12:552
2122581210,0sleep10-21swapper/123:11:061
2120672210,0sleep20-21swapper/223:07:392
2118858210,0sleep10-21swapper/123:05:021
2117455210,0sleep00-21swapper/023:00:160
2117049210,0sleep20-21swapper/223:00:132
2117020210,0sleep10-21swapper/123:00:131
2116481210,0sleep12116533-21ssh23:00:011
2116217210,0sleep00-21swapper/022:57:400
2115364210,0sleep20-21swapper/222:55:182
2113917210,0sleep00-21swapper/022:50:520
2113070210,0sleep10-21swapper/122:50:171
2112304210,0sleep20-21swapper/222:50:022
2108980210,0sleep10-21swapper/122:40:161
2107176210,0sleep10-21swapper/122:35:191
2107048210,0sleep00-21swapper/022:35:180
2104303210,0sleep00-21swapper/022:30:150
2102255210,0sleep10-21swapper/122:25:151
2102084210,0sleep20-21swapper/222:25:122
2101842210,0sleep00-21swapper/022:25:020
2100436210,0sleep20-21swapper/222:20:162
2100436210,0sleep20-21swapper/222:20:152
2100225210,0sleep10-21swapper/122:20:151
2100225210,0sleep10-21swapper/122:20:151
2100087210,0sleep00-21swapper/022:20:140
2100087210,0sleep00-21swapper/022:20:140
2097891210,0sleep20-21swapper/222:15:162
2096598210,0sleep00-21swapper/022:10:210
2096315210,0sleep20-21swapper/222:10:192
2096233210,0sleep10-21swapper/122:10:181
2094724210,0sleep20-21swapper/222:05:232
2093459210,0sleep10-21swapper/122:05:021
2093257210,0sleep10-21swapper/122:03:171
2093143211,0sleep0171ktimers/022:02:160
2091997210,0sleep20-21swapper/222:00:152
2087425210,0sleep20-21swapper/221:50:142
2085080210,0sleep20-21swapper/221:45:012
2084940210,0sleep10-21swapper/121:43:491
2083530210,0sleep00-21swapper/021:40:150
2081451210,0sleep20-21swapper/221:35:172
2081367210,0sleep10-21swapper/121:35:161
2081119210,0sleep00-21swapper/021:35:150
2079425210,0sleep20-21swapper/221:30:162
2079067210,0sleep00-21swapper/021:30:150
2078608210,0sleep20-21swapper/221:29:092
2078434210,0sleep00-21swapper/021:27:230
2077715210,0sleep10-21swapper/121:25:191
2076336210,0sleep10-21swapper/121:21:271
2076300210,0sleep20-21swapper/221:21:002
2076107210,0sleep00-21swapper/021:20:210
2074044210,0sleep10-21swapper/121:16:271
2072814210,0sleep20-21swapper/221:15:162
2072649210,0sleep00-21swapper/021:15:140
2071598210,0sleep20-21swapper/221:10:202
2068981210,0sleep20-21swapper/221:05:142
2068748210,0sleep00-21swapper/021:05:130
2066863210,0sleep10-21swapper/121:00:141
2066834210,0sleep00-21swapper/021:00:130
2064931210,0sleep20-21swapper/220:55:152
2062735210,0sleep10-21swapper/120:50:011
2061970210,0sleep20-21swapper/220:45:182
2061059210,1sleep00-21swapper/020:45:040
2060823210,0sleep10-21swapper/120:45:011
2060532210,0sleep10-21swapper/120:41:211
2059686210,0sleep22059898-21ntpq20:40:182
2058465210,0sleep20-21swapper/220:35:212
2055738210,0sleep00-21swapper/020:30:150
2055306210,0sleep10-21swapper/120:30:121
2055058210,0sleep20-21swapper/220:30:012
2053431210,0sleep10-21swapper/120:25:111
2053391210,0sleep00-21swapper/020:25:110
2052454210,0sleep00-21swapper/020:20:220
2052200210,0sleep20-21swapper/220:20:202
2050475210,0sleep00-21swapper/020:15:180
2050205210,0sleep20-21swapper/220:15:162
2049236210,0sleep20-21swapper/220:14:152
2047753210,0sleep00-21swapper/020:10:140
2046399210,0sleep20-21swapper/220:05:202
2043992210,0sleep20-21swapper/220:00:152
2043537210,0sleep00-21swapper/020:00:000
2041871210,0sleep00-21swapper/019:55:130
2040509210,0sleep00-21swapper/019:50:170
2040214210,0sleep20-21swapper/219:50:152
2040018210,0sleep10-21swapper/119:50:141
2038946210,0sleep20-21swapper/219:45:182
2037800210,0sleep10-21swapper/119:45:001
2037415210,0sleep00-21swapper/019:40:200
2036875210,0sleep10-21swapper/119:40:171
2035678210,0sleep20-21swapper/219:37:382
2035103210,0sleep00-21swapper/019:35:200
2033610210,0sleep00-21swapper/019:30:390
2032750210,0sleep10-21swapper/119:30:161
2032075210,0sleep20-21swapper/219:30:002
2031195210,0sleep20-21swapper/219:25:172
2030912210,0sleep00-21swapper/019:25:150
2030424210,0sleep10-21swapper/119:25:121
2029184210,0sleep00-21swapper/019:20:180
2028119210,0sleep20-21swapper/219:20:012
2028111210,0sleep10-21swapper/119:19:561
2027752210,0sleep00-21swapper/019:15:220
2024643210,1sleep00-21swapper/019:10:180
2024332210,0sleep20-21swapper/219:10:162
2024197210,0sleep10-21swapper/119:10:151
2148409200,0sleep20-21swapper/200:15:032
2142146200,0sleep20-21swapper/200:00:022
2133504200,0sleep20-21swapper/223:40:002
2129559200,0sleep00-21swapper/023:30:010
2110452200,0sleep20-21swapper/222:45:062
2110451200,0sleep10-21swapper/122:45:051
2108361200,0sleep00-21swapper/022:40:030
2104163200,0sleep10-21swapper/122:30:041
2097867200,0sleep10-21swapper/122:15:031
2089258200,0sleep10-21swapper/121:55:011
2087410200,0sleep10-21swapper/121:50:051
2078933200,0sleep10-21swapper/121:30:041
2070413200,0sleep00-21swapper/021:10:010
2062970200,0sleep00-21swapper/020:50:040
2062967200,0sleep20-21swapper/220:50:022
2059146200,0sleep00-21swapper/020:40:030
2057211200,0sleep00-21swapper/020:35:030
2053383200,0sleep20-21swapper/220:25:042
2047619200,0sleep10-21swapper/120:10:031
2043793200,0sleep10-21swapper/120:00:021
2036119200,0sleep20-21swapper/219:40:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional