You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-16 - 14:48
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Thu Oct 16, 2025 00:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
206627721080,1sleep20-21swapper/223:23:002
1704077992512,5cyclictest1873806-21cpuspeed_turbos21:25:122
1704077992410,5cyclictest3818356-21systemd-journal21:25:002
1704068992420,3cyclictest514-21systemd-logind23:00:010
1704077992317,4cyclictest0-21swapper/222:40:002
1704077992313,2cyclictest1-21systemd20:30:012
1704073992319,2cyclictest2091390-21cron23:40:001
1704082992219,1cyclictest0-21swapper/320:45:003
1704082992219,1cyclictest0-21swapper/300:00:013
1704077992218,3cyclictest761-21cups-browsed23:00:002
1704073992219,2cyclictest1929866-21sendmail22:00:011
1704073992219,2cyclictest1929866-21sendmail22:00:001
1704073992219,2cyclictest0-21swapper/100:00:011
1704068992219,2cyclictest500-21avahi-daemon20:55:010
1704068992219,2cyclictest2002621-21idleruntime-cro22:45:010
1704068992219,2cyclictest1945976-21grep22:10:000
1704068992219,2cyclictest1897557-21sendmail21:39:590
1704068992219,2cyclictest0-21swapper/000:00:000
1704068992218,3cyclictest3004-21cron22:35:010
1704068992218,3cyclictest2132765-21irqcore00:05:200
1704082992118,1cyclictest0-21swapper/321:00:013
1704077992118,2cyclictest2148038-21apt-get00:15:012
1704077992116,3cyclictest1774113-21latency_hist20:05:012
1704077992116,3cyclictest1732375-21kthreadcore19:30:192
1704082992017,1cyclictest0-21swapper/320:40:013
1704077992017,2cyclictest0-21swapper/221:20:002
1704077992012,4cyclictest1792539-21systemd-run20:20:012
1704073992017,2cyclictest0-21swapper/100:30:001
1704073992015,3cyclictest1774450-21cpuspeed_turbos20:05:141
1704068992017,2cyclictest2180857-21latency_hist00:35:010
1704068992016,2cyclictest1867001-21kthreadcore21:20:190
170407799193,6cyclictest2057814-21sh23:17:072
1704077991914,3cyclictest2148322-21apt-key00:15:012
1704077991914,3cyclictest0-21swapper/219:55:012
1704077991914,2cyclictest0-21swapper/200:00:012
1704077991913,4cyclictest1971558-21irqrtprio22:25:142
1704073991916,2cyclictest1962090-21sendmail22:20:001
1704073991916,2cyclictest1961830-21kworker/u8:1-rpciod23:20:001
1704073991916,2cyclictest1898856-21irqrtprio21:40:171
1704073991916,2cyclictest1724728-21latency_hist19:25:011
1704068991916,2cyclictest2059699-21fschecks_count23:20:150
1704068991916,2cyclictest0-21swapper/020:20:010
1704068991916,2cyclictest0-21swapper/019:45:010
1704068991915,3cyclictest500-21avahi-daemon20:00:010
1704068991915,3cyclictest2099526-21idleruntime-cro23:45:000
1704082991815,1cyclictest0-21swapper/321:10:243
170407799181,4cyclictest1986246-21idleruntime-cro22:35:012
1704077991814,3cyclictest1962706-21ssh22:20:182
1704077991813,4cyclictest1823782-21idleruntime-cro20:45:002
1704077991813,3cyclictest1712674-21cpuspeed_turbos19:15:132
1704077991812,5cyclictest2085012-21irqrtprio23:35:152
1704077991812,4cyclictest1843272-21cpuspeed_turbos21:05:102
1704073991815,2cyclictest2076051-21irqcore23:30:141
1704073991814,3cyclictest1976555-21sshd22:26:211
1704068991815,2cyclictest522-21in:imuxsock21:35:000
1704068991814,3cyclictest1824409-21gltestperf20:45:130
1704068991814,2cyclictest2148006-21idleruntime-cro00:15:010
1704068991812,2cyclictest471rcuog/222:25:010
1704082991714,1cyclictest0-21swapper/321:20:003
1704082991714,1cyclictest0-21swapper/320:20:013
1704082991713,1cyclictest0-21swapper/319:20:003
1704077991714,2cyclictest1894845-21ntp_kernel_err21:35:192
1704077991713,3cyclictest2104972-21ntp_kernel_pll_23:45:212
1704077991713,3cyclictest1940038-21kthreadcore22:05:182
1704077991713,3cyclictest1918869-21kthreadcore21:50:192
1704077991713,3cyclictest1743650-21gltestperf19:40:182
1704077991713,2cyclictest0-21swapper/223:35:002
1704077991712,4cyclictest1923213-21irqrtprio21:55:172
1704077991712,4cyclictest1923213-21irqrtprio21:55:162
1704077991712,3cyclictest1849027-21diskmemload23:40:182
170407799171,10cyclictest1841369-21ntp_states20:55:222
170407799170,6cyclictest1779828-21runrttasks20:05:412
1704073991714,2cyclictest508-21polkitd21:15:001
1704073991714,2cyclictest2043200-21latency_hist23:10:011
1704073991714,2cyclictest1963087-21irqcore22:20:191
1704073991714,2cyclictest0-21swapper/120:25:001
1704073991713,3cyclictest1805113-21unin-run20:30:011
1704073991713,3cyclictest1762909-21cut19:55:151
1704073991712,3cyclictest1911521-21grep21:45:241
1704073991710,5cyclictest1718392-21cron19:19:591
1704068991714,2cyclictest1774028-21cron20:05:000
1704068991714,2cyclictest1730877-21latency_hist19:30:000
1704068991713,3cyclictest2088261-21kthreadcore23:35:170
1704068991713,3cyclictest2080793-21processes23:30:210
1704068991713,3cyclictest2048904-21unixbench_multi23:10:240
1704082991613,1cyclictest0-21swapper/323:30:013
1704082991613,1cyclictest0-21swapper/323:15:153
1704082991613,1cyclictest0-21swapper/322:35:013
1704082991613,1cyclictest0-21swapper/320:30:013
1704082991613,1cyclictest0-21swapper/319:50:003
1704082991613,1cyclictest0-21swapper/319:21:403
1704082991613,1cyclictest0-21swapper/319:15:003
170407799164,4cyclictest769-21runrttasks22:46:202
170407799163,6cyclictest2011078-21diskmemload22:50:122
170407799163,4cyclictest2187524-21sh00:36:482
1704077991613,2cyclictest2137770-21unixbench_multi00:05:292
1704077991613,2cyclictest1813122-21tr20:35:142
1704077991613,2cyclictest1786318-21cron20:15:002
1704077991612,3cyclictest1793411-21idleruntime20:20:162
1704077991611,3cyclictest0-21swapper/219:15:012
1704077991610,4cyclictest2000513-21unixbench_multi22:40:242
170407799160,2cyclictest1937912-21grep22:05:002
1704073991614,1cyclictest1761710-21cron19:55:011
1704073991613,2cyclictest2010916-21latency_hist22:50:001
1704073991613,2cyclictest0-21swapper/100:10:181
1704073991612,3cyclictest1747681-21kthreadcore19:40:221
1704073991612,2cyclictest524-21rs:main1
1704073991612,2cyclictest2132061-21latency_hist00:05:011
1704073991612,2cyclictest0-21swapper/120:20:011
1704073991612,2cyclictest0-21swapper/119:15:011
170407399160,15cyclictest1879703-21kworker/u8:3+rpciod22:35:011
170407399160,14cyclictest1937911-21munin-run22:05:011
170406899163,11cyclictest2099844-21/usr/sbin/munin23:45:190
1704068991613,2cyclictest2150001-21kthreadcore00:15:160
1704068991613,2cyclictest1860533-21kthreadcore21:15:190
1704068991613,2cyclictest1808540-21kthreadcore20:30:180
1704068991612,3cyclictest1817993-21cpuspeed_turbos20:40:160
1704068991612,2cyclictest1938061-21latency_hist22:05:010
1704082991512,1cyclictest0-21swapper/323:50:003
1704082991512,1cyclictest0-21swapper/323:40:213
1704082991512,1cyclictest0-21swapper/323:30:173
1704082991512,1cyclictest0-21swapper/323:25:003
1704082991512,1cyclictest0-21swapper/322:10:213
1704082991512,1cyclictest0-21swapper/321:57:573
1704082991512,1cyclictest0-21swapper/321:57:563
1704082991512,1cyclictest0-21swapper/321:39:593
1704082991512,1cyclictest0-21swapper/321:25:133
1704082991512,1cyclictest0-21swapper/319:35:203
1704082991512,1cyclictest0-21swapper/319:35:003
1704082991512,1cyclictest0-21swapper/300:27:123
170407799159,4cyclictest2032408-21ntpq23:00:232
170407799157,5cyclictest3818356-21systemd-journal23:10:002
170407799152,5cyclictest1738463-21irqstats19:35:162
170407799151,9cyclictest0-21swapper/221:47:192
1704077991513,1cyclictest1849027-21diskmemload22:19:442
1704077991512,2cyclictest1849027-21diskmemload23:50:172
1704077991512,2cyclictest0-21swapper/200:20:142
1704077991511,3cyclictest2072219-21kthreadcore23:25:212
1704077991511,3cyclictest1899554-21kthreadcore21:40:182
1704077991511,3cyclictest1854238-21latency21:10:202
1704077991510,3cyclictest0-21swapper/223:10:272
1704077991510,3cyclictest0-21swapper/200:00:172
1704073991512,2cyclictest2186324-21ntp_offset00:35:191
1704073991512,2cyclictest2139993-21latency_hist00:10:001
1704073991512,2cyclictest2099945-21cpuspeed_turbos23:45:131
1704073991512,2cyclictest1843403-21fschecks_time21:05:111
1704073991512,2cyclictest1730245-21unixbench_multi19:25:241
1704073991511,3cyclictest2063931-21kthreadcore23:20:191
1704073991511,2cyclictest1780908-21sed20:10:161
1704073991511,2cyclictest0-21swapper/122:35:161
170407399150,2cyclictest1866950-21irqrtprio21:20:191
1704068991512,2cyclictest2184186-21kthreadcore00:35:170
1704068991512,2cyclictest2112348-21kthreadcore23:50:190
1704068991512,2cyclictest1953970-21cron22:15:010
1704068991512,2cyclictest1921728-21cron21:55:000
1704068991512,2cyclictest1842107-21kworker/0:0+kacpid22:29:040
1704068991512,2cyclictest1780673-21fschecks_count20:10:150
1704068991512,2cyclictest1761771-21kworker/0:2+kacpid20:58:200
1704068991512,2cyclictest1731455-21irqcore19:30:170
1704068991512,2cyclictest1712864-21gltestperf19:15:150
1704068991512,2cyclictest1666265-21kworker/0:1+kacpid19:45:030
1704068991512,2cyclictest0-21swapper/022:40:000
1704068991511,3cyclictest2069847-21kthreadcore23:25:190
1704068991511,3cyclictest2040661-21timerandwakeup23:05:240
1704068991511,3cyclictest2011818-21irqcore22:50:160
1704068991511,2cyclictest0-21swapper/021:05:130
1704068991510,4cyclictest1852278-21kthreadcore21:10:180
170406899150,14cyclictest744-21snmpd22:45:220
1704082991412,1cyclictest0-21swapper/300:19:503
1704082991412,1cyclictest0-21swapper/300:15:003
1704082991411,1cyclictest0-21swapper/323:10:213
1704082991411,1cyclictest0-21swapper/323:05:143
1704082991411,1cyclictest0-21swapper/322:50:243
1704082991411,1cyclictest0-21swapper/322:40:183
1704082991411,1cyclictest0-21swapper/322:39:213
1704082991411,1cyclictest0-21swapper/321:51:173
1704082991411,1cyclictest0-21swapper/320:45:233
1704082991411,1cyclictest0-21swapper/300:05:193
1704082991410,1cyclictest0-21swapper/319:55:103
170407799145,3cyclictest0-21swapper/222:12:032
170407799142,3cyclictest1754710-21ntp_states19:45:192
1704077991411,2cyclictest0-21swapper/221:30:172
1704077991410,3cyclictest2168153-21kthreadcore00:25:182
1704077991410,2cyclictest1704081-21latency_hist19:10:012
170407799140,9cyclictest1811174-21taskset20:31:232
170407799140,5cyclictest1831550-21irqstats20:50:172
170407799140,4cyclictest1723522-21memory19:20:192
1704073991412,1cyclictest1741562-21kthreadcore19:35:181
1704073991411,2cyclictest2159701-21kthreadcore00:20:181
1704073991411,2cyclictest2029499-21kthreadcore23:00:181
1704073991411,2cyclictest1939490-21irqrtprio22:05:171
1704073991411,2cyclictest1824027-21/usr/sbin/munin20:45:151
1704073991411,2cyclictest1816792-21proc_pri20:35:191
1704073991411,2cyclictest0-21swapper/122:50:171
1704073991411,2cyclictest0-21swapper/121:30:161
1704073991411,2cyclictest0-21swapper/120:35:001
1704073991411,2cyclictest0-21swapper/119:35:011
1704073991410,3cyclictest2072558-21grep23:25:221
1704073991410,3cyclictest2043327-21/usr/sbin/munin23:10:171
1704073991410,3cyclictest1755141-21unixbench_singl19:45:221
170406899143,3cyclictest0-21swapper/020:08:200
1704068991411,2cyclictest2159527-21kthreadcore00:20:180
1704068991411,2cyclictest1925959-21kthreadcore21:55:190
1704068991411,2cyclictest1925959-21kthreadcore21:55:180
1704068991411,2cyclictest1876695-21kthreadcore21:25:180
1704068991411,2cyclictest1842107-21kworker/0:0+kacpid22:15:110
1704068991411,2cyclictest1811983-21cut20:35:120
1704068991411,2cyclictest1761727-21munin-plugin-st19:55:000
1704068991410,3cyclictest500-21avahi-daemon19:15:010
170406899140,5cyclictest2168672-21ssh00:25:180
1704082991310,1cyclictest0-21swapper/323:52:103
1704082991310,1cyclictest0-21swapper/323:35:223
1704082991310,1cyclictest0-21swapper/323:00:223
1704082991310,1cyclictest0-21swapper/322:08:273
1704082991310,1cyclictest0-21swapper/321:45:143
1704082991310,1cyclictest0-21swapper/321:40:153
1704082991310,1cyclictest0-21swapper/321:35:003
1704082991310,1cyclictest0-21swapper/321:05:153
1704082991310,1cyclictest0-21swapper/320:05:153
1704082991310,1cyclictest0-21swapper/319:30:003
1704082991310,1cyclictest0-21swapper/300:30:183
170407799131,3cyclictest1824428-21hddtemp_smartct20:45:142
170407799130,3cyclictest1767011-21ntp_states19:55:192
170407799130,3cyclictest1726187-21irqstats19:25:162
170407399138,3cyclictest501-21dbus-daemon22:55:211
1704073991311,1cyclictest1995102-21tr22:40:131
1704073991310,2cyclictest2108093-21fschecks_count23:50:131
1704073991310,2cyclictest1951074-21kthreadcore22:10:221
1704073991310,2cyclictest1889898-21cpuspeed_turbos21:35:121
1704073991310,2cyclictest1874830-21irqrtprio21:25:161
1704073991310,2cyclictest1833620-21kthreadcore20:50:191
170406899139,3cyclictest64450irq/159-eno2:rx-021:40:220
170406899139,3cyclictest2121999-21kworker/0:2+kacpi_notify00:00:240
170406899139,2cyclictest0-21swapper/021:45:230
1704068991310,2cyclictest2043053-21kworker/0:0+kacpi_notify23:17:320
1704068991310,2cyclictest1804556-21unixbench-2d20:25:220
1704068991310,2cyclictest1761771-21kworker/0:2+kacpid20:23:000
16981192131,5sleep00-21swapper/019:05:120
170408299129,1cyclictest0-21swapper/323:00:003
170408299129,1cyclictest0-21swapper/322:29:243
170408299129,1cyclictest0-21swapper/322:20:203
170408299129,1cyclictest0-21swapper/321:20:243
170408299129,1cyclictest0-21swapper/319:45:013
170408299129,1cyclictest0-21swapper/300:38:033
170408299129,1cyclictest0-21swapper/300:00:203
1704082991210,1cyclictest0-21swapper/320:30:003
170407799121,3cyclictest2173028-21df_inode00:30:152
170407399129,2cyclictest331ktimers/100:18:441
170407399129,2cyclictest2099526-21idleruntime-cro23:44:591
170407399129,2cyclictest1769080-21tr20:00:181
170407399128,3cyclictest1863138-21unixbench_singl21:15:251
170407399126,4cyclictest1919657-21awk21:50:251
1704073991210,1cyclictest0-21swapper/120:40:241
170406899127,4cyclictest1718701-21apt-get19:20:130
170406899126,4cyclictest501-21dbus-daemon19:40:010
170408299119,1cyclictest0-21swapper/322:15:173
170408299119,1cyclictest0-21swapper/300:20:173
170408299118,1cyclictest0-21swapper/322:45:143
170408299118,1cyclictest0-21swapper/322:00:173
170408299118,1cyclictest0-21swapper/320:15:013
170408299118,1cyclictest0-21swapper/320:04:523
170407399117,2cyclictest1839205-21kthreadcore20:55:191
170406899118,2cyclictest2029052-21kthreadcore23:00:180
170406899110,3cyclictest0-21swapper/021:01:500
170408299108,1cyclictest0-21swapper/319:55:013
170407799100,6cyclictest1842900-21basename21:05:002
170407399100,8cyclictest1704137-21cat19:10:001
170407399100,1cyclictest0-21swapper/121:01:401
17040829997,1cyclictest0-21swapper/320:50:153
17040829997,1cyclictest0-21swapper/320:20:013
17040829986,1cyclictest0-21swapper/321:01:013
1703937280,2sleep30-21swapper/319:09:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional