You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-18 - 08:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Thu Sep 18, 2025 00:43:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22974442980,1sleep72297446-21cut07:58:127
177843926656,7sleep50-21swapper/503:27:045
177842126238,7sleep60-21swapper/603:26:496
177834125747,7sleep40-21swapper/403:25:434
177823225622,29sleep70-21swapper/703:24:187
1778766994038,2cyclictest1880541-21kworker/u16:5+flush-8:004:28:094
1778766993937,2cyclictest2210252-21kworker/u16:0+events_unbound07:13:264
1778766993937,2cyclictest2196401-21kworker/u16:1+flush-8:007:53:204
1778766993937,2cyclictest2196401-21kworker/u16:1+events_unbound07:51:234
1778766993836,2cyclictest2058214-21kworker/u16:2+events_unbound06:23:104
1778766993836,2cyclictest2058214-21kworker/u16:2+events_unbound06:23:104
1778766993835,2cyclictest2333938-21kworker/u16:4+flush-8:008:18:054
1778766993634,2cyclictest2210252-21kworker/u16:0+events_unbound08:43:084
1778766993634,2cyclictest2210252-21kworker/u16:0+events_unbound08:43:084
1778766993633,2cyclictest2127824-21kworker/u16:3+flush-8:006:33:194
1778766993533,2cyclictest2282686-21kworker/u16:3+events_unbound08:07:034
1778766993432,2cyclictest2210252-21kworker/u16:0+flush-8:007:28:204
1778766993432,2cyclictest2058214-21kworker/u16:2+flush-8:006:43:184
1778772993328,4cyclictest2167360-21cat06:52:536
1778766993331,2cyclictest1929715-21kworker/u16:0+flush-8:004:43:144
1778772993230,2cyclictest2008425-21kworker/u16:0+flush-8:005:22:486
1778766993230,2cyclictest2068263-21kworker/u16:4+flush-8:006:00:024
1778766993230,2cyclictest2058214-21kworker/u16:2+flush-8:006:28:084
1778766993129,2cyclictest2226236-21kworker/u16:2+events_unbound07:24:244
1778766993129,2cyclictest1749036-21kworker/u16:2+events_unbound04:53:364
177877699300,28cyclictest589235-21systemd-journal04:38:217
1778766993028,2cyclictest2314245-21kworker/u16:1+events_unbound08:30:124
1778766993028,2cyclictest2282686-21kworker/u16:3+flush-8:008:23:254
1778766993028,2cyclictest2038562-21kworker/u16:4+events_unbound05:42:244
1778766993028,2cyclictest2008425-21kworker/u16:0+events_unbound07:06:084
1778766993028,2cyclictest1959175-21kworker/u16:5+events_unbound05:06:104
1778766993028,2cyclictest1929715-21kworker/u16:0+events_unbound05:12:574
1778766993028,2cyclictest1929715-21kworker/u16:0+events_unbound04:49:124
1778766993028,2cyclictest1778500-21kworker/u16:1+flush-8:004:03:204
1778766992927,2cyclictest2196699-21kworker/u16:2+events_unbound07:11:094
1778766992927,2cyclictest2157122-21kworker/u16:5+flush-8:007:01:314
1778766992927,2cyclictest1778500-21kworker/u16:1+events_unbound05:24:054
1778772992826,2cyclictest2215614-21kworker/u16:6+flush-8:008:07:486
1778766992826,2cyclictest2215614-21kworker/u16:6+events_unbound08:12:314
1778766992826,2cyclictest2157122-21kworker/u16:5+events_unbound06:50:184
1778766992826,2cyclictest2068263-21kworker/u16:4+events_unbound06:18:484
1778766992826,2cyclictest2068263-21kworker/u16:4+events_unbound06:18:484
1778766992826,2cyclictest2008425-21kworker/u16:0+events_unbound05:31:094
1778766992826,2cyclictest1778500-21kworker/u16:1+events_unbound05:52:204
1778766992826,2cyclictest1778500-21kworker/u16:1+events_unbound04:42:164
1778766992826,2cyclictest1749036-21kworker/u16:2+flush-8:004:33:154
1778766992826,2cyclictest1749036-21kworker/u16:2+events_unbound03:44:214
21784692270,1chrt2178326-21irqcore06:58:117
1778772992725,2cyclictest2363735-21kworker/u16:2+flush-8:008:37:486
1778772992725,2cyclictest2108202-21kworker/u16:1+flush-8:006:22:486
1778772992725,2cyclictest2108202-21kworker/u16:1+flush-8:006:22:486
1778772992725,2cyclictest1749036-21kworker/u16:2+flush-8:003:37:486
1778766992725,2cyclictest2335758-21kworker/u16:5+events_unbound08:37:404
1778766992725,2cyclictest2210252-21kworker/u16:0+events_unbound07:21:034
1778766992725,2cyclictest1998447-21kworker/u16:3+flush-8:006:05:324
1778766992725,2cyclictest1978900-21kworker/u16:2+flush-8:005:18:034
1778766992725,2cyclictest1929715-21kworker/u16:0+events_unbound05:12:124
1778766992725,2cyclictest1778500-21kworker/u16:1+events_unbound04:24:454
1778772992624,2cyclictest2157122-21kworker/u16:5+flush-8:007:02:496
1778772992624,2cyclictest2058214-21kworker/u16:2+flush-8:006:12:486
1778772992624,2cyclictest1978913-21kworker/u16:4+flush-8:005:27:536
1778766992624,2cyclictest2226236-21kworker/u16:2+flush-8:007:42:264
1778766992624,2cyclictest2210252-21kworker/u16:0+events_unbound07:43:574
1778766992624,2cyclictest2147288-21kworker/u16:1+flush-8:006:40:354
1778766992624,2cyclictest2068263-21kworker/u16:4+flush-8:006:15:204
1778766992624,2cyclictest1730079-21kworker/u16:0+flush-8:003:51:264
1778766992624,2cyclictest1730079-21kworker/u16:0+events_unbound03:41:164
1778766992624,2cyclictest1084147-21kworker/u16:6+events_unbound03:59:204
1778766992624,2cyclictest1084147-21kworker/u16:6+events_unbound03:59:204
1778766992523,2cyclictest2314245-21kworker/u16:1+events_unbound08:42:274
1778766992523,2cyclictest1998447-21kworker/u16:3+flush-8:005:46:314
1778766992523,2cyclictest1959161-21kworker/u16:3+events_unbound05:02:024
1778766992523,2cyclictest1084147-21kworker/u16:6+flush-8:003:37:374
1778766992523,2cyclictest1084147-21kworker/u16:6+events_unbound03:54:134
18629112240,0chrt65-21ksoftirqd/404:08:134
1778772992422,2cyclictest2058214-21kworker/u16:2+events_unbound06:43:066
1778766992422,2cyclictest2412566-21kworker/u16:2+events_unbound08:55:014
1778766992422,2cyclictest2210252-21kworker/u16:0+events_unbound08:00:514
1778766992422,2cyclictest2058214-21kworker/u16:2+events_unbound06:54:194
1778766992422,2cyclictest1778500-21kworker/u16:1+events_unbound04:18:064
1778772992321,2cyclictest2215614-21kworker/u16:6+flush-8:007:32:496
1778772992321,2cyclictest1998447-21kworker/u16:3+flush-8:006:03:156
1778766992321,2cyclictest2058214-21kworker/u16:2+events_unbound06:08:474
1778766992321,2cyclictest2018240-21kworker/u16:2+events_unbound05:33:244
1778766992321,2cyclictest1880541-21kworker/u16:5+events_unbound04:16:224
1778766992321,2cyclictest1778500-21kworker/u16:1+flush-8:005:53:294
1778772992220,2cyclictest2196401-21kworker/u16:1+flush-8:007:33:226
1778772992220,2cyclictest1929715-21kworker/u16:0+flush-8:004:58:096
1778772992220,2cyclictest1084147-21kworker/u16:6+events_unbound03:28:126
1778766992220,2cyclictest2314245-21kworker/u16:1+events_unbound08:49:434
1778766992220,2cyclictest2314245-21kworker/u16:1+events_unbound08:49:434
1778766992220,2cyclictest2210252-21kworker/u16:0+events_unbound08:13:434
1778766992220,2cyclictest2196401-21kworker/u16:1+events_unbound07:36:494
1778766992220,2cyclictest1084147-21kworker/u16:6+events_unbound03:28:414
23946202210,1chrt0-21swapper/608:48:116
23946202210,1chrt0-21swapper/608:48:116
1778772992119,2cyclictest2018240-21kworker/u16:2+events_unbound05:33:206
1778772992119,2cyclictest1778500-21kworker/u16:1+events_unbound03:43:036
1778772992118,2cyclictest1959175-21kworker/u16:5+flush-8:004:57:546
1778776992019,0cyclictest0-21swapper/708:52:157
1778776992019,0cyclictest0-21swapper/708:52:157
1778776992017,2cyclictest1313-21snmpd07:15:037
1778772992018,2cyclictest1900296-21kworker/u16:3+flush-8:004:48:166
1778776991915,3cyclictest1313-21snmpd05:25:517
1778772991917,2cyclictest2335758-21kworker/u16:5+events_unbound08:31:306
1778772991917,2cyclictest2038562-21kworker/u16:4+flush-8:005:43:196
1778772991914,4cyclictest1910289-21latency_hist04:32:546
1778776991813,4cyclictest589235-21systemd-journal05:12:497
1778772991816,2cyclictest2314245-21kworker/u16:1+flush-8:008:55:356
1778772991816,2cyclictest2314245-21kworker/u16:1+flush-8:008:43:186
1778772991816,2cyclictest2314245-21kworker/u16:1+flush-8:008:43:186
1778772991816,2cyclictest2314245-21kworker/u16:1+events_unbound08:22:506
1778772991816,2cyclictest2215614-21kworker/u16:6+events_unbound07:20:246
1778772991816,2cyclictest2210252-21kworker/u16:0+flush-8:007:42:536
1778776991716,0cyclictest0-21swapper/707:32:537
1778776991713,3cyclictest1313-21snmpd07:24:067
1778776991713,2cyclictest1912374-21grep04:33:127
177877699170,16cyclictest1313-21snmpd04:56:117
1778772991715,2cyclictest2282686-21kworker/u16:3+flush-8:008:23:386
1778772991715,2cyclictest2282686-21kworker/u16:3+flush-8:008:12:536
1778772991715,2cyclictest2210252-21kworker/u16:0+events_unbound07:41:236
1778772991715,2cyclictest2176853-21kworker/u16:3+events_unbound07:13:126
1778772991715,2cyclictest2068263-21kworker/u16:4+events_unbound05:58:136
1778772991715,2cyclictest1880541-21kworker/u16:5+events_unbound04:38:066
1778776991613,2cyclictest1313-21snmpd04:22:597
1778776991610,5cyclictest589235-21systemd-journal04:17:537
1778772991614,2cyclictest2196699-21kworker/u16:2+events_unbound07:11:036
1778772991614,2cyclictest2196401-21kworker/u16:1+flush-8:007:53:186
1778772991614,2cyclictest2147288-21kworker/u16:1+flush-8:006:56:206
1778772991614,2cyclictest2058214-21kworker/u16:2+events_unbound06:29:436
1778772991614,2cyclictest2058214-21kworker/u16:2+events_unbound06:27:426
1778772991614,2cyclictest2058214-21kworker/u16:2+events_unbound06:27:426
1778772991614,2cyclictest1778500-21kworker/u16:1+events_unbound04:18:086
1778772991614,2cyclictest1749036-21kworker/u16:2+events_unbound04:37:316
1778768991612,3cyclictest1969186-21latency_hist05:02:545
1778772991513,2cyclictest2215614-21kworker/u16:6+events_unbound08:08:076
1778772991513,2cyclictest1978900-21kworker/u16:2+events_unbound05:12:246
1778772991513,2cyclictest1900296-21kworker/u16:3+events_unbound04:27:076
1778772991513,2cyclictest1778500-21kworker/u16:1+events_unbound04:08:066
1778772991513,2cyclictest1749036-21kworker/u16:2+events_unbound03:56:526
1778768991513,1cyclictest2255748-21/usr/sbin/munin07:37:525
177877699149,4cyclictest2108708-21/usr/sbin/munin06:23:207
177877699149,4cyclictest2108708-21/usr/sbin/munin06:23:207
177877699148,4cyclictest589235-21systemd-journal06:04:217
1778776991412,1cyclictest0-21swapper/707:34:197
1778776991410,3cyclictest2037833-21ntp_states05:38:207
1778776991410,3cyclictest1313-21snmpd04:00:347
1778776991410,3cyclictest1313-21snmpd04:00:347
177877699140,13cyclictest589235-21systemd-journal05:42:547
177877699140,12cyclictest2362796-21ntpq08:28:217
177877699140,12cyclictest2345246-21cat08:23:117
177877699140,12cyclictest1945056-21kthreadcore04:48:137
1778772991413,1cyclictest0-21swapper/606:38:176
1778772991412,2cyclictest2058214-21kworker/u16:2+events_unbound05:57:096
1778772991412,2cyclictest2008425-21kworker/u16:0+flush-8:005:50:136
1778772991412,2cyclictest1778500-21kworker/u16:1+flush-8:003:58:206
1778772991412,2cyclictest1778500-21kworker/u16:1+flush-8:003:58:206
1778772991412,2cyclictest1749036-21kworker/u16:2+events_unbound03:42:106
1778772991412,2cyclictest1084147-21kworker/u16:6+events_unbound04:03:036
1778768991413,1cyclictest2177168-21latency_hist06:57:535
1778768991413,1cyclictest1841851-21latency_hist03:57:545
1778768991413,1cyclictest0-21swapper/507:43:105
177876899140,1cyclictest0-21swapper/505:48:145
177876899140,1cyclictest0-21swapper/505:37:035
177877699139,3cyclictest1822219-21/usr/sbin/munin03:48:267
177877699139,3cyclictest1313-21snmpd06:43:447
177877699139,3cyclictest1313-21snmpd06:35:207
177877699138,4cyclictest589235-21systemd-journal05:22:537
177877699138,4cyclictest2062512-21tr05:53:157
177877699138,4cyclictest1813640-21kthreadcore03:43:107
1778776991310,2cyclictest2331969-21tr08:13:177
1778776991310,2cyclictest2265918-21/usr/sbin/munin07:43:237
1778776991310,2cyclictest1997723-21ntpq05:13:187
1778776991310,2cyclictest1313-21snmpd03:36:277
177877699130,12cyclictest2078107-21turbostat06:02:537
177877699130,11cyclictest2225303-21ntpq07:18:207
1778772991311,2cyclictest2226236-21kworker/u16:2+flush-8:007:23:036
1778772991311,2cyclictest2196401-21kworker/u16:1+events_unbound07:51:136
1778772991311,2cyclictest2127824-21kworker/u16:3+flush-8:006:36:416
1778772991311,2cyclictest1998447-21kworker/u16:3+flush-8:005:38:166
1778772991311,2cyclictest1959161-21kworker/u16:3+flush-8:005:03:166
1778772991311,2cyclictest1900296-21kworker/u16:3+events_unbound04:44:036
177877299130,13cyclictest0-21swapper/606:13:226
1778768991312,1cyclictest0-21swapper/508:26:025
1778768991312,1cyclictest0-21swapper/507:17:535
1778768991312,1cyclictest0-21swapper/505:22:235
1778768991312,1cyclictest0-21swapper/504:27:035
1778768991312,1cyclictest0-21swapper/503:40:235
1778768991311,1cyclictest0-21swapper/506:47:535
177876899130,1cyclictest0-21swapper/508:28:055
177876899130,1cyclictest0-21swapper/508:22:015
177876899130,1cyclictest0-21swapper/506:29:235
177876899130,13cyclictest0-21swapper/503:47:105
177877699129,2cyclictest1909344-21ntpq04:28:207
177877699129,2cyclictest1313-21snmpd07:12:187
177877699129,2cyclictest1313-21snmpd03:32:007
177877699129,2cyclictest0-21swapper/708:33:237
177877699128,3cyclictest2383823-21/usr/sbin/munin08:43:177
177877699128,3cyclictest2383823-21/usr/sbin/munin08:43:177
177877699128,3cyclictest1313-21snmpd05:33:127
177877699126,5cyclictest27204-21gmain04:18:327
177877699125,6cyclictest0-21swapper/704:06:317
177877699124,7cyclictest0-21swapper/708:08:237
177877699121,9cyclictest2013099-21kthreadcore05:28:107
177877699121,3cyclictest2382445-21ntpq08:38:197
177877699120,4cyclictest2343088-21ntpq08:18:167
177877699120,4cyclictest2106247-21kthreadcore06:18:167
177877699120,4cyclictest2106247-21kthreadcore06:18:167
177877699120,11cyclictest1806872-21kthreadcore03:38:167
177877699120,11cyclictest0-21swapper/708:03:057
177877699120,10cyclictest2187320-21/usr/sbin/munin07:03:057
177877699120,10cyclictest2168178-21df_inode06:53:087
1778772991210,2cyclictest2210252-21kworker/u16:0+events_unbound08:41:466
1778772991210,2cyclictest2008425-21kworker/u16:0+flush-8:007:04:586
1778772991210,2cyclictest1821784-21kworker/u16:3+flush-8:003:49:426
1778772991210,2cyclictest1778500-21kworker/u16:1+flush-8:005:23:566
1778772991210,2cyclictest1778500-21kworker/u16:1+events_unbound05:16:066
177876899122,6cyclictest76-21ksoftirqd/504:37:535
1778768991211,1cyclictest0-21swapper/508:15:485
1778768991211,1cyclictest0-21swapper/506:51:275
1778768991211,1cyclictest0-21swapper/506:02:545
1778768991211,1cyclictest0-21swapper/504:45:585
1778768991211,1cyclictest0-21swapper/504:01:505
1778768991211,1cyclictest0-21swapper/504:01:505
1778768991211,1cyclictest0-21swapper/503:29:215
177876899120,1cyclictest0-21swapper/508:03:305
177876899120,1cyclictest0-21swapper/505:15:275
177876899120,12cyclictest0-21swapper/503:48:545
177877699118,2cyclictest2285462-21/usr/sbin/munin07:53:087
177877699118,2cyclictest2052583-21cut05:48:157
177877699118,2cyclictest1313-21snmpd06:11:167
177877699117,3cyclictest1832046-21/usr/sbin/munin03:53:107
177877699117,3cyclictest1313-21snmpd07:38:077
177877699117,3cyclictest1313-21snmpd04:42:597
177877699111,9cyclictest1969581-21/usr/sbin/munin05:03:187
177877699110,4cyclictest2118537-21/usr/sbin/munin06:28:117
177877699110,3cyclictest2159068-21kthreadcore06:48:127
177877699110,3cyclictest1864165-21kthreadcore04:08:147
177877699110,2cyclictest2403457-21/usr/sbin/munin08:53:187
177877699110,2cyclictest2278073-21cut07:48:097
177877699110,2cyclictest1960137-21cstates04:58:067
177877699110,10cyclictest2144490-21grep06:38:187
177877299119,2cyclictest2210252-21kworker/u16:0+events_unbound08:02:276
177877299119,2cyclictest1850324-21kworker/u16:0+flush-8:004:13:536
177876899119,1cyclictest2098126-21pool-udisksd06:13:375
1778768991110,1cyclictest0-21swapper/508:09:395
1778768991110,1cyclictest0-21swapper/507:52:185
1778768991110,1cyclictest0-21swapper/507:31:495
1778768991110,1cyclictest0-21swapper/506:23:235
1778768991110,1cyclictest0-21swapper/506:23:235
1778768991110,1cyclictest0-21swapper/506:18:215
1778768991110,1cyclictest0-21swapper/506:18:215
1778768991110,1cyclictest0-21swapper/505:45:055
1778768991110,1cyclictest0-21swapper/505:42:535
1778768991110,1cyclictest0-21swapper/505:30:165
1778768991110,1cyclictest0-21swapper/505:06:285
1778768991110,1cyclictest0-21swapper/504:56:325
177876899110,11cyclictest0-21swapper/508:50:285
177876899110,11cyclictest0-21swapper/508:50:285
177876899110,11cyclictest0-21swapper/507:27:435
177876899110,11cyclictest0-21swapper/505:26:195
177876899110,11cyclictest0-21swapper/504:11:365
177876899110,0cyclictest0-21swapper/504:04:305
177877699100,9cyclictest2097512-21ntp_states06:13:177
177876899109,1cyclictest76-21ksoftirqd/508:34:145
177876899109,1cyclictest2197033-21latency_hist07:07:535
177876899109,1cyclictest0-21swapper/508:54:305
177876899109,1cyclictest0-21swapper/508:41:205
177876899109,1cyclictest0-21swapper/507:00:365
177876899109,1cyclictest0-21swapper/506:34:265
177876899109,1cyclictest0-21swapper/505:12:275
177876899109,1cyclictest0-21swapper/504:37:565
177876899109,1cyclictest0-21swapper/503:33:365
177876899108,1cyclictest2226295-21latency_hist07:22:535
177876899108,1cyclictest1-21systemd08:02:525
177876899102,4cyclictest0-21swapper/504:52:545
17787689998,1cyclictest1880446-21pool-udisksd04:13:375
17787689998,1cyclictest0-21swapper/507:56:245
17787689998,1cyclictest0-21swapper/507:39:045
17787689998,1cyclictest0-21swapper/505:57:535
17787689996,1cyclictest201rcu_preempt05:58:145
17787689994,3cyclictest0-21swapper/506:38:195
17787689994,3cyclictest0-21swapper/504:18:135
17787689987,1cyclictest0-21swapper/507:09:125
17787689987,1cyclictest0-21swapper/506:09:485
17787689987,1cyclictest0-21swapper/504:30:285
17787689983,3cyclictest0-21swapper/508:43:065
17787689983,3cyclictest0-21swapper/508:43:065
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional