You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-30 - 23:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Mon Jun 30, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
50834921090,1sleep686-21ksoftirqd/616:23:376
39454899500,48cyclictest964969-21latency_hist20:08:127
394520993634,2cyclictest871913-21kworker/u16:2+events_unbound19:58:314
394520993533,2cyclictest871913-21kworker/u16:2+events_unbound19:23:324
394520993432,2cyclictest574079-21kworker/u16:0+flush-8:017:18:324
394520993432,2cyclictest401477-21kworker/u16:1+events_unbound16:18:164
394520993331,2cyclictest439157-21kworker/u16:2+events_unbound16:13:494
394520993230,2cyclictest758582-21kworker/u16:2+flush-8:019:03:344
394520993230,2cyclictest552473-21kworker/u16:1+events_unbound17:24:144
394520993028,2cyclictest656457-21kworker/u16:3+events_unbound17:53:484
10316122300,1chrt0-21swapper/620:38:336
394520992926,2cyclictest902924-21kworker/u16:0+flush-8:020:18:274
394520992926,2cyclictest902924-21kworker/u16:0+flush-8:020:18:274
39454899280,26cyclictest563014-21latency_hist16:53:147
394520992826,2cyclictest769038-21kworker/u16:0+events_unbound18:33:444
394520992826,2cyclictest393744-21kworker/u16:0+events_unbound16:03:414
394520992725,2cyclictest959485-21kworker/u16:5+flush-8:020:03:394
394520992725,2cyclictest881275-21kworker/u16:3+events_unbound19:38:334
39454899260,24cyclictest635088-21latency_hist17:28:137
394526992623,2cyclictest902924-21kworker/u16:0+events_unbound20:03:195
394520992624,2cyclictest1000673-21kworker/u16:2+events_unbound20:58:154
394520992624,1cyclictest799787-21kworker/u16:4+flush-8:018:48:334
39454899250,23cyclictest758843-21latency_hist18:28:147
394520992523,2cyclictest902924-21kworker/u16:0+flush-8:019:48:124
394520992523,2cyclictest779367-21kworker/u16:1+events_unbound19:29:484
394520992523,2cyclictest718294-21kworker/u16:3+flush-8:018:13:394
394520992523,2cyclictest696721-21kworker/u16:0+flush-8:018:23:524
394520992523,2cyclictest552473-21kworker/u16:1+flush-8:016:51:444
394520992523,2cyclictest439157-21kworker/u16:2+flush-8:016:30:454
394520992522,2cyclictest871913-21kworker/u16:2+flush-8:019:53:234
394520992422,2cyclictest552473-21kworker/u16:1+events_unbound17:03:394
394520992321,2cyclictest963992-21kworker/u16:6+events_unbound20:28:354
394520992321,2cyclictest924451-21kworker/u16:1+flush-8:019:48:354
394520992321,2cyclictest393744-21kworker/u16:0+events_unbound16:36:484
394526992220,2cyclictest779367-21kworker/u16:1+events_unbound19:08:085
394526992220,2cyclictest401477-21kworker/u16:1+flush-8:016:14:125
394520992220,2cyclictest963992-21kworker/u16:6+events_unbound20:47:234
394520992220,2cyclictest574079-21kworker/u16:0+flush-8:016:58:444
394520992220,2cyclictest490061-21kworker/u16:3+events_unbound16:25:204
394520992219,2cyclictest781416-21kworker/u16:3+flush-8:018:43:234
394520992219,2cyclictest781416-21kworker/u16:3+flush-8:018:43:234
394526992119,2cyclictest781416-21kworker/u16:3+events_unbound18:43:195
394526992119,2cyclictest781416-21kworker/u16:3+events_unbound18:43:195
394526992119,2cyclictest582599-21kworker/u16:2+flush-8:017:14:165
394520992119,2cyclictest779367-21kworker/u16:1+flush-8:018:38:394
394520992119,2cyclictest439157-21kworker/u16:2+flush-8:016:43:344
394520992119,2cyclictest401477-21kworker/u16:1+flush-8:015:34:484
394520992119,2cyclictest401477-21kworker/u16:1+flush-8:015:34:484
394526992018,2cyclictest696721-21kworker/u16:0+events_unbound18:21:125
394526992018,2cyclictest439157-21kworker/u16:2+events_unbound16:49:155
394520992018,2cyclictest871913-21kworker/u16:2+events_unbound19:19:244
394520992018,2cyclictest843293-21kworker/u16:0+events_unbound19:08:414
394520992018,2cyclictest645130-21kworker/u16:2+events_unbound17:39:234
394520992018,2cyclictest574079-21kworker/u16:0+flush-8:017:13:344
394520992018,2cyclictest317563-21kworker/u16:3+flush-8:015:43:444
394526991917,2cyclictest799787-21kworker/u16:4+events_unbound19:22:245
394526991917,2cyclictest393744-21kworker/u16:0+events_unbound15:39:375
394520991917,2cyclictest718294-21kworker/u16:3+flush-8:018:08:444
394520991917,2cyclictest645130-21kworker/u16:2+flush-8:017:58:464
394520991917,2cyclictest401477-21kworker/u16:1+flush-8:016:08:394
39454899180,12cyclictest892843-21latency_hist19:33:137
394526991816,2cyclictest902924-21kworker/u16:0+events_unbound20:10:155
394526991816,2cyclictest645130-21kworker/u16:2+events_unbound18:05:245
394526991816,2cyclictest582599-21kworker/u16:2+events_unbound17:09:235
394526991816,2cyclictest582599-21kworker/u16:2+events_unbound17:09:235
394526991816,2cyclictest439157-21kworker/u16:2+events_unbound16:26:135
394520991816,2cyclictest758582-21kworker/u16:2+flush-8:019:02:484
394520991816,2cyclictest574079-21kworker/u16:0+flush-8:017:12:234
394520991816,2cyclictest574079-21kworker/u16:0+flush-8:017:12:234
394520991816,2cyclictest401477-21kworker/u16:1+events_unbound15:38:174
394520991816,2cyclictest393744-21kworker/u16:0+flush-8:015:58:434
7808342170,0sleep7780832-21cut18:38:317
39454899170,16cyclictest643-21systemd-journal20:18:127
394537991714,2cyclictest634190-21kworker/u16:4+flush-8:017:24:446
394526991715,2cyclictest552473-21kworker/u16:1+events_unbound18:16:485
394520991715,2cyclictest902924-21kworker/u16:0+events_unbound20:16:464
394520991715,2cyclictest799787-21kworker/u16:4+flush-8:019:16:484
394520991715,2cyclictest393744-21kworker/u16:0+events_unbound15:53:474
394520991715,2cyclictest393744-21kworker/u16:0+events_unbound15:52:484
39454899160,15cyclictest1313-21snmpd18:48:187
39454899160,10cyclictest1313-21snmpd17:57:327
394537991614,2cyclictest871913-21kworker/u16:2+flush-8:019:28:446
394526991614,2cyclictest799787-21kworker/u16:4+events_unbound18:49:165
394526991614,2cyclictest779367-21kworker/u16:1+events_unbound19:24:215
394526991614,2cyclictest758582-21kworker/u16:2+events_unbound18:55:005
394526991614,2cyclictest574079-21kworker/u16:0+events_unbound17:25:205
394526991614,2cyclictest439157-21kworker/u16:2+events_unbound16:42:235
394526991614,2cyclictest1000673-21kworker/u16:2+events_unbound20:55:555
394526991613,2cyclictest552473-21kworker/u16:1+events_unbound17:03:155
394520991614,2cyclictest963992-21kworker/u16:6+flush-8:020:33:234
394520991614,2cyclictest871913-21kworker/u16:2+flush-8:019:33:474
394520991614,2cyclictest562840-21kworker/u16:3+events_unbound16:53:474
39454899158,1cyclictest573369-21basename16:58:137
39454899151,8cyclictest27174-21gnome-shell19:25:127
394548991514,0cyclictest0-21swapper/720:00:447
39454899150,9cyclictest822283-21cat18:58:327
39454899150,9cyclictest667567-21cat17:43:357
39454899150,8cyclictest933034-21ntpq19:48:427
39454899150,14cyclictest913423-21latency_hist19:43:137
39454899150,14cyclictest779470-21turbostat18:38:127
39454899150,14cyclictest1067977-21latency_hist20:58:127
39454899150,10cyclictest1313-21snmpd20:42:157
39454899150,10cyclictest1313-21snmpd19:34:147
39454899150,10cyclictest1313-21snmpd19:15:347
39454899150,10cyclictest1313-21snmpd19:03:557
39454899150,10cyclictest1313-21snmpd17:03:467
39454899150,10cyclictest1313-21snmpd15:44:207
394526991513,2cyclictest902924-21kworker/u16:0+events_unbound20:20:195
394526991513,2cyclictest902924-21kworker/u16:0+events_unbound20:20:195
394526991513,2cyclictest902924-21kworker/u16:0+events_unbound19:45:005
394526991513,2cyclictest656457-21kworker/u16:3+events_unbound17:57:415
394526991513,2cyclictest645130-21kworker/u16:2+events_unbound18:01:245
394526991513,2cyclictest393744-21kworker/u16:0+events_unbound15:59:135
394520991513,2cyclictest747699-21kworker/u16:5+events_unbound18:32:484
394520991513,2cyclictest1000673-21kworker/u16:2+events_unbound20:53:464
39454899146,3cyclictest27924-21gmain16:32:597
39454899146,2cyclictest27642-21gmain15:38:447
39454899144,5cyclictest27978-21gmain18:53:437
39454899143,6cyclictest27304-21gsd-housekeepin17:58:467
394548991411,2cyclictest1026848-21munin-run20:38:137
394548991410,3cyclictest975352-21basename20:13:127
39454899140,9cyclictest1313-21snmpd17:52:327
39454899140,9cyclictest1313-21snmpd17:01:227
39454899140,9cyclictest1015248-21tr20:28:437
39454899140,8cyclictest27174-21gnome-shell19:47:127
39454899140,8cyclictest27174-21gnome-shell19:09:127
39454899140,8cyclictest1313-21snmpd16:37:067
39454899140,8cyclictest1313-21snmpd16:19:037
39454899140,12cyclictest624812-21latency_hist17:23:137
394537991411,2cyclictest317563-21kworker/u16:3+events_unbound15:38:156
39453799140,14cyclictest0-21swapper/620:18:126
39453799140,14cyclictest0-21swapper/617:43:526
39453799140,14cyclictest0-21swapper/616:48:426
39453799140,14cyclictest0-21swapper/616:03:376
394526991412,2cyclictest963992-21kworker/u16:6+events_unbound20:35:005
394526991412,2cyclictest871913-21kworker/u16:2+events_unbound19:55:005
394526991412,2cyclictest843293-21kworker/u16:0+events_unbound19:14:105
394526991412,2cyclictest799787-21kworker/u16:4+events_unbound19:11:595
394526991412,2cyclictest779367-21kworker/u16:1+events_unbound19:01:085
394526991412,2cyclictest696721-21kworker/u16:0+events_unbound18:11:415
394526991412,2cyclictest656457-21kworker/u16:3+events_unbound17:39:445
394526991412,2cyclictest645130-21kworker/u16:2+events_unbound17:44:335
394526991412,2cyclictest440224-21kworker/u16:4+events_unbound15:55:235
394526991412,2cyclictest401477-21kworker/u16:1+events_unbound16:12:235
39452699140,14cyclictest0-21swapper/520:43:405
394520991412,2cyclictest696721-21kworker/u16:0+flush-8:018:18:264
39454899133,5cyclictest27763-21gvfs-afc-volume20:23:147
39454899130,8cyclictest27174-21gnome-shell17:42:127
39454899130,8cyclictest1313-21snmpd19:57:387
39454899130,8cyclictest1313-21snmpd17:10:377
39454899130,8cyclictest1313-21snmpd17:10:377
39454899130,3cyclictest606882-21kthreadcore17:13:357
39454899130,2cyclictest718308-21/usr/sbin/munin18:08:317
39454899130,2cyclictest501951-21/usr/sbin/munin16:23:297
394537991312,1cyclictest0-21swapper/620:19:566
394537991312,1cyclictest0-21swapper/620:19:566
394537991311,2cyclictest779367-21kworker/u16:1+flush-8:019:08:446
39453799130,1cyclictest0-21swapper/620:01:496
39453799130,1cyclictest0-21swapper/618:12:156
39453799130,13cyclictest0-21swapper/620:54:006
39453799130,13cyclictest0-21swapper/619:08:136
39453799130,13cyclictest0-21swapper/619:02:136
39453799130,13cyclictest0-21swapper/615:56:036
394526991311,2cyclictest963992-21kworker/u16:6+events_unbound20:15:295
394526991311,2cyclictest881275-21kworker/u16:3+events_unbound19:37:235
394526991311,2cyclictest871913-21kworker/u16:2+events_unbound19:39:485
394526991311,2cyclictest512245-21kworker/u16:4+events_unbound16:34:535
394526991311,2cyclictest439157-21kworker/u16:2+events_unbound16:29:255
394526991311,2cyclictest1058400-21kworker/u16:3+events_unbound21:02:245
394526991311,2cyclictest1000673-21kworker/u16:2+events_unbound20:30:235
394526991311,1cyclictest403416-21missed_timers15:33:355
394526991311,1cyclictest403416-21missed_timers15:33:355
394520991312,1cyclictest0-21swapper/420:38:274
39452099130,13cyclictest0-21swapper/416:40:054
39454899129,2cyclictest0-21swapper/716:05:037
394548991212,0cyclictest0-21swapper/720:21:237
394548991212,0cyclictest0-21swapper/720:21:237
39454899120,2cyclictest790527-21/usr/sbin/munin18:43:477
39454899120,2cyclictest790527-21/usr/sbin/munin18:43:477
39454899120,2cyclictest728596-21/usr/sbin/munin18:13:357
39454899120,2cyclictest450460-21/usr/sbin/munin15:58:437
39454899120,2cyclictest429864-21/usr/sbin/munin15:48:347
39454899120,2cyclictest1068719-21/usr/sbin/munin20:58:477
39454899120,2cyclictest1037883-21/usr/sbin/munin20:43:477
39454899120,11cyclictest1313-21snmpd20:51:197
39454899120,11cyclictest1313-21snmpd16:41:097
39454899120,10cyclictest707996-21/usr/sbin/munin18:03:317
39454899120,10cyclictest0-21swapper/715:33:257
39454899120,10cyclictest0-21swapper/715:33:257
394537991211,1cyclictest0-21swapper/620:47:516
394537991211,1cyclictest0-21swapper/620:29:336
394537991211,1cyclictest0-21swapper/620:08:456
394537991211,1cyclictest0-21swapper/619:27:556
394537991211,1cyclictest0-21swapper/618:37:476
394537991211,1cyclictest0-21swapper/617:42:086
394537991211,1cyclictest0-21swapper/616:46:016
394537991211,1cyclictest0-21swapper/616:38:556
39453799120,1cyclictest0-21swapper/618:42:126
39453799120,1cyclictest0-21swapper/616:22:226
39453799120,12cyclictest0-21swapper/619:54:526
39453799120,12cyclictest0-21swapper/618:44:536
39453799120,12cyclictest0-21swapper/618:44:536
39453799120,12cyclictest0-21swapper/618:24:336
39453799120,12cyclictest0-21swapper/617:49:526
39453799120,12cyclictest0-21swapper/617:10:466
39453799120,12cyclictest0-21swapper/617:10:466
39453799120,12cyclictest0-21swapper/616:00:466
394526991210,2cyclictest924451-21kworker/u16:1+events_unbound19:50:445
394526991210,2cyclictest758582-21kworker/u16:2+events_unbound18:39:285
394526991210,2cyclictest747699-21kworker/u16:5+events_unbound18:30:015
394526991210,2cyclictest732676-21kworker/u16:4+events_unbound18:25:205
394526991210,2cyclictest645130-21kworker/u16:2+events_unbound17:49:245
394526991210,2cyclictest562863-21kworker/u16:5+events_unbound16:53:565
394526991210,2cyclictest552473-21kworker/u16:1+events_unbound17:00:295
394526991210,2cyclictest439157-21kworker/u16:2+events_unbound15:51:245
394526991210,2cyclictest401477-21kworker/u16:1+events_unbound16:08:115
394526991210,2cyclictest393744-21kworker/u16:0+events_unbound16:19:485
394526991210,2cyclictest1036833-21kworker/u16:0+events_unbound20:42:375
39452699120,12cyclictest0-21swapper/519:33:125
39452099129,2cyclictest656457-21kworker/u16:3+flush-8:017:43:174
394520991210,2cyclictest963992-21kworker/u16:6+flush-8:020:27:274
394520991210,2cyclictest963992-21kworker/u16:6+events_unbound20:48:234
394520991210,2cyclictest902924-21kworker/u16:0+flush-8:020:09:424
394520991210,2cyclictest799774-21kworker/u16:0+flush-8:018:55:484
394520991210,2cyclictest552473-21kworker/u16:1+events_unbound18:03:174
39454899111,4cyclictest1260-21containerd16:13:327
39454899110,9cyclictest440175-21/usr/sbin/munin15:53:317
39454899110,2cyclictest759583-21/usr/sbin/munin18:28:287
39454899110,10cyclictest96-21ksoftirqd/718:18:307
39454899110,10cyclictest1313-21snmpd16:08:487
394537991110,1cyclictest0-21swapper/619:19:246
394537991110,1cyclictest0-21swapper/617:57:356
394537991110,1cyclictest0-21swapper/616:36:156
39453799110,11cyclictest0-21swapper/620:05:176
39452699119,2cyclictest747699-21kworker/u16:5+events_unbound18:35:245
39452699119,2cyclictest582599-21kworker/u16:2+events_unbound17:23:035
39452699119,2cyclictest552473-21kworker/u16:1+events_unbound16:47:005
39452699119,2cyclictest393744-21kworker/u16:0+events_unbound15:47:085
39452699119,2cyclictest1036833-21kworker/u16:0+events_unbound20:52:245
39452699110,1cyclictest0-21swapper/520:23:485
39454899100,2cyclictest862671-21/usr/sbin/munin19:18:287
39454899100,2cyclictest543175-21/usr/sbin/munin16:43:347
39453799109,1cyclictest0-21swapper/619:49:406
39453799109,1cyclictest0-21swapper/618:20:086
39453799109,1cyclictest0-21swapper/618:04:316
39453799109,1cyclictest0-21swapper/618:01:036
3945379998,1cyclictest0-21swapper/619:13:256
3945379998,1cyclictest0-21swapper/618:57:586
3945379998,1cyclictest0-21swapper/617:17:526
3945379998,1cyclictest0-21swapper/616:29:566
3945269997,2cyclictest871913-21kworker/u16:2+events_unbound20:00:035
3945209997,2cyclictest656457-21kworker/u16:3+events_unbound17:49:284
3945379987,1cyclictest0-21swapper/617:06:216
3945379987,1cyclictest0-21swapper/616:08:206
3945379987,1cyclictest0-21swapper/615:34:086
3945379987,1cyclictest0-21swapper/615:34:086
3945379976,1cyclictest0-21swapper/617:01:566
3945269972,3cyclictest0-21swapper/517:38:145
3945269972,3cyclictest0-21swapper/517:38:145
912496260,1sleep60-21swapper/619:38:416
3945489960,5cyclictest656382-21exim417:38:147
3945489960,5cyclictest656382-21exim417:38:147
3945379965,1cyclictest0-21swapper/617:18:486
3945379960,6cyclictest0-21swapper/618:33:126
3945269962,1cyclictest0-21swapper/515:33:145
3945489950,4cyclictest645402-21latency_hist17:33:147
3945489950,4cyclictest645402-21latency_hist17:33:147
3945489950,4cyclictest394872-21sed15:33:147
3945379954,1cyclictest0-21swapper/620:48:476
3945379951,1cyclictest681rcuog/419:43:436
3945379950,4cyclictest997138-21taskset20:23:276
3945379950,4cyclictest902798-21chrt19:33:596
3945379950,4cyclictest810111-21chrt18:48:566
3945379950,4cyclictest573036-21taskset16:53:586
3945379950,4cyclictest490625-21taskset16:13:526
3945379950,4cyclictest439238-21chrt15:50:366
3945379950,4cyclictest419546-21chrt15:43:236
3945379950,4cyclictest1077967-21chrt20:58:526
3945379950,4cyclictest1017263-21taskset20:33:206
3945379950,1cyclictest0-21swapper/618:14:456
3945269940,1cyclictest201rcu_preempt17:33:145
3945269940,1cyclictest201rcu_preempt17:33:145
3945209930,2cyclictest646095-21taskset17:33:144
3945209930,1cyclictest991rcuop/717:38:144
3945209930,1cyclictest991rcuop/717:38:144
3945379920,2cyclictest991rcuop/715:33:146
3945379910,1cyclictest0-21swapper/617:38:146
3945379910,1cyclictest0-21swapper/617:38:146
3945379910,1cyclictest0-21swapper/617:33:146
3945379910,1cyclictest0-21swapper/617:33:146
3945209910,1cyclictest0-21swapper/417:33:144
3945209910,1cyclictest0-21swapper/415:33:144
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional