You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 14:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Sat Nov 29, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24455125848,7sleep40-21swapper/415:23:004
24455125848,7sleep40-21swapper/415:23:004
24449925848,7sleep60-21swapper/615:22:226
24449925848,7sleep60-21swapper/615:22:226
24486199540,52cyclictest27944-21gnome-software19:29:347
24432625320,29sleep70-21swapper/715:19:597
24432625320,29sleep70-21swapper/715:19:597
244861995044,5cyclictest268863-21latency_hist15:33:047
244854994946,2cyclictest883827-21kworker/u16:2+events_unbound20:42:254
244856994844,3cyclictest280152-21iostat_ios15:38:205
244861994742,4cyclictest773128-21latency_hist19:43:037
24444024737,7sleep50-21swapper/515:21:335
24444024737,7sleep50-21swapper/515:21:335
244854994543,2cyclictest470293-21kworker/u16:4+events_unbound19:21:254
244861994338,4cyclictest278943-21latency_hist15:38:037
244854994341,2cyclictest288871-21kworker/u16:4+events_unbound16:32:264
24486199420,40cyclictest540922-21latency_hist17:48:047
244860994240,2cyclictest470293-21kworker/u16:4+events_unbound19:53:226
244854994237,4cyclictest752517-21pool-/usr/libex19:29:264
244854994139,2cyclictest288871-21kworker/u16:4+events_unbound17:03:324
244861994034,5cyclictest732637-21latency_hist19:23:027
244854994038,2cyclictest853576-21kworker/u16:4+events_unbound20:32:254
244861993934,4cyclictest389783-21latency_hist16:33:037
244860993836,2cyclictest601890-21kworker/u16:3+flush-8:019:58:216
244860993836,2cyclictest470293-21kworker/u16:4+flush-8:019:32:576
244854993836,2cyclictest540670-21kworker/u16:1+flush-8:017:58:294
244854993836,2cyclictest470293-21kworker/u16:4+events_unbound18:48:374
244854993836,2cyclictest470293-21kworker/u16:4+events_unbound18:48:374
244854993836,2cyclictest288871-21kworker/u16:4+events_unbound16:57:214
244861993731,5cyclictest833682-21latency_hist20:13:027
244854993735,2cyclictest793482-21kworker/u16:1+flush-8:020:12:574
244854993735,2cyclictest470293-21kworker/u16:4+flush-8:017:20:294
244861993631,4cyclictest702304-21latency_hist19:08:037
244860993634,2cyclictest470293-21kworker/u16:4+flush-8:019:17:586
244856993632,3cyclictest3732922-21kworker/u16:3+events_unbound15:58:185
244854993634,2cyclictest591286-21kworker/u16:1+events_unbound18:16:144
244854993633,2cyclictest601890-21kworker/u16:3+flush-8:020:18:134
244854993633,2cyclictest601890-21kworker/u16:3+flush-8:020:18:134
244861993529,5cyclictest470387-21latency_hist17:13:037
244856993532,2cyclictest853576-21kworker/u16:4+flush-8:020:43:035
244854993533,2cyclictest410144-21kworker/u16:0+events_unbound16:43:304
244861993433,0cyclictest0-21swapper/719:58:217
244860993432,2cyclictest591286-21kworker/u16:1+flush-8:018:18:196
244854993432,2cyclictest359962-21kworker/u16:3+flush-8:016:21:064
244854993331,2cyclictest591286-21kworker/u16:1+events_unbound18:39:244
244854993331,2cyclictest349307-21kworker/u16:2+flush-8:016:12:544
244854993331,2cyclictest288871-21kworker/u16:4+flush-8:015:42:214
244861993231,0cyclictest0-21swapper/715:57:517
244856993230,2cyclictest652320-21kworker/u16:0+flush-8:019:29:405
244854993230,2cyclictest3732922-21kworker/u16:3+events_unbound15:55:144
244854993230,2cyclictest288871-21kworker/u16:4+events_unbound16:07:314
244854993229,2cyclictest359962-21kworker/u16:3+flush-8:017:08:474
24486199310,29cyclictest631786-21systemd-run18:33:037
244860993129,2cyclictest601890-21kworker/u16:3+flush-8:020:28:306
244860993129,2cyclictest410144-21kworker/u16:0+events_unbound16:53:306
244854993129,2cyclictest652320-21kworker/u16:0+events_unbound19:11:334
244854993129,2cyclictest288871-21kworker/u16:4+events_unbound16:40:274
244854993129,2cyclictest288871-21kworker/u16:4+events_unbound15:53:034
244854993128,3cyclictest601890-21kworker/u16:3+flush-8:018:23:384
244854993028,2cyclictest752576-21kworker/u16:6+events_unbound19:57:014
244854993028,2cyclictest652320-21kworker/u16:0+flush-8:019:13:384
244854993028,2cyclictest601693-21kworker/u16:2+events_unbound18:33:284
244854993028,2cyclictest591286-21kworker/u16:1+flush-8:018:20:254
244854993028,2cyclictest470293-21kworker/u16:4+events_unbound19:24:584
244854992927,2cyclictest843623-21kworker/u16:2+flush-8:020:27:214
244854992927,2cyclictest793482-21kworker/u16:1+flush-8:020:01:074
244854992927,2cyclictest601693-21kworker/u16:2+flush-8:018:54:294
244854992927,2cyclictest540670-21kworker/u16:1+events_unbound17:44:584
244854992927,2cyclictest359962-21kworker/u16:3+flush-8:017:25:364
244854992927,2cyclictest359962-21kworker/u16:3+flush-8:016:34:144
244854992927,2cyclictest359962-21kworker/u16:3+events_unbound16:51:064
244860992826,2cyclictest883827-21kworker/u16:2+flush-8:020:36:266
244860992826,2cyclictest540670-21kworker/u16:1+events_unbound17:58:276
244860992826,2cyclictest509802-21kworker/u16:0+flush-8:017:48:206
244854992826,2cyclictest470293-21kworker/u16:4+events_unbound19:43:064
244854992826,2cyclictest288871-21kworker/u16:4+flush-8:016:58:334
244854992826,2cyclictest278675-21kworker/u16:1+events_unbound15:59:134
244854992825,2cyclictest601890-21kworker/u16:3+flush-8:020:03:174
244860992725,2cyclictest883815-21kworker/u16:0+events_unbound20:38:196
244860992725,2cyclictest783181-21kworker/u16:0+flush-8:020:07:596
244860992725,2cyclictest591286-21kworker/u16:1+flush-8:018:33:216
244860992725,2cyclictest470293-21kworker/u16:4+flush-8:019:22:586
244860992725,2cyclictest3732922-21kworker/u16:3+events_unbound16:08:306
244860992725,2cyclictest188864-21kworker/u16:2+events_unbound15:23:276
244856992725,2cyclictest864336-21kworker/u16:0+events_unbound20:28:215
244856992725,2cyclictest652320-21kworker/u16:0+flush-8:019:18:195
244854992725,2cyclictest752576-21kworker/u16:6+events_unbound19:52:064
244854992725,2cyclictest601693-21kworker/u16:2+events_unbound18:32:044
244854992725,2cyclictest509802-21kworker/u16:0+events_unbound18:13:014
244854992725,2cyclictest509802-21kworker/u16:0+events_unbound17:42:144
244854992725,2cyclictest470293-21kworker/u16:4+events_unbound19:41:344
244854992725,2cyclictest359962-21kworker/u16:3+events_unbound17:15:314
244860992624,2cyclictest853576-21kworker/u16:4+flush-8:020:47:586
244860992624,2cyclictest652320-21kworker/u16:0+flush-8:019:27:576
244860992624,2cyclictest601890-21kworker/u16:3+events_unbound20:08:216
244860992624,2cyclictest410144-21kworker/u16:0+events_unbound16:58:276
244854992624,2cyclictest652320-21kworker/u16:0+flush-8:019:02:244
244854992624,2cyclictest470216-21kworker/u16:2+flush-8:017:32:264
244861992519,5cyclictest589235-21systemd-journal17:27:227
244860992523,2cyclictest470293-21kworker/u16:4+flush-8:019:43:306
244860992523,2cyclictest470293-21kworker/u16:4+events_unbound17:43:186
244860992523,2cyclictest470216-21kworker/u16:2+flush-8:017:27:586
244860992523,2cyclictest359962-21kworker/u16:3+events_unbound17:13:176
244860992523,2cyclictest288871-21kworker/u16:4+flush-8:015:53:276
244860992523,2cyclictest288871-21kworker/u16:4+events_unbound17:03:216
244856992522,2cyclictest188864-21kworker/u16:2+flush-8:015:33:235
244854992523,2cyclictest883827-21kworker/u16:2+events_unbound20:45:334
244854992523,2cyclictest268041-21kworker/u16:0+flush-8:015:35:504
244854992523,2cyclictest268041-21kworker/u16:0+flush-8:015:28:504
244860992422,2cyclictest470293-21kworker/u16:4+flush-8:018:50:256
244860992422,2cyclictest470293-21kworker/u16:4+flush-8:018:50:256
244860992422,2cyclictest470293-21kworker/u16:4+events_unbound19:33:286
244860992422,2cyclictest470293-21kworker/u16:4+events_unbound19:33:286
244860992422,2cyclictest359962-21kworker/u16:3+events_unbound16:18:186
244856992421,3cyclictest843623-21kworker/u16:2+flush-8:020:23:335
244854992422,2cyclictest783181-21kworker/u16:0+flush-8:020:13:064
244854992421,2cyclictest652320-21kworker/u16:0+flush-8:019:03:134
244861992321,1cyclictest0-21swapper/716:08:237
244860992321,2cyclictest359439-21kworker/u16:1+flush-8:016:27:586
244854992321,2cyclictest601890-21kworker/u16:3+events_unbound18:47:464
244854992321,2cyclictest509802-21kworker/u16:0+events_unbound17:56:164
244854992321,2cyclictest470293-21kworker/u16:4+events_unbound19:36:024
244854992321,2cyclictest470293-21kworker/u16:4+events_unbound19:36:024
244854992321,2cyclictest288871-21kworker/u16:4+flush-8:016:25:544
244861992217,4cyclictest581278-21latency_hist18:08:037
244861992217,4cyclictest581278-21latency_hist18:08:037
244860992220,2cyclictest3732922-21kworker/u16:3+events_unbound15:38:346
244856992219,2cyclictest359439-21kworker/u16:1+events_unbound16:58:185
244854992220,2cyclictest883827-21kworker/u16:2+flush-8:020:35:464
244854992219,3cyclictest3732922-21kworker/u16:3+flush-8:015:24:344
244861992116,4cyclictest853869-21latency_hist20:23:037
244861992116,4cyclictest853869-21latency_hist20:23:037
244861992116,4cyclictest520800-21latency_hist17:38:037
244860992119,2cyclictest288871-21kworker/u16:4+flush-8:016:17:596
244854992119,2cyclictest883815-21kworker/u16:0+flush-8:020:50:134
244854992119,2cyclictest509802-21kworker/u16:0+flush-8:017:34:184
244854992119,2cyclictest3732922-21kworker/u16:3+flush-8:015:48:034
244861992019,0cyclictest0-21swapper/719:13:257
244861992019,0cyclictest0-21swapper/715:25:527
244861992018,1cyclictest0-21swapper/720:43:227
244860992018,2cyclictest601890-21kworker/u16:3+flush-8:020:23:276
244860992018,2cyclictest601693-21kworker/u16:2+flush-8:018:42:246
244860992018,2cyclictest288871-21kworker/u16:4+events_unbound16:48:296
244854992018,2cyclictest540670-21kworker/u16:1+flush-8:017:52:344
244861991919,0cyclictest0-21swapper/715:48:247
244861991914,4cyclictest339386-21latency_hist16:08:047
24486199190,17cyclictest460291-21latency_hist17:08:037
244856991917,2cyclictest601890-21kworker/u16:3+flush-8:018:23:035
244856991915,3cyclictest288871-21kworker/u16:4+flush-8:016:18:355
244861991817,0cyclictest0-21swapper/720:51:177
244861991817,0cyclictest0-21swapper/720:38:227
244861991813,4cyclictest288998-21latency_hist15:43:037
244860991816,2cyclictest601890-21kworker/u16:3+events_unbound19:08:316
244860991816,2cyclictest601693-21kworker/u16:2+flush-8:018:59:296
244860991816,2cyclictest601693-21kworker/u16:2+events_unbound18:28:546
244860991816,2cyclictest3732922-21kworker/u16:3+events_unbound16:03:346
244861991714,2cyclictest4175659-21snmpd18:57:077
244861991711,5cyclictest409911-21latency_hist16:43:037
244860991715,2cyclictest853576-21kworker/u16:4+flush-8:020:48:106
244860991715,2cyclictest783181-21kworker/u16:0+flush-8:019:49:386
244860991715,2cyclictest470293-21kworker/u16:4+flush-8:018:56:266
244860991715,2cyclictest359962-21kworker/u16:3+flush-8:017:10:586
244856991713,3cyclictest359962-21kworker/u16:3+flush-8:017:23:285
244861991615,0cyclictest0-21swapper/720:33:327
244861991614,1cyclictest0-21swapper/720:28:247
244861991613,1cyclictest0-21swapper/719:08:337
244860991614,2cyclictest601890-21kworker/u16:3+flush-8:020:13:166
244860991614,2cyclictest470293-21kworker/u16:4+flush-8:018:11:316
244860991614,2cyclictest470293-21kworker/u16:4+flush-8:017:36:136
244860991614,2cyclictest470293-21kworker/u16:4+events_unbound17:21:066
244860991614,2cyclictest359962-21kworker/u16:3+events_unbound17:41:136
244860991614,2cyclictest278675-21kworker/u16:1+flush-8:015:44:596
244856991613,2cyclictest601693-21kworker/u16:2+flush-8:018:23:225
24485699160,15cyclictest0-21swapper/517:38:225
244861991513,1cyclictest0-21swapper/720:03:347
244861991512,2cyclictest490595-21journal-offline17:23:037
244861991512,2cyclictest0-21swapper/717:48:047
24486199150,13cyclictest589235-21systemd-journal19:57:257
244860991513,2cyclictest591286-21kworker/u16:1+flush-8:018:27:136
244860991513,2cyclictest470293-21kworker/u16:4+events_unbound18:44:096
244860991513,2cyclictest3732922-21kworker/u16:3+events_unbound15:31:496
244854991513,2cyclictest329178-21kworker/u16:0+events_unbound16:13:214
24486199148,5cyclictest510726-21latency_hist17:33:037
244861991413,0cyclictest0-21swapper/718:18:237
244861991412,1cyclictest0-21swapper/717:38:217
24486199140,13cyclictest0-21swapper/718:08:307
244860991412,2cyclictest470293-21kworker/u16:4+flush-8:018:04:026
244860991412,2cyclictest470293-21kworker/u16:4+flush-8:018:04:026
244860991412,2cyclictest3732922-21kworker/u16:3+events_unbound15:33:386
244860991412,2cyclictest359962-21kworker/u16:3+events_unbound17:28:216
244860991412,2cyclictest288871-21kworker/u16:4+flush-8:016:30:196
244856991412,2cyclictest601890-21kworker/u16:3+flush-8:020:33:305
244856991412,2cyclictest410144-21kworker/u16:0+events_unbound17:18:205
24486199139,3cyclictest4175659-21snmpd16:20:427
24486199137,5cyclictest589235-21systemd-journal18:38:027
24486199136,6cyclictest0-21swapper/716:46:157
24486199130,1cyclictest0-21swapper/716:48:287
24486199130,13cyclictest0-21swapper/718:38:297
24486199130,12cyclictest621438-21taskset18:24:547
24486199130,12cyclictest0-21swapper/718:43:257
244860991311,2cyclictest509802-21kworker/u16:0+events_unbound17:56:036
244860991311,2cyclictest470293-21kworker/u16:4+flush-8:019:38:416
244860991311,2cyclictest470293-21kworker/u16:4+flush-8:018:16:286
244860991311,2cyclictest359962-21kworker/u16:3+events_unbound16:34:516
244860991311,2cyclictest288871-21kworker/u16:4+flush-8:016:47:026
244856991312,1cyclictest0-21swapper/519:52:125
244856991312,1cyclictest0-21swapper/517:13:275
244856991310,2cyclictest883827-21kworker/u16:2+flush-8:020:43:185
244856991310,2cyclictest391184-21irqcore16:33:205
244854991311,2cyclictest509802-21kworker/u16:0+flush-8:018:08:034
244854991311,2cyclictest509802-21kworker/u16:0+flush-8:018:08:034
24486199129,2cyclictest662446-21/usr/sbin/munin18:48:297
24486199129,2cyclictest662446-21/usr/sbin/munin18:48:297
24486199129,2cyclictest4175659-21snmpd18:14:217
24486199129,2cyclictest4175659-21snmpd16:54:377
24486199129,2cyclictest4175659-21snmpd16:34:307
24486199129,2cyclictest4175659-21snmpd15:44:087
24486199127,4cyclictest450193-21latency_hist17:03:047
24486199121,3cyclictest774757-21kthreadcore19:43:227
244861991212,0cyclictest0-21swapper/716:27:587
24486199120,2cyclictest0-21swapper/716:13:237
24486199120,11cyclictest4175659-21snmpd20:13:547
244860991210,2cyclictest601890-21kworker/u16:3+flush-8:020:18:066
244860991210,2cyclictest601890-21kworker/u16:3+flush-8:020:18:066
244860991210,2cyclictest3732922-21kworker/u16:3+events_unbound15:58:516
244860991210,2cyclictest359439-21kworker/u16:1+events_unbound16:42:246
24485699129,3cyclictest783181-21kworker/u16:0+flush-8:019:58:345
24485699129,2cyclictest3732922-21kworker/u16:3+flush-8:016:03:335
244856991210,2cyclictest509802-21kworker/u16:0+flush-8:017:43:315
24485699120,1cyclictest0-21swapper/519:37:135
24485699120,1cyclictest0-21swapper/519:37:135
24485699120,1cyclictest0-21swapper/517:08:155
24486199119,1cyclictest0-21swapper/718:58:267
24486199118,2cyclictest4175659-21snmpd17:17:407
24486199117,3cyclictest854265-21/usr/sbin/munin20:23:357
24486199116,4cyclictest739932-21kthreadcore19:23:267
24486199114,6cyclictest0-21swapper/718:01:177
24486199113,7cyclictest948-21NetworkManager19:48:167
24486199110,10cyclictest4175659-21snmpd19:35:207
24486199110,10cyclictest4175659-21snmpd19:35:207
24486099119,2cyclictest652320-21kworker/u16:0+events_unbound19:03:386
244860991110,1cyclictest0-21swapper/615:48:066
24485699119,2cyclictest601693-21kworker/u16:2+flush-8:018:58:345
24485699117,3cyclictest793482-21kworker/u16:1+flush-8:020:08:375
24485699117,3cyclictest3732922-21kworker/u16:3+flush-8:015:43:205
244856991110,1cyclictest0-21swapper/520:22:475
244856991110,1cyclictest0-21swapper/520:22:475
244856991110,1cyclictest0-21swapper/520:17:545
244856991110,1cyclictest0-21swapper/519:09:385
244856991110,1cyclictest0-21swapper/518:15:345
244856991110,1cyclictest0-21swapper/517:58:425
244856991110,1cyclictest0-21swapper/515:48:485
24485699110,11cyclictest0-21swapper/520:04:395
24485699110,11cyclictest0-21swapper/518:03:445
24485699110,11cyclictest0-21swapper/518:03:445
24485699110,11cyclictest0-21swapper/517:56:585
24485699110,11cyclictest0-21swapper/516:57:515
24485699110,11cyclictest0-21swapper/516:39:525
24485699110,11cyclictest0-21swapper/516:14:205
24485699110,0cyclictest0-21swapper/518:51:025
24485699110,0cyclictest0-21swapper/518:51:025
24486199107,2cyclictest0-21swapper/715:58:197
24486199100,2cyclictest1131-21lldpd17:57:167
24485699109,1cyclictest0-21swapper/520:49:075
24485699109,1cyclictest0-21swapper/519:54:435
24485699109,1cyclictest0-21swapper/519:04:365
24485699109,1cyclictest0-21swapper/518:34:575
24485699109,1cyclictest0-21swapper/516:12:365
24485699109,1cyclictest0-21swapper/515:24:035
24485699108,2cyclictest601890-21kworker/u16:3+flush-8:019:43:185
24485699107,3cyclictest246128-21kworker/u16:1+flush-8:015:28:065
24485699105,4cyclictest713601-21cat19:13:195
24485699100,10cyclictest0-21swapper/517:04:475
24485699100,0cyclictest0-21swapper/519:26:215
2448569998,1cyclictest0-21swapper/518:55:175
2448569998,1cyclictest0-21swapper/518:39:135
2448569998,1cyclictest0-21swapper/517:49:245
2448569994,4cyclictest380265-21grep16:28:165
2448569990,8cyclictest582033-21expr18:08:185
2448569987,1cyclictest0-21swapper/519:38:585
2448569987,1cyclictest0-21swapper/517:34:065
2448569987,1cyclictest0-21swapper/515:56:125
2448569986,2cyclictest591286-21kworker/u16:1+flush-8:018:28:195
2448569984,3cyclictest421106-21tune2fs16:48:195
2448569983,4cyclictest652372-21cpuspeed18:43:155
2448569983,4cyclictest501067-21cpu17:28:155
2448569983,4cyclictest410629-21expr16:43:165
2448569972,4cyclictest370408-21expr16:23:195
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional