You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-30 - 21:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Thu Oct 30, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
289958925418,30sleep70-21swapper/715:20:417
2900819994240,2cyclictest3247591-21kworker/u16:1+flush-8:018:50:024
290041124232,7sleep50-21swapper/515:24:245
2900821994038,2cyclictest3116791-21kworker/u16:5+flush-8:017:19:375
290082399390,37cyclictest3339379-21latency_hist19:00:037
290023123929,7sleep60-21swapper/615:21:536
290082399380,36cyclictest3480891-21latency_hist20:10:027
2900821993836,2cyclictest2910555-21kworker/u16:0+events_unbound15:46:385
2900821993835,2cyclictest3187572-21kworker/u16:4+flush-8:017:50:055
2900821993735,2cyclictest2823807-21kworker/u16:1+flush-8:015:35:055
2900823993631,4cyclictest2955276-21latency_hist15:50:037
2900821993634,2cyclictest3339206-21kworker/u16:2+events_unbound19:10:475
2900821993634,2cyclictest3208118-21kworker/u16:0+flush-8:018:55:375
2900819993633,2cyclictest3146440-21kworker/u16:4+events_unbound17:25:044
2900823993530,4cyclictest3278817-21latency_hist18:30:027
2900822993533,2cyclictest3036108-21kworker/u16:2+flush-8:016:36:376
2900821993533,2cyclictest3187572-21kworker/u16:4+flush-8:017:58:375
2900821993533,2cyclictest3187572-21kworker/u16:4+flush-8:017:58:375
2900821993533,2cyclictest3016289-21kworker/u16:3+events_unbound16:20:385
2900819993533,2cyclictest3308748-21kworker/u16:3+flush-8:018:50:334
2900823993433,0cyclictest0-21swapper/717:10:287
2900821993431,2cyclictest3187565-21kworker/u16:1+events_unbound17:45:065
289078423423,8sleep40-21swapper/415:20:144
2900822993331,2cyclictest3430191-21kworker/u16:3+flush-8:020:10:286
2900822993331,2cyclictest3389910-21kworker/u16:2+events_unbound19:29:296
2900822993331,2cyclictest3308748-21kworker/u16:3+events_unbound19:01:296
2900822993331,2cyclictest2910555-21kworker/u16:0+events_unbound16:57:346
2900821993331,2cyclictest3430191-21kworker/u16:3+events_unbound20:17:365
2900821993331,2cyclictest3247591-21kworker/u16:1+events_unbound18:21:135
2900821993331,2cyclictest3116791-21kworker/u16:5+events_unbound17:14:385
2900821993330,2cyclictest3187572-21kworker/u16:4+flush-8:018:00:135
2900823993227,4cyclictest3258671-21latency_hist18:20:037
290082399320,30cyclictest3359615-21latency_hist19:10:037
2900822993230,2cyclictest3208118-21kworker/u16:0+events_unbound18:50:366
2900822993229,2cyclictest3460430-21kworker/u16:2+events_unbound20:00:146
2900821993230,2cyclictest3208118-21kworker/u16:0+flush-8:018:40:055
2900821993230,2cyclictest3116791-21kworker/u16:5+flush-8:017:08:375
2900819993229,3cyclictest3400522-21kworker/u16:1+events_unbound19:35:274
2900822993129,2cyclictest3500758-21kworker/u16:1+events_unbound20:20:346
2900822993129,2cyclictest3156866-21kworker/u16:2+events_unbound17:27:466
2900822993129,2cyclictest3107019-21kworker/u16:3+events_unbound17:34:176
2900822993129,2cyclictest3076954-21kworker/u16:2+events_unbound17:03:466
2900822993129,2cyclictest2208190-21kworker/u16:7+flush-8:016:09:136
2900821993129,2cyclictest3308748-21kworker/u16:3+flush-8:019:07:425
2900821993129,2cyclictest3005755-21kworker/u16:1+flush-8:016:49:305
2900821993129,2cyclictest2823807-21kworker/u16:1+events_unbound15:56:385
2900819993129,2cyclictest3460430-21kworker/u16:2+flush-8:020:09:574
2900822993028,2cyclictest3500757-21kworker/u16:0+events_unbound20:54:066
2900822993028,2cyclictest3460430-21kworker/u16:2+flush-8:020:15:316
2900822993028,2cyclictest3460430-21kworker/u16:2+events_unbound20:32:176
2900822993028,2cyclictest3430191-21kworker/u16:3+events_unbound19:52:096
2900822993028,2cyclictest3247591-21kworker/u16:1+events_unbound18:24:256
2900822993028,2cyclictest2823807-21kworker/u16:1+events_unbound15:26:426
2900821993028,2cyclictest3460430-21kworker/u16:2+events_unbound20:09:025
2900821993028,2cyclictest3208118-21kworker/u16:0+flush-8:018:38:025
2900821993028,2cyclictest2208190-21kworker/u16:7+events_unbound16:34:415
2900821993028,2cyclictest2208190-21kworker/u16:7+events_unbound16:17:035
2900821993027,2cyclictest3400522-21kworker/u16:1+flush-8:019:48:415
2900819993028,2cyclictest3208118-21kworker/u16:0+flush-8:019:30:024
2900822992927,2cyclictest3460430-21kworker/u16:2+events_unbound20:28:366
2900822992927,2cyclictest3430191-21kworker/u16:3+events_unbound19:44:306
2900822992927,2cyclictest3361213-21kworker/u16:1+events_unbound19:21:056
2900822992927,2cyclictest3247591-21kworker/u16:1+flush-8:018:48:066
2900822992927,2cyclictest3208118-21kworker/u16:0+flush-8:018:57:296
2900822992927,2cyclictest3208118-21kworker/u16:0+events_unbound19:38:346
2900822992927,2cyclictest3208118-21kworker/u16:0+events_unbound18:29:186
2900822992927,2cyclictest3187572-21kworker/u16:4+events_unbound18:19:356
2900822992927,2cyclictest3107019-21kworker/u16:3+events_unbound17:37:266
2900822992927,2cyclictest3036108-21kworker/u16:2+events_unbound16:42:376
2900822992927,2cyclictest2975347-21kworker/u16:3+events_unbound16:04:026
2900822992927,2cyclictest2910555-21kworker/u16:0+events_unbound16:47:466
2900822992927,2cyclictest2910555-21kworker/u16:0+events_unbound15:34:496
2900822992927,2cyclictest2823807-21kworker/u16:1+events_unbound15:46:096
2900822992927,2cyclictest2823807-21kworker/u16:1+events_unbound15:37:076
2900821992927,2cyclictest3400522-21kworker/u16:1+events_unbound19:36:185
2900821992927,2cyclictest3187572-21kworker/u16:4+events_unbound18:07:135
2900821992927,2cyclictest3066112-21kworker/u16:3+events_unbound16:51:155
2900821992927,2cyclictest2208190-21kworker/u16:7+events_unbound15:51:235
2900821992926,2cyclictest3156866-21kworker/u16:2+events_unbound18:15:135
2900822992826,2cyclictest3550438-21kworker/u16:2+flush-8:020:48:456
2900822992826,2cyclictest3430191-21kworker/u16:3+events_unbound20:07:106
2900822992826,2cyclictest3208118-21kworker/u16:0+flush-8:018:13:416
2900822992826,2cyclictest3208118-21kworker/u16:0+events_unbound19:46:536
2900822992826,2cyclictest3208118-21kworker/u16:0+events_unbound19:18:256
2900822992826,2cyclictest3187565-21kworker/u16:1+events_unbound17:53:226
2900822992826,2cyclictest3116791-21kworker/u16:5+events_unbound17:18:336
2900822992826,2cyclictest2995569-21kworker/u16:2+flush-8:016:14:106
2900822992826,2cyclictest2910555-21kworker/u16:0+events_unbound16:53:546
2900821992826,2cyclictest3137141-21kworker/u16:0+events_unbound17:21:335
2900821992826,2cyclictest2924673-21kworker/u16:3+events_unbound15:34:235
2900821992826,2cyclictest2823807-21kworker/u16:1+events_unbound16:03:305
2900821992825,2cyclictest3208118-21kworker/u16:0+events_unbound19:50:065
2900819992826,2cyclictest3187565-21kworker/u16:1+flush-8:018:03:384
2900823992726,0cyclictest0-21swapper/719:25:177
2900823992721,5cyclictest589235-21systemd-journal18:54:367
2900822992725,2cyclictest3500757-21kworker/u16:0+events_unbound20:42:546
2900822992725,2cyclictest3339206-21kworker/u16:2+events_unbound19:11:546
2900822992725,2cyclictest3228731-21kworker/u16:3+flush-8:018:06:186
2900822992725,2cyclictest3156866-21kworker/u16:2+flush-8:017:48:026
2900822992725,2cyclictest3005755-21kworker/u16:1+events_unbound16:16:426
2900821992725,2cyclictest3450565-21kworker/u16:1+flush-8:019:55:335
2900821992725,2cyclictest3208118-21kworker/u16:0+events_unbound19:33:025
2900821992725,2cyclictest3167412-21kworker/u16:0+events_unbound17:35:415
2900821992725,2cyclictest3156866-21kworker/u16:2+flush-8:018:45:365
2900823992625,0cyclictest0-21swapper/717:20:267
2900822992624,2cyclictest3187572-21kworker/u16:4+events_unbound18:02:316
2900822992624,2cyclictest3156866-21kworker/u16:2+flush-8:018:37:426
2900822992624,2cyclictest3005755-21kworker/u16:1+events_unbound17:08:196
2900822992624,2cyclictest2208190-21kworker/u16:7+flush-8:015:44:186
2900822992624,2cyclictest2208190-21kworker/u16:7+flush-8:015:44:186
2900822992624,2cyclictest2208190-21kworker/u16:7+events_unbound16:32:186
2900821992624,2cyclictest3500757-21kworker/u16:0+flush-8:020:35:285
2900821992624,2cyclictest3379647-21kworker/u16:4+events_unbound19:25:375
2900821992624,2cyclictest3247591-21kworker/u16:1+flush-8:018:51:215
2900821992624,2cyclictest3156866-21kworker/u16:2+flush-8:018:14:415
2900821992624,2cyclictest3036108-21kworker/u16:2+events_unbound16:38:025
2900821992624,2cyclictest3005755-21kworker/u16:1+events_unbound16:29:305
2900821992624,2cyclictest2995569-21kworker/u16:2+events_unbound16:10:545
2900821992624,2cyclictest2910555-21kworker/u16:0+flush-8:016:58:225
2900821992624,2cyclictest2910555-21kworker/u16:0+events_unbound17:03:345
2900819992624,2cyclictest3561120-21kworker/u16:1+events_unbound20:50:264
2900819992624,2cyclictest3460430-21kworker/u16:2+events_unbound20:00:024
2900819992623,2cyclictest3308748-21kworker/u16:3+flush-8:019:15:244
2900822992523,2cyclictest3440457-21kworker/u16:4+events_unbound19:59:396
2900822992523,2cyclictest3208118-21kworker/u16:0+events_unbound18:32:276
2900822992523,2cyclictest3187572-21kworker/u16:4+events_unbound17:59:306
2900822992523,2cyclictest3187572-21kworker/u16:4+events_unbound17:59:306
2900822992523,2cyclictest3156866-21kworker/u16:2+events_unbound18:44:096
2900822992523,2cyclictest3005755-21kworker/u16:1+flush-8:017:12:376
2900821992523,2cyclictest3500757-21kworker/u16:0+flush-8:020:31:035
2900821992523,2cyclictest3500757-21kworker/u16:0+events_unbound20:48:095
2900821992523,2cyclictest3430191-21kworker/u16:3+flush-8:019:43:345
2900821992523,2cyclictest3208118-21kworker/u16:0+flush-8:018:25:575
2900821992523,2cyclictest3156866-21kworker/u16:2+flush-8:017:41:025
2900821992523,2cyclictest3156866-21kworker/u16:2+flush-8:017:27:025
2900821992523,2cyclictest2823807-21kworker/u16:1+events_unbound15:44:235
2900821992523,2cyclictest2823807-21kworker/u16:1+events_unbound15:44:235
2900819992523,2cyclictest3500757-21kworker/u16:0+events_unbound20:45:194
2900819992523,2cyclictest3500757-21kworker/u16:0+events_unbound20:25:314
2900819992522,3cyclictest2823807-21kworker/u16:1+events_unbound15:31:294
2900819992522,2cyclictest2823807-21kworker/u16:1+flush-8:016:05:324
2900823992423,0cyclictest0-21swapper/720:34:597
2900823992423,0cyclictest0-21swapper/719:45:317
2900822992422,2cyclictest3500758-21kworker/u16:1+events_unbound20:37:066
2900822992422,2cyclictest3339206-21kworker/u16:2+events_unbound19:07:326
2900822992422,2cyclictest3308748-21kworker/u16:3+flush-8:019:32:186
2900822992422,2cyclictest3137141-21kworker/u16:0+flush-8:017:23:026
2900822992422,2cyclictest3005755-21kworker/u16:1+flush-8:016:26:076
2900822992422,2cyclictest2208190-21kworker/u16:7+events_unbound16:21:336
2900821992422,2cyclictest3500757-21kworker/u16:0+events_unbound20:42:295
2900821992422,2cyclictest3500757-21kworker/u16:0+events_unbound20:27:385
2900821992422,2cyclictest3107019-21kworker/u16:3+events_unbound17:30:505
2900821992421,2cyclictest3361213-21kworker/u16:1+events_unbound19:20:145
2900819992422,2cyclictest3016289-21kworker/u16:3+flush-8:016:25:314
2900819992421,2cyclictest3208118-21kworker/u16:0+flush-8:018:25:144
2900823992317,5cyclictest3015895-21latency_hist16:20:037
2900822992321,2cyclictest2910555-21kworker/u16:0+events_unbound15:51:376
2900822992321,2cyclictest2208190-21kworker/u16:7+events_unbound15:58:536
2900821992321,2cyclictest3500758-21kworker/u16:1+events_unbound20:24:595
2900821992321,2cyclictest3430191-21kworker/u16:3+events_unbound20:13:495
2900821992321,2cyclictest3339206-21kworker/u16:2+flush-8:019:02:375
2900821992321,2cyclictest3005755-21kworker/u16:1+flush-8:016:44:125
2900821992321,2cyclictest2975347-21kworker/u16:3+flush-8:016:08:435
2900819992321,2cyclictest3500757-21kworker/u16:0+events_unbound20:36:124
2900819992321,2cyclictest3187572-21kworker/u16:4+events_unbound18:07:384
2900819992321,2cyclictest2910555-21kworker/u16:0+flush-8:015:35:384
2900823992221,0cyclictest0-21swapper/720:25:157
2900823992221,0cyclictest0-21swapper/719:40:267
2900823992221,0cyclictest0-21swapper/718:14:587
2900822992220,2cyclictest3156866-21kworker/u16:2+events_unbound17:44:446
2900821992220,2cyclictest3379647-21kworker/u16:4+events_unbound19:19:235
2900819992220,2cyclictest3208118-21kworker/u16:0+events_unbound19:51:384
2900819992220,2cyclictest3187565-21kworker/u16:1+events_unbound17:43:384
2900819992220,2cyclictest3156866-21kworker/u16:2+events_unbound17:35:214
2900819992220,2cyclictest2910555-21kworker/u16:0+flush-8:016:55:244
2900819992220,2cyclictest2208190-21kworker/u16:7+events_unbound16:37:384
2900823992116,4cyclictest589235-21systemd-journal16:10:387
2900821992119,2cyclictest3440457-21kworker/u16:4+events_unbound20:00:535
2900821992119,2cyclictest1477038-21kworker/u16:6+events_unbound15:27:235
2900819992119,2cyclictest2910555-21kworker/u16:0+events_unbound17:03:384
2900819992119,2cyclictest2208190-21kworker/u16:7+events_unbound16:30:284
2900819992117,3cyclictest3208118-21kworker/u16:0+flush-8:019:20:284
2900823992019,0cyclictest0-21swapper/717:56:167
2900823992019,0cyclictest0-21swapper/717:56:167
2900823992015,4cyclictest589235-21systemd-journal20:02:367
2900823992015,4cyclictest3551369-21latency_hist20:45:027
2900819992018,2cyclictest3247591-21kworker/u16:1+flush-8:019:00:224
2900819992018,2cyclictest3208118-21kworker/u16:0+events_unbound19:30:204
2900819992018,2cyclictest2910555-21kworker/u16:0+events_unbound15:55:284
2900823991918,0cyclictest0-21swapper/719:15:277
2900823991918,0cyclictest0-21swapper/718:35:297
2900823991918,0cyclictest0-21swapper/716:03:217
2900821991917,2cyclictest3550438-21kworker/u16:2+events_unbound20:53:475
2900819991917,2cyclictest3460430-21kworker/u16:2+events_unbound20:03:374
2900819991917,2cyclictest3208118-21kworker/u16:0+events_unbound18:35:214
2900819991917,2cyclictest3107019-21kworker/u16:3+events_unbound17:31:374
2900819991917,2cyclictest2208190-21kworker/u16:7+flush-8:016:40:294
2900819991916,2cyclictest3187565-21kworker/u16:1+events_unbound17:50:224
2900823991812,5cyclictest3390013-21latency_hist19:25:027
2900819991816,2cyclictest3208118-21kworker/u16:0+flush-8:018:55:184
2900819991816,2cyclictest3156866-21kworker/u16:2+events_unbound18:15:374
2900819991815,2cyclictest2910555-21kworker/u16:0+events_unbound15:45:244
2900819991815,2cyclictest2208190-21kworker/u16:7+flush-8:015:25:194
2900823991716,0cyclictest0-21swapper/715:42:387
2900823991716,0cyclictest0-21swapper/715:42:387
2900821991715,2cyclictest3208118-21kworker/u16:0+events_unbound18:33:325
2900819991715,2cyclictest3500758-21kworker/u16:1+events_unbound20:30:204
2900819991715,2cyclictest3187565-21kworker/u16:1+flush-8:017:45:184
2900819991715,2cyclictest3156866-21kworker/u16:2+flush-8:017:59:584
2900819991715,2cyclictest3156866-21kworker/u16:2+flush-8:017:59:584
2900819991715,2cyclictest3005755-21kworker/u16:1+flush-8:016:52:384
2900823991615,0cyclictest0-21swapper/717:05:187
2900823991615,0cyclictest0-21swapper/716:06:387
2900823991610,5cyclictest3309072-21latency_hist18:45:037
2900823991610,5cyclictest3096786-21latency_hist17:00:037
290082399160,15cyclictest4175659-21snmpd15:59:167
2900819991614,2cyclictest3208118-21kworker/u16:0+events_unbound18:10:204
2900819991614,2cyclictest3005755-21kworker/u16:1+flush-8:016:45:334
2900819991614,2cyclictest2208190-21kworker/u16:7+events_unbound16:18:394
2900823991514,0cyclictest0-21swapper/720:54:187
2900823991512,2cyclictest4175659-21snmpd19:39:187
2900823991512,2cyclictest0-21swapper/718:50:027
2900823991512,2cyclictest0-21swapper/718:30:037
290082399150,13cyclictest27944-21gnome-software15:31:257
2900819991513,2cyclictest3339206-21kworker/u16:2+events_unbound19:10:164
2900819991513,2cyclictest3308748-21kworker/u16:3+events_unbound19:05:294
290082399148,5cyclictest3106840-21latency_hist17:05:037
290082399146,7cyclictest27394-21gmain20:15:217
2900823991413,0cyclictest0-21swapper/716:38:127
290082399140,1cyclictest0-21swapper/719:55:267
290082399140,1cyclictest0-21swapper/717:40:297
290082399140,13cyclictest0-21swapper/717:45:277
2900819991411,3cyclictest2975347-21kworker/u16:3+flush-8:016:00:314
2900819991411,2cyclictest3156866-21kworker/u16:2+flush-8:018:40:194
2900819991411,2cyclictest2910555-21kworker/u16:0+flush-8:015:40:224
2900819991411,2cyclictest2910555-21kworker/u16:0+flush-8:015:40:224
290082399139,3cyclictest4175659-21snmpd17:34:337
290082399138,4cyclictest3066421-21latency_hist16:45:037
290082399136,6cyclictest965-21gmain17:35:217
290082399136,6cyclictest0-21swapper/715:30:007
290082399130,1cyclictest0-21swapper/717:25:327
2900819991312,1cyclictest0-21swapper/417:05:394
2900819991312,1cyclictest0-21swapper/416:12:224
2900819991311,2cyclictest3430191-21kworker/u16:3+flush-8:020:13:564
2900819991311,2cyclictest3208118-21kworker/u16:0+events_unbound19:45:294
2900819991311,2cyclictest3208118-21kworker/u16:0+events_unbound18:30:284
290081999130,1cyclictest0-21swapper/419:43:274
290081999130,13cyclictest0-21swapper/417:13:044
290081999130,13cyclictest0-21swapper/416:21:024
290082399129,2cyclictest4175659-21snmpd20:45:567
290082399129,2cyclictest4175659-21snmpd19:30:457
290082399129,2cyclictest4175659-21snmpd17:53:427
290082399129,2cyclictest4175659-21snmpd16:33:407
290082399128,3cyclictest4175659-21snmpd19:14:327
290082399128,3cyclictest4175659-21snmpd16:20:257
290082399128,3cyclictest3127508-21/usr/sbin/munin17:15:327
2900823991212,0cyclictest0-21swapper/719:02:297
2900823991212,0cyclictest0-21swapper/716:45:297
2900823991211,0cyclictest0-21swapper/716:50:137
2900823991210,1cyclictest0-21swapper/720:10:207
2900823991210,1cyclictest0-21swapper/719:50:167
290082399120,11cyclictest3510137-21ntp_offset20:20:307
290081999128,3cyclictest3500757-21kworker/u16:0+flush-8:020:40:324
2900819991211,1cyclictest0-21swapper/417:20:284
2900819991210,2cyclictest3430191-21kworker/u16:3+events_unbound20:20:164
290082399118,2cyclictest4175659-21snmpd18:21:407
290082399118,2cyclictest4175659-21snmpd18:02:257
290082399118,2cyclictest3030752-21kthreadcore16:25:227
290082399117,3cyclictest3228734-21/usr/sbin/munin18:05:267
290082399117,3cyclictest27642-21gmain20:35:247
2900823991110,0cyclictest0-21swapper/715:50:217
290082399110,10cyclictest4175659-21snmpd15:38:127
290081999119,2cyclictest3258654-21kworker/u16:3+flush-8:018:22:444
290081999117,3cyclictest2910555-21kworker/u16:0+flush-8:015:50:294
2900819991110,1cyclictest0-21swapper/417:16:224
290081999107,2cyclictest3460430-21kworker/u16:2+events_unbound20:15:154
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional