You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 12:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Sun Nov 30, 2025 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
292938326050,7sleep60-21swapper/603:22:366
292934325949,7sleep50-21swapper/503:22:035
292909825421,29sleep70-21swapper/703:18:447
2929778994842,5cyclictest2939762-21latency_hist03:28:027
292936924737,7sleep40-21swapper/403:22:264
2929765994643,2cyclictest2888047-21kworker/u16:5+events_unbound04:33:044
2929765994643,2cyclictest2888047-21kworker/u16:5+events_unbound04:33:044
2929765994442,2cyclictest3104963-21kworker/u16:3+events_unbound04:59:354
2929765994441,2cyclictest3406965-21kworker/u16:4+events_unbound07:28:284
2929765993836,2cyclictest3306242-21kworker/u16:2+events_unbound06:55:344
2929765993836,2cyclictest3053616-21kworker/u16:2+flush-8:004:53:444
2929778993732,4cyclictest589235-21systemd-journal03:53:017
2929778993732,4cyclictest589235-21systemd-journal03:53:017
2929765993735,2cyclictest2888047-21kworker/u16:5+events_unbound04:31:324
2929765993734,2cyclictest3477617-21kworker/u16:1+events_unbound08:43:114
2929765993634,2cyclictest3053616-21kworker/u16:2+events_unbound05:26:354
2929778993534,0cyclictest0-21swapper/705:08:327
2929778993530,4cyclictest3407303-21latency_hist07:18:017
2929769993432,2cyclictest3134310-21kworker/u16:1+flush-8:005:03:005
2929765993432,2cyclictest3477617-21kworker/u16:1+events_unbound08:16:114
292977899330,31cyclictest3417513-21date07:23:007
2929769993331,2cyclictest3406965-21kworker/u16:4+flush-8:007:48:285
2929769993330,2cyclictest2963689-21kworker/u16:2+flush-8:003:48:025
2929769993330,2cyclictest2963689-21kworker/u16:2+flush-8:003:48:025
2929765993331,2cyclictest3568469-21kworker/u16:0+events_unbound08:36:524
2929765993331,2cyclictest3497841-21kworker/u16:2+events_unbound08:02:344
2929765993331,2cyclictest3266122-21kworker/u16:0+events_unbound06:17:374
2929765993331,2cyclictest2973964-21kworker/u16:1+events_unbound03:56:474
2929765993331,2cyclictest1504635-21kworker/u16:0+flush-8:003:48:284
2929765993331,2cyclictest1504635-21kworker/u16:0+flush-8:003:48:284
2929778993227,4cyclictest589235-21systemd-journal07:28:007
2929778993227,4cyclictest3387210-21latency_hist07:08:017
2929773993227,2cyclictest3053616-21kworker/u16:2+flush-8:005:33:326
2929769993230,2cyclictest3134310-21kworker/u16:1+events_unbound05:16:335
2929769993230,2cyclictest3053616-21kworker/u16:2+events_unbound05:11:365
2929769993230,2cyclictest3053616-21kworker/u16:2+events_unbound04:22:055
2929765993230,2cyclictest3594356-21kworker/u16:4+flush-8:008:51:434
2929765993230,2cyclictest3397164-21kworker/u16:2+events_unbound07:19:114
2929765993230,2cyclictest3276229-21kworker/u16:1+events_unbound06:51:344
2929765993230,2cyclictest3276229-21kworker/u16:1+events_unbound06:51:344
2929765993230,2cyclictest3185667-21kworker/u16:3+events_unbound05:33:334
2929778993126,4cyclictest589235-21systemd-journal06:05:337
2929778993126,4cyclictest589235-21systemd-journal06:05:337
2929769993129,2cyclictest3406965-21kworker/u16:4+events_unbound08:10:135
2929769993129,2cyclictest3134310-21kworker/u16:1+events_unbound05:21:165
2929769993129,2cyclictest3034606-21kworker/u16:3+events_unbound04:14:445
2929769993129,2cyclictest2919021-21kworker/u16:1+events_unbound03:35:175
2929769993129,2cyclictest2888047-21kworker/u16:5+events_unbound03:57:485
2929765993129,2cyclictest3406965-21kworker/u16:4+events_unbound07:50:354
2929765993129,2cyclictest3134310-21kworker/u16:1+flush-8:005:21:114
2929765993129,2cyclictest3134310-21kworker/u16:1+events_unbound05:17:124
2929765993129,2cyclictest3084516-21kworker/u16:0+flush-8:004:42:084
2929765993129,2cyclictest3053616-21kworker/u16:2+flush-8:005:50:344
2929765993129,2cyclictest3053616-21kworker/u16:2+events_unbound05:31:344
2929765993129,2cyclictest3053616-21kworker/u16:2+events_unbound05:12:354
2929765993129,2cyclictest2973964-21kworker/u16:1+flush-8:004:03:034
2929765993129,2cyclictest2963689-21kworker/u16:2+flush-8:003:42:564
2929765993129,2cyclictest2919021-21kworker/u16:1+events_unbound03:31:484
2929765993129,2cyclictest1504635-21kworker/u16:0+events_unbound03:44:304
2929769993028,2cyclictest3594356-21kworker/u16:4+events_unbound08:50:155
2929769993028,2cyclictest3477617-21kworker/u16:1+events_unbound08:06:525
2929769993028,2cyclictest3306242-21kworker/u16:2+events_unbound06:40:125
2929769993028,2cyclictest3165114-21kworker/u16:4+flush-8:005:45:175
2929769993028,2cyclictest3053616-21kworker/u16:2+flush-8:005:53:045
2929769993028,2cyclictest2929269-21kworker/u16:4+flush-8:003:38:275
2929769993028,2cyclictest2888047-21kworker/u16:5+events_unbound03:58:035
2929769993027,2cyclictest3084516-21kworker/u16:0+flush-8:004:53:015
2929765993028,2cyclictest3331775-21kworker/u16:3+flush-8:007:06:214
2929765993028,2cyclictest3331775-21kworker/u16:3+flush-8:006:58:114
2929765993028,2cyclictest3331775-21kworker/u16:3+events_unbound07:46:344
2929765993028,2cyclictest3104963-21kworker/u16:3+flush-8:005:07:344
2929765993028,2cyclictest2973964-21kworker/u16:1+flush-8:004:25:044
2929765993028,2cyclictest2888047-21kworker/u16:5+events_unbound03:59:314
2929778992927,1cyclictest0-21swapper/707:53:237
2929769992927,2cyclictest3397164-21kworker/u16:2+flush-8:007:17:085
2929769992927,2cyclictest3376804-21kworker/u16:0+events_unbound07:00:005
2929769992927,2cyclictest3266122-21kworker/u16:0+flush-8:006:36:085
2929769992927,2cyclictest3185667-21kworker/u16:3+flush-8:006:18:435
2929769992927,2cyclictest3185667-21kworker/u16:3+events_unbound05:40:125
2929769992927,2cyclictest2888047-21kworker/u16:5+events_unbound04:35:355
2929769992927,2cyclictest2888047-21kworker/u16:5+events_unbound04:35:355
2929765992927,2cyclictest3477617-21kworker/u16:1+events_unbound08:07:294
2929765992927,2cyclictest3417126-21kworker/u16:0+flush-8:007:38:364
2929765992927,2cyclictest3417126-21kworker/u16:0+flush-8:007:24:164
2929765992927,2cyclictest3331775-21kworker/u16:3+events_unbound07:11:164
2929765992927,2cyclictest2919021-21kworker/u16:1+events_unbound03:35:334
2929778992823,4cyclictest589235-21systemd-journal08:35:327
2929769992826,2cyclictest3417126-21kworker/u16:0+events_unbound07:35:245
2929769992826,2cyclictest3417126-21kworker/u16:0+events_unbound07:35:245
2929769992826,2cyclictest3306242-21kworker/u16:2+flush-8:006:46:205
2929769992826,2cyclictest3053616-21kworker/u16:2+events_unbound06:01:535
2929769992826,2cyclictest3053616-21kworker/u16:2+events_unbound05:50:415
2929769992826,2cyclictest2963689-21kworker/u16:2+flush-8:004:10:115
2929769992825,2cyclictest3185667-21kworker/u16:3+events_unbound06:23:085
2929765992826,2cyclictest3266122-21kworker/u16:0+events_unbound06:33:294
2929765992826,2cyclictest3185667-21kworker/u16:3+flush-8:006:10:124
2929765992824,3cyclictest3406965-21kworker/u16:4+flush-8:008:28:164
2929773992724,2cyclictest3134310-21kworker/u16:1+events_unbound05:03:166
2929769992725,2cyclictest3518556-21kworker/u16:3+events_unbound08:36:235
2929769992725,2cyclictest3331775-21kworker/u16:3+events_unbound07:45:015
2929769992725,2cyclictest3276229-21kworker/u16:1+events_unbound06:17:575
2929769992725,2cyclictest3266122-21kworker/u16:0+events_unbound06:11:105
2929769992725,2cyclictest3185667-21kworker/u16:3+flush-8:005:37:165
2929769992725,2cyclictest3104963-21kworker/u16:3+events_unbound05:03:315
2929769992725,2cyclictest3104963-21kworker/u16:3+events_unbound04:57:495
2929769992725,2cyclictest3053616-21kworker/u16:2+events_unbound04:32:295
2929769992725,2cyclictest2973964-21kworker/u16:1+events_unbound04:44:205
2929769992725,2cyclictest2973964-21kworker/u16:1+events_unbound04:24:285
2929769992725,2cyclictest2888047-21kworker/u16:5+events_unbound03:32:085
2929765992725,2cyclictest3477617-21kworker/u16:1+events_unbound08:41:474
2929765992725,2cyclictest3331775-21kworker/u16:3+flush-8:007:15:444
2929765992725,2cyclictest3306242-21kworker/u16:2+events_unbound06:44:244
2929765992725,2cyclictest3053616-21kworker/u16:2+flush-8:004:19:454
2929769992624,2cyclictest3477617-21kworker/u16:1+flush-8:008:22:275
2929769992624,2cyclictest3477617-21kworker/u16:1+flush-8:008:22:275
2929769992624,2cyclictest3417126-21kworker/u16:0+events_unbound08:27:315
2929769992624,2cyclictest3406965-21kworker/u16:4+events_unbound07:24:515
2929769992624,2cyclictest3276229-21kworker/u16:1+flush-8:007:09:155
2929769992624,2cyclictest3165114-21kworker/u16:4+events_unbound05:32:175
2929769992624,2cyclictest3053616-21kworker/u16:2+events_unbound04:39:175
2929769992624,2cyclictest2888047-21kworker/u16:5+events_unbound04:04:245
2929765992624,2cyclictest3185668-21kworker/u16:5+flush-8:005:55:434
2929765992623,2cyclictest2973964-21kworker/u16:1+flush-8:004:43:124
2929778992520,4cyclictest2963863-21latency_hist03:38:017
2929769992523,2cyclictest3557711-21kworker/u16:2+events_unbound08:40:275
2929769992523,2cyclictest3406965-21kworker/u16:4+events_unbound07:55:035
2929769992523,2cyclictest3406965-21kworker/u16:4+events_unbound07:41:245
2929769992523,2cyclictest3276229-21kworker/u16:1+flush-8:007:05:435
2929769992523,2cyclictest2963689-21kworker/u16:2+events_unbound03:44:095
2929769992523,2cyclictest2888047-21kworker/u16:5+events_unbound03:25:025
2929769992522,2cyclictest3185668-21kworker/u16:5+events_unbound06:03:105
2929769992522,2cyclictest3185668-21kworker/u16:5+events_unbound06:03:105
2929765992523,2cyclictest3276229-21kworker/u16:1+events_unbound06:25:014
2929765992522,2cyclictest3084516-21kworker/u16:0+events_unbound04:48:034
2929769992422,2cyclictest3557711-21kworker/u16:2+events_unbound08:45:535
2929769992422,2cyclictest3518556-21kworker/u16:3+flush-8:008:32:325
2929769992422,2cyclictest3406965-21kworker/u16:4+flush-8:008:01:485
2929769992422,2cyclictest3406965-21kworker/u16:4+events_unbound07:29:075
2929769992422,2cyclictest3331775-21kworker/u16:3+flush-8:006:55:565
2929769992422,2cyclictest3276229-21kworker/u16:1+events_unbound06:29:485
2929769992422,2cyclictest3165114-21kworker/u16:4+events_unbound05:25:035
2929765992422,2cyclictest3518556-21kworker/u16:3+flush-8:008:27:554
2929765992422,2cyclictest3518556-21kworker/u16:3+flush-8:008:19:384
2929765992422,2cyclictest3518556-21kworker/u16:3+flush-8:008:19:384
2929765992422,2cyclictest1504635-21kworker/u16:0+events_unbound03:27:364
292977899230,21cyclictest589235-21systemd-journal08:28:017
2929765992321,2cyclictest3417126-21kworker/u16:0+events_unbound07:33:524
2929765992321,2cyclictest3417126-21kworker/u16:0+events_unbound07:33:524
2929765992321,2cyclictest3306242-21kworker/u16:2+events_unbound06:39:364
2929765992321,2cyclictest3185667-21kworker/u16:3+events_unbound06:21:044
2929765992321,2cyclictest3185667-21kworker/u16:3+events_unbound06:05:404
2929765992321,2cyclictest3185667-21kworker/u16:3+events_unbound06:05:404
2929765992321,2cyclictest3034606-21kworker/u16:3+events_unbound04:17:204
2929773992219,3cyclictest3165114-21kworker/u16:4+events_unbound05:43:296
2929773992218,3cyclictest3406965-21kworker/u16:4+events_unbound07:18:306
2929769992220,2cyclictest3417126-21kworker/u16:0+events_unbound08:14:515
2929769992220,2cyclictest3406965-21kworker/u16:4+events_unbound07:22:115
2929778992116,4cyclictest3457766-21latency_hist07:43:017
2929778992116,4cyclictest3236024-21latency_hist05:53:017
2929773992118,3cyclictest3276229-21kworker/u16:1+events_unbound06:33:176
2929773992118,3cyclictest3185668-21kworker/u16:5+events_unbound05:58:306
2929769992119,2cyclictest3276229-21kworker/u16:1+flush-8:006:51:405
2929769992119,2cyclictest3276229-21kworker/u16:1+flush-8:006:51:405
2929765992119,2cyclictest3417126-21kworker/u16:0+flush-8:008:10:424
2929765992119,2cyclictest3417126-21kworker/u16:0+events_unbound07:54:514
2929778991918,0cyclictest0-21swapper/707:33:187
2929778991918,0cyclictest0-21swapper/707:33:187
2929778991913,5cyclictest3205833-21latency_hist05:38:017
2929773991917,2cyclictest3104963-21kworker/u16:3+flush-8:005:13:326
2929773991917,2cyclictest2973964-21kworker/u16:1+flush-8:004:13:336
2929773991915,3cyclictest3266122-21kworker/u16:0+flush-8:006:18:016
2929773991915,3cyclictest3084516-21kworker/u16:0+flush-8:004:48:176
2929765991917,2cyclictest3053616-21kworker/u16:2+flush-8:006:00:084
2929765991916,2cyclictest3165114-21kworker/u16:4+events_unbound05:38:054
2929778991817,0cyclictest0-21swapper/704:06:137
2929778991816,1cyclictest0-21swapper/706:18:237
2929778991815,2cyclictest4175659-21snmpd08:30:497
2929778991813,4cyclictest3084850-21latency_hist04:38:017
2929778991813,4cyclictest3084850-21latency_hist04:38:017
2929765991816,2cyclictest3266122-21kworker/u16:0+events_unbound06:29:134
2929778991716,0cyclictest0-21swapper/705:53:187
2929778991715,1cyclictest0-21swapper/706:39:047
2929778991714,2cyclictest3499401-21irqcore08:03:187
2929778991712,4cyclictest3135261-21latency_hist05:03:027
2929773991715,2cyclictest3406965-21kworker/u16:4+flush-8:007:41:096
2929765991716,1cyclictest2888047-21kworker/u16:5+events_unbound04:09:194
2929765991715,2cyclictest3185668-21kworker/u16:5+events_unbound05:47:534
2929778991615,0cyclictest0-21swapper/705:58:257
2929778991611,4cyclictest3467840-21latency_hist07:48:017
2929778991610,5cyclictest3588798-21latency_hist08:48:017
2929778991610,5cyclictest3346882-21latency_hist06:48:017
292977899160,14cyclictest3004224-21latency_hist03:58:027
292977899160,14cyclictest2967494-21cut03:38:217
292977899160,14cyclictest1021-21rs:main7
292977899160,14cyclictest0-21swapper/706:53:257
2929773991613,2cyclictest2888047-21kworker/u16:5+flush-8:003:53:186
292977899159,5cyclictest3185630-21latency_hist05:28:017
2929778991514,0cyclictest0-21swapper/708:42:037
2929778991513,1cyclictest0-21swapper/707:08:237
2929778991510,4cyclictest3518160-21latency_hist08:13:007
292977899150,14cyclictest3528255-21latency_hist08:18:017
2929773991513,2cyclictest3266122-21kworker/u16:0+flush-8:006:28:196
2929778991413,0cyclictest0-21swapper/706:08:267
2929778991411,2cyclictest0-21swapper/704:18:027
292977899140,13cyclictest0-21swapper/705:28:277
2929773991413,1cyclictest0-21swapper/607:33:246
2929773991413,1cyclictest0-21swapper/607:33:246
2929773991412,2cyclictest3185667-21kworker/u16:3+events_unbound06:23:296
2929773991410,3cyclictest3519337-21cat08:13:196
2929773991410,3cyclictest3075948-21cat04:33:186
2929773991410,3cyclictest3075948-21cat04:33:186
292977899139,3cyclictest4175659-21snmpd08:21:077
292977899139,3cyclictest4175659-21snmpd08:21:077
292977899138,4cyclictest3377131-21latency_hist07:03:017
292977899138,4cyclictest3215875-21latency_hist05:43:027
292977899137,5cyclictest27763-21gvfs-afc-volume05:03:267
292977899135,7cyclictest27763-21gvfs-afc-volume03:58:277
2929778991312,1cyclictest0-21swapper/705:43:257
2929778991312,0cyclictest0-21swapper/707:58:117
2929778991312,0cyclictest0-21swapper/706:13:127
2929778991311,1cyclictest0-21swapper/704:48:237
292977899130,7cyclictest984960-21gvfsd-wsdd03:47:447
2929773991312,1cyclictest0-21swapper/608:28:166
2929773991310,2cyclictest0-21swapper/608:10:366
292977899129,3cyclictest96-21ksoftirqd/705:13:257
292977899128,3cyclictest96-21ksoftirqd/704:13:337
292977899127,4cyclictest589235-21systemd-journal04:56:487
292977899125,5cyclictest4035268-21containerd06:28:277
292977899120,3cyclictest3093111-21cut04:38:237
292977899120,11cyclictest3060280-21kthreadcore04:23:247
292977899120,11cyclictest2953809-21latency_hist03:33:027
292977899120,10cyclictest3468234-21/usr/sbin/munin07:48:157
292977399128,3cyclictest3539506-21cat08:23:176
292977399128,3cyclictest3417126-21kworker/u16:0+flush-8:007:58:206
292977399120,1cyclictest0-21swapper/608:47:396
292977399120,1cyclictest0-21swapper/604:54:306
292977399120,1cyclictest0-21swapper/604:02:096
292977899119,1cyclictest3437562-21munin-run07:33:007
292977899118,2cyclictest4175659-21snmpd06:50:027
292977899118,2cyclictest4175659-21snmpd06:50:027
292977899118,2cyclictest4175659-21snmpd04:11:017
292977899110,9cyclictest0-21swapper/705:18:177
292977899110,10cyclictest4175659-21snmpd04:32:027
292977399117,3cyclictest1504635-21kworker/u16:0+flush-8:003:28:326
2929773991110,1cyclictest0-21swapper/606:07:016
2929773991110,1cyclictest0-21swapper/606:07:016
2929773991110,1cyclictest0-21swapper/605:55:496
2929773991110,1cyclictest0-21swapper/605:32:296
2929773991110,1cyclictest0-21swapper/604:05:376
2929773991110,1cyclictest0-21swapper/603:44:216
2929773991110,1cyclictest0-21swapper/603:34:156
292977399110,11cyclictest0-21swapper/608:40:056
292977399110,11cyclictest0-21swapper/607:55:006
292977399110,11cyclictest0-21swapper/607:47:356
292977399110,11cyclictest0-21swapper/607:29:096
292977399110,11cyclictest0-21swapper/607:00:186
292977399110,11cyclictest0-21swapper/606:47:326
292977399110,11cyclictest0-21swapper/606:39:486
292977399110,11cyclictest0-21swapper/605:39:256
292977399110,11cyclictest0-21swapper/604:39:036
292977899100,8cyclictest0-21swapper/706:23:147
292977399109,1cyclictest0-21swapper/608:49:236
292977399109,1cyclictest0-21swapper/608:06:306
292977399109,1cyclictest0-21swapper/607:50:546
292977399109,1cyclictest0-21swapper/607:04:336
292977399109,1cyclictest0-21swapper/606:10:296
292977399109,1cyclictest0-21swapper/605:27:596
292977399109,1cyclictest0-21swapper/604:46:466
292977399107,3cyclictest3104963-21kworker/u16:3+flush-8:004:58:326
292977399105,4cyclictest3559459-21tune2fs08:33:166
29297789995,4cyclictest27763-21gvfs-afc-volume06:33:237
29297789990,8cyclictest984960-21gvfsd-wsdd08:49:137
29297739998,1cyclictest0-21swapper/608:18:586
29297739998,1cyclictest0-21swapper/608:18:586
29297739998,1cyclictest0-21swapper/607:08:496
29297739998,1cyclictest0-21swapper/604:29:356
29297739996,2cyclictest0-21swapper/605:08:186
29297739995,3cyclictest3287615-21cat06:18:186
29297739987,1cyclictest0-21swapper/607:17:586
29297739987,1cyclictest0-21swapper/604:09:066
29297739986,1cyclictest0-21swapper/604:23:256
29297739983,4cyclictest3419100-21grep07:23:186
29297739983,4cyclictest3165953-21sort05:18:176
29297739983,4cyclictest2986452-21cat03:48:216
29297739983,4cyclictest2986452-21cat03:48:216
29297739983,4cyclictest2938483-21cut03:23:246
29297739976,1cyclictest0-21swapper/606:51:006
29297739976,1cyclictest0-21swapper/606:51:006
29297739976,1cyclictest0-21swapper/605:48:066
29297739976,1cyclictest0-21swapper/603:41:496
29297739975,1cyclictest0-21swapper/604:18:226
29297739972,4cyclictest3357720-21expr06:53:146
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional