You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 01:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot2.osadl.org (updated Thu Nov 27, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8072940,1sleep60-21swapper/606:33:256
3826531995954,4cyclictest4113823-21latency_hist05:53:137
3826518995654,2cyclictest3961108-21kworker/u16:2+flush-8:004:28:135
382592125520,30sleep70-21swapper/703:18:567
382592125520,30sleep70-21swapper/703:18:567
3826531995044,5cyclictest3931111-21latency_hist04:13:147
3826527995048,2cyclictest3961108-21kworker/u16:2+events_unbound04:33:376
3826531994943,5cyclictest232255-21latency_hist08:28:137
3826527994846,2cyclictest4133695-21kworker/u16:2+flush-8:006:18:396
382593724737,7sleep40-21swapper/403:19:094
382593724737,7sleep40-21swapper/403:19:094
3826527994643,2cyclictest3992715-21kworker/u16:4+flush-8:006:43:126
3826531994539,5cyclictest589235-21systemd-journal03:43:137
382619324535,7sleep50-21swapper/503:22:395
382619324535,7sleep50-21swapper/503:22:395
3826518994341,2cyclictest4093455-21kworker/u16:1+flush-8:007:28:125
3826517994138,2cyclictest4093455-21kworker/u16:1+flush-8:007:33:294
3826518993937,2cyclictest3992715-21kworker/u16:4+flush-8:005:53:345
3826517993937,2cyclictest41456-21kworker/u16:2+events_unbound08:35:454
3826531993837,0cyclictest0-21swapper/705:30:337
3826527993836,2cyclictest3992715-21kworker/u16:4+events_unbound06:28:426
3826527993836,2cyclictest3836267-21kworker/u16:1+flush-8:003:58:376
3826527993836,2cyclictest162030-21kworker/u16:1+flush-8:008:08:426
382594823828,7sleep60-21swapper/603:19:196
382594823828,7sleep60-21swapper/603:19:196
41021562370,1chrt4102157-21tr05:43:347
3826517993735,2cyclictest4014935-21kworker/u16:3+flush-8:005:28:334
3826517993734,2cyclictest3836267-21kworker/u16:1+flush-8:003:58:424
3826531993635,0cyclictest0-21swapper/705:16:137
3826531993635,0cyclictest0-21swapper/703:53:167
3826527993634,2cyclictest3872027-21kworker/u16:0+flush-8:003:46:376
3826517993634,2cyclictest4002891-21kworker/u16:0+events_unbound04:59:524
3826527993533,2cyclictest151694-21kworker/u16:4+events_unbound07:59:246
3826517993533,2cyclictest3941324-21kworker/u16:5+events_unbound04:29:464
3826531993433,0cyclictest0-21swapper/706:26:367
3826531993433,0cyclictest0-21swapper/704:27:247
3826527993432,2cyclictest41456-21kworker/u16:2+events_unbound08:17:436
3826527993432,2cyclictest4093455-21kworker/u16:1+events_unbound05:49:346
3826527993432,2cyclictest3992715-21kworker/u16:4+flush-8:005:14:346
3826531993332,0cyclictest0-21swapper/707:33:427
3826531993332,0cyclictest0-21swapper/706:36:487
3826527993331,2cyclictest4072647-21kworker/u16:0+flush-8:005:32:136
3826527993331,2cyclictest3750195-21kworker/u16:2+events_unbound03:24:346
3826527993330,2cyclictest4072647-21kworker/u16:0+flush-8:006:08:166
3826518993331,2cyclictest60181-21kworker/u16:4+events_unbound07:00:135
3826517993331,2cyclictest232435-21kworker/u16:1+events_unbound08:51:434
3826517993331,2cyclictest232435-21kworker/u16:1+events_unbound08:42:454
3826531993231,0cyclictest0-21swapper/708:08:127
3826531993231,0cyclictest0-21swapper/706:22:247
3826527993230,2cyclictest4093455-21kworker/u16:1+flush-8:005:47:246
3826527993230,2cyclictest4093455-21kworker/u16:1+events_unbound07:17:356
3826531993130,0cyclictest0-21swapper/707:09:527
3826531993130,0cyclictest0-21swapper/703:37:247
3826527993129,2cyclictest60181-21kworker/u16:4+events_unbound07:26:406
3826527993129,2cyclictest60181-21kworker/u16:4+events_unbound07:07:176
3826527993129,2cyclictest3941324-21kworker/u16:5+flush-8:004:31:486
3826527993129,2cyclictest3880579-21kworker/u16:4+events_unbound04:10:246
3826527993129,2cyclictest30840-21kworker/u16:0+flush-8:007:10:246
3826527993129,2cyclictest151616-21kworker/u16:3+flush-8:008:30:196
3826527993129,2cyclictest151616-21kworker/u16:3+events_unbound08:05:246
3826517993129,2cyclictest3836267-21kworker/u16:1+events_unbound03:56:334
3826531993029,0cyclictest0-21swapper/708:52:087
3826527993028,2cyclictest41456-21kworker/u16:2+events_unbound08:34:436
3826527993028,2cyclictest41456-21kworker/u16:2+events_unbound08:24:236
3826527993028,2cyclictest3992715-21kworker/u16:4+events_unbound05:06:216
3826527993028,2cyclictest3940214-21kworker/u16:1+flush-8:004:16:346
3826527993028,2cyclictest3900958-21kworker/u16:2+events_unbound04:06:086
3826527993028,2cyclictest3836267-21kworker/u16:1+events_unbound03:54:366
3826527993027,2cyclictest3850375-21kworker/u16:2+events_unbound03:42:336
3826518993028,2cyclictest60181-21kworker/u16:4+flush-8:007:13:085
3826518993028,2cyclictest41456-21kworker/u16:2+flush-8:007:43:125
3826517993028,2cyclictest4093455-21kworker/u16:1+events_unbound06:31:324
3826531992929,0cyclictest0-21swapper/704:00:377
3826531992928,0cyclictest0-21swapper/706:40:137
3826527992927,2cyclictest60181-21kworker/u16:4+events_unbound07:02:166
3826527992927,2cyclictest41456-21kworker/u16:2+flush-8:006:54:406
3826527992927,2cyclictest4002891-21kworker/u16:0+flush-8:004:58:436
3826527992927,2cyclictest3128588-21kworker/u16:3+events_unbound03:36:216
3826527992926,2cyclictest4093455-21kworker/u16:1+events_unbound07:38:166
3826527992926,2cyclictest3930870-21kworker/u16:0+events_unbound04:23:236
3826518992926,2cyclictest151694-21kworker/u16:4+flush-8:008:18:125
3826517992926,2cyclictest3900958-21kworker/u16:2+events_unbound04:08:424
3826531992827,0cyclictest0-21swapper/707:57:237
3826531992827,0cyclictest0-21swapper/705:06:137
3826527992826,2cyclictest4072647-21kworker/u16:0+events_unbound06:05:146
3826527992826,2cyclictest3992715-21kworker/u16:4+events_unbound04:54:166
3826527992826,2cyclictest3128588-21kworker/u16:3+events_unbound03:51:186
3826527992826,2cyclictest30840-21kworker/u16:0+events_unbound06:52:406
3826527992826,2cyclictest232435-21kworker/u16:1+events_unbound08:39:536
3826527992826,2cyclictest151694-21kworker/u16:4+events_unbound07:54:476
3826518992826,2cyclictest4093455-21kworker/u16:1+flush-8:005:53:095
3826518992826,2cyclictest4072647-21kworker/u16:0+flush-8:005:38:315
3826518992826,2cyclictest162030-21kworker/u16:1+flush-8:007:58:245
3826518992825,2cyclictest232435-21kworker/u16:1+events_unbound08:38:335
3826518992824,3cyclictest3992715-21kworker/u16:4+flush-8:004:53:245
3826517992826,2cyclictest4013218-21kworker/u16:1+events_unbound05:13:274
3826517992826,2cyclictest3836267-21kworker/u16:1+events_unbound03:32:294
3826517992826,2cyclictest162030-21kworker/u16:1+flush-8:008:07:434
3826531992726,0cyclictest0-21swapper/707:04:377
3826531992726,0cyclictest0-21swapper/704:18:157
3826527992725,2cyclictest41456-21kworker/u16:2+flush-8:008:22:046
3826527992725,2cyclictest4133695-21kworker/u16:2+flush-8:006:00:246
3826527992725,2cyclictest4014935-21kworker/u16:3+flush-8:005:33:266
3826527992724,2cyclictest3961108-21kworker/u16:2+flush-8:005:23:236
3826527992724,2cyclictest151616-21kworker/u16:3+events_unbound07:48:226
3826517992725,2cyclictest3992715-21kworker/u16:4+events_unbound06:35:074
3826531992625,0cyclictest0-21swapper/706:59:527
3826531992625,0cyclictest0-21swapper/704:54:147
3826531992625,0cyclictest0-21swapper/703:48:237
3826527992624,2cyclictest41456-21kworker/u16:2+flush-8:007:34:366
3826527992624,2cyclictest4093455-21kworker/u16:1+events_unbound05:41:206
3826527992624,2cyclictest4072647-21kworker/u16:0+flush-8:006:14:166
3826527992624,2cyclictest3940214-21kworker/u16:1+events_unbound04:21:096
3826527992624,2cyclictest232435-21kworker/u16:1+events_unbound08:48:436
3826527992624,2cyclictest151694-21kworker/u16:4+events_unbound07:48:006
3826527992623,2cyclictest3961108-21kworker/u16:2+flush-8:004:52:446
3826518992624,2cyclictest4174237-21kworker/u16:3+flush-8:006:23:315
3826518992622,3cyclictest4133695-21kworker/u16:2+flush-8:006:18:445
3826517992624,2cyclictest41456-21kworker/u16:2+events_unbound07:41:294
3826517992624,2cyclictest3900958-21kworker/u16:2+flush-8:004:03:274
3826517992623,2cyclictest3961108-21kworker/u16:2+flush-8:004:33:394
3826531992522,2cyclictest241386-21ntpq08:28:397
3826527992523,2cyclictest4183243-21kworker/u16:5+events_unbound06:26:196
3826527992523,2cyclictest3992715-21kworker/u16:4+events_unbound05:20:336
3826527992523,2cyclictest3836267-21kworker/u16:1+flush-8:003:31:566
3826527992523,2cyclictest232435-21kworker/u16:1+flush-8:008:47:216
3826518992523,2cyclictest3992715-21kworker/u16:4+flush-8:006:33:425
3826518992523,2cyclictest30840-21kworker/u16:0+flush-8:006:58:085
3826518992523,2cyclictest151694-21kworker/u16:4+events_unbound07:48:415
3826518992521,3cyclictest3128588-21kworker/u16:3+flush-8:003:53:295
3826517992523,2cyclictest4014935-21kworker/u16:3+flush-8:005:23:294
3826531992424,0cyclictest0-21swapper/703:30:247
3826531992423,0cyclictest0-21swapper/706:51:247
3826531992423,0cyclictest0-21swapper/706:07:257
3826527992422,2cyclictest4093455-21kworker/u16:1+flush-8:005:55:356
3826527992421,2cyclictest3961108-21kworker/u16:2+events_unbound04:41:286
3826531992322,0cyclictest0-21swapper/704:46:417
3826531992322,0cyclictest0-21swapper/704:41:057
3826527992321,2cyclictest3961108-21kworker/u16:2+flush-8:004:46:316
3826518992320,2cyclictest4013218-21kworker/u16:1+events_unbound05:13:255
3826517992321,2cyclictest3992715-21kworker/u16:4+flush-8:006:03:384
3826517992321,2cyclictest3992715-21kworker/u16:4+flush-8:005:52:344
3826517992321,2cyclictest141416-21kworker/u16:0+events_unbound07:50:214
3826517992320,2cyclictest3992715-21kworker/u16:4+events_unbound05:08:164
3826531992221,0cyclictest0-21swapper/704:14:437
3826531992220,1cyclictest0-21swapper/706:43:407
382653199220,20cyclictest4133907-21latency_hist06:03:137
3826527992220,2cyclictest3992715-21kworker/u16:4+events_unbound06:44:416
3826527992220,2cyclictest30840-21kworker/u16:0+flush-8:007:19:486
3826527992219,2cyclictest3992715-21kworker/u16:4+flush-8:005:08:226
3826518992220,2cyclictest41456-21kworker/u16:2+flush-8:008:46:505
3826518992220,2cyclictest4093455-21kworker/u16:1+flush-8:007:18:345
3826518992220,2cyclictest3930870-21kworker/u16:0+events_unbound04:13:435
3826518992220,2cyclictest30840-21kworker/u16:0+flush-8:007:15:035
3826517992220,2cyclictest3961108-21kworker/u16:2+events_unbound05:18:384
3826517992220,2cyclictest3961108-21kworker/u16:2+events_unbound04:25:344
3826517992219,2cyclictest3850375-21kworker/u16:2+flush-8:003:45:064
3826531992120,0cyclictest0-21swapper/708:10:437
3826531992116,4cyclictest161801-21latency_hist07:53:137
3826518992118,2cyclictest3992715-21kworker/u16:4+flush-8:006:03:265
3826517992119,2cyclictest4133695-21kworker/u16:2+flush-8:006:18:354
3826517992118,2cyclictest41456-21kworker/u16:2+flush-8:008:33:014
3826517992118,2cyclictest3940214-21kworker/u16:1+flush-8:004:41:344
3826517992118,2cyclictest30840-21kworker/u16:0+flush-8:006:53:234
3826531992019,0cyclictest0-21swapper/708:41:337
3826531992019,0cyclictest0-21swapper/707:18:437
3826531992019,0cyclictest0-21swapper/706:54:147
3826531992019,0cyclictest0-21swapper/705:27:287
3826531992017,2cyclictest4175659-21snmpd04:37:177
3826527992018,2cyclictest41456-21kworker/u16:2+events_unbound07:29:186
3826518992018,2cyclictest4174237-21kworker/u16:3+events_unbound06:28:365
3826518992018,2cyclictest41456-21kworker/u16:2+events_unbound07:46:285
3826518992018,2cyclictest3128588-21kworker/u16:3+events_unbound04:08:325
3826517992018,2cyclictest4174237-21kworker/u16:3+flush-8:006:38:424
3826517992018,2cyclictest4093455-21kworker/u16:1+flush-8:007:26:174
3826517992017,2cyclictest3981980-21kworker/u16:3+events_unbound04:48:524
3826531991918,0cyclictest0-21swapper/708:33:427
382653199190,18cyclictest1297940-21gmain04:52:527
3826531991817,0cyclictest0-21swapper/707:29:287
3826531991813,4cyclictest4013185-21latency_hist05:03:147
3826518991816,2cyclictest4093455-21kworker/u16:1+flush-8:005:58:325
3826518991816,2cyclictest3992715-21kworker/u16:4+flush-8:005:10:145
3826518991816,2cyclictest30840-21kworker/u16:0+flush-8:006:48:275
3826518991815,2cyclictest3850375-21kworker/u16:2+flush-8:003:45:055
3826517991816,2cyclictest4072647-21kworker/u16:0+events_unbound06:16:234
3826517991816,2cyclictest4014935-21kworker/u16:3+events_unbound05:43:334
3826517991816,2cyclictest151616-21kworker/u16:3+flush-8:007:53:354
3826517991815,2cyclictest60181-21kworker/u16:4+flush-8:007:13:034
3826531991716,0cyclictest0-21swapper/708:18:377
3826531991716,0cyclictest0-21swapper/708:15:177
3826531991716,0cyclictest0-21swapper/707:40:487
3826531991716,0cyclictest0-21swapper/704:04:017
3826531991716,0cyclictest0-21swapper/703:47:027
3826531991712,4cyclictest4164054-21latency_hist06:18:137
3826518991715,2cyclictest3961108-21kworker/u16:2+flush-8:004:38:475
3826518991715,2cyclictest3880579-21kworker/u16:4+flush-8:004:03:295
3826517991715,2cyclictest60181-21kworker/u16:4+flush-8:007:28:364
3826517991715,2cyclictest3941324-21kworker/u16:5+flush-8:004:18:394
3826517991714,2cyclictest3850375-21kworker/u16:2+flush-8:003:33:454
3826531991613,2cyclictest4175659-21snmpd06:29:267
3826518991614,2cyclictest41456-21kworker/u16:2+flush-8:008:12:145
3826518991614,2cyclictest4072647-21kworker/u16:0+flush-8:006:13:425
3826518991614,2cyclictest3992715-21kworker/u16:4+flush-8:005:02:095
3826518991614,2cyclictest3961108-21kworker/u16:2+flush-8:005:28:055
3826518991614,2cyclictest3930870-21kworker/u16:0+flush-8:004:33:335
3826518991614,2cyclictest232435-21kworker/u16:1+flush-8:008:29:445
3826518991614,2cyclictest162030-21kworker/u16:1+flush-8:008:03:385
3826518991614,2cyclictest151616-21kworker/u16:3+events_unbound08:23:345
3826518991613,2cyclictest3128588-21kworker/u16:3+events_unbound03:28:385
3826518991612,3cyclictest3961108-21kworker/u16:2+flush-8:004:53:135
3826517991614,2cyclictest4093455-21kworker/u16:1+flush-8:005:59:234
3826517991614,2cyclictest4093455-21kworker/u16:1+events_unbound05:55:014
3826517991614,2cyclictest151616-21kworker/u16:3+events_unbound08:16:134
3826517991613,2cyclictest41456-21kworker/u16:2+flush-8:007:13:404
3826517991613,2cyclictest3940214-21kworker/u16:1+flush-8:004:43:254
382653199159,5cyclictest589235-21systemd-journal06:13:137
3826531991512,2cyclictest4175659-21snmpd07:28:067
3826518991513,2cyclictest60181-21kworker/u16:4+events_unbound07:28:375
3826518991513,2cyclictest4072647-21kworker/u16:0+flush-8:005:34:565
3826518991513,2cyclictest4014935-21kworker/u16:3+flush-8:005:43:285
3826518991513,2cyclictest3128588-21kworker/u16:3+flush-8:003:27:045
3826518991512,2cyclictest3941324-21kworker/u16:5+flush-8:004:18:375
3826517991513,2cyclictest41456-21kworker/u16:2+flush-8:007:43:444
3826517991513,2cyclictest3128588-21kworker/u16:3+flush-8:003:39:004
3826517991513,2cyclictest151616-21kworker/u16:3+flush-8:008:23:364
3826517991512,2cyclictest4174237-21kworker/u16:3+flush-8:006:24:084
3826517991512,2cyclictest3992715-21kworker/u16:4+flush-8:006:45:404
3826517991512,2cyclictest3992715-21kworker/u16:4+events_unbound04:53:234
3826517991512,2cyclictest162030-21kworker/u16:1+flush-8:007:58:304
382653199143,9cyclictest948-21NetworkManager07:58:287
3826531991414,0cyclictest0-21swapper/707:18:037
3826531991414,0cyclictest0-21swapper/703:23:437
3826531991413,0cyclictest0-21swapper/707:43:327
3826518991412,2cyclictest41456-21kworker/u16:2+events_unbound08:53:095
3826518991412,2cyclictest3940214-21kworker/u16:1+flush-8:004:43:245
382651899140,13cyclictest0-21swapper/503:48:285
3826517991412,2cyclictest4014935-21kworker/u16:3+events_unbound05:42:344
3826517991412,2cyclictest3992715-21kworker/u16:4+events_unbound05:07:244
3826517991411,2cyclictest3128588-21kworker/u16:3+flush-8:003:51:134
382651799140,13cyclictest0-21swapper/403:23:384
382653199137,5cyclictest3971419-21latency_hist04:33:137
382653199137,5cyclictest272612-21latency_hist08:48:137
3826518991311,2cyclictest4174237-21kworker/u16:3+flush-8:006:43:295
3826518991311,2cyclictest3992715-21kworker/u16:4+flush-8:005:28:275
3826518991311,2cyclictest3128588-21kworker/u16:3+flush-8:003:33:395
3826517991312,1cyclictest0-21swapper/407:18:374
3826517991311,2cyclictest4093455-21kworker/u16:1+events_unbound06:58:284
382651799130,13cyclictest0-21swapper/404:13:424
382653199129,2cyclictest4175659-21snmpd05:21:547
382653199125,6cyclictest27642-21gmain05:39:037
3826531991212,0cyclictest0-21swapper/705:13:067
382651899129,2cyclictest3961108-21kworker/u16:2+flush-8:005:18:425
382651899120,12cyclictest0-21swapper/507:53:275
3826517991210,2cyclictest30840-21kworker/u16:0+events_unbound07:03:354
3826517991210,2cyclictest151694-21kworker/u16:4+flush-8:008:18:364
382653199117,3cyclictest4175659-21snmpd05:54:027
382653199110,10cyclictest0-21swapper/705:33:357
3826518991110,1cyclictest0-21swapper/508:20:435
3826518991110,1cyclictest0-21swapper/505:06:305
382651899110,1cyclictest0-21swapper/503:42:095
382651899110,11cyclictest0-21swapper/506:39:215
382651799118,2cyclictest4014935-21kworker/u16:3+events_unbound05:33:294
382651799110,0cyclictest0-21swapper/408:45:004
382651899108,2cyclictest232069-21kworker/u16:0+events_unbound08:33:385
382651899100,9cyclictest0-21swapper/504:28:325
382651799108,2cyclictest4133695-21kworker/u16:2+flush-8:006:13:134
382651799108,2cyclictest30840-21kworker/u16:0+events_unbound06:51:514
38265189998,1cyclictest0-21swapper/507:37:505
38265189997,2cyclictest60181-21kworker/u16:4+flush-8:007:05:035
38265189994,4cyclictest3902090-21if_err_p4p103:58:285
38265189986,1cyclictest0-21swapper/506:08:245
38265179983,4cyclictest196201-21grep08:08:314
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional