You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-07 - 20:08
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Fri Nov 07, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4079132990,1sleep7407615-21latency_hist20:54:307
3931659995449,4cyclictest4177513-21latency_hist17:24:317
393114625444,7sleep60-21swapper/615:22:106
393104525444,7sleep50-21swapper/515:20:485
3931656995351,2cyclictest74567-21kworker/u16:2+flush-8:018:39:306
393165999510,49cyclictest367216-21latency_hist20:34:317
393165999490,48cyclictest125242-21latency_hist18:34:307
3931659994742,4cyclictest4066545-21latency_hist16:29:317
393114424737,7sleep40-21swapper/415:22:084
3931648994240,2cyclictest54598-21kworker/u16:0+events_unbound18:19:314
3931656994138,2cyclictest74567-21kworker/u16:2+flush-8:018:24:416
3931648994139,2cyclictest3941307-21kworker/u16:3+events_unbound15:45:004
3931659994034,5cyclictest24046-21latency_hist17:44:307
3931651993937,2cyclictest145228-21kworker/u16:1+flush-8:018:44:565
39216612393,30sleep70-21swapper/715:19:387
3931659993833,4cyclictest246138-21latency_hist19:34:317
3931651993835,2cyclictest3985618-21kworker/u16:4+flush-8:016:04:575
3931656993735,2cyclictest3941307-21kworker/u16:3+flush-8:015:34:316
3931659993631,4cyclictest589235-21systemd-journal20:09:307
3931659993630,5cyclictest4137144-21latency_hist17:04:317
3931648993634,2cyclictest4096514-21kworker/u16:0+events_unbound16:44:174
3931648993431,2cyclictest4096686-21kworker/u16:3+flush-8:017:54:374
3931648993431,2cyclictest4096686-21kworker/u16:3+flush-8:017:54:374
3931659993226,5cyclictest286468-21latency_hist19:54:317
3931656993230,2cyclictest185410-21kworker/u16:4+events_unbound19:09:316
3931651993230,2cyclictest306421-21kworker/u16:0+flush-8:020:04:555
3931659993129,1cyclictest0-21swapper/719:54:567
3931651993129,2cyclictest4106533-21kworker/u16:5+events_unbound17:09:595
3931648993128,2cyclictest266193-21kworker/u16:1+flush-8:020:24:054
3931656993027,2cyclictest236447-21kworker/u16:3+flush-8:019:59:496
3931651993028,2cyclictest54598-21kworker/u16:0+flush-8:018:31:075
3931651993028,2cyclictest54598-21kworker/u16:0+events_unbound17:59:455
3931648993028,2cyclictest3941307-21kworker/u16:3+flush-8:016:20:004
3931656992926,2cyclictest3985618-21kworker/u16:4+flush-8:015:59:386
3931651992927,2cyclictest3845173-21kworker/u16:0+flush-8:016:09:565
3931651992927,2cyclictest316367-21kworker/u16:2+flush-8:020:41:095
3931651992927,2cyclictest316367-21kworker/u16:2+events_unbound20:44:595
3931651992927,2cyclictest185410-21kworker/u16:4+flush-8:019:09:575
3931648992927,2cyclictest54598-21kworker/u16:0+flush-8:018:04:494
3931648992927,2cyclictest34399-21kworker/u16:2+flush-8:017:51:414
3931659992827,0cyclictest0-21swapper/719:15:057
3931656992825,2cyclictest145228-21kworker/u16:1+events_unbound18:44:466
3931651992826,2cyclictest4056153-21kworker/u16:2+events_unbound16:34:585
3931651992826,2cyclictest236447-21kworker/u16:3+flush-8:019:50:015
3931651992826,2cyclictest225871-21kworker/u16:0+flush-8:019:44:565
3931648992826,2cyclictest4096686-21kworker/u16:3+events_unbound17:39:314
3931648992825,2cyclictest3845173-21kworker/u16:0+flush-8:015:33:294
3931659992726,0cyclictest0-21swapper/719:04:467
3931659992726,0cyclictest0-21swapper/718:59:497
3931656992725,2cyclictest54598-21kworker/u16:0+flush-8:018:09:566
3931651992725,2cyclictest145228-21kworker/u16:1+flush-8:019:04:565
3931651992724,3cyclictest4167385-21kworker/u16:1+flush-8:018:34:575
3931648992725,2cyclictest54598-21kworker/u16:0+flush-8:018:39:554
3931648992725,2cyclictest54598-21kworker/u16:0+flush-8:018:39:554
3931648992725,2cyclictest3845173-21kworker/u16:0+events_unbound16:14:524
3931648992724,2cyclictest54598-21kworker/u16:0+events_unbound18:49:334
3931659992625,0cyclictest0-21swapper/717:24:417
3931659992624,1cyclictest0-21swapper/715:59:437
3931656992624,2cyclictest4147631-21kworker/u16:2+events_unbound17:19:496
3931656992623,3cyclictest3845173-21kworker/u16:0+flush-8:016:24:306
3931656992622,3cyclictest4096514-21kworker/u16:0+events_unbound17:09:476
3931651992624,2cyclictest3985618-21kworker/u16:4+flush-8:015:49:465
3931651992624,2cyclictest377013-21kworker/u16:0+events_unbound20:49:565
3931651992624,2cyclictest266193-21kworker/u16:1+flush-8:020:14:565
3931651992624,2cyclictest225871-21kworker/u16:0+flush-8:019:24:505
3931648992624,2cyclictest4096514-21kworker/u16:0+events_unbound17:34:304
3931648992624,2cyclictest4056153-21kworker/u16:2+events_unbound16:39:164
3931648992624,2cyclictest236447-21kworker/u16:3+flush-8:019:41:294
3931648992624,2cyclictest225871-21kworker/u16:0+flush-8:019:54:514
3931659992525,0cyclictest0-21swapper/719:44:597
3931659992524,0cyclictest0-21swapper/720:34:497
3931659992524,0cyclictest0-21swapper/719:24:487
3931659992523,1cyclictest0-21swapper/716:09:527
3931656992522,2cyclictest4096686-21kworker/u16:3+events_unbound16:59:456
3931651992523,2cyclictest4167385-21kworker/u16:1+flush-8:018:19:545
3931651992523,2cyclictest3845173-21kworker/u16:0+flush-8:016:29:545
3931648992523,2cyclictest4147631-21kworker/u16:2+events_unbound17:40:014
3931648992523,2cyclictest266193-21kworker/u16:1+events_unbound19:51:464
3931648992522,2cyclictest3941307-21kworker/u16:3+flush-8:015:44:304
3931656992422,2cyclictest377013-21kworker/u16:0+events_unbound20:49:446
3931651992422,2cyclictest4096686-21kworker/u16:3+events_unbound17:08:255
3931651992422,2cyclictest225871-21kworker/u16:0+events_unbound19:54:445
3931656992321,2cyclictest4096686-21kworker/u16:3+events_unbound17:14:586
3931651992321,2cyclictest4096686-21kworker/u16:3+flush-8:017:54:025
3931651992321,2cyclictest34399-21kworker/u16:2+flush-8:017:49:305
3931651992321,2cyclictest3221334-21kworker/u16:4+flush-8:015:31:325
3931651992320,2cyclictest54598-21kworker/u16:0+events_unbound18:39:335
3931651992320,2cyclictest54598-21kworker/u16:0+events_unbound18:39:335
3931648992321,2cyclictest3985618-21kworker/u16:4+flush-8:015:54:514
3931656992219,2cyclictest4147631-21kworker/u16:2+flush-8:017:29:576
3931651992220,2cyclictest4106533-21kworker/u16:5+flush-8:017:18:125
3931651992220,2cyclictest266193-21kworker/u16:1+events_unbound20:10:005
3931651992219,2cyclictest145228-21kworker/u16:1+flush-8:019:34:425
3931648992220,2cyclictest3845173-21kworker/u16:0+events_unbound16:11:354
3931648992219,2cyclictest185410-21kworker/u16:4+flush-8:019:14:594
3931648992219,2cyclictest145228-21kworker/u16:1+events_unbound19:29:334
3931659992116,4cyclictest589235-21systemd-journal18:37:587
3931656992119,2cyclictest4096514-21kworker/u16:0+flush-8:016:44:496
3931656992118,2cyclictest377230-21kworker/u16:3+flush-8:020:45:336
3931656992118,2cyclictest266193-21kworker/u16:1+events_unbound20:29:446
3931651992119,2cyclictest4147631-21kworker/u16:2+events_unbound17:30:125
3931651992119,2cyclictest4096686-21kworker/u16:3+flush-8:017:55:555
3931651992119,2cyclictest4096686-21kworker/u16:3+flush-8:017:55:555
3931651992119,2cyclictest3985618-21kworker/u16:4+flush-8:015:54:455
3931651992119,2cyclictest3845173-21kworker/u16:0+events_unbound16:16:575
3931651992119,2cyclictest185409-21kworker/u16:3+events_unbound19:01:175
3931648992119,2cyclictest54598-21kworker/u16:0+flush-8:018:30:004
3931648992119,2cyclictest4096686-21kworker/u16:3+flush-8:017:09:264
3931648992119,2cyclictest316367-21kworker/u16:2+events_unbound20:25:584
3931648992119,2cyclictest276756-21kworker/u16:4+events_unbound20:34:554
3931648992118,3cyclictest54598-21kworker/u16:0+flush-8:018:56:294
3931648992118,3cyclictest377230-21kworker/u16:3+flush-8:020:49:454
3931656992017,2cyclictest4096514-21kworker/u16:0+flush-8:016:54:596
3931651992018,2cyclictest4147631-21kworker/u16:2+events_unbound17:39:475
3931651992018,2cyclictest4096514-21kworker/u16:0+events_unbound17:02:105
3931651992018,2cyclictest4056153-21kworker/u16:2+flush-8:016:25:485
3931651992018,2cyclictest4056153-21kworker/u16:2+events_unbound16:48:005
3931651992018,2cyclictest4036034-21kworker/u16:1+events_unbound16:20:445
3931651992018,2cyclictest266193-21kworker/u16:1+events_unbound20:35:095
3931648992018,2cyclictest4006360-21kworker/u16:1+events_unbound16:02:594
3931659991914,4cyclictest4006146-21latency_hist15:59:317
3931659991913,5cyclictest387391-21latency_hist20:44:307
3931656991917,2cyclictest54598-21kworker/u16:0+events_unbound19:00:006
3931656991917,2cyclictest4147631-21kworker/u16:2+events_unbound17:24:446
3931656991915,3cyclictest4147631-21kworker/u16:2+flush-8:017:40:006
3931651991917,2cyclictest4167385-21kworker/u16:1+events_unbound18:14:445
3931651991917,2cyclictest4147631-21kworker/u16:2+events_unbound17:29:005
3931651991917,2cyclictest3941307-21kworker/u16:3+events_unbound15:44:175
3931651991917,2cyclictest185409-21kworker/u16:3+flush-8:019:18:005
3931651991917,2cyclictest145228-21kworker/u16:1+flush-8:018:55:085
3931651991917,2cyclictest145228-21kworker/u16:1+events_unbound19:33:595
3931651991916,2cyclictest3221334-21kworker/u16:4+events_unbound15:34:445
3931648991917,2cyclictest74567-21kworker/u16:2+flush-8:018:24:524
3931648991917,2cyclictest4056153-21kworker/u16:2+events_unbound16:31:014
3931648991917,2cyclictest3845173-21kworker/u16:0+events_unbound15:39:064
3931648991917,2cyclictest266193-21kworker/u16:1+events_unbound20:09:464
3931659991817,0cyclictest0-21swapper/717:59:497
3931659991817,0cyclictest0-21swapper/717:54:557
3931659991817,0cyclictest0-21swapper/717:54:557
3931659991817,0cyclictest0-21swapper/716:04:527
3931656991816,2cyclictest54598-21kworker/u16:0+events_unbound18:50:006
3931656991816,2cyclictest4167385-21kworker/u16:1+flush-8:018:30:546
3931656991816,2cyclictest335971-21kworker/u16:0+events_unbound20:19:566
3931651991816,2cyclictest3845173-21kworker/u16:0+events_unbound16:01:055
3931651991816,2cyclictest316367-21kworker/u16:2+events_unbound20:25:465
3931648991815,2cyclictest3985618-21kworker/u16:4+flush-8:016:06:304
3931648991814,3cyclictest14377-21kworker/u16:0+flush-8:017:47:114
3931659991716,0cyclictest0-21swapper/718:04:457
3931659991711,5cyclictest94960-21latency_hist18:19:317
3931656991715,2cyclictest54598-21kworker/u16:0+events_unbound18:04:466
3931656991715,2cyclictest3941307-21kworker/u16:3+flush-8:015:57:046
3931656991715,2cyclictest3845173-21kworker/u16:0+flush-8:016:14:566
3931656991715,2cyclictest225871-21kworker/u16:0+flush-8:019:39:566
3931656991714,3cyclictest266193-21kworker/u16:1+flush-8:020:34:446
3931656991713,3cyclictest4056153-21kworker/u16:2+flush-8:016:39:466
3931651991715,2cyclictest74567-21kworker/u16:2+flush-8:018:25:045
3931651991715,2cyclictest4147631-21kworker/u16:2+flush-8:017:24:255
3931651991715,2cyclictest4106533-21kworker/u16:5+events_unbound16:57:545
3931651991715,2cyclictest4056153-21kworker/u16:2+events_unbound16:39:555
3931651991715,2cyclictest3941307-21kworker/u16:3+events_unbound15:28:395
3931651991715,2cyclictest266193-21kworker/u16:1+flush-8:020:31:235
3931651991715,2cyclictest236447-21kworker/u16:3+flush-8:020:01:195
3931651991715,2cyclictest145228-21kworker/u16:1+events_unbound18:54:285
3931648991715,2cyclictest4106533-21kworker/u16:5+events_unbound17:04:254
3931648991715,2cyclictest316367-21kworker/u16:2+events_unbound20:29:564
3931659991613,2cyclictest4175659-21snmpd18:50:337
3931659991613,2cyclictest4175659-21snmpd17:19:077
3931659991611,4cyclictest44565-21latency_hist17:54:317
3931656991614,2cyclictest54606-21kworker/u16:4+flush-8:017:59:576
3931656991614,2cyclictest3221334-21kworker/u16:4+events_unbound15:34:546
3931656991613,3cyclictest316367-21kworker/u16:2+flush-8:020:14:556
393165699160,2cyclictest218607-21grep19:19:496
3931651991614,2cyclictest4167385-21kworker/u16:1+flush-8:017:36:245
3931651991614,2cyclictest4106533-21kworker/u16:5+events_unbound16:49:475
3931651991613,2cyclictest3845173-21kworker/u16:0+events_unbound15:44:425
3931648991614,2cyclictest185409-21kworker/u16:3+events_unbound19:06:434
3931659991514,0cyclictest0-21swapper/716:35:007
3931659991514,0cyclictest0-21swapper/716:30:017
3931659991512,2cyclictest0-21swapper/715:54:317
393165999150,14cyclictest0-21swapper/719:09:577
393165999150,13cyclictest4175659-21snmpd18:20:457
3931656991513,2cyclictest54598-21kworker/u16:0+flush-8:018:39:596
3931656991513,2cyclictest54598-21kworker/u16:0+flush-8:018:39:596
3931651991513,2cyclictest54598-21kworker/u16:0+events_unbound18:14:135
3931651991513,2cyclictest4167385-21kworker/u16:1+flush-8:018:04:535
3931651991513,2cyclictest266193-21kworker/u16:1+flush-8:020:24:085
3931648991513,2cyclictest4096686-21kworker/u16:3+events_unbound17:09:514
3931648991513,2cyclictest3985618-21kworker/u16:4+events_unbound15:54:094
3931648991512,2cyclictest54598-21kworker/u16:0+flush-8:018:12:114
3931648991512,2cyclictest4106533-21kworker/u16:5+flush-8:016:49:454
393164899150,14cyclictest0-21swapper/417:19:514
393165999147,6cyclictest0-21swapper/717:39:297
393165999147,6cyclictest0-21swapper/716:17:597
393165999141,3cyclictest256223-21munin-run19:39:307
3931659991413,1cyclictest0-21swapper/718:24:577
393165999140,1cyclictest0-21swapper/718:54:557
393165999140,1cyclictest0-21swapper/716:44:547
3931656991412,2cyclictest4167385-21kworker/u16:1+events_unbound17:44:446
3931656991410,3cyclictest21492-21rs:main6
3931651991412,2cyclictest225871-21kworker/u16:0+flush-8:019:39:515
3931648991412,2cyclictest54598-21kworker/u16:0+events_unbound19:00:594
3931648991411,2cyclictest266193-21kworker/u16:1+flush-8:020:39:514
393165999139,3cyclictest4175659-21snmpd20:28:217
393165999138,4cyclictest223574-21cut19:19:527
393165999137,4cyclictest4095808-21ntp_states16:39:587
393165999136,6cyclictest27763-21gvfs-afc-volume18:44:507
3931659991312,1cyclictest0-21swapper/720:44:527
3931659991310,2cyclictest0-21swapper/715:44:597
393165999130,12cyclictest0-21swapper/719:59:587
3931656991311,2cyclictest3985618-21kworker/u16:4+events_unbound16:04:586
3931656991311,2cyclictest236447-21kworker/u16:3+flush-8:020:09:476
3931651991311,2cyclictest185410-21kworker/u16:4+events_unbound19:20:085
3931648991311,2cyclictest4096514-21kworker/u16:0+flush-8:016:54:574
3931648991311,2cyclictest4096514-21kworker/u16:0+events_unbound17:14:544
393164899130,6cyclictest0-21swapper/417:24:444
393164899130,1cyclictest0-21swapper/420:44:574
393165999129,2cyclictest4175659-21snmpd18:44:277
393165999129,2cyclictest4175659-21snmpd18:44:277
393165999128,3cyclictest4175659-21snmpd20:22:217
3931659991211,0cyclictest0-21swapper/716:54:567
393165999120,11cyclictest4175659-21snmpd17:06:557
393165999120,11cyclictest4175659-21snmpd15:26:207
393165699129,3cyclictest3985618-21kworker/u16:4+flush-8:016:10:016
3931656991211,1cyclictest0-21swapper/620:05:416
3931656991211,1cyclictest0-21swapper/617:54:506
3931656991211,1cyclictest0-21swapper/617:54:506
3931656991210,2cyclictest4056153-21kworker/u16:2+events_unbound16:29:456
393165699120,1cyclictest0-21swapper/615:28:426
393164899129,2cyclictest335971-21kworker/u16:0+flush-8:020:16:334
3931648991210,2cyclictest4167385-21kworker/u16:1+flush-8:018:34:464
3931648991210,2cyclictest4096686-21kworker/u16:3+events_unbound18:00:594
3931648991210,2cyclictest4066328-21kworker/u16:3+events_unbound16:29:264
3931648991210,2cyclictest145228-21kworker/u16:1+flush-8:019:25:004
393165999118,2cyclictest4175659-21snmpd17:30:267
393165999117,3cyclictest4175659-21snmpd17:47:477
393165999116,4cyclictest84921-21latency_hist18:14:317
3931659991110,1cyclictest0-21swapper/715:29:567
393165999110,9cyclictest331255-21cut20:14:507
393165999110,4cyclictest326614-21sshd-session20:11:467
393165999110,10cyclictest96-21ksoftirqd/719:40:007
393165999110,10cyclictest4175659-21snmpd16:51:017
393165699119,2cyclictest4167385-21kworker/u16:1+flush-8:018:19:596
393165699119,2cyclictest4096686-21kworker/u16:3+flush-8:017:52:046
393165699110,1cyclictest0-21swapper/618:15:206
393165699110,0cyclictest0-21swapper/619:48:406
393165699110,0cyclictest0-21swapper/617:09:176
393164899119,2cyclictest185409-21kworker/u16:3+events_unbound19:14:264
3931648991110,1cyclictest0-21swapper/420:08:134
3931648991110,1cyclictest0-21swapper/419:20:374
3931648991110,1cyclictest0-21swapper/416:44:424
393164899110,11cyclictest0-21swapper/418:48:084
393165999107,2cyclictest4175659-21snmpd16:20:547
393165999107,2cyclictest4175659-21snmpd15:43:477
393165999101,2cyclictest0-21swapper/717:09:497
393165999100,5cyclictest4175659-21snmpd15:35:137
393165699109,1cyclictest0-21swapper/620:42:256
393165699109,1cyclictest0-21swapper/620:29:206
393165699109,1cyclictest0-21swapper/616:37:366
393165699109,1cyclictest0-21swapper/616:29:146
393165699109,1cyclictest0-21swapper/615:52:216
393165699109,1cyclictest0-21swapper/615:45:346
393165699109,1cyclictest0-21swapper/615:41:196
393165699108,2cyclictest4147631-21kworker/u16:2+events_unbound17:34:466
393164899109,1cyclictest0-21swapper/419:47:064
39316569998,1cyclictest0-21swapper/619:39:036
39316569998,1cyclictest0-21swapper/619:29:266
39316569997,1cyclictest174098-21kthreadcore18:54:526
39316569996,2cyclictest236447-21kworker/u16:3+flush-8:019:49:516
39316489998,1cyclictest0-21swapper/418:19:554
39316489998,1cyclictest0-21swapper/415:24:464
39316489996,2cyclictest255183-21needreboot19:34:534
39316569983,4cyclictest290861-21grep19:54:486
39316569983,4cyclictest197899-21cut19:09:506
39316569973,3cyclictest236616-21expr19:29:446
39316569973,3cyclictest206248-21grep19:14:446
39316489973,3cyclictest297011-21cat19:59:454
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional