You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 04:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Mon Nov 17, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
431148995651,4cyclictest505672-21latency_hist03:58:537
43073925320,29sleep70-21swapper/703:22:577
43065725141,7sleep40-21swapper/403:21:504
43058325140,7sleep50-21swapper/503:20:475
431148994843,4cyclictest829415-21latency_hist06:38:537
431148994843,4cyclictest697911-21latency_hist05:33:537
431128994744,2cyclictest848619-21kworker/u16:0+events_unbound07:00:434
431148994645,0cyclictest0-21swapper/707:56:357
431128994543,2cyclictest667139-21kworker/u16:2+events_unbound05:22:434
431148994443,0cyclictest0-21swapper/708:32:117
431140994442,2cyclictest3185608-21kworker/u16:5+events_unbound05:53:435
431128994442,2cyclictest951029-21kworker/u16:0+flush-8:007:49:194
431140994341,2cyclictest1021902-21kworker/u16:5+events_unbound08:20:115
431148994241,0cyclictest0-21swapper/704:25:547
431140994240,2cyclictest526077-21kworker/u16:2+events_unbound04:52:435
431140994240,2cyclictest526077-21kworker/u16:2+events_unbound04:52:435
431128994240,2cyclictest698298-21kworker/u16:4+events_unbound05:46:434
431128994240,2cyclictest454192-21kworker/u16:0+events_unbound03:42:414
431140994139,2cyclictest556134-21kworker/u16:0+events_unbound04:31:445
431128994139,2cyclictest768540-21kworker/u16:2+events_unbound06:08:444
431148994035,4cyclictest859710-21latency_hist06:53:527
431140994038,2cyclictest3185608-21kworker/u16:5+events_unbound04:04:245
431140994038,2cyclictest3185608-21kworker/u16:5+events_unbound04:04:245
431140994038,2cyclictest1041714-21kworker/u16:1+events_unbound08:28:555
431140993937,2cyclictest920032-21kworker/u16:3+events_unbound08:11:435
43078423929,7sleep60-21swapper/603:23:366
43114899380,36cyclictest799082-21latency_hist06:23:537
43114899380,36cyclictest799082-21latency_hist06:23:537
431128993836,2cyclictest566352-21kworker/u16:1+events_unbound04:28:094
431148993736,0cyclictest0-21swapper/703:46:417
431140993735,2cyclictest889998-21kworker/u16:0+flush-8:007:21:285
431140993735,2cyclictest779255-21kworker/u16:1+events_unbound06:16:155
431140993735,2cyclictest505541-21kworker/u16:0+events_unbound04:02:535
431148993635,0cyclictest0-21swapper/704:13:557
431140993634,2cyclictest874587-21kworker/u16:4+flush-8:007:30:435
431140993634,2cyclictest505541-21kworker/u16:0+events_unbound03:57:445
431140993634,2cyclictest1051976-21kworker/u16:2+events_unbound08:35:125
431128993634,2cyclictest3185608-21kworker/u16:5+events_unbound04:43:304
431140993533,2cyclictest874587-21kworker/u16:4+events_unbound07:08:425
431140993533,2cyclictest874587-21kworker/u16:4+events_unbound07:08:425
431140993533,2cyclictest454192-21kworker/u16:0+events_unbound03:37:325
431140993533,2cyclictest391527-21kworker/u16:1+events_unbound04:13:005
431128993533,2cyclictest920032-21kworker/u16:3+events_unbound08:01:164
431128993533,2cyclictest3185608-21kworker/u16:5+events_unbound04:04:194
431128993533,2cyclictest3185608-21kworker/u16:5+events_unbound04:04:194
43114899340,32cyclictest589235-21systemd-journal05:23:537
431143993431,2cyclictest1092401-21kworker/u16:2+events_unbound08:49:116
431140993432,2cyclictest526077-21kworker/u16:2+events_unbound05:03:435
431140993432,2cyclictest3185608-21kworker/u16:5+events_unbound04:56:025
431128993432,2cyclictest1021902-21kworker/u16:5+events_unbound08:39:154
431128993431,2cyclictest934644-21kworker/u16:2+flush-8:007:58:524
431128993331,2cyclictest737654-21kworker/u16:2+events_unbound05:50:434
431140993230,2cyclictest556134-21kworker/u16:0+flush-8:004:34:435
431140993230,2cyclictest3185608-21kworker/u16:5+events_unbound03:48:415
431148993130,0cyclictest0-21swapper/708:13:527
431140993129,2cyclictest934644-21kworker/u16:2+events_unbound07:51:345
431140993129,2cyclictest627007-21kworker/u16:0+events_unbound05:30:555
431140993129,2cyclictest526077-21kworker/u16:2+events_unbound04:18:315
431128993129,2cyclictest627007-21kworker/u16:0+events_unbound05:24:284
431128993129,2cyclictest495225-21kworker/u16:3+flush-8:004:13:404
431128993129,2cyclictest374548-21kworker/u16:2+events_unbound03:52:044
431140993028,2cyclictest779255-21kworker/u16:1+flush-8:006:23:525
431140993028,2cyclictest779255-21kworker/u16:1+flush-8:006:23:525
431128993028,2cyclictest990949-21kworker/u16:0+events_unbound08:43:544
431128993028,2cyclictest627007-21kworker/u16:0+flush-8:005:29:594
431128993028,2cyclictest627007-21kworker/u16:0+events_unbound05:06:434
431128993028,2cyclictest3185608-21kworker/u16:5+events_unbound04:56:434
431140992927,2cyclictest3185608-21kworker/u16:5+events_unbound05:16:335
431128992927,2cyclictest667139-21kworker/u16:2+events_unbound05:38:444
431128992927,2cyclictest667139-21kworker/u16:2+events_unbound05:38:444
431128992927,2cyclictest1021902-21kworker/u16:5+events_unbound08:14:264
431148992827,0cyclictest0-21swapper/704:04:427
431148992827,0cyclictest0-21swapper/704:04:427
431140992826,2cyclictest920032-21kworker/u16:3+events_unbound08:16:355
431140992826,2cyclictest859467-21kworker/u16:1+events_unbound06:51:035
431128992826,2cyclictest667139-21kworker/u16:2+events_unbound05:14:484
431148992725,1cyclictest0-21swapper/703:59:207
431143992723,3cyclictest768540-21kworker/u16:2+flush-8:006:19:126
431143992723,3cyclictest768540-21kworker/u16:2+flush-8:006:19:126
431140992725,2cyclictest990949-21kworker/u16:0+events_unbound08:46:115
431140992725,2cyclictest779255-21kworker/u16:1+flush-8:006:26:025
431140992725,2cyclictest758386-21kworker/u16:3+events_unbound06:55:425
431140992725,2cyclictest758386-21kworker/u16:3+events_unbound06:38:425
431140992725,2cyclictest454192-21kworker/u16:0+flush-8:003:42:435
431140992725,2cyclictest3185608-21kworker/u16:5+flush-8:005:23:455
431128992725,2cyclictest758386-21kworker/u16:3+events_unbound06:28:424
431148992625,0cyclictest0-21swapper/705:23:437
431148992625,0cyclictest0-21swapper/704:41:447
431148992624,1cyclictest976-21in:imjournal04:33:437
431140992624,2cyclictest990949-21kworker/u16:0+events_unbound07:58:425
431140992624,2cyclictest748240-21kworker/u16:1+events_unbound05:58:445
431140992624,2cyclictest1092401-21kworker/u16:2+flush-8:008:49:035
431140992624,2cyclictest1021902-21kworker/u16:5+events_unbound08:42:095
431128992624,2cyclictest899987-21kworker/u16:1+events_unbound07:23:434
431128992624,2cyclictest859467-21kworker/u16:1+events_unbound06:55:444
431128992624,2cyclictest606582-21kworker/u16:3+flush-8:004:50:484
431128992624,2cyclictest606582-21kworker/u16:3+flush-8:004:50:484
431128992624,2cyclictest556134-21kworker/u16:0+flush-8:004:36:324
431128992624,2cyclictest3185608-21kworker/u16:5+events_unbound06:19:274
431128992624,2cyclictest3185608-21kworker/u16:5+events_unbound06:19:274
431128992624,2cyclictest1031308-21kworker/u16:6+events_unbound08:21:104
431128992624,2cyclictest1021902-21kworker/u16:5+events_unbound08:31:104
431128992623,2cyclictest758386-21kworker/u16:3+flush-8:006:17:444
431143992523,2cyclictest454192-21kworker/u16:0+events_unbound03:34:096
431140992523,2cyclictest688172-21kworker/u16:3+events_unbound05:44:365
431140992523,2cyclictest3185608-21kworker/u16:5+events_unbound06:47:045
431140992523,2cyclictest3185608-21kworker/u16:5+events_unbound05:10:435
431140992522,2cyclictest3185608-21kworker/u16:5+flush-8:006:03:545
431140992522,2cyclictest3185608-21kworker/u16:5+flush-8:004:18:555
431128992523,2cyclictest810816-21kworker/u16:2+flush-8:007:16:024
431128992523,2cyclictest606582-21kworker/u16:3+events_unbound04:47:424
431128992522,2cyclictest990949-21kworker/u16:0+flush-8:008:33:554
431128992522,2cyclictest810816-21kworker/u16:2+events_unbound06:48:564
431148992423,0cyclictest0-21swapper/705:44:427
43114899240,23cyclictest738443-21latency_hist05:53:537
431140992422,2cyclictest810816-21kworker/u16:2+events_unbound06:32:525
431140992422,2cyclictest688172-21kworker/u16:3+events_unbound05:40:045
431140992422,2cyclictest1041714-21kworker/u16:1+events_unbound08:24:515
431140992421,2cyclictest899987-21kworker/u16:1+flush-8:007:39:205
431128992422,2cyclictest920032-21kworker/u16:3+events_unbound07:34:374
431128992422,2cyclictest526077-21kworker/u16:2+events_unbound04:14:114
431148992322,0cyclictest0-21swapper/708:18:037
431148992322,0cyclictest0-21swapper/707:50:417
431148992322,0cyclictest0-21swapper/707:09:437
431148992322,0cyclictest0-21swapper/704:10:517
43114899230,22cyclictest495522-21latency_hist03:53:537
431143992320,2cyclictest454192-21kworker/u16:0+events_unbound03:44:116
431140992321,2cyclictest951029-21kworker/u16:0+flush-8:007:44:365
431140992321,2cyclictest889998-21kworker/u16:0+events_unbound07:25:425
431140992321,2cyclictest454192-21kworker/u16:0+events_unbound03:32:045
431140992321,2cyclictest1001606-21kworker/u16:4+events_unbound08:04:185
431128992321,2cyclictest951029-21kworker/u16:0+flush-8:007:41:314
431128992321,2cyclictest374548-21kworker/u16:2+events_unbound03:38:444
431128992320,2cyclictest3185608-21kworker/u16:5+flush-8:005:54:544
431148992221,0cyclictest0-21swapper/707:16:417
431148992221,0cyclictest0-21swapper/706:28:447
431148992221,0cyclictest0-21swapper/705:43:437
431148992221,0cyclictest0-21swapper/703:29:197
431140992220,2cyclictest874587-21kworker/u16:4+events_unbound07:11:285
431140992220,2cyclictest810816-21kworker/u16:2+flush-8:006:43:475
431140992220,2cyclictest758386-21kworker/u16:3+events_unbound06:11:425
431140992220,2cyclictest3185608-21kworker/u16:5+events_unbound04:40:455
431140992220,2cyclictest3185608-21kworker/u16:5+events_unbound03:24:155
431128992220,2cyclictest899987-21kworker/u16:1+flush-8:008:09:094
431128992220,2cyclictest616779-21kworker/u16:1+flush-8:005:09:424
431128992220,2cyclictest495225-21kworker/u16:3+events_unbound03:59:224
431128992220,2cyclictest1001606-21kworker/u16:4+flush-8:008:04:194
431148992120,0cyclictest0-21swapper/706:00:457
431148992120,0cyclictest0-21swapper/705:07:567
431140992119,2cyclictest899987-21kworker/u16:1+events_unbound07:35:155
431140992119,2cyclictest848619-21kworker/u16:0+events_unbound07:02:035
431128992119,2cyclictest990949-21kworker/u16:0+events_unbound08:28:514
431128992119,2cyclictest768540-21kworker/u16:2+events_unbound06:13:374
431128992119,2cyclictest758386-21kworker/u16:3+events_unbound07:04:204
431128992119,2cyclictest758386-21kworker/u16:3+events_unbound07:04:204
431128992119,2cyclictest495225-21kworker/u16:3+events_unbound03:56:334
431143992018,2cyclictest737654-21kworker/u16:2+flush-8:005:51:406
431143992018,2cyclictest616779-21kworker/u16:1+events_unbound05:19:196
431140992018,2cyclictest374548-21kworker/u16:2+events_unbound03:51:565
431128992018,2cyclictest889998-21kworker/u16:0+flush-8:007:26:084
431128992018,2cyclictest374548-21kworker/u16:2+events_unbound03:30:484
431128992017,2cyclictest874587-21kworker/u16:4+flush-8:007:29:124
431148991918,0cyclictest0-21swapper/705:14:227
431148991918,0cyclictest0-21swapper/704:52:027
431148991918,0cyclictest0-21swapper/704:52:027
431148991914,4cyclictest960749-21latency_hist07:43:527
43114899190,18cyclictest0-21swapper/707:34:177
43114899190,18cyclictest0-21swapper/706:09:227
43114899190,17cyclictest1098513-21cut08:49:167
431143991917,2cyclictest874587-21kworker/u16:4+events_unbound07:24:226
431143991917,2cyclictest374548-21kworker/u16:2+events_unbound03:29:086
431143991917,2cyclictest1041714-21kworker/u16:1+events_unbound08:39:216
431143991915,3cyclictest3185608-21kworker/u16:5+flush-8:004:43:536
431140991917,2cyclictest667139-21kworker/u16:2+flush-8:005:28:035
431140991917,2cyclictest627007-21kworker/u16:0+events_unbound05:07:085
431128991917,2cyclictest990949-21kworker/u16:0+events_unbound08:52:284
431128991917,2cyclictest810816-21kworker/u16:2+flush-8:006:43:484
431128991917,2cyclictest810816-21kworker/u16:2+events_unbound06:34:184
431148991816,1cyclictest0-21swapper/706:30:587
431143991815,2cyclictest391527-21kworker/u16:1+flush-8:003:25:046
431140991816,2cyclictest3185608-21kworker/u16:5+events_unbound05:34:245
431140991816,2cyclictest3185608-21kworker/u16:5+events_unbound05:34:245
431140991816,2cyclictest3185608-21kworker/u16:5+events_unbound04:26:455
431128991816,2cyclictest951029-21kworker/u16:0+events_unbound07:47:554
431128991815,2cyclictest566352-21kworker/u16:1+flush-8:004:28:554
431148991716,1cyclictest0-21swapper/708:39:187
431148991716,0cyclictest0-21swapper/707:48:417
431148991714,2cyclictest0-21swapper/705:33:537
431148991714,2cyclictest0-21swapper/705:33:537
431143991715,2cyclictest758386-21kworker/u16:3+flush-8:006:38:496
431143991713,3cyclictest737654-21kworker/u16:2+flush-8:005:59:096
431140991715,2cyclictest737654-21kworker/u16:2+flush-8:005:59:185
431128991715,2cyclictest758386-21kworker/u16:3+flush-8:006:29:084
431128991714,2cyclictest3185608-21kworker/u16:5+flush-8:006:44:114
431148991616,0cyclictest0-21swapper/704:34:187
431148991615,0cyclictest0-21swapper/708:07:047
431148991615,0cyclictest0-21swapper/706:14:567
431148991615,0cyclictest0-21swapper/704:43:557
431148991615,0cyclictest0-21swapper/703:42:047
431143991614,2cyclictest505541-21kworker/u16:0+events_unbound04:04:096
431143991614,2cyclictest505541-21kworker/u16:0+events_unbound04:04:096
431143991613,2cyclictest647046-21kworker/u16:3+flush-8:005:09:076
431143991613,2cyclictest506196-21/usr/sbin/munin03:59:076
431140991614,2cyclictest920032-21kworker/u16:3+events_unbound08:01:555
431128991614,2cyclictest810816-21kworker/u16:2+events_unbound07:12:194
431128991614,2cyclictest616779-21kworker/u16:1+flush-8:004:59:174
431148991515,0cyclictest0-21swapper/707:28:437
431148991514,0cyclictest0-21swapper/706:45:437
431140991513,2cyclictest810816-21kworker/u16:2+events_unbound07:18:035
431128991513,2cyclictest3185608-21kworker/u16:5+events_unbound05:41:204
431148991413,0cyclictest0-21swapper/704:23:127
431148991411,2cyclictest4175659-21snmpd05:03:177
43114899140,1cyclictest0-21swapper/708:19:207
431143991412,2cyclictest566352-21kworker/u16:1+flush-8:004:28:536
43112899140,7cyclictest0-21swapper/404:19:124
43114899138,4cyclictest940535-21latency_hist07:33:537
43114899138,4cyclictest879931-21latency_hist07:03:527
43114899132,3cyclictest27642-21gmain08:24:237
431148991312,0cyclictest0-21swapper/707:19:117
43114899130,11cyclictest431368-21/usr/sbin/munin03:24:237
43114399139,3cyclictest1012568-21cat08:09:106
431143991312,1cyclictest0-21swapper/604:09:226
431128991311,2cyclictest728173-21kworker/u16:0+events_unbound06:03:294
431128991310,2cyclictest374548-21kworker/u16:2+flush-8:003:24:104
43114899129,2cyclictest4175659-21snmpd07:59:577
43114899128,3cyclictest948-21NetworkManager04:57:077
43114899127,4cyclictest4175659-21snmpd08:44:107
431148991212,0cyclictest0-21swapper/707:04:187
431148991212,0cyclictest0-21swapper/707:04:187
43114399129,2cyclictest874587-21kworker/u16:4+flush-8:007:31:126
431143991211,1cyclictest0-21swapper/608:31:416
431143991211,1cyclictest0-21swapper/608:15:556
43112899120,1cyclictest0-21swapper/403:46:174
43114899118,2cyclictest4175659-21snmpd08:34:167
43114899118,2cyclictest4175659-21snmpd06:53:597
43114899118,2cyclictest4175659-21snmpd05:54:067
43114899118,2cyclictest4175659-21snmpd03:35:577
43114899110,2cyclictest829810-21/usr/sbin/munin06:39:217
43114899110,2cyclictest0-21swapper/706:04:197
43114399118,2cyclictest964603-21cut07:44:146
43114399118,2cyclictest3185608-21kworker/u16:5+flush-8:005:54:256
43114399116,4cyclictest608031-21cat04:49:106
43114399116,4cyclictest608031-21cat04:49:106
431143991110,1cyclictest0-21swapper/608:00:096
431143991110,1cyclictest0-21swapper/607:57:296
431143991110,1cyclictest0-21swapper/606:41:486
431143991110,1cyclictest0-21swapper/606:32:496
431143991110,1cyclictest0-21swapper/606:09:486
431143991110,1cyclictest0-21swapper/605:28:116
431143991110,1cyclictest0-21swapper/604:59:016
43114399110,4cyclictest0-21swapper/605:39:106
43114399110,1cyclictest0-21swapper/603:41:186
43114399110,11cyclictest0-21swapper/607:43:176
43114399110,11cyclictest0-21swapper/607:00:346
43114399110,11cyclictest0-21swapper/605:35:176
43114399110,11cyclictest0-21swapper/605:35:176
43114399110,11cyclictest0-21swapper/605:16:416
43114399110,11cyclictest0-21swapper/604:56:216
43114399110,11cyclictest0-21swapper/604:48:186
43114399110,11cyclictest0-21swapper/604:36:106
43114099110,1cyclictest0-21swapper/504:46:065
43114899100,2cyclictest0-21swapper/705:09:217
43114399109,1cyclictest0-21swapper/607:20:166
43114399109,1cyclictest0-21swapper/606:52:326
43114399109,1cyclictest0-21swapper/606:47:106
43114399109,1cyclictest0-21swapper/604:21:306
43114399109,1cyclictest0-21swapper/604:18:026
43114399108,2cyclictest688172-21kworker/u16:3+events_unbound05:29:116
43114399108,1cyclictest1042876-21fschecks_time08:24:106
43114399100,9cyclictest0-21swapper/608:44:206
43114399100,1cyclictest0-21swapper/608:34:176
4311439998,1cyclictest0-21swapper/607:05:566
4311439998,1cyclictest0-21swapper/607:05:566
4311439998,1cyclictest0-21swapper/606:57:536
4311439997,1cyclictest0-21swapper/606:04:136
4311439996,3cyclictest3185608-21kworker/u16:5+flush-8:005:04:196
4311439987,1cyclictest0-21swapper/606:18:386
4311439987,1cyclictest0-21swapper/603:50:366
4311439985,2cyclictest900536-21/usr/sbin/munin07:14:156
4311439984,3cyclictest497905-21cut03:54:166
4311439983,4cyclictest946169-21cut07:34:136
4311439983,4cyclictest802020-21grep06:24:126
4311439983,4cyclictest720267-21cut05:44:116
4311439980,7cyclictest0-21swapper/608:19:136
4311439975,1cyclictest0-21swapper/608:04:076
4311439974,2cyclictest971438-21expr07:49:066
4311439973,3cyclictest893956-21grep07:09:136
4311439972,4cyclictest567302-21expr04:29:076
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional