You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 07:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Thu Nov 20, 2025 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
399086825948,7sleep50-21swapper/503:18:455
399086825948,7sleep50-21swapper/503:18:455
4000903995550,4cyclictest200216-21latency_hist07:13:417
4000901995349,3cyclictest109414-21kworker/u16:4+flush-8:006:38:416
400047825218,30sleep70-21swapper/703:23:047
400047825218,30sleep70-21swapper/703:23:047
4000900994946,3cyclictest290808-21kworker/u16:3+flush-8:008:48:405
4000901994642,3cyclictest109414-21kworker/u16:4+flush-8:007:03:416
4000900994442,2cyclictest4049775-21kworker/u16:1+flush-8:004:38:415
4000901994037,2cyclictest109414-21kworker/u16:4+events_unbound07:53:516
400022124030,7sleep40-21swapper/403:19:244
400022124030,7sleep40-21swapper/403:19:244
4000900993937,2cyclictest4171469-21kworker/u16:3+flush-8:006:24:115
400090399370,35cyclictest589235-21systemd-journal04:53:417
4000900993735,2cyclictest4171717-21kworker/u16:5+flush-8:005:31:195
4000903993631,4cyclictest4008359-21latency_hist03:48:417
4000903993629,6cyclictest589235-21systemd-journal07:08:157
4000900993634,2cyclictest4160898-21kworker/u16:0+flush-8:006:21:175
4000900993633,2cyclictest321118-21kworker/u16:1+events_unbound08:48:505
4000903993528,5cyclictest17965-21latency_hist05:43:417
4000900993533,2cyclictest3290230-21kworker/u16:4+flush-8:004:29:195
399086923525,7sleep60-21swapper/603:18:466
399086923525,7sleep60-21swapper/603:18:466
4000900993432,2cyclictest4171469-21kworker/u16:3+flush-8:007:09:515
4000900993432,2cyclictest109414-21kworker/u16:4+events_unbound06:33:515
4000900993331,2cyclictest4171717-21kworker/u16:5+events_unbound05:37:185
4000900993331,2cyclictest4171469-21kworker/u16:3+events_unbound06:09:195
4000900993331,2cyclictest4100446-21kworker/u16:0+events_unbound05:08:195
4000900993331,2cyclictest4005926-21kworker/u16:2+flush-8:003:47:515
4000900993331,2cyclictest331185-21kworker/u16:2+events_unbound08:39:155
4000900993331,2cyclictest321118-21kworker/u16:1+flush-8:008:10:505
4000900993331,2cyclictest290808-21kworker/u16:3+events_unbound08:01:515
4000900993331,2cyclictest109414-21kworker/u16:4+events_unbound06:49:515
4000900993330,2cyclictest3978854-21kworker/u16:1+flush-8:003:33:525
4000900993330,2cyclictest3290230-21kworker/u16:4+events_unbound03:57:445
4000900993230,2cyclictest4160898-21kworker/u16:0+flush-8:006:31:005
4000900993230,2cyclictest331185-21kworker/u16:2+events_unbound08:36:195
4000900993230,2cyclictest28825-21kworker/u16:2+events_unbound05:49:105
4000900993230,2cyclictest109414-21kworker/u16:4+events_unbound06:46:275
4000896993228,3cyclictest4171717-21kworker/u16:5+events_unbound05:39:004
4000903993126,4cyclictest68815-21latency_hist06:08:417
4000901993128,2cyclictest4160898-21kworker/u16:0+flush-8:006:03:406
4000900993129,2cyclictest4171469-21kworker/u16:3+flush-8:007:06:515
4000900993129,2cyclictest4171469-21kworker/u16:3+events_unbound05:40:595
4000900993129,2cyclictest4110718-21kworker/u16:4+flush-8:004:57:195
4000900993129,2cyclictest3978854-21kworker/u16:1+events_unbound03:33:035
4000900993129,2cyclictest331185-21kworker/u16:2+flush-8:008:19:195
4000900993129,2cyclictest331185-21kworker/u16:2+flush-8:008:17:265
4000900993129,2cyclictest3290230-21kworker/u16:4+events_unbound03:25:455
4000900993129,2cyclictest109414-21kworker/u16:4+events_unbound07:43:185
4000900993129,2cyclictest109414-21kworker/u16:4+events_unbound07:28:565
4000903993029,0cyclictest0-21swapper/706:59:477
4000900993028,2cyclictest4160898-21kworker/u16:0+events_unbound05:26:185
4000900993028,2cyclictest4049775-21kworker/u16:1+events_unbound05:13:005
4000900993028,2cyclictest4049775-21kworker/u16:1+events_unbound04:51:445
4000900993028,2cyclictest4049775-21kworker/u16:1+events_unbound04:23:285
4000900993028,2cyclictest230942-21kworker/u16:1+events_unbound07:34:345
4000900993028,2cyclictest109414-21kworker/u16:4+events_unbound07:56:155
4000900993027,2cyclictest3290230-21kworker/u16:4+events_unbound03:51:435
4000903992924,4cyclictest4029877-21latency_hist04:13:417
4000901992927,2cyclictest331185-21kworker/u16:2+flush-8:008:18:566
4000901992927,2cyclictest109414-21kworker/u16:4+flush-8:007:41:426
4000900992927,2cyclictest4171469-21kworker/u16:3+events_unbound07:01:115
4000900992927,2cyclictest4171469-21kworker/u16:3+events_unbound06:16:275
4000900992927,2cyclictest4160898-21kworker/u16:0+flush-8:006:04:315
4000900992927,2cyclictest4110718-21kworker/u16:4+events_unbound06:00:555
4000900992927,2cyclictest3934268-21kworker/u16:3+events_unbound04:18:235
4000900992927,2cyclictest3290230-21kworker/u16:4+flush-8:004:06:275
4000900992927,2cyclictest3290230-21kworker/u16:4+events_unbound04:12:205
4000900992927,2cyclictest290808-21kworker/u16:3+events_unbound08:30:235
4000900992927,2cyclictest109414-21kworker/u16:4+events_unbound07:26:235
4000896992926,2cyclictest109414-21kworker/u16:4+events_unbound07:28:544
4000903992827,0cyclictest0-21swapper/708:00:167
4000903992827,0cyclictest0-21swapper/707:49:187
4000901992826,2cyclictest4100446-21kworker/u16:0+flush-8:005:04:076
4000901992826,2cyclictest3934268-21kworker/u16:3+flush-8:004:39:066
4000901992825,2cyclictest4049775-21kworker/u16:1+flush-8:005:09:076
4000900992826,2cyclictest4150308-21kworker/u16:2+events_unbound05:14:175
4000900992826,2cyclictest4005926-21kworker/u16:2+flush-8:004:41:165
4000900992826,2cyclictest3934268-21kworker/u16:3+flush-8:004:46:035
4000900992826,2cyclictest109414-21kworker/u16:4+events_unbound07:44:515
4000896992825,2cyclictest4049775-21kworker/u16:1+flush-8:005:03:414
4000903992723,2cyclictest78944-21munin-run06:13:407
4000901992725,2cyclictest4005926-21kworker/u16:2+flush-8:004:44:076
4000901992723,3cyclictest230533-21kworker/u16:0+events_unbound07:29:056
4000900992725,2cyclictest4049775-21kworker/u16:1+flush-8:005:21:435
4000900992725,2cyclictest4049775-21kworker/u16:1+events_unbound04:24:365
4000900992725,2cyclictest4005926-21kworker/u16:2+events_unbound03:41:275
4000900992724,2cyclictest3290230-21kworker/u16:4+events_unbound04:02:085
4000896992724,3cyclictest4110718-21kworker/u16:4+events_unbound05:14:094
4000903992625,0cyclictest0-21swapper/704:15:437
4000900992624,2cyclictest4171469-21kworker/u16:3+flush-8:007:15:235
4000896992623,2cyclictest109414-21kworker/u16:4+events_unbound06:33:554
4000903992525,0cyclictest0-21swapper/707:54:367
4000903992524,0cyclictest0-21swapper/708:24:427
4000903992520,4cyclictest210372-21latency_hist07:18:417
4000901992523,2cyclictest290808-21kworker/u16:3+events_unbound07:59:116
4000901992522,2cyclictest3290230-21kworker/u16:4+flush-8:003:49:126
4000901992521,3cyclictest109414-21kworker/u16:4+flush-8:008:33:576
4000900992523,2cyclictest4171469-21kworker/u16:3+events_unbound05:54:545
4000900992523,2cyclictest4160898-21kworker/u16:0+events_unbound05:45:315
4000900992523,2cyclictest290808-21kworker/u16:3+flush-8:008:06:435
4000900992522,2cyclictest271357-21kworker/u16:1+events_unbound07:48:505
4000896992523,2cyclictest3934268-21kworker/u16:3+events_unbound04:18:564
4000896992522,2cyclictest4171469-21kworker/u16:3+flush-8:007:08:414
4000903992423,0cyclictest0-21swapper/707:24:107
4000903992423,0cyclictest0-21swapper/706:46:257
4000903992423,0cyclictest0-21swapper/704:28:267
4000903992423,0cyclictest0-21swapper/704:06:077
4000903992423,0cyclictest0-21swapper/703:36:407
4000903992418,5cyclictest321438-21latency_hist08:13:407
4000901992422,2cyclictest79310-21kworker/u16:1+flush-8:006:13:566
4000901992422,2cyclictest4110718-21kworker/u16:4+flush-8:005:14:026
4000900992422,2cyclictest3934268-21kworker/u16:3+events_unbound05:01:315
4000896992422,2cyclictest290808-21kworker/u16:3+flush-8:007:59:144
4000903992322,0cyclictest0-21swapper/708:07:107
4000900992321,2cyclictest149453-21kworker/u16:2+events_unbound06:55:585
4000900992321,2cyclictest109414-21kworker/u16:4+events_unbound07:22:155
4000896992319,3cyclictest331185-21kworker/u16:2+flush-8:008:19:094
4000903992222,0cyclictest0-21swapper/703:26:407
4000903992221,0cyclictest0-21swapper/704:56:117
400090399220,20cyclictest169962-21latency_hist06:58:417
4000901992219,3cyclictest4171469-21kworker/u16:3+events_unbound06:23:536
4000900992220,2cyclictest4171469-21kworker/u16:3+events_unbound06:40:395
4000896992220,2cyclictest230942-21kworker/u16:1+flush-8:007:33:544
4000896992218,3cyclictest4190868-21smart_sda05:29:084
4000903992120,0cyclictest0-21swapper/707:46:067
4000903992120,0cyclictest0-21swapper/706:36:067
4000903992120,0cyclictest0-21swapper/706:13:447
4000901992119,2cyclictest4171469-21kworker/u16:3+events_unbound06:39:066
4000901992119,2cyclictest3934268-21kworker/u16:3+flush-8:004:54:086
4000901992119,2cyclictest290808-21kworker/u16:3+events_unbound08:44:106
4000896992119,2cyclictest290808-21kworker/u16:3+events_unbound08:49:094
4000903992019,0cyclictest0-21swapper/708:40:007
4000903992019,0cyclictest0-21swapper/708:35:387
4000903992019,0cyclictest0-21swapper/705:46:457
4000903992019,0cyclictest0-21swapper/705:18:497
4000901992015,4cyclictest109134-21kworker/u16:1+events_unbound06:30:046
4000900992018,2cyclictest290808-21kworker/u16:3+flush-8:008:28:155
4000896992018,2cyclictest3934268-21kworker/u16:3+flush-8:004:39:084
4000896992018,2cyclictest300991-21kworker/u16:0+flush-8:008:08:564
4000903991918,0cyclictest0-21swapper/705:38:097
4000903991918,0cyclictest0-21swapper/704:20:307
4000903991918,0cyclictest0-21swapper/703:29:567
4000903991914,4cyclictest402376-21latency_hist08:53:417
4000903991913,5cyclictest38562-21latency_hist05:53:417
400090399190,17cyclictest589356-21tuned-ppd04:30:427
4000901991917,2cyclictest149453-21kworker/u16:2+events_unbound06:53:596
4000903991817,0cyclictest0-21swapper/708:19:537
4000903991817,0cyclictest0-21swapper/707:36:537
4000903991817,0cyclictest0-21swapper/704:38:417
4000903991817,0cyclictest0-21swapper/703:57:507
4000901991816,2cyclictest3934268-21kworker/u16:3+flush-8:004:19:076
4000896991815,3cyclictest4110718-21kworker/u16:4+events_unbound06:04:054
4000903991716,0cyclictest0-21swapper/705:54:327
4000903991714,2cyclictest4175659-21snmpd06:02:557
4000903991711,5cyclictest4080412-21latency_hist04:38:417
4000901991715,2cyclictest83197-21kworker/u16:2+flush-8:006:19:066
4000901991715,2cyclictest200604-21kworker/u16:2+events_unbound07:23:536
4000901991714,2cyclictest4049775-21kworker/u16:1+flush-8:005:24:086
4000896991715,2cyclictest331185-21kworker/u16:2+flush-8:008:24:074
4000896991713,3cyclictest3934268-21kworker/u16:3+flush-8:004:54:074
400089699170,16cyclictest4142092-21perl05:08:534
4000903991616,0cyclictest0-21swapper/706:23:307
4000903991615,0cyclictest0-21swapper/708:30:357
4000903991615,0cyclictest0-21swapper/707:42:157
4000903991615,0cyclictest0-21swapper/706:29:057
4000903991615,0cyclictest0-21swapper/705:27:317
4000903991615,0cyclictest0-21swapper/705:14:087
4000903991615,0cyclictest0-21swapper/705:10:537
4000903991611,4cyclictest4141049-21latency_hist05:08:417
400090399160,14cyclictest0-21swapper/708:14:067
4000896991613,2cyclictest149453-21kworker/u16:2+flush-8:006:55:234
4000903991515,0cyclictest0-21swapper/705:01:257
4000903991514,0cyclictest0-21swapper/706:42:007
4000903991512,2cyclictest4175659-21snmpd08:47:227
400090399150,14cyclictest4175659-21snmpd03:51:167
4000896991513,2cyclictest4171469-21kworker/u16:3+events_unbound07:39:094
4000903991414,0cyclictest0-21swapper/707:19:397
4000903991414,0cyclictest0-21swapper/704:44:597
4000901991412,2cyclictest290808-21kworker/u16:3+flush-8:008:09:056
400090199140,7cyclictest0-21swapper/606:03:576
400089699147,6cyclictest0-21swapper/404:05:294
4000896991412,2cyclictest290808-21kworker/u16:3+flush-8:008:29:064
4000896991412,2cyclictest109134-21kworker/u16:1+events_unbound06:38:544
4000896991411,3cyclictest4049775-21kworker/u16:1+flush-8:005:24:104
4000896991411,2cyclictest321118-21kworker/u16:1+flush-8:008:38:584
4000896991410,3cyclictest4171469-21kworker/u16:3+flush-8:007:14:074
4000903991310,2cyclictest0-21swapper/703:58:417
400089699139,3cyclictest221660-21cat07:23:554
400090399129,2cyclictest0-21swapper/706:24:057
400090199128,3cyclictest3290230-21kworker/u16:4+flush-8:004:08:566
4000901991211,1cyclictest0-21swapper/607:13:546
4000901991210,2cyclictest4160898-21kworker/u16:0+flush-8:006:11:376
4000901991210,2cyclictest4005926-21kworker/u16:2+events_unbound04:49:066
4000901991210,2cyclictest321118-21kworker/u16:1+flush-8:008:32:236
4000901991210,2cyclictest179691-21kworker/u16:1+flush-8:007:04:116
400090199120,1cyclictest0-21swapper/604:24:356
400090199120,12cyclictest0-21swapper/606:52:186
400089699129,2cyclictest4160898-21kworker/u16:0+flush-8:006:44:094
400089699128,3cyclictest4101782-21cat04:48:574
400089699120,12cyclictest0-21swapper/408:05:184
400089699120,12cyclictest0-21swapper/404:24:554
400089699120,11cyclictest0-21swapper/403:27:324
400090399110,10cyclictest96-21ksoftirqd/706:49:007
400090199119,2cyclictest4171717-21kworker/u16:5+flush-8:005:29:046
4000901991110,1cyclictest0-21swapper/608:51:486
4000901991110,1cyclictest0-21swapper/605:47:496
4000901991110,1cyclictest0-21swapper/605:41:026
4000901991110,1cyclictest0-21swapper/605:35:136
4000901991110,1cyclictest0-21swapper/603:43:556
400090199110,11cyclictest0-21swapper/606:47:156
400090199110,11cyclictest0-21swapper/604:06:376
400090199110,11cyclictest0-21swapper/603:29:246
400090199110,0cyclictest0-21swapper/605:52:056
400089699119,2cyclictest79310-21kworker/u16:1+flush-8:006:13:574
400089699118,3cyclictest3934268-21kworker/u16:3+flush-8:004:14:194
400089699117,3cyclictest49828-21cat05:58:544
400089699117,3cyclictest4162383-21cat05:18:574
400089699116,4cyclictest211545-21cat07:18:554
4000896991110,1cyclictest0-21swapper/404:11:594
400089699110,1cyclictest0-21swapper/406:51:124
400089699110,11cyclictest0-21swapper/407:50:104
400089699110,11cyclictest0-21swapper/407:03:114
400089699110,11cyclictest0-21swapper/406:33:334
400089699110,11cyclictest0-21swapper/405:04:574
400089699110,11cyclictest0-21swapper/404:34:414
400090399107,2cyclictest96-21ksoftirqd/705:28:527
400090399107,2cyclictest4175659-21snmpd03:40:377
400090399105,4cyclictest589235-21systemd-journal07:29:157
400090199109,1cyclictest0-21swapper/607:37:046
400090199109,1cyclictest0-21swapper/607:12:106
400090199109,1cyclictest0-21swapper/605:03:036
400090199109,1cyclictest0-21swapper/603:38:146
400090199100,0cyclictest0-21swapper/604:36:536
400089699109,1cyclictest0-21swapper/406:21:434
400089699109,1cyclictest0-21swapper/405:37:444
400089699109,1cyclictest0-21swapper/403:43:364
40009019997,2cyclictest109414-21kworker/u16:4+flush-8:008:04:116
40009019997,2cyclictest109414-21kworker/u16:4+events_unbound07:18:546
40009019995,3cyclictest0-21swapper/605:19:026
40009019994,4cyclictest342851-21cat08:23:566
40009019990,8cyclictest4010516-21taskset04:02:446
40009019990,1cyclictest0-21swapper/604:28:596
40008969998,1cyclictest0-21swapper/408:37:464
40008969998,1cyclictest0-21swapper/408:15:234
40008969998,1cyclictest0-21swapper/405:49:344
40008969998,1cyclictest0-21swapper/403:53:134
40008969996,2cyclictest0-21swapper/404:44:024
40008969995,3cyclictest100302-21cat06:23:534
40009019987,1cyclictest0-21swapper/608:15:136
40009019987,1cyclictest0-21swapper/607:46:326
40009019984,3cyclictest376228-21cut08:39:006
40009019983,4cyclictest4037697-21grep04:14:066
40008969987,1cyclictest0-21swapper/405:56:214
40008969987,1cyclictest0-21swapper/405:47:504
40008969986,2cyclictest4110718-21kworker/u16:4+events_unbound06:08:564
40008969986,1cyclictest0-21swapper/408:44:024
40008969984,3cyclictest4067177-21cut04:29:034
40008969983,4cyclictest261269-21cpu07:43:534
40008969983,4cyclictest199221-21awk07:09:064
40009019972,4cyclictest39787-21irqcore05:53:566
40009019972,4cyclictest271801-21expr07:48:526
40008969975,1cyclictest0-21swapper/407:53:554
40008969973,3cyclictest4002887-21grep03:28:444
40008969965,1cyclictest0-21swapper/403:58:544
40008969964,2cyclictest0-21swapper/403:54:194
40008969964,1cyclictest1-21systemd03:38:424
40009019950,5cyclictest0-21swapper/603:24:026
40009019950,4cyclictest4009500-21taskset03:53:556
40009019950,4cyclictest4007333-21chrt03:39:456
40008969950,4cyclictest4007999-21taskset03:43:434
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional