You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 16:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Tue Dec 02, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
377339025949,7sleep50-21swapper/515:19:305
3773948995553,2cyclictest3898482-21kworker/u16:2+events_unbound16:57:516
377341925419,30sleep70-21swapper/715:19:567
3773948994441,3cyclictest3970246-21kworker/u16:5+flush-8:017:27:516
377394999410,39cyclictest4130964-21latency_hist18:17:517
3773944994138,2cyclictest88584-21kworker/u16:2+flush-8:019:53:135
3773949994035,4cyclictest4191451-21latency_hist18:47:517
377394999380,36cyclictest3888715-21latency_hist16:17:527
377352523828,7sleep40-21swapper/415:21:224
3773949993734,2cyclictest4090621-21latency_hist17:57:527
3773948993735,2cyclictest4020308-21kworker/u16:2+flush-8:017:29:296
3773944993734,2cyclictest3773446-21kworker/u16:1+events_unbound15:22:545
3773949993635,0cyclictest0-21swapper/719:43:057
3773949993529,5cyclictest68579-21latency_hist19:22:517
3773944993533,2cyclictest98913-21kworker/u16:1+events_unbound20:08:155
3773949993428,5cyclictest589235-21systemd-journal16:04:117
3773944993431,2cyclictest3868596-21kworker/u16:0+events_unbound16:48:105
377351823323,7sleep60-21swapper/615:21:166
3773944993230,2cyclictest168420-21kworker/u16:4+flush-8:020:28:145
2486582320,1chrt248661-21kthreadcore20:48:157
3773944993129,2cyclictest26862-21kworker/u16:5+flush-8:019:28:225
3773937993128,2cyclictest98913-21kworker/u16:1+flush-8:020:08:164
3773949993025,4cyclictest3929022-21latency_hist16:37:517
3773949993024,5cyclictest4120875-21latency_hist18:12:517
377394999300,28cyclictest209744-21latency_hist20:32:517
3773944993028,2cyclictest4080956-21kworker/u16:0+flush-8:018:08:215
3773944993028,2cyclictest26862-21kworker/u16:5+events_unbound19:08:215
3773944993027,3cyclictest3969718-21kworker/u16:4+flush-8:016:58:075
3773944993026,3cyclictest3797749-21kworker/u16:3+flush-8:015:42:525
3773948992927,2cyclictest3797750-21kworker/u16:4+flush-8:015:43:266
3773948992927,2cyclictest3797749-21kworker/u16:3+events_unbound15:53:176
3773937992927,2cyclictest168420-21kworker/u16:4+flush-8:020:43:224
3773948992825,2cyclictest88584-21kworker/u16:2+flush-8:019:48:066
3773948992825,2cyclictest3797749-21kworker/u16:3+flush-8:015:38:056
3773944992826,2cyclictest4080956-21kworker/u16:0+flush-8:019:13:065
3773944992825,2cyclictest7467-21kworker/u16:1+flush-8:018:53:125
3773949992725,1cyclictest0-21swapper/718:18:097
3773948992724,2cyclictest98913-21kworker/u16:1+events_unbound20:12:516
3773944992725,2cyclictest88584-21kworker/u16:2+flush-8:019:43:215
3773944992725,2cyclictest3773446-21kworker/u16:1+events_unbound15:28:205
3773944992724,2cyclictest3832787-21kworker/u16:1+flush-8:016:18:075
3773944992723,3cyclictest3832787-21kworker/u16:1+flush-8:015:58:235
3773944992723,3cyclictest3832787-21kworker/u16:1+flush-8:015:58:235
3773949992625,0cyclictest0-21swapper/717:33:157
3773944992624,2cyclictest3832787-21kworker/u16:1+flush-8:016:13:185
3773937992624,2cyclictest3970246-21kworker/u16:5+flush-8:017:29:214
3773948992523,2cyclictest3784069-21kworker/u16:2+flush-8:015:28:076
3773944992523,2cyclictest4020308-21kworker/u16:2+flush-8:018:48:225
3773948992421,2cyclictest4130751-21kworker/u16:1+flush-8:018:38:176
3773944992422,2cyclictest98913-21kworker/u16:1+flush-8:019:52:515
3773944992421,2cyclictest4130751-21kworker/u16:1+flush-8:018:37:515
3773949992319,3cyclictest3949175-21awk16:47:517
3773949992319,3cyclictest3949175-21awk16:47:517
3773949992318,4cyclictest589235-21systemd-journal20:17:507
3773949992318,4cyclictest4009831-21latency_hist17:17:517
3773944992321,2cyclictest3797749-21kworker/u16:3+events_unbound15:43:215
3773937992319,3cyclictest3907924-21kworker/u16:3+flush-8:016:53:234
377394999220,20cyclictest3818166-21latency_hist15:42:527
3773948992220,2cyclictest4050151-21kworker/u16:1+flush-8:017:43:166
3773948992218,3cyclictest26862-21kworker/u16:5+flush-8:019:13:196
3773944992220,2cyclictest4080956-21kworker/u16:0+flush-8:019:38:125
3773944992220,2cyclictest3907924-21kworker/u16:3+flush-8:016:56:475
3773944992219,2cyclictest168420-21kworker/u16:4+flush-8:020:18:165
3773937992220,2cyclictest4030413-21kworker/u16:0+flush-8:017:33:584
3773937992215,6cyclictest0-21swapper/418:08:064
3773948992119,2cyclictest3868596-21kworker/u16:0+flush-8:016:47:516
3773948992119,2cyclictest3868596-21kworker/u16:0+flush-8:016:47:516
3773937992119,2cyclictest4080956-21kworker/u16:0+events_unbound19:28:064
3773944992018,2cyclictest4020308-21kworker/u16:2+flush-8:017:38:095
3773944992018,2cyclictest3898482-21kworker/u16:2+events_unbound16:33:115
3773944992018,2cyclictest3868596-21kworker/u16:0+flush-8:017:18:215
3773944992018,2cyclictest3832787-21kworker/u16:1+flush-8:017:08:225
3773944992018,2cyclictest3784069-21kworker/u16:2+events_unbound15:33:115
3773944992018,2cyclictest168420-21kworker/u16:4+flush-8:020:48:225
3773944992017,2cyclictest3832787-21kworker/u16:1+events_unbound16:08:165
3773944992015,4cyclictest4039881-21packagekitd17:29:145
3773949991918,0cyclictest0-21swapper/719:48:197
3773948991916,2cyclictest3832787-21kworker/u16:1+events_unbound16:08:126
3773944991917,2cyclictest4130751-21kworker/u16:1+flush-8:018:38:075
3773944991917,2cyclictest4130751-21kworker/u16:1+events_unbound18:23:055
3773944991917,2cyclictest3832787-21kworker/u16:1+flush-8:016:28:185
3773944991917,2cyclictest26862-21kworker/u16:5+flush-8:019:23:175
3773937991915,3cyclictest3970246-21kworker/u16:5+flush-8:017:23:074
3773949991813,4cyclictest17671-21latency_hist18:57:517
3773949991812,5cyclictest3797969-21latency_hist15:32:527
377394999180,17cyclictest4184977-17auditd16:27:527
377394999180,16cyclictest589235-21systemd-journal16:18:547
377394999180,16cyclictest4151116-21latency_hist18:27:517
3773948991814,3cyclictest3898482-21kworker/u16:2+flush-8:016:49:236
3773944991816,2cyclictest3848507-21kworker/u16:2+flush-8:016:03:045
3773944991815,2cyclictest26862-21kworker/u16:5+flush-8:019:03:175
3773937991816,2cyclictest26862-21kworker/u16:5+flush-8:019:58:334
3773949991716,0cyclictest0-21swapper/720:43:057
3773949991714,2cyclictest4175659-21snmpd18:27:597
3773949991711,5cyclictest4181349-21latency_hist18:42:527
377394999170,15cyclictest4175659-21snmpd17:46:467
3773948991715,2cyclictest3907924-21kworker/u16:3+events_unbound16:33:156
3773944991715,2cyclictest26862-21kworker/u16:5+events_unbound20:23:085
3773944991714,2cyclictest4020308-21kworker/u16:2+flush-8:017:36:505
3773937991714,2cyclictest3868596-21kworker/u16:0+events_unbound16:48:164
3773949991615,0cyclictest0-21swapper/719:08:177
3773948991614,2cyclictest3848507-21kworker/u16:2+flush-8:015:58:136
3773948991614,2cyclictest3848507-21kworker/u16:2+flush-8:015:58:136
3773948991614,2cyclictest3784069-21kworker/u16:2+events_unbound15:33:076
3773948991613,3cyclictest3868596-21kworker/u16:0+flush-8:016:14:546
3773948991613,2cyclictest26862-21kworker/u16:5+flush-8:019:53:116
3773944991614,2cyclictest4050151-21kworker/u16:1+flush-8:017:53:105
3773937991614,2cyclictest4080956-21kworker/u16:0+flush-8:018:33:074
3773937991614,2cyclictest3898482-21kworker/u16:2+events_unbound16:43:154
3773937991614,2cyclictest3898482-21kworker/u16:2+events_unbound16:43:154
3773937991614,2cyclictest26862-21kworker/u16:5+flush-8:019:43:154
3773937991613,2cyclictest4171346-21kworker/u16:3+flush-8:018:47:514
3773949991513,1cyclictest0-21swapper/716:28:147
3773949991512,2cyclictest0-21swapper/717:07:527
3773949991511,3cyclictest589235-21systemd-journal17:22:467
3773948991513,2cyclictest3832787-21kworker/u16:1+events_unbound17:08:206
3773944991513,2cyclictest3832787-21kworker/u16:1+flush-8:016:23:165
3773944991512,2cyclictest209462-21kworker/u16:0+flush-8:020:33:195
3773937991513,2cyclictest88584-21kworker/u16:2+events_unbound19:33:054
377394999140,1cyclictest0-21swapper/717:03:177
3773948991412,2cyclictest88584-21kworker/u16:2+flush-8:019:35:236
3773948991412,2cyclictest26862-21kworker/u16:5+events_unbound19:43:206
3773948991412,2cyclictest209462-21kworker/u16:0+events_unbound20:33:206
3773948991412,2cyclictest17885-21kworker/u16:4+events_unbound18:58:166
3773948991411,2cyclictest3774401-21cpuspeed_turbos15:23:036
3773944991412,2cyclictest3990502-21kworker/u16:3+flush-8:017:58:185
3773944991412,2cyclictest3970246-21kworker/u16:5+flush-8:017:23:215
3773944991412,2cyclictest3907924-21kworker/u16:3+flush-8:016:43:105
3773944991412,2cyclictest3907924-21kworker/u16:3+flush-8:016:43:105
3773944991412,2cyclictest3907924-21kworker/u16:3+flush-8:016:38:185
377394499140,14cyclictest0-21swapper/518:43:095
3773937991413,1cyclictest0-21swapper/417:53:054
3773937991411,2cyclictest3072107-21kworker/u16:5+events_unbound15:23:154
377394999139,3cyclictest4175659-21snmpd20:12:377
377394999139,2cyclictest0-21swapper/716:08:147
377394999138,4cyclictest7621-21latency_hist18:52:517
377394999137,5cyclictest27174-21gnome-shell19:36:507
3773949991311,1cyclictest0-21swapper/720:23:177
377394999130,12cyclictest4040215-21turbostat17:32:517
377394899139,3cyclictest3969718-21kworker/u16:4+flush-8:017:14:426
377394899135,8cyclictest0-21swapper/618:17:516
3773948991311,2cyclictest4080956-21kworker/u16:0+events_unbound17:53:156
3773948991311,2cyclictest3898482-21kworker/u16:2+flush-8:017:03:226
3773948991310,3cyclictest4020308-21kworker/u16:2+flush-8:017:58:116
3773944991311,2cyclictest88584-21kworker/u16:2+events_unbound20:03:225
3773944991311,2cyclictest4080956-21kworker/u16:0+flush-8:019:33:155
377394999129,2cyclictest4175659-21snmpd18:34:207
377394999129,2cyclictest4175659-21snmpd18:03:227
377394999128,3cyclictest96-21ksoftirqd/720:33:047
377394999128,3cyclictest948-21NetworkManager20:03:457
377394999128,3cyclictest3798370-21/usr/sbin/munin15:33:157
377394999126,5cyclictest3858484-21latency_hist16:02:517
377394999126,5cyclictest3858484-21latency_hist16:02:517
377394999125,6cyclictest27394-21gmain20:38:097
377394999125,6cyclictest0-21swapper/716:40:267
377394999123,8cyclictest0-21swapper/719:04:117
377394999123,7cyclictest0-21swapper/716:48:177
377394999120,4cyclictest4175659-21snmpd15:43:117
377394999120,2cyclictest0-21swapper/716:53:227
377394999120,11cyclictest4175659-21snmpd20:19:317
377394999120,11cyclictest0-21swapper/715:48:127
377394899129,2cyclictest9833-21kthreadcore18:53:096
377394899127,4cyclictest59695-21cat19:18:086
377394899127,4cyclictest3970505-21cat16:58:096
3773948991210,2cyclictest4130751-21kworker/u16:1+flush-8:018:32:516
3773948991210,2cyclictest168420-21kworker/u16:4+flush-8:020:23:216
377394899120,1cyclictest0-21swapper/620:45:176
377394899120,1cyclictest0-21swapper/618:51:276
377394899120,1cyclictest0-21swapper/616:26:546
377394899120,12cyclictest0-21swapper/618:27:016
377394899120,12cyclictest0-21swapper/616:04:406
377394499129,2cyclictest4050151-21kworker/u16:1+flush-8:017:44:365
3773944991211,1cyclictest0-21swapper/518:17:565
3773944991210,2cyclictest190502-21kworker/u16:2+events_unbound20:38:055
377394499120,1cyclictest0-21swapper/519:18:475
3773937991211,1cyclictest0-21swapper/420:24:544
377393799120,5cyclictest0-21swapper/416:03:044
377393799120,1cyclictest0-21swapper/417:18:344
377393799120,1cyclictest0-21swapper/416:12:594
377393799120,12cyclictest0-21swapper/415:45:524
377393799120,12cyclictest0-21swapper/415:41:464
377394999118,2cyclictest4175659-21snmpd19:29:127
377394999118,2cyclictest4175659-21snmpd19:23:307
377394999118,2cyclictest4175659-21snmpd15:55:347
377394999117,3cyclictest948-21NetworkManager19:58:177
377394999116,4cyclictest4020294-21sendmail17:22:517
377394999115,5cyclictest589235-21systemd-journal17:48:307
377394999115,5cyclictest27174-21gnome-shell19:01:507
377394999114,6cyclictest0-21swapper/719:41:117
377394999110,9cyclictest0-21swapper/717:58:157
377394999110,5cyclictest50045-21kthreadcore19:13:127
377394999110,10cyclictest0-21swapper/716:58:177
377394899119,2cyclictest4080956-21kworker/u16:0+flush-8:019:40:536
377394899119,2cyclictest26862-21kworker/u16:5+flush-8:019:23:186
377394899119,2cyclictest168420-21kworker/u16:4+flush-8:020:28:066
3773948991110,1cyclictest0-21swapper/618:12:126
3773948991110,1cyclictest0-21swapper/618:05:166
377394899110,1cyclictest0-21swapper/615:50:486
377394899110,11cyclictest0-21swapper/620:13:266
377394899110,11cyclictest0-21swapper/618:46:446
377394899110,0cyclictest0-21swapper/618:35:426
3773944991110,1cyclictest0-21swapper/515:52:165
377394499110,11cyclictest0-21swapper/517:49:335
377394499110,0cyclictest0-21swapper/518:05:285
3773937991110,1cyclictest0-21swapper/418:44:194
3773937991110,1cyclictest0-21swapper/416:28:454
3773937991110,1cyclictest0-21swapper/415:58:574
3773937991110,1cyclictest0-21swapper/415:58:574
377393799110,1cyclictest0-21swapper/416:18:184
377393799110,11cyclictest0-21swapper/420:33:444
377393799110,11cyclictest0-21swapper/417:02:194
377393799110,11cyclictest0-21swapper/415:37:404
377393799110,0cyclictest0-21swapper/416:22:554
377393799110,0cyclictest0-21swapper/415:49:584
377394999104,5cyclictest27763-21gvfs-afc-volume15:23:097
377394999104,5cyclictest27174-21gnome-shell17:40:507
377394899109,1cyclictest0-21swapper/616:19:296
377394899107,2cyclictest4020308-21kworker/u16:2+flush-8:017:35:286
377394899100,1cyclictest0-21swapper/617:51:346
377394499109,1cyclictest0-21swapper/520:43:355
377394499109,1cyclictest0-21swapper/518:16:215
377394499109,1cyclictest0-21swapper/517:06:595
377394499107,3cyclictest4130751-21kworker/u16:1+flush-8:018:30:565
377393799109,1cyclictest0-21swapper/419:38:054
377393799109,1cyclictest0-21swapper/419:15:134
377393799109,1cyclictest0-21swapper/416:39:284
377393799108,2cyclictest228984-21kworker/u16:3+flush-8:020:39:274
377393799107,2cyclictest209462-21kworker/u16:0+flush-8:020:48:184
377393799107,2cyclictest0-21swapper/418:58:064
377393799105,4cyclictest155428-21kthreadcore20:03:124
37739499990,1cyclictest0-21swapper/719:53:197
37739489998,1cyclictest0-21swapper/616:42:306
37739449998,1cyclictest0-21swapper/515:57:095
37739379998,1cyclictest0-21swapper/419:57:474
37739379998,1cyclictest0-21swapper/417:12:064
37739379998,1cyclictest0-21swapper/415:54:044
37739379997,2cyclictest4171346-21kworker/u16:3+flush-8:019:03:064
37739379997,1cyclictest0-21swapper/418:53:104
37739379997,1cyclictest0-21swapper/417:43:124
37739379996,2cyclictest4171346-21kworker/u16:3+flush-8:019:18:174
37739379996,2cyclictest4123904-21kthreadcore18:13:094
37739379995,3cyclictest180656-21cat20:18:094
37739379994,4cyclictest3784483-21munin-node15:27:534
37739379993,5cyclictest984965-21wsdd20:30:384
37739379990,4cyclictest4077350-21grep17:48:134
37739489986,2cyclictest4080956-21kworker/u16:0+flush-8:018:18:076
37739489986,1cyclictest0-21swapper/620:03:106
37739489986,1cyclictest0-21swapper/619:03:136
37739489984,3cyclictest4051970-21irqstats17:38:086
37739489983,4cyclictest4013951-21tr17:18:106
37739489983,4cyclictest40103-21grep19:08:096
37739489983,4cyclictest3916355-21cut16:28:116
37739489983,4cyclictest181236-21cut20:18:116
37739489983,4cyclictest146563-21cut19:58:126
37739449986,1cyclictest0-21swapper/519:58:095
37739449984,3cyclictest4009033-21sendmail_mailtr17:13:195
37739449983,4cyclictest18977-21sed18:58:085
37739449983,4cyclictest178548-21ntpq20:13:155
37739379986,1cyclictest0-21swapper/418:28:044
37739379986,1cyclictest0-21swapper/418:18:054
37739379986,1cyclictest0-21swapper/417:38:124
37739379985,2cyclictest0-21swapper/419:08:124
37739379984,3cyclictest4172903-21kthreadcore18:38:104
37739379984,3cyclictest4142242-21if_p4p118:23:074
37739379983,4cyclictest76446-21tr19:23:154
37739379983,4cyclictest4104486-21cut18:03:104
37739379983,4cyclictest4096488-21cut17:58:114
37739379983,4cyclictest3983143-21cat17:03:114
37739379983,4cyclictest3923640-21kthreadcore16:33:094
37739379983,4cyclictest3869407-21expr16:08:104
37739379980,7cyclictest0-21swapper/419:48:104
37739489975,1cyclictest0-21swapper/620:48:076
37739489973,3cyclictest79737-21sh19:28:076
37739489972,4cyclictest222069-21kthreadcore20:38:096
37739379975,1cyclictest0-21swapper/420:13:034
37739379972,4cyclictest4000243-21cpuspeed_turbos17:13:044
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional