You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-29 - 09:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot4.osadl.org (updated Wed Oct 29, 2025 00:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
339365923610,12sleep13393661-21ntp_kernel_err19:05:261
33942482262189,27sleep00-21swapper/019:05:410
33947649914335,88cyclictest3472951-21gpg-connect-age21:50:020
33947669912361,50cyclictest3523742-21grep23:00:001
3394764991191,104cyclictest3527760-21cstates23:05:150
339476699853,69cyclictest3472919-21apt-key21:50:021
35691022640,2sleep10-21swapper/100:01:241
3394764996016,10cyclictest3566581-21python300:00:010
125899600,4rtkit-daemon1257-21rtkit-daemon22:07:221
3394764995916,11cyclictest3509457-21/usr/sbin/munin22:40:010
125899570,4rtkit-daemon1257-21rtkit-daemon20:21:361
3394766995652,2cyclictest3527738-21cstates23:05:151
3394764995617,11cyclictest3523707-21munin-run23:00:000
125899560,4rtkit-daemon1257-21rtkit-daemon00:06:000
3394764995514,11cyclictest3465415-21latency_hist21:40:000
125899540,5rtkit-daemon1257-21rtkit-daemon23:10:541
125899540,4rtkit-daemon1257-21rtkit-daemon23:16:111
3394764995322,11cyclictest3502569-21ssh22:30:170
3394764995318,10cyclictest3477698-21sh21:55:220
3394764995315,11cyclictest3466594-21mailstats21:40:250
3394764995314,11cyclictest3458097-21apt-get21:30:140
125899530,4rtkit-daemon1257-21rtkit-daemon23:46:490
3394764995216,11cyclictest3589968-21sed00:30:270
3394764995215,9cyclictest1-21systemd19:35:010
3394764995215,11cyclictest3399805-21latency_hist19:20:000
3394764995214,9cyclictest1-21systemd23:30:000
3394764995214,11cyclictest3488370-21ntpq22:10:230
125899520,4rtkit-daemon1257-21rtkit-daemon23:42:440
3394764995117,9cyclictest1-21systemd20:35:010
3394764995116,9cyclictest3567063-21apt-config00:00:020
3394764995116,10cyclictest3443180-21diskmemload00:35:150
3394764995115,12cyclictest3501065-21chrt22:27:220
3394764995115,12cyclictest3395352-21tar19:10:050
3394764995115,10cyclictest3534677-21apt-get23:15:130
3394764995114,9cyclictest3527356-21cat23:05:010
3394764995113,12cyclictest3443328-21latency_hist21:10:000
3394764995017,14cyclictest3547795-21rm23:32:490
3394764995016,9cyclictest3431599-21latency_hist20:40:000
3394764995016,12cyclictest3584365-21apt-get00:25:060
3394764995015,9cyclictest672-21dbus-daemon19:45:030
3394764995015,9cyclictest1-21systemd20:05:010
3394764995015,14cyclictest3436292-21/usr/sbin/munin20:50:240
3394764995014,9cyclictest1-21systemd22:45:000
3394764995014,21cyclictest3559758-21apt-get23:50:150
3394764995013,9cyclictest3450867-21latency_hist21:20:020
3394764995013,12cyclictest3487437-21latency_hist22:10:000
3394764995012,12cyclictest3517289-21fschecks_count22:50:190
125899500,4rtkit-daemon1257-21rtkit-daemon22:39:541
3394764994919,11cyclictest3443180-21diskmemload22:45:430
3394764994918,10cyclictest3577416-21/usr/sbin/munin00:15:130
3394764994917,9cyclictest3453516-21ssh21:23:060
3394764994915,15cyclictest3534421-21sh23:15:000
3394764994915,14cyclictest3496013-21ntpq22:20:240
3394764994915,13cyclictest3457637-21rm21:29:330
3394764994915,13cyclictest3415795-21systemctl19:59:590
3394764994915,12cyclictest3419132-21sed20:05:290
3394764994915,11cyclictest3552369-21date23:40:010
3394764994915,10cyclictest1-21systemd20:45:000
3394764994914,13cyclictest3491579-21cut22:15:190
3394764994914,11cyclictest3481680-21unixbench_multi22:00:280
3394764994914,10cyclictest3447120-21wc21:15:000
3394764994913,12cyclictest3425655-21latency_hist20:25:000
3394764994913,11cyclictest3582120-21grep00:20:260
3394764994818,10cyclictest3576810-21python300:15:010
3394764994818,10cyclictest3576810-21python300:15:000
3394764994815,10cyclictest3443180-21diskmemload21:45:260
3394764994814,13cyclictest3422089-21grep20:15:190
3394764994814,12cyclictest3538576-21grep23:20:210
3394764994814,11cyclictest1-21systemd19:45:000
3394764994814,10cyclictest1-21systemd20:50:010
3394764994813,11cyclictest3402439-21grep19:25:190
3394764994813,10cyclictest3421803-21latency_hist20:15:000
3394764994812,11cyclictest3401684-21latency_hist19:25:010
3394764994717,10cyclictest3427243-21taskset20:27:080
3394764994714,10cyclictest672-21dbus-daemon19:40:010
3394764994713,11cyclictest3437806-21fschecks_time20:55:210
3394764994615,10cyclictest3441362-21latency_hist21:05:010
3394764994614,12cyclictest3413985-21date19:55:010
125899440,3rtkit-daemon1257-21rtkit-daemon20:55:001
125899380,6rtkit-daemon3549925-21/usr/sbin/munin23:35:231
125899380,4rtkit-daemon3455514-21ssh21:25:241
125899380,4rtkit-daemon1257-21rtkit-daemon22:15:011
125899370,3rtkit-daemon0-21swapper/021:56:141
125899350,5rtkit-daemon3588521-21ssh00:30:021
125899350,3rtkit-daemon0-21swapper/023:49:501
125899350,3rtkit-daemon0-21swapper/000:24:531
125899340,3rtkit-daemon0-21swapper/019:45:331
3394766993330,2cyclictest63750irq/132-enp2s022:18:301
125899330,4rtkit-daemon1257-21rtkit-daemon20:31:361
125899330,3rtkit-daemon1257-21rtkit-daemon23:20:351
125899330,3rtkit-daemon0-21swapper/023:52:081
125899330,3rtkit-daemon0-21swapper/020:09:361
3394766993215,11cyclictest3395417-21apt-get19:10:251
125899320,3rtkit-daemon0-21swapper/020:16:431
3394766993129,1cyclictest0-21swapper/119:40:061
125899310,3rtkit-daemon0-21swapper/021:31:431
125899310,3rtkit-daemon0-21swapper/020:35:381
125899310,3rtkit-daemon0-21swapper/019:19:491
3394766993026,3cyclictest63750irq/132-enp2s022:33:441
125899300,3rtkit-daemon0-21swapper/000:36:251
125899290,3rtkit-daemon0-21swapper/023:44:371
125899290,3rtkit-daemon0-21swapper/020:41:301
339476699288,15cyclictest26-21ksoftirqd/121:35:141
3394766992824,2cyclictest34274142sleep120:28:581
339476699280,9cyclictest3469361-21grep21:45:181
3394766992620,4cyclictest3438647-21cat20:55:291
3394766992612,12cyclictest26-21ksoftirqd/100:25:031
339476699260,6cyclictest0-21swapper/123:28:341
339476699260,6cyclictest0-21swapper/121:17:131
339476699260,12cyclictest3566549-40mandb00:00:001
125899260,1rtkit-daemon1257-21rtkit-daemon19:25:301
339476699259,13cyclictest26-21ksoftirqd/122:00:151
339476699258,14cyclictest26-21ksoftirqd/100:10:121
339476699258,14cyclictest26-21ksoftirqd/100:10:111
339476699255,5cyclictest26-21ksoftirqd/100:06:111
339476699250,6cyclictest0-21swapper/122:28:051
339476699250,6cyclictest0-21swapper/121:21:571
339476699250,5cyclictest0-21swapper/122:46:551
339476699248,2cyclictest141rcu_preempt23:00:491
339476699248,12cyclictest3509690-21apt-get22:40:151
3394766992421,2cyclictest63750irq/132-enp2s022:52:281
3394766992416,6cyclictest3433920-21fschecks_time20:45:191
339476699240,6cyclictest0-21swapper/121:10:441
339476699240,6cyclictest0-21swapper/119:30:131
339476699240,5cyclictest0-21swapper/119:20:181
339476699234,18cyclictest3548723-21/usr/sbin/munin23:35:011
339476699232,15cyclictest3494478-21apt-get22:20:061
125899230,19rtkit-daemon1257-21rtkit-daemon20:04:161
339476699228,5cyclictest0-21swapper/121:43:491
339476699215,4cyclictest26-21ksoftirqd/100:19:111
125899210,3rtkit-daemon1257-21rtkit-daemon20:13:531
125899210,13rtkit-daemon0-21swapper/119:38:371
339476699201,6cyclictest3442678-21unixbench_singl21:05:251
339476699201,17cyclictest3412387-21tune2fs19:50:171
3394766991911,6cyclictest3415334-21sshd19:55:311
3394766991812,4cyclictest849-21snmpd21:01:571
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:543
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
none10:34:542
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional