You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 07:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Thu Nov 20, 2025 00:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
307662304271,21sleep20-21swapper/219:08:122
307272269235,22sleep30-21swapper/319:07:413
305602268234,22sleep00-21swapper/019:05:300
306692258225,21sleep10-21swapper/119:06:551
74712660,0sleep20-21swapper/223:04:182
154382630,0sleep00-21swapper/022:24:440
34052620,0sleep10-21swapper/100:08:151
272932590,0sleep30-21swapper/300:02:093
271942580,0sleep00-21swapper/022:32:240
231822550,0sleep30-21swapper/323:37:133
115432530,0sleep30-21swapper/321:12:023
106942370,0sleep20-21swapper/221:35:022
86932190,0sleep00-21swapper/023:50:000
32652190,0sleep00-21swapper/021:52:240
60362180,0sleep00-21swapper/022:40:210
203172150,0sleep30-21swapper/323:13:043
163142140,0sleep00-21swapper/023:32:510
48262130,0sleep30-21swapper/300:09:343
3104399120,3cyclictest27294-21date22:10:012
296092120,0sleep30-21swapper/321:25:153
109492120,0sleep20-21swapper/221:58:032
3103699119,1cyclictest12-21ksoftirqd/021:40:010
3104399109,1cyclictest28-21ksoftirqd/200:35:022
3104399106,3cyclictest25096-21apt-get00:25:112
3104399106,3cyclictest25096-21apt-get00:25:112
3104399105,4cyclictest19187-21apt-get23:35:022
3103699109,1cyclictest12-21ksoftirqd/000:00:000
310479994,3cyclictest19982-21systemd-journal20:50:123
310479994,3cyclictest19982-21systemd-journal20:06:443
310439994,3cyclictest19982-21systemd-journal20:18:012
310439993,4cyclictest19982-21systemd-journal20:13:252
310439991,5cyclictest6613-21wc00:10:152
310439990,6cyclictest31636-21ssh23:20:252
310439990,6cyclictest21611-21ssh21:19:362
310439990,6cyclictest16464-21ssh22:01:402
310439990,6cyclictest14754-21ssh23:53:552
310439990,2cyclictest0-21swapper/200:20:002
310369994,3cyclictest19982-21systemd-journal20:59:190
310369994,3cyclictest19982-21systemd-journal00:33:210
310369990,8cyclictest5691-21ssh00:35:060
310479986,1cyclictest34-21ksoftirqd/300:35:003
310479985,2cyclictest34-21ksoftirqd/300:15:013
310479984,3cyclictest19982-21systemd-journal22:17:203
310479983,3cyclictest19982-21systemd-journal22:48:133
310479983,3cyclictest19982-21systemd-journal20:45:083
310479982,3cyclictest19982-21systemd-journal19:55:053
310439986,1cyclictest31870-21apt-get21:50:142
310439985,2cyclictest26441-21grep21:45:192
310439984,3cyclictest4243-21smartctl21:00:202
310439984,3cyclictest3029-21idleruntime-cro19:40:002
310439984,3cyclictest19982-21systemd-journal20:52:372
310439983,4cyclictest7250-21wc21:10:002
310439983,4cyclictest434-21cron19:20:012
310439983,3cyclictest19982-21systemd-journal20:09:402
310439982,5cyclictest27300-21ssh23:40:112
310439982,4cyclictest0-21swapper/222:47:412
310439982,4cyclictest0-21swapper/222:21:292
310439980,6cyclictest7999-21sh23:49:292
310439980,6cyclictest13384-21ssh21:13:422
310439980,5cyclictest28447-21taskset20:44:552
310439980,5cyclictest28447-21taskset20:44:552
310439980,5cyclictest21526-21ssh23:14:122
310439980,5cyclictest12119-21ssh23:06:512
310439980,4cyclictest508-21dbus-daemon20:55:332
310439980,4cyclictest27240-21ssh00:02:052
310439980,3cyclictest32125-21perl00:30:182
310439980,3cyclictest19210-21cstates00:20:112
310399984,3cyclictest19982-21systemd-journal00:12:281
310399983,3cyclictest19982-21systemd-journal22:35:321
310399983,3cyclictest19982-21systemd-journal22:35:321
310399983,3cyclictest19982-21systemd-journal20:40:141
310399983,3cyclictest19982-21systemd-journal20:40:131
310399983,3cyclictest19982-21systemd-journal20:25:011
310399981,1cyclictest31413-21idleruntime-cro21:50:011
310369987,1cyclictest12-21ksoftirqd/020:25:000
310369986,1cyclictest1403-21ssh22:14:360
310369984,3cyclictest19982-21systemd-journal19:32:320
310369980,7cyclictest7227-21diskmemload22:15:200
310369980,7cyclictest0-21swapper/023:35:120
310479976,1cyclictest34-21ksoftirqd/322:45:003
310479975,1cyclictest11290-21cron00:40:003
310479974,2cyclictest508-21dbus-daemon23:16:103
310479974,2cyclictest34-21ksoftirqd/319:25:013
310479973,3cyclictest508-21dbus-daemon22:07:453
310479973,3cyclictest32100-21tune2fs19:15:013
310479973,3cyclictest30654-21snmpd20:37:063
310479973,3cyclictest19982-21systemd-journal22:39:023
310479973,3cyclictest19982-21systemd-journal22:39:013
310479973,3cyclictest19982-21systemd-journal21:06:483
310479973,3cyclictest12831-21gdbus00:15:013
310479973,3cyclictest11736-21in:imuxsock21:59:233
310479973,2cyclictest19982-21systemd-journal22:20:413
310479973,1cyclictest0-21swapper/323:31:143
310479973,1cyclictest0-21swapper/322:59:353
310479972,3cyclictest19982-21systemd-journal22:10:073
310479971,5cyclictest0-21swapper/321:54:233
310479970,5cyclictest7661-21lldpd21:02:103
310479970,2cyclictest1-21systemd22:04:243
310439975,1cyclictest27638-21ssh22:10:102
310439974,2cyclictest2399-21latency_hist19:35:012
310439974,2cyclictest22463-21apt21:20:102
310439974,2cyclictest19522-21apt20:25:012
310439973,3cyclictest8863-21expr20:00:162
310439973,3cyclictest8786-21ls22:20:012
310439973,3cyclictest6825-21fschecks_count19:55:082
310439973,3cyclictest5071-21sort23:25:162
310439973,3cyclictest31906-21latency_hist19:15:012
310439973,3cyclictest30654-21snmpd21:30:122
310439973,3cyclictest23965-21sed23:15:182
310439973,3cyclictest23665-21ssh22:30:132
310439973,3cyclictest22996-21uname20:30:212
310439973,3cyclictest21218-21sshd22:50:322
310439973,3cyclictest11499-21gpg-connect-age23:30:012
310439973,3cyclictest0-21swapper/219:50:172
310439972,3cyclictest0-21swapper/221:41:262
310439971,5cyclictest16645-21cut22:25:122
310439971,4cyclictest0-21swapper/222:55:102
310439971,3cyclictest21758-21dump-pmu-power20:30:012
310439971,3cyclictest0-21swapper/221:27:392
310439971,3cyclictest0-21swapper/200:05:252
310439970,5cyclictest7268-21ssh22:40:562
310439970,5cyclictest18400-21ssh23:55:392
310439970,4cyclictest0-21swapper/222:35:132
310439970,4cyclictest0-21swapper/222:35:132
310399974,2cyclictest19982-21systemd-journal21:27:151
310399973,3cyclictest30654-21snmpd22:18:181
310399973,3cyclictest19982-21systemd-journal22:03:341
310399973,3cyclictest19982-21systemd-journal19:43:221
310399973,1cyclictest0-21swapper/123:51:071
310399973,1cyclictest0-21swapper/122:25:301
310399973,1cyclictest0-21swapper/122:10:041
310399973,1cyclictest0-21swapper/122:08:391
310399973,1cyclictest0-21swapper/121:43:451
310399973,1cyclictest0-21swapper/100:04:171
310399972,3cyclictest19982-21systemd-journal23:15:011
310399971,3cyclictest15312-21munin-run23:09:591
310399970,3cyclictest3065450snmpd22:49:071
310369975,1cyclictest12-21ksoftirqd/021:09:590
310369974,2cyclictest11738-21rs:main0
310369974,2cyclictest11738-21rs:main0
310369973,3cyclictest508-21dbus-daemon21:15:340
310369973,3cyclictest30654-21snmpd21:00:220
310369973,3cyclictest0-21swapper/022:29:020
310369973,3cyclictest0-21swapper/022:09:490
310369973,3cyclictest0-21swapper/021:34:030
310369973,2cyclictest19982-21systemd-journal19:54:390
310369973,1cyclictest0-21swapper/022:55:250
310369973,1cyclictest0-21swapper/000:21:220
310369972,3cyclictest19982-21systemd-journal23:10:370
310369972,3cyclictest19982-21systemd-journal19:58:240
310369970,3cyclictest29260-21sshd20:45:140
310479964,1cyclictest16536-21smart_sda20:15:213
310479963,2cyclictest668-21runrttasks19:26:433
310479963,2cyclictest508-21dbus-daemon19:40:033
310479963,2cyclictest34-21ksoftirqd/323:00:153
310479963,2cyclictest34-21ksoftirqd/319:15:013
310479963,2cyclictest3129-21swap23:45:243
310479963,2cyclictest30408-21gltestperf23:20:123
310479963,2cyclictest24362-21cut22:30:193
310479963,2cyclictest23293-21rm22:52:243
310479963,2cyclictest19982-21systemd-journal22:26:313
310479963,2cyclictest18256-21cat20:20:183
310479963,2cyclictest1357-21hddtemp_smartct20:55:123
310479963,1cyclictest0-21swapper/323:56:043
310479963,1cyclictest0-21swapper/321:15:093
310479962,3cyclictest9462-21cat23:05:153
310479962,3cyclictest4969-21cat23:25:153
310479962,3cyclictest30654-21snmpd20:11:073
310479962,3cyclictest30654-21snmpd20:04:373
310479962,3cyclictest30654-21snmpd19:45:083
310479962,3cyclictest30654-21snmpd19:35:463
310479962,3cyclictest30654-21snmpd00:26:403
310479962,3cyclictest30654-21snmpd00:26:393
310479962,3cyclictest27179-21cat20:40:163
310479962,3cyclictest27179-21cat20:40:163
310479962,3cyclictest26713-21apt23:40:083
310479962,3cyclictest25515-21cat21:45:133
310479962,3cyclictest22355-21cat20:30:173
310479962,3cyclictest19982-21systemd-journal19:52:583
310479962,3cyclictest19194-21grep21:40:183
310479962,1cyclictest0-21swapper/323:51:393
310479961,3cyclictest19982-21systemd-journal21:30:203
310479960,6cyclictest0-21swapper/300:21:213
310479960,0cyclictest0-21swapper/321:21:243
310439962,3cyclictest4048-21chrt19:46:322
310439962,3cyclictest30654-21snmpd19:26:282
310439962,3cyclictest28722-21/usr/sbin/munin20:45:112
310439961,3cyclictest19982-21systemd-journal20:35:112
310439961,3cyclictest19982-21systemd-journal19:23:472
310439960,2cyclictest3366-21taskset19:40:572
310399964,1cyclictest2421-21hddtemp_smartct23:45:201
310399964,1cyclictest18750irq/134-ahci[0020:25:111
310399963,2cyclictest9186-21proc_pri21:10:191
310399963,2cyclictest508-21dbus-daemon00:20:061
310399963,2cyclictest4861-21chrt19:51:001
310399963,2cyclictest309402sleep120:50:151
310399963,2cyclictest23984-21/usr/sbin/munin20:35:001
310399963,2cyclictest17416-21ntpq23:55:161
310399963,2cyclictest1738-21cat23:00:121
310399963,2cyclictest17077-21awk23:10:181
310399963,1cyclictest0-21swapper/123:30:051
310399962,3cyclictest7895-21sed21:55:201
310399962,3cyclictest4218-21expr21:30:121
310399962,3cyclictest3802-21expr21:00:161
310399962,3cyclictest327-21seq19:19:031
310399962,3cyclictest30654-21snmpd23:26:181
310399962,3cyclictest30654-21snmpd22:41:161
310399962,3cyclictest30654-21snmpd21:50:241
310399962,3cyclictest30654-21snmpd21:38:051
310399962,3cyclictest30654-21snmpd20:15:051
310399962,3cyclictest30654-21snmpd20:00:411
310399962,3cyclictest30654-21snmpd19:25:541
310399962,3cyclictest29821-21apt-get23:20:111
310399962,3cyclictest14311-21sensors20:10:221
310399962,3cyclictest1256-21fschecks_time20:55:121
310399962,3cyclictest1-21systemd21:15:051
310399962,3cyclictest10256-21ntpq22:20:161
310399962,2cyclictest508-21dbus-daemon19:23:461
310399962,1cyclictest0-21swapper/123:39:281
310399962,1cyclictest0-21swapper/122:55:021
310399962,1cyclictest0-21swapper/100:26:471
310399962,1cyclictest0-21swapper/100:26:471
310399961,4cyclictest0-21swapper/120:45:131
310399961,3cyclictest508-21dbus-daemon23:42:021
310399961,3cyclictest19982-21systemd-journal22:30:151
310399961,3cyclictest19982-21systemd-journal20:38:261
310399961,2cyclictest22005-21ssh22:51:131
310399960,6cyclictest0-21swapper/100:31:531
310399960,2cyclictest6367-21/usr/sbin/munin19:55:551
310399960,2cyclictest12994-21/usr/sbin/munin00:15:121
310369963,2cyclictest643-21ssh00:05:430
310369963,2cyclictest5979-21cron21:55:000
310369963,2cyclictest5164-21ntpd00:01:230
310369963,2cyclictest508-21dbus-daemon20:53:030
310369963,2cyclictest4727-21grep23:25:150
310369963,2cyclictest32290-21rm21:26:310
310369963,2cyclictest30654-21snmpd20:41:480
310369963,2cyclictest30654-21snmpd20:41:480
310369963,2cyclictest25204-21awk20:35:170
310369963,2cyclictest22627-21ssh22:51:460
310369963,2cyclictest1720-21cat19:29:560
310369963,2cyclictest13380-21cat22:45:190
310369963,2cyclictest0-21swapper/000:10:550
310369962,3cyclictest508-21dbus-daemon19:48:230
310369962,3cyclictest30654-21snmpd23:00:500
310369962,3cyclictest30654-21snmpd21:05:120
310369962,3cyclictest30654-21snmpd20:13:590
310369962,3cyclictest30654-21snmpd20:07:590
310369962,3cyclictest30654-21snmpd19:21:310
310369962,3cyclictest30654-21snmpd00:17:130
310369962,3cyclictest22727-21irqstats20:30:180
310369962,3cyclictest20701-21cut20:25:160
310369962,3cyclictest14311-21cat22:00:170
310369962,3cyclictest10304-21sed23:05:200
310369962,3cyclictest0-21swapper/023:50:560
310369962,3cyclictest0-21swapper/023:18:520
310369962,3cyclictest0-21swapper/021:24:280
310369962,1cyclictest0-21swapper/021:45:590
310369961,3cyclictest19982-21systemd-journal23:21:100
310369960,3cyclictest30350-21rm23:42:010
310369960,0cyclictest0-21swapper/021:40:060
310369960,0cyclictest0-21swapper/000:25:140
310369960,0cyclictest0-21swapper/000:25:140
310479953,1cyclictest10639-21sh21:35:003
310479952,2cyclictest19869-21cpuspeed20:25:133
310479951,3cyclictest29741-11brltty19:34:123
310399953,1cyclictest11469-21hddtemp_smartct20:05:151
310399953,1cyclictest0-21swapper/119:45:041
310399952,2cyclictest5457-21cmp00:35:011
310399952,2cyclictest30654-21snmpd19:10:061
310399951,3cyclictest7227-21diskmemload21:20:021
310399951,3cyclictest5447-21/usr/sbin/munin21:05:141
310399951,3cyclictest508-21dbus-daemon19:35:511
310399951,2cyclictest19982-21systemd-journal19:32:311
310369952,2cyclictest32589-21taskset19:16:220
310369952,2cyclictest31314-21taskset19:11:220
310369952,2cyclictest15684-21expr20:15:140
310369951,3cyclictest30654-21snmpd19:40:460
310369950,3cyclictest19982-21systemd-journal20:00:580
310369941,2cyclictest29741-11brltty19:35:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional