You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 16:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Sat Nov 29, 2025 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
254122305277,17sleep20-21swapper/207:05:452
255392271236,22sleep30-21swapper/307:07:223
255462266233,21sleep00-21swapper/007:07:280
254062257224,21sleep10-21swapper/107:05:401
41942640,0sleep10-21swapper/107:30:151
152332600,0sleep228-21ksoftirqd/212:05:212
30032570,3sleep228-21ksoftirqd/210:15:212
184252570,0sleep30-21swapper/309:10:583
32742550,0sleep30-21swapper/311:06:083
19422550,0sleep00-21swapper/012:39:340
134062390,0sleep10-21swapper/110:24:181
277082170,0sleep1211rcuc/112:35:151
240952170,0sleep20-21swapper/211:48:592
169712140,0sleep30-21swapper/310:26:153
123862120,0sleep10-21swapper/111:14:271
257052110,0chrt0-21swapper/111:50:101
307222100,0sleep00-21swapper/011:02:450
2589599105,3cyclictest19982-21systemd-journal09:47:360
2589599104,3cyclictest32402-21munin-run11:55:010
259099994,3cyclictest19982-21systemd-journal07:29:053
259019993,3cyclictest28118-21munin-run12:15:012
258999995,2cyclictest19982-21systemd-journal07:24:041
258999990,7cyclictest19982-21systemd-journal12:13:381
258959995,3cyclictest508-21dbus-daemon07:18:390
32480280,0sleep20-21swapper/210:39:092
26105280,0sleep20-21swapper/210:08:532
259099984,3cyclictest19982-21systemd-journal08:12:543
259099984,2cyclictest19982-21systemd-journal08:54:583
259019984,3cyclictest19982-21systemd-journal12:35:192
259019983,3cyclictest19982-21systemd-journal11:21:292
259019983,3cyclictest19982-21systemd-journal11:09:222
259019982,3cyclictest4464-21awk08:45:012
259019982,3cyclictest11206-21munin-run09:00:002
258999984,3cyclictest508-21dbus-daemon07:16:341
258999983,3cyclictest502-21systemd-logind09:24:401
258959984,3cyclictest508-21dbus-daemon10:32:400
258959983,3cyclictest7661-21lldpd11:10:080
258959983,3cyclictest19982-21systemd-journal10:38:550
258959983,3cyclictest19982-21systemd-journal09:07:330
11836280,0sleep00-21swapper/012:25:210
8237270,1sleep325887-21cyclictest07:40:133
259099974,2cyclictest34-21ksoftirqd/307:20:023
259099974,2cyclictest29455-21sshd09:20:023
259099974,2cyclictest16745-21cron08:00:013
259099973,3cyclictest508-21dbus-daemon11:40:013
259099973,3cyclictest508-21dbus-daemon09:55:013
259099973,3cyclictest30749-21sshd12:15:453
259099973,3cyclictest30654-21snmpd12:21:173
259099973,3cyclictest30654-21snmpd11:50:333
259099973,3cyclictest30654-21snmpd07:54:483
259099973,3cyclictest28832-21users11:00:223
259099973,3cyclictest25002-21cat08:15:203
259099973,3cyclictest21297-21cron09:40:023
259099973,3cyclictest19982-21systemd-journal11:41:313
259099973,3cyclictest19982-21systemd-journal10:10:443
259099973,3cyclictest19982-21systemd-journal10:10:003
259099973,3cyclictest19904-21diskstats11:20:143
259099973,3cyclictest19019-21grep12:30:163
259099973,3cyclictest0-21swapper/311:33:363
259099973,3cyclictest0-21swapper/310:18:593
259099973,2cyclictest19982-21systemd-journal08:39:163
259099973,1cyclictest0-21swapper/312:37:213
259099973,1cyclictest0-21swapper/310:00:063
259099973,1cyclictest0-21swapper/309:56:393
259099972,3cyclictest19982-21systemd-journal09:15:293
259099972,3cyclictest19982-21systemd-journal09:15:293
259099972,3cyclictest19982-21systemd-journal09:01:183
259019974,2cyclictest30654-21snmpd09:55:182
259019973,3cyclictest7682-21cut11:10:172
259019973,3cyclictest508-21dbus-daemon08:02:522
259019973,3cyclictest30654-21snmpd12:02:582
259019973,3cyclictest2227-21awk08:40:012
259019973,3cyclictest19982-21systemd-journal09:50:342
259019973,3cyclictest0-21swapper/210:22:242
259019973,3cyclictest0-21swapper/210:10:222
259019973,2cyclictest19982-21systemd-journal07:19:542
259019973,1cyclictest0-21swapper/212:26:182
259019973,1cyclictest0-21swapper/209:14:502
259019972,3cyclictest7661-21lldpd10:55:382
259019970,4cyclictest15205-21munin-run09:35:012
259019970,4cyclictest12316-21dump-pmu-power07:50:012
258999976,1cyclictest0-21swapper/111:59:481
258999974,2cyclictest508-21dbus-daemon08:33:461
258999974,2cyclictest508-21dbus-daemon07:48:171
258999974,2cyclictest11736-21in:imuxsock09:25:551
258999973,3cyclictest19982-21systemd-journal10:15:011
258999973,3cyclictest0-21swapper/111:30:131
258999973,3cyclictest0-21swapper/110:25:241
258999973,3cyclictest0-21swapper/109:12:271
258999973,1cyclictest0-21swapper/111:40:501
258999973,1cyclictest0-21swapper/109:55:021
258999972,3cyclictest19982-21systemd-journal09:15:291
258999972,3cyclictest19982-21systemd-journal09:15:291
258999972,3cyclictest19982-21systemd-journal09:08:471
258999971,3cyclictest7661-21lldpd12:08:561
258999970,5cyclictest19982-21systemd-journal09:32:351
258959976,1cyclictest12-21ksoftirqd/009:20:000
258959974,2cyclictest508-21dbus-daemon08:45:260
258959974,2cyclictest11736-21in:imuxsock11:55:170
258959973,3cyclictest6587-21awk11:35:120
258959973,3cyclictest508-21dbus-daemon09:27:090
258959973,3cyclictest4938-21sshd09:52:190
258959973,3cyclictest30654-21snmpd10:55:450
258959973,3cyclictest30654-21snmpd10:40:250
258959973,3cyclictest30654-21snmpd07:31:470
258959973,3cyclictest14730-21df10:50:150
258959973,3cyclictest13568-21sshd09:33:170
258959973,3cyclictest0-21swapper/011:27:500
258959973,1cyclictest0-21swapper/012:08:200
258959973,1cyclictest0-21swapper/010:14:560
258959972,3cyclictest19982-21systemd-journal11:30:140
258959972,3cyclictest19982-21systemd-journal11:06:530
258959970,3cyclictest27951-21munin-run08:25:010
258959970,3cyclictest14777-21/usr/sbin/munin07:55:170
23430270,0sleep10-21swapper/110:06:111
16049270,0sleep00-21swapper/011:16:260
2790260,0sleep30-21swapper/310:40:223
259099966,0cyclictest0-21swapper/311:45:383
259099964,1cyclictest11736-21in:imuxsock08:20:143
259099963,2cyclictest9910-21vmstat10:20:223
259099963,2cyclictest909-21fschecks_count11:55:153
259099963,2cyclictest7799-21chrt10:45:063
259099963,2cyclictest7518-21ssh12:00:093
259099963,2cyclictest7166-21ls11:10:143
259099963,2cyclictest30950-21gltestperf08:30:143
259099963,2cyclictest28957-21ssh09:45:183
259099963,2cyclictest1-21systemd12:05:173
259099963,2cyclictest0-21swapper/307:10:213
259099962,3cyclictest508-21dbus-daemon07:25:013
259099962,3cyclictest3205-21apt09:25:113
259099962,3cyclictest30654-21snmpd10:30:313
259099962,3cyclictest30654-21snmpd09:40:313
259099962,3cyclictest30654-21snmpd08:55:043
259099962,3cyclictest30654-21snmpd08:45:283
259099962,3cyclictest30654-21snmpd08:27:183
259099962,3cyclictest30654-21snmpd07:46:383
259099962,3cyclictest30654-21snmpd07:30:133
259099962,3cyclictest17344-21cat08:00:153
259099962,3cyclictest15251-21cat10:50:173
259099962,3cyclictest10720-21expr12:25:153
259099962,1cyclictest0-21swapper/312:13:113
259099962,1cyclictest0-21swapper/310:55:243
259099961,4cyclictest7846-21turbostat07:40:013
259099961,3cyclictest19982-21systemd-journal10:36:013
259099961,3cyclictest19982-21systemd-journal08:05:113
259099960,6cyclictest0-21swapper/311:17:443
259099960,5cyclictest13683-21/usr/sbin/munin09:05:173
259099960,4cyclictest19982-21systemd-journal11:29:003
259099960,4cyclictest19982-21systemd-journal08:42:063
259099960,0cyclictest0-21swapper/309:31:523
259019966,0cyclictest0-21swapper/212:32:572
259019964,1cyclictest11736-21in:imuxsock08:50:012
259019963,2cyclictest8969-21latency_hist08:55:002
259019963,2cyclictest668-21runrttasks08:19:082
259019963,2cyclictest668-21runrttasks07:28:252
259019963,2cyclictest6566-21grep07:35:202
259019963,2cyclictest4071-21cat12:20:172
259019963,2cyclictest32309-21ssh09:21:492
259019963,2cyclictest30654-21snmpd08:10:022
259019963,2cyclictest30533-21ssh09:46:352
259019963,2cyclictest28518-21ssh12:15:142
259019963,2cyclictest17165-21cat09:35:222
259019963,2cyclictest0-21swapper/211:15:162
259019962,3cyclictest7907-21smart_sda11:35:192
259019962,3cyclictest3767-21grep09:25:152
259019962,3cyclictest30654-21snmpd11:56:152
259019962,3cyclictest30654-21snmpd11:41:422
259019962,3cyclictest30654-21snmpd11:30:102
259019962,3cyclictest30654-21snmpd11:26:252
259019962,3cyclictest30654-21snmpd10:52:532
259019962,3cyclictest30654-21snmpd08:25:032
259019962,3cyclictest30654-21snmpd08:24:482
259019962,3cyclictest30654-21snmpd08:07:592
259019962,3cyclictest30654-21snmpd07:21:132
259019962,3cyclictest30449-21seq08:30:122
259019962,3cyclictest27867-21cat11:00:162
259019962,3cyclictest19982-21systemd-journal10:01:222
259019962,3cyclictest14704-21sshd09:05:202
259019962,3cyclictest12548-21sshd09:00:192
259019962,3cyclictest0-21swapper/210:46:442
259019962,1cyclictest0-21swapper/210:41:022
259019962,1cyclictest0-21swapper/210:31:012
259019962,1cyclictest0-21swapper/209:15:222
259019962,1cyclictest0-21swapper/209:15:222
259019960,6cyclictest0-21swapper/210:25:502
259019960,4cyclictest19982-21systemd-journal09:42:362
258999964,1cyclictest3477-21sshd08:40:181
258999964,1cyclictest27756-21grep11:00:161
258999963,2cyclictest668-21runrttasks07:59:321
258999963,2cyclictest508-21dbus-daemon10:55:111
258999963,2cyclictest508-21dbus-daemon08:55:261
258999963,2cyclictest3300-21ssh12:20:111
258999963,2cyclictest15365-21ntpq10:50:181
258999963,2cyclictest15244-21rm11:15:381
258999963,2cyclictest11736-21in:imuxsock08:17:031
258999962,3cyclictest6749-21munin-run08:50:001
258999962,3cyclictest508-21dbus-daemon07:43:171
258999962,3cyclictest30654-21snmpd12:01:261
258999962,3cyclictest29276-21cat12:15:171
258999962,3cyclictest29095-21awk10:35:221
258999962,3cyclictest19982-21systemd-journal09:35:031
258999962,3cyclictest15277-21cat10:00:161
258999962,3cyclictest11770-21expr09:00:141
258999962,3cyclictest1014-21ls08:35:161
258999962,3cyclictest0-21swapper/110:45:411
258999962,3cyclictest0-21swapper/110:30:561
258999962,3cyclictest0-21swapper/109:43:201
258999962,2cyclictest8953-1kworker/u9:111:35:131
258999962,1cyclictest0-21swapper/112:31:081
258999962,1cyclictest0-21swapper/112:27:371
258999962,1cyclictest0-21swapper/111:08:111
258999962,1cyclictest0-21swapper/110:15:501
258999962,1cyclictest0-21swapper/109:53:521
258999961,3cyclictest1347-21rm10:40:131
258999960,6cyclictest0-21swapper/109:46:461
258999960,5cyclictest21511-21sshd08:10:141
258999960,3cyclictest21706-21rm11:46:051
258999960,3cyclictest21371-21rm11:20:341
258959964,1cyclictest9808-21grep08:55:150
258959963,2cyclictest7828-21sshd09:55:080
258959963,2cyclictest5406-21sh12:20:430
258959963,2cyclictest30654-21snmpd07:20:080
258959963,2cyclictest23341-21ssh11:22:450
258959963,2cyclictest19895-21cut11:45:170
258959963,2cyclictest18312-21sshd11:44:210
258959963,2cyclictest17295-21expr08:00:140
258959963,2cyclictest15465-21sshd09:35:030
258959963,2cyclictest15193-21if_enp4s010:25:170
258959963,2cyclictest15122-21tr10:00:160
258959963,2cyclictest12305-21turbostat.cron07:50:000
258959963,2cyclictest11764-21sh10:22:340
258959962,3cyclictest508-21dbus-daemon08:44:360
258959962,3cyclictest30654-21snmpd12:15:430
258959962,3cyclictest30654-21snmpd10:46:460
258959962,3cyclictest30654-21snmpd09:40:180
258959962,3cyclictest30654-21snmpd08:36:090
258959962,3cyclictest28876-21cat08:25:170
258959962,3cyclictest22373-21uname08:10:180
258959962,3cyclictest21854-21if_enp4s012:10:160
258959962,3cyclictest19185-21cat12:30:160
258959962,3cyclictest13456-21pluginstate09:05:010
258959962,1cyclictest32425-21turbostat08:35:000
258959962,1cyclictest0-21swapper/010:06:140
258959961,2cyclictest17261-21/usr/sbin/munin09:10:190
258959961,2cyclictest12015-21ssh12:03:430
258959960,3cyclictest1696-21/usr/sbin/munin10:15:210
259019953,1cyclictest12549-21sshd07:50:102
259019953,1cyclictest0-21swapper/207:55:212
259019952,2cyclictest8122-21awk07:40:132
259019952,2cyclictest26688-21needreboot11:50:162
259019951,3cyclictest3853-21cat07:30:122
259019951,3cyclictest26249-21expr07:10:142
258999952,2cyclictest6454-21cat07:35:201
258999952,2cyclictest26188-21wc07:10:131
258999952,2cyclictest25985-21apt08:20:121
258999952,2cyclictest25913-21mktemp11:25:011
258999952,2cyclictest19574-21expr08:05:141
258999952,2cyclictest1735-21grep07:25:151
258999951,3cyclictest29741-11brltty08:52:561
258999951,3cyclictest17724-21cat08:00:161
258999951,3cyclictest13148-21sed07:50:161
258999950,3cyclictest28183-21sshd08:25:101
258959953,1cyclictest6988-21cat08:50:120
258959953,1cyclictest20603-21timerandwakeup08:05:210
258959952,2cyclictest8710-21gltestperf07:40:150
258959952,2cyclictest1419-21cpuspeed_turbos07:25:140
258959952,2cyclictest13362-21date07:50:170
258959951,3cyclictest6309-21grep07:35:180
258959951,3cyclictest30654-21snmpd09:15:020
258959951,3cyclictest30654-21snmpd09:15:020
258959951,3cyclictest26323-21expr07:10:130
258959950,4cyclictest24228-21sshd08:15:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional