You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-14 - 00:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Thu Nov 13, 2025 12:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
287502305271,21sleep20-21swapper/207:08:272
288342288253,23sleep10-21swapper/107:09:301
288332256224,20sleep00-21swapper/007:09:290
285762255221,22sleep30-21swapper/307:06:093
102712710,0sleep20-21swapper/211:12:512
4152680,0sleep30-21swapper/312:21:273
314062660,0sleep20-21swapper/210:38:162
287712600,0sleep10-21swapper/112:19:141
282462550,0sleep30-21swapper/309:30:003
147552530,0sleep119099-21diskmemload12:32:021
26672400,0sleep00-21swapper/012:23:320
28975993634,1cyclictest14727-21lspci09:59:210
212482190,0sleep20-21swapper/210:30:152
228792180,0sleep10-21swapper/111:23:001
219812150,0sleep00-21swapper/012:37:380
198062150,1sleep219807-21pmu-power11:45:182
131132150,2sleep213109-21latency_hist07:45:012
40952100,0sleep10-21swapper/112:24:541
2898399100,7cyclictest6962-21ssh12:26:012
289879993,5cyclictest0-21swapper/309:59:323
289839991,4cyclictest5461-1kworker/u9:010:25:062
289839990,6cyclictest9835-21ssh10:46:282
289839990,6cyclictest7044-21ssh09:53:422
289839990,6cyclictest5621-21ssh11:09:392
289839990,6cyclictest3729-21ssh12:24:342
289839990,6cyclictest17056-21sh10:52:242
289839990,6cyclictest14793-21ssh11:41:302
289839990,6cyclictest10806-21sh09:55:512
289839990,5cyclictest27186-21ssh12:17:382
289839990,4cyclictest3541-21rm11:33:212
289839990,4cyclictest25272-21dpkg11:50:112
289839990,4cyclictest21378-21smart_sda08:00:202
289839990,4cyclictest19935-21cat10:55:142
289839990,4cyclictest18722-21expr12:35:152
289839990,4cyclictest17562-21ssh12:34:382
289839990,4cyclictest16777-21sh11:18:132
289839990,4cyclictest11738-21rs:main2
289839990,4cyclictest11738-21rs:main2
289839990,4cyclictest0-21swapper/211:01:232
289839990,3cyclictest18775-21sh11:20:112
289759996,2cyclictest12-21ksoftirqd/008:40:010
289759995,3cyclictest502-21systemd-logind11:54:170
289879985,2cyclictest508-21dbus-daemon08:20:413
289879984,3cyclictest19982-21systemd-journal09:22:253
289879983,3cyclictest7661-21lldpd11:14:093
289879983,3cyclictest19982-21systemd-journal12:09:543
289879983,3cyclictest19982-21systemd-journal12:04:413
289879983,3cyclictest19982-21systemd-journal10:38:183
289879983,3cyclictest19982-21systemd-journal08:53:373
289879983,3cyclictest19982-21systemd-journal07:21:013
289879980,7cyclictest29741-11brltty09:37:253
289839983,3cyclictest19982-21systemd-journal09:40:422
289839983,3cyclictest19982-21systemd-journal08:48:122
289839981,3cyclictest4436-21ssh10:16:172
289839980,6cyclictest8148-21ssh12:01:452
289839980,5cyclictest7484-21sh11:35:392
289839980,5cyclictest28435-21ssh11:27:242
289839980,5cyclictest28435-21ssh11:27:242
289839980,5cyclictest24751-21ssh10:06:292
289839980,5cyclictest16113-21taskset10:00:012
289839980,5cyclictest1310-21ssh11:56:122
289839980,4cyclictest668-21runrttasks08:50:162
289839980,4cyclictest17562-21ssh12:10:102
289839980,3cyclictest29680-21grep10:10:192
289839980,3cyclictest25871-21sed08:10:202
289839980,3cyclictest19099-21diskmemload10:44:162
289839980,3cyclictest10694-21sh10:21:062
289799984,3cyclictest508-21dbus-daemon07:21:501
289799984,3cyclictest19982-21systemd-journal09:12:191
289759984,3cyclictest12098-21sshd08:51:530
289759983,3cyclictest19982-21systemd-journal11:12:420
289759983,1cyclictest0-21swapper/010:15:170
289759982,3cyclictest24581-21apt-get11:50:010
289879976,1cyclictest18477-21apt-get12:35:143
289879975,1cyclictest22304-21apt-get10:05:163
289879974,2cyclictest8317-21cron07:35:013
289879974,2cyclictest508-21dbus-daemon09:00:003
289879974,2cyclictest25152-21gltestperf08:10:153
289879974,2cyclictest11736-21in:imuxsock08:06:293
289879973,3cyclictest508-21dbus-daemon11:36:203
289879973,3cyclictest508-21dbus-daemon10:10:243
289879973,3cyclictest19982-21systemd-journal10:28:023
289879973,1cyclictest0-21swapper/311:41:183
289879973,1cyclictest0-21swapper/311:34:483
289879973,1cyclictest0-21swapper/311:26:373
289879973,1cyclictest0-21swapper/311:26:373
289879973,1cyclictest0-21swapper/310:04:433
289839973,3cyclictest508-21dbus-daemon09:21:592
289839973,3cyclictest26062-21apt-get09:25:132
289839973,3cyclictest26062-21apt-get09:25:132
289839973,3cyclictest19982-21systemd-journal08:24:002
289839972,3cyclictest19982-21systemd-journal09:39:322
289839972,3cyclictest19982-21systemd-journal07:55:142
289839971,4cyclictest3428-21sh08:35:002
289839970,5cyclictest24129-21taskset08:07:102
289839970,5cyclictest13777-21ssh12:06:142
289839970,3cyclictest12808-21expr08:55:132
289839970,2cyclictest1-21systemd08:36:062
289799974,2cyclictest502-21systemd-logind12:07:381
289799974,2cyclictest11736-21in:imuxsock09:40:021
289799973,3cyclictest508-21dbus-daemon09:57:531
289799973,3cyclictest508-21dbus-daemon09:10:001
289799973,3cyclictest30654-21snmpd10:20:091
289799973,3cyclictest2367-21cut10:15:171
289799973,3cyclictest19982-21systemd-journal11:45:551
289799973,3cyclictest19982-21systemd-journal11:28:421
289799973,3cyclictest19982-21systemd-journal11:28:411
289799973,3cyclictest19982-21systemd-journal10:42:311
289799973,3cyclictest19982-21systemd-journal09:42:421
289799973,1cyclictest0-21swapper/111:55:001
289799973,1cyclictest0-21swapper/110:34:591
289799972,4cyclictest0-21swapper/110:35:141
289799971,1cyclictest19982-21systemd-journal07:13:301
289759974,2cyclictest508-21dbus-daemon11:55:210
289759974,2cyclictest508-21dbus-daemon09:37:250
289759973,3cyclictest32041-21cron11:05:010
289759973,3cyclictest30654-21snmpd12:10:160
289759973,3cyclictest29741-11brltty10:07:550
289759973,3cyclictest27110-21packagekitd08:15:000
289759973,3cyclictest2410-21ntpq08:30:190
289759973,3cyclictest19982-21systemd-journal09:50:010
289759973,3cyclictest1-21systemd08:20:060
289759973,3cyclictest0-21swapper/012:05:200
289759973,2cyclictest19982-21systemd-journal12:02:120
289759973,1cyclictest0-21swapper/012:33:190
289759973,1cyclictest0-21swapper/010:29:450
289759972,3cyclictest19982-21systemd-journal10:32:550
289759972,3cyclictest19982-21systemd-journal10:10:500
289759972,3cyclictest19982-21systemd-journal07:34:520
21928270,0sleep10-21swapper/112:13:171
289879964,1cyclictest28981-21irqstats09:30:153
289879963,2cyclictest8898-21cut08:45:173
289879963,2cyclictest668-21runrttasks12:26:093
289879963,2cyclictest508-21dbus-daemon07:45:123
289879963,2cyclictest31524-21tr11:55:143
289879963,2cyclictest30654-21snmpd11:50:223
289879963,2cyclictest27706-21taskset09:26:223
289879963,2cyclictest19982-21systemd-journal07:15:023
289879963,2cyclictest19201-21uniq11:45:153
289879963,2cyclictest11736-21in:imuxsock09:46:413
289879963,2cyclictest11525-21cat12:30:133
289879963,2cyclictest0-21swapper/311:00:053
289879963,1cyclictest0-21swapper/310:45:463
289879962,3cyclictest9259-21cat07:35:163
289879962,3cyclictest6623-21cat08:40:173
289879962,3cyclictest5111-21cat08:35:213
289879962,3cyclictest30654-21snmpd10:57:553
289879962,3cyclictest30654-21snmpd10:50:043
289879962,3cyclictest30654-21snmpd10:40:083
289879962,3cyclictest30654-21snmpd09:52:203
289879962,3cyclictest29741-11brltty08:00:133
289879962,3cyclictest27003-21sed08:15:013
289879962,3cyclictest19982-21systemd-journal11:06:173
289879962,3cyclictest18420-21fw_packets07:55:133
289879962,3cyclictest18334-21cat12:10:153
289879962,3cyclictest0-21swapper/311:23:033
289879962,1cyclictest0-21swapper/311:15:263
289879961,4cyclictest0-21swapper/312:18:273
289879961,3cyclictest19982-21systemd-journal10:15:493
289879961,3cyclictest19982-21systemd-journal09:02:323
289879961,3cyclictest19982-21systemd-journal07:41:023
289879960,5cyclictest8248-21/usr/sbin/munin10:20:183
289879960,5cyclictest22173-21which09:15:153
289879960,3cyclictest21935-21grep10:30:183
289879960,2cyclictest4090-21/usr/sbin/munin07:25:133
289879960,2cyclictest4090-21/usr/sbin/munin07:25:133
289839966,0cyclictest0-21swapper/208:15:182
289839964,1cyclictest14426-21ntp_states07:45:172
289839963,2cyclictest508-21dbus-daemon09:47:532
289839962,3cyclictest9087-21fw_packets07:35:152
289839962,3cyclictest6927-21expr07:30:132
289839962,3cyclictest32623-21grep07:15:152
289839962,3cyclictest28455-21tr09:30:122
289839962,3cyclictest16700-21cut07:50:192
289839961,3cyclictest19982-21systemd-journal07:23:062
289839960,2cyclictest21571-21apt-get09:15:112
289799966,0cyclictest0-21swapper/109:30:461
289799965,1cyclictest22-21ksoftirqd/107:30:011
289799965,1cyclictest22-21ksoftirqd/107:30:011
289799964,1cyclictest7379-21unixbench-2d08:40:221
289799963,2cyclictest7416-21grep07:30:161
289799963,2cyclictest580-21fschecks_time11:05:161
289799963,2cyclictest25129-21grep08:10:141
289799963,2cyclictest22375-21/usr/sbin/munin08:05:181
289799963,2cyclictest20717-21gltestperf08:00:151
289799963,2cyclictest12172-21cstates11:40:121
289799963,2cyclictest11736-21in:imuxsock09:22:491
289799962,3cyclictest508-21dbus-daemon10:05:231
289799962,3cyclictest4265-21sed08:35:161
289799962,3cyclictest27546-21sh08:15:131
289799962,3cyclictest26828-21cat11:00:161
289799962,3cyclictest19982-21systemd-journal08:50:431
289799962,3cyclictest19982-21systemd-journal07:46:281
289799962,3cyclictest18670-21ssh10:27:531
289799962,3cyclictest16298-21date09:00:201
289799962,3cyclictest1-21systemd09:52:531
289799962,2cyclictest19982-21systemd-journal08:47:411
289799962,1cyclictest0-21swapper/112:35:301
289799962,1cyclictest0-21swapper/111:37:361
289799962,1cyclictest0-21swapper/111:31:401
289799962,1cyclictest0-21swapper/110:56:011
289799962,1cyclictest0-21swapper/110:52:291
289799962,1cyclictest0-21swapper/110:00:211
289799961,4cyclictest12720-21ssh11:15:131
289799961,3cyclictest19982-21systemd-journal11:50:011
289799961,3cyclictest19982-21systemd-journal09:46:121
289799961,3cyclictest19982-21systemd-journal07:38:581
289799961,1cyclictest19982-21systemd-journal12:27:381
289799960,6cyclictest0-21swapper/111:12:031
289799960,5cyclictest28339-21ssh10:10:091
289799960,2cyclictest6047-21memory12:00:161
289799960,0cyclictest0-21swapper/110:47:521
289799960,0cyclictest0-21swapper/109:26:111
289799960,0cyclictest0-21swapper/109:26:111
289759966,0cyclictest0-21swapper/009:30:200
289759963,2cyclictest8986-21sh10:45:330
289759963,2cyclictest508-21dbus-daemon08:55:180
289759963,2cyclictest508-21dbus-daemon08:28:360
289759963,2cyclictest508-21dbus-daemon08:07:190
289759963,2cyclictest508-21dbus-daemon07:38:330
289759963,2cyclictest30654-21snmpd11:35:340
289759963,2cyclictest30654-21snmpd10:22:030
289759963,2cyclictest27854-21sed10:35:160
289759963,2cyclictest27441-21cat09:25:210
289759963,2cyclictest27441-21cat09:25:210
289759963,2cyclictest19982-21systemd-journal10:58:100
289759963,2cyclictest17966-21date09:05:170
289759963,2cyclictest17220-21ssh11:18:450
289759963,2cyclictest16056-21cat07:50:140
289759963,2cyclictest15747-21needreboot09:00:170
289759963,2cyclictest12384-21fschecks_count11:40:130
289759963,2cyclictest11737-21in:imklog08:10:130
289759963,2cyclictest0-21swapper/012:25:120
289759962,3cyclictest32185-21cat11:30:140
289759962,3cyclictest29741-11brltty11:05:280
289759962,3cyclictest20357-21expr08:00:130
289759962,3cyclictest19589-21ls09:10:120
289759962,3cyclictest17258-21ntpq10:00:170
289759962,3cyclictest15109-21grep10:50:210
289759962,3cyclictest1213-21sshd07:15:280
289759962,3cyclictest11868-21ntp_kernel_pll_07:40:190
289759961,3cyclictest7687-21sshd08:42:200
289759961,3cyclictest502-21systemd-logind07:45:120
289759961,3cyclictest19982-21systemd-journal09:52:500
289759961,3cyclictest19982-21systemd-journal08:48:370
289759961,3cyclictest19982-21systemd-journal07:10:100
289759960,6cyclictest0-21swapper/010:41:030
289759960,4cyclictest19982-21systemd-journal09:19:040
289759960,3cyclictest31063-21turbostat11:30:000
289759960,3cyclictest31063-21turbostat11:30:000
289759960,3cyclictest19059-21expr11:20:130
289759960,2cyclictest23423-21/usr/sbin/munin12:15:130
289879953,1cyclictest1936-21fschecks_time08:30:153
289879952,2cyclictest32158-21cut08:25:163
289879952,2cyclictest31966-21cat07:15:023
289879952,2cyclictest17286-21grep09:05:153
289879952,2cyclictest15928-21expr07:50:133
289879951,3cyclictest922-21gltestperf09:40:143
289879951,3cyclictest19841-21perl09:10:143
289839953,1cyclictest29731-21gltestperf07:10:182
289839952,2cyclictest856-21seq08:26:072
289839952,2cyclictest6016-21cat08:40:142
289839952,2cyclictest19858-21grep09:10:142
289839952,2cyclictest17476-21expr09:05:152
289839951,3cyclictest14883-21apt-get09:00:132
289799952,2cyclictest627-21timerwakeupswit08:25:211
289799952,2cyclictest21821-21ls09:15:121
289799952,2cyclictest18044-21grep07:55:131
289799952,2cyclictest15911-21expr07:50:141
289799952,2cyclictest14021-21grep08:55:191
289799952,2cyclictest11083-21expr07:40:151
289799951,3cyclictest31523-21df07:15:011
289799951,3cyclictest1909-21/usr/sbin/munin08:30:151
289799950,3cyclictest19982-21systemd-journal08:21:301
289759953,1cyclictest23933-21cpuspeed_turbos09:20:140
289759952,2cyclictest3418-21unixbench_singl07:20:220
289759952,2cyclictest316-21acpi09:40:120
289759952,2cyclictest19982-21systemd-journal07:25:120
289759952,2cyclictest19982-21systemd-journal07:25:120
289759951,3cyclictest18043-21cpuspeed_turbos07:55:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional