You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 05:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Wed Nov 19, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
243902305271,21sleep10-21swapper/119:05:361
244662282215,20sleep00-21swapper/019:06:270
227752274222,22sleep20-21swapper/219:05:132
244932271237,21sleep30-21swapper/319:06:373
3792610,0sleep00-21swapper/023:54:300
155072590,0sleep30-21swapper/321:31:093
320782560,0sleep30-21swapper/323:53:443
156922560,0sleep20-21swapper/223:20:172
208832540,0sleep30-21swapper/323:45:203
28302360,0sleep00-21swapper/023:55:230
51262220,0sleep20-21swapper/221:45:472
168722210,0sleep10-21swapper/122:59:541
151872210,0sleep00-21swapper/021:09:590
32502170,0sleep10-21swapper/100:19:421
107832170,3sleep122-21ksoftirqd/121:00:001
101862160,0sleep30-21swapper/322:10:503
302302150,0sleep10-21swapper/122:24:531
45042130,0sleep30-21swapper/321:45:233
2486699129,2cyclictest22-21ksoftirqd/123:19:591
24866991211,1cyclictest22-21ksoftirqd/100:40:001
24866991210,1cyclictest22-21ksoftirqd/123:45:011
24866991210,1cyclictest22-21ksoftirqd/122:15:011
24866991210,1cyclictest22-21ksoftirqd/121:20:001
24866991210,1cyclictest22-21ksoftirqd/120:20:011
24866991210,1cyclictest22-21ksoftirqd/119:50:011
149062120,0sleep30-21swapper/300:29:093
2486699119,2cyclictest22-21ksoftirqd/120:10:011
2486699115,1cyclictest22-21ksoftirqd/120:15:011
24866991110,1cyclictest22-21ksoftirqd/100:25:011
2487399103,6cyclictest0-21swapper/323:25:143
2486699109,1cyclictest22-21ksoftirqd/122:00:011
2486699109,1cyclictest22-21ksoftirqd/121:50:011
2486699109,1cyclictest22-21ksoftirqd/119:15:011
2486699108,2cyclictest22-21ksoftirqd/120:25:001
2486699108,1cyclictest22-21ksoftirqd/121:29:591
2486699108,1cyclictest22-21ksoftirqd/119:55:011
2486699107,2cyclictest22-21ksoftirqd/120:30:001
2486699103,4cyclictest15755-21munin-run00:05:011
216522100,0sleep30-21swapper/320:10:213
248739994,3cyclictest19982-21systemd-journal23:40:363
248739994,3cyclictest19982-21systemd-journal22:20:013
248739994,3cyclictest19982-21systemd-journal20:06:063
248669998,1cyclictest22-21ksoftirqd/121:40:011
248669998,1cyclictest22-21ksoftirqd/119:15:011
248669997,1cyclictest22-21ksoftirqd/123:30:001
248669997,1cyclictest211rcuc/120:45:011
248669997,1cyclictest211rcuc/100:00:011
248669996,2cyclictest22-21ksoftirqd/122:20:001
248619990,8cyclictest15528-21rm22:15:020
23435290,1sleep023534-21ntp_states00:35:200
26666280,0sleep30-21swapper/323:05:323
248739984,2cyclictest19982-21systemd-journal00:32:133
248739983,3cyclictest19982-21systemd-journal22:10:013
248739983,3cyclictest19982-21systemd-journal20:33:013
248739983,3cyclictest19982-21systemd-journal00:37:133
248709985,2cyclictest29904-21cron21:20:002
248709984,3cyclictest11736-21in:imuxsock20:26:202
248709983,3cyclictest19982-21systemd-journal23:25:062
248669987,1cyclictest22-21ksoftirqd/100:30:011
248669986,2cyclictest22-21ksoftirqd/123:10:001
248669986,2cyclictest22-21ksoftirqd/119:40:001
248669986,1cyclictest22-21ksoftirqd/122:45:011
248669985,2cyclictest22-21ksoftirqd/122:40:011
248669985,2cyclictest22-21ksoftirqd/121:05:001
248669983,3cyclictest19982-21systemd-journal00:11:381
248669982,4cyclictest22-21ksoftirqd/122:00:201
248669981,4cyclictest1-21systemd00:32:281
248669980,7cyclictest29066-21missed_timers22:45:201
248619983,4cyclictest3546-21grep00:20:000
248619983,3cyclictest19982-21systemd-journal22:27:200
248619983,3cyclictest19982-21systemd-journal22:27:200
248619983,3cyclictest19982-21systemd-journal21:22:140
10921280,0sleep30-21swapper/321:28:413
248739976,1cyclictest2200-21kworker/3:021:21:543
248739973,3cyclictest7941-21ntpq19:40:173
248739973,3cyclictest508-21dbus-daemon19:28:463
248739973,3cyclictest508-21dbus-daemon00:12:283
248739973,3cyclictest502-21systemd-logind22:34:263
248739973,3cyclictest19982-21systemd-journal20:48:513
248739973,3cyclictest11736-21in:imuxsock21:00:583
248739973,3cyclictest0-21swapper/322:39:113
248739973,1cyclictest0-21swapper/322:45:463
248739973,1cyclictest0-21swapper/322:40:163
248739973,1cyclictest0-21swapper/322:04:583
248739973,1cyclictest0-21swapper/321:54:213
248739972,4cyclictest34-21ksoftirqd/321:55:223
248739972,3cyclictest19982-21systemd-journal21:38:553
248739972,3cyclictest19982-21systemd-journal21:10:083
248739972,3cyclictest19982-21systemd-journal21:05:493
248739972,3cyclictest19982-21systemd-journal20:26:463
248739972,3cyclictest19982-21systemd-journal19:45:023
248739970,3cyclictest1422-21/usr/sbin/munin23:55:183
248739970,2cyclictest29741-11brltty00:03:423
248739970,1cyclictest9572-21sh22:54:553
248709974,2cyclictest4436-21wc20:45:152
248709974,2cyclictest29741-11brltty23:13:392
248709974,2cyclictest24907-21cron20:20:012
248709974,2cyclictest11736-21in:imuxsock21:11:232
248709974,2cyclictest11198-21ntpq22:55:192
248709973,3cyclictest30654-21snmpd22:03:422
248709973,3cyclictest30654-21snmpd00:32:342
248709973,3cyclictest30654-21snmpd00:14:282
248709973,3cyclictest27494-21irqstats23:50:152
248709973,3cyclictest20238-21cat22:40:002
248709973,3cyclictest19982-21systemd-journal22:40:362
248709973,3cyclictest0-21swapper/222:28:182
248709973,3cyclictest0-21swapper/222:28:172
248709973,2cyclictest19982-21systemd-journal00:08:432
248709973,1cyclictest0-21swapper/222:49:092
248709973,1cyclictest0-21swapper/221:31:282
248709972,3cyclictest19982-21systemd-journal21:25:532
248709970,4cyclictest32176-21apt19:25:012
248709970,4cyclictest27176-21idleruntime-cro20:25:012
248709970,2cyclictest9676-21users22:10:242
248669975,1cyclictest22-21ksoftirqd/121:30:001
248669974,2cyclictest22-21ksoftirqd/123:50:011
248669974,2cyclictest22-21ksoftirqd/100:10:011
248669974,1cyclictest141rcu_preempt19:22:061
248669973,3cyclictest19982-21systemd-journal20:50:011
248669973,3cyclictest19982-21systemd-journal20:01:441
248669973,3cyclictest0-21swapper/122:28:361
248669973,3cyclictest0-21swapper/122:28:351
248669973,3cyclictest0-21swapper/121:23:591
248669973,1cyclictest0-21swapper/123:50:121
248669973,1cyclictest0-21swapper/123:35:211
248669971,1cyclictest141rcu_preempt21:14:441
248669970,1cyclictest141rcu_preempt20:36:211
248619974,2cyclictest508-21dbus-daemon20:23:250
248619974,2cyclictest508-21dbus-daemon19:55:280
248619974,2cyclictest19982-21systemd-journal22:56:440
248619973,3cyclictest508-21dbus-daemon21:35:000
248619973,3cyclictest508-21dbus-daemon19:20:000
248619973,3cyclictest29741-11brltty21:16:480
248619973,3cyclictest19982-21systemd-journal21:29:450
248619973,3cyclictest0-21swapper/021:54:060
248619973,3cyclictest0-21swapper/021:40:140
248619972,3cyclictest1973-21ls19:30:010
248619970,3cyclictest31277-21sshd22:46:430
21312270,0sleep10-21swapper/123:02:181
248739964,1cyclictest28323-21df19:15:163
248739963,2cyclictest4699-21interrupts00:20:163
248739963,2cyclictest31569-21cat19:20:233
248739963,2cyclictest16999-1kworker/u9:222:55:153
248739963,2cyclictest14798-21missed_timers19:55:163
248739963,2cyclictest11736-21in:imuxsock20:00:033
248739963,2cyclictest0-21swapper/323:38:103
248739963,1cyclictest0-21swapper/323:01:333
248739963,1cyclictest0-21swapper/322:17:463
248739962,3cyclictest691-21ntpq20:35:193
248739962,3cyclictest30654-21snmpd20:22:153
248739962,3cyclictest30654-21snmpd19:39:443
248739962,3cyclictest28923-21ntpq21:40:183
248739962,3cyclictest2844-21cat20:40:213
248739962,3cyclictest16490-21expr00:05:133
248739962,1cyclictest0-21swapper/323:10:173
248739962,1cyclictest0-21swapper/321:15:053
248739961,3cyclictest7661-21lldpd23:32:483
248739961,2cyclictest2398-21ssh00:18:413
248739960,6cyclictest0-21swapper/322:25:273
248739960,6cyclictest0-21swapper/322:25:273
248739960,6cyclictest0-21swapper/320:50:023
248739960,0cyclictest0-21swapper/323:20:033
248709964,1cyclictest25380-21sed23:05:162
248709963,2cyclictest7147-21cat20:50:182
248709963,2cyclictest31145-21taskset23:30:182
248709963,2cyclictest22608-21cpuspeed_turbos00:35:152
248709963,2cyclictest19982-21systemd-journal21:38:042
248709963,2cyclictest19982-21systemd-journal20:36:452
248709963,2cyclictest16949-21/usr/sbin/munin20:00:202
248709963,2cyclictest11736-21in:imuxsock19:51:202
248709963,2cyclictest0-21swapper/221:51:492
248709963,1cyclictest0-21swapper/222:32:062
248709962,3cyclictest8217-21sensors_temp19:40:192
248709962,3cyclictest665-21cat19:25:172
248709962,3cyclictest4246-21expr00:20:142
248709962,3cyclictest3226-21smartctl20:40:232
248709962,3cyclictest30654-21snmpd23:00:142
248709962,3cyclictest30654-21snmpd20:57:512
248709962,3cyclictest28289-21cat19:15:162
248709962,3cyclictest23605-21expr22:20:152
248709962,3cyclictest21703-21cut20:10:222
248709962,3cyclictest15269-21latency_hist21:10:002
248709962,3cyclictest12952-21hostname23:40:182
248709962,3cyclictest1124-21which22:05:172
248709962,3cyclictest0-21swapper/221:40:492
248709962,1cyclictest0-21swapper/223:48:342
248709962,1cyclictest0-21swapper/222:50:452
248709962,1cyclictest0-21swapper/200:17:242
248709962,1cyclictest0-21swapper/200:00:312
248709961,4cyclictest19982-21systemd-journal19:35:012
248709961,4cyclictest0-21swapper/222:16:152
248709961,3cyclictest19982-21systemd-journal23:55:472
248709961,3cyclictest19982-21systemd-journal20:34:412
248709961,2cyclictest7031-21ssh23:15:052
248709961,2cyclictest22019-21ssh21:57:312
248709960,5cyclictest9940-21/usr/sbin/munin00:25:152
248709960,5cyclictest5581-21gzip19:35:212
248709960,1cyclictest0-21swapper/223:36:352
248669965,1cyclictest22-21ksoftirqd/123:30:011
248669965,1cyclictest22-21ksoftirqd/123:20:001
248669963,2cyclictest307-21ssh22:05:091
248669963,2cyclictest19982-21systemd-journal19:30:501
248669963,1cyclictest0-21swapper/122:50:081
248669962,3cyclictest8694-21sshd19:41:381
248669962,3cyclictest15665-21seq19:56:331
248669962,3cyclictest14110-21ntpq21:05:191
248669962,3cyclictest0-21swapper/121:52:541
248669962,3cyclictest0-21swapper/121:40:021
248669962,1cyclictest0-21swapper/122:30:371
248669961,3cyclictest22-21ksoftirqd/123:10:181
248669961,3cyclictest141rcu_preempt20:31:461
248619964,1cyclictest16489-21df00:30:140
248619963,2cyclictest7987-21sh22:30:500
248619963,2cyclictest7366-21cut20:50:190
248619963,2cyclictest32741-21diskstats23:10:210
248619963,2cyclictest30654-21snmpd22:11:590
248619963,2cyclictest30654-21snmpd20:16:010
248619963,2cyclictest30654-21snmpd19:42:020
248619963,2cyclictest24473-21ssh23:25:360
248619963,2cyclictest23629-21expr22:20:150
248619963,2cyclictest12691-21sh23:40:160
248619963,2cyclictest0-21swapper/023:30:370
248619962,3cyclictest5451-21fschecks_count23:35:170
248619962,3cyclictest508-21dbus-daemon20:35:560
248619962,3cyclictest4715-21expr19:35:150
248619962,3cyclictest451-21apt22:05:140
248619962,3cyclictest30654-21snmpd23:05:010
248619962,3cyclictest30654-21snmpd23:01:310
248619962,3cyclictest30654-21snmpd20:56:450
248619962,3cyclictest30654-21snmpd20:32:450
248619962,3cyclictest30654-21snmpd19:22:120
248619962,3cyclictest30654-21snmpd00:25:450
248619962,3cyclictest29741-11brltty20:29:400
248619962,3cyclictest26990-21grep22:00:230
248619962,3cyclictest25056-21awk00:10:250
248619962,3cyclictest2494-21tune2fs20:40:190
248619962,3cyclictest11141-21seq21:00:140
248619962,3cyclictest0-21swapper/023:48:510
248619962,1cyclictest0-21swapper/022:40:460
248619962,1cyclictest0-21swapper/021:46:200
248619962,1cyclictest0-21swapper/021:10:120
248619961,5cyclictest8524-21sendmail_mailst23:15:220
248619961,4cyclictest0-21swapper/022:50:160
248619961,3cyclictest19982-21systemd-journal00:00:220
248619960,5cyclictest4999-21ntp_states00:20:170
248619960,4cyclictest19982-21systemd-journal22:35:160
248619960,4cyclictest19982-21systemd-journal19:50:530
248619960,3cyclictest23125-21rm21:36:150
248619960,3cyclictest20259-21rm21:56:010
248619960,3cyclictest18215-21rm00:05:370
248619960,3cyclictest14823-21rm23:20:060
248619960,2cyclictest19982-21systemd-journal19:11:180
248739952,2cyclictest25465-21df19:10:143
248739952,2cyclictest23481-21expr20:15:193
248739952,2cyclictest12123-21expr19:50:143
248739952,2cyclictest0-21swapper/320:55:143
248739951,2cyclictest19982-21systemd-journal19:30:013
248739950,4cyclictest7249-21ssh23:15:153
248709952,2cyclictest25753-21cat19:10:152
248709952,2cyclictest18906-21sort20:05:162
248709952,2cyclictest14148-21cut19:55:142
248709952,2cyclictest0-21swapper/221:20:142
248709951,3cyclictest9291-21/usr/sbin/munin19:45:152
248709951,3cyclictest30654-21snmpd21:00:122
248669954,1cyclictest22-21ksoftirqd/120:50:011
248669952,2cyclictest32410-21seq19:25:131
248619952,2cyclictest30654-21snmpd20:45:060
248619952,2cyclictest30654-21snmpd19:45:110
248619952,2cyclictest16408-21grep20:00:170
248619951,3cyclictest30654-21snmpd19:30:100
248619951,3cyclictest20915-21expr20:10:160
248619950,3cyclictest19982-21systemd-journal20:06:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional