You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-02 - 19:24
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackaslot8.osadl.org (updated Sat Dec 02, 2023 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
223382324291,20sleep20-21swapper/207:08:392
221732279213,22sleep30-21swapper/307:06:303
221262277242,23sleep10-21swapper/107:05:521
222272257224,21sleep00-21swapper/007:07:120
128022610,0sleep20-21swapper/209:31:012
42252600,0sleep10-21swapper/109:25:311
99672560,0sleep30-21swapper/312:14:443
98032150,0sleep00-21swapper/011:50:580
76022140,0chrt0-21swapper/211:50:252
226712140,0sleep30-21swapper/310:41:303
22562991312,1cyclictest12-21ksoftirqd/007:40:010
187202130,0sleep20-21swapper/211:37:072
147652130,0sleep10-21swapper/109:32:401
189432120,0sleep10-21swapper/110:19:541
2256299119,2cyclictest12-21ksoftirqd/012:35:010
2257099108,2cyclictest28-21ksoftirqd/212:30:012
2256299109,1cyclictest12-21ksoftirqd/011:10:000
2256299109,1cyclictest12-21ksoftirqd/010:45:000
2256299109,1cyclictest12-21ksoftirqd/009:35:000
2256299108,1cyclictest12-21ksoftirqd/011:20:000
225749994,3cyclictest19982-21systemd-journal09:48:193
225709994,3cyclictest19982-21systemd-journal10:23:212
225709994,3cyclictest19982-21systemd-journal08:19:012
225669994,3cyclictest19982-21systemd-journal10:01:141
225669994,3cyclictest19982-21systemd-journal09:55:491
225669994,3cyclictest19982-21systemd-journal08:21:571
225629997,2cyclictest12-21ksoftirqd/011:00:010
225749986,1cyclictest0-21swapper/309:17:183
225749983,3cyclictest19982-21systemd-journal11:58:033
225749983,3cyclictest19982-21systemd-journal09:56:403
225709985,2cyclictest27016-21cat09:19:592
225709984,3cyclictest19982-21systemd-journal10:25:022
225709983,3cyclictest19982-21systemd-journal08:22:462
225709982,3cyclictest1897-21basename07:35:012
225709980,7cyclictest9116-21ssh09:50:352
225669986,1cyclictest14031-21diskstats11:15:231
225669984,3cyclictest19982-21systemd-journal10:44:381
225629988,0cyclictest121ksoftirqd/011:50:010
225629987,1cyclictest12-21ksoftirqd/008:00:000
225629986,1cyclictest12-21ksoftirqd/011:15:010
225629986,1cyclictest12-21ksoftirqd/009:55:000
225629984,3cyclictest508-21dbus-daemon10:07:560
225629984,3cyclictest19982-21systemd-journal07:25:010
225629984,3cyclictest12-21ksoftirqd/009:30:000
225629983,3cyclictest19982-21systemd-journal11:31:220
225629980,0cyclictest141rcu_preempt08:20:190
4957270,0sleep10-21swapper/111:48:391
26311270,0sleep00-21swapper/009:19:220
225749975,1cyclictest14122-21ls11:34:593
225749973,3cyclictest508-21dbus-daemon10:16:413
225749973,3cyclictest5000-21fw_packets07:40:233
225749973,3cyclictest4749-21awk10:10:303
225749973,3cyclictest3758-21snmpd11:43:513
225749973,3cyclictest3758-21snmpd11:27:093
225749973,3cyclictest23206-21fschecks_count07:10:263
225749973,3cyclictest1805-21sshd07:34:223
225749973,1cyclictest0-21swapper/312:26:023
225749973,1cyclictest0-21swapper/311:38:213
225749973,1cyclictest0-21swapper/311:15:153
225749972,3cyclictest19982-21systemd-journal12:35:123
225749972,3cyclictest19982-21systemd-journal10:10:013
225749972,3cyclictest19982-21systemd-journal09:52:303
225749970,3cyclictest4918-21ntp_states10:50:263
225749970,2cyclictest19982-21systemd-journal10:40:013
225749970,2cyclictest19982-21systemd-journal10:40:003
225709974,2cyclictest508-21dbus-daemon09:11:112
225709974,2cyclictest24025-21sshd10:42:332
225709973,3cyclictest3758-21snmpd09:07:412
225709973,3cyclictest29377-21awk09:20:312
225709973,3cyclictest26520-21pluginstate08:30:012
225709973,3cyclictest0-21swapper/211:15:232
225709973,3cyclictest0-21swapper/211:06:142
225709973,1cyclictest0-21swapper/210:52:062
225709972,3cyclictest19982-21systemd-journal11:57:132
225709972,1cyclictest0-21swapper/210:36:132
225709972,1cyclictest0-21swapper/210:36:132
225709971,4cyclictest19982-21systemd-journal10:12:052
225709971,4cyclictest19982-21systemd-journal08:50:192
225709970,6cyclictest32118-21dump-pmu-power09:45:012
225709970,0cyclictest0-21swapper/212:19:142
225669974,2cyclictest11736-21in:imuxsock07:47:451
225669973,3cyclictest3758-21snmpd12:32:081
225669973,3cyclictest3758-21snmpd09:43:281
225669973,3cyclictest29741-11brltty07:19:471
225669973,3cyclictest19982-21systemd-journal10:47:591
225669973,3cyclictest0-21swapper/112:09:271
225669973,3cyclictest0-21swapper/110:06:401
225669973,1cyclictest0-21swapper/112:00:191
225669973,1cyclictest0-21swapper/109:16:341
225669972,3cyclictest19982-21systemd-journal12:20:351
225669972,3cyclictest19982-21systemd-journal10:50:021
225629976,1cyclictest12243-21diskmemload11:03:340
225629976,1cyclictest12-21ksoftirqd/012:30:010
225629976,1cyclictest12-21ksoftirqd/012:00:000
225629976,1cyclictest12-21ksoftirqd/007:20:010
225629976,1cyclictest12-21ksoftirqd/007:15:000
225629975,2cyclictest12-21ksoftirqd/010:25:010
225629975,1cyclictest12-21ksoftirqd/010:30:010
225629974,2cyclictest12-21ksoftirqd/010:35:010
225629974,2cyclictest11736-21in:imuxsock08:39:530
225629974,2cyclictest11736-21in:imuxsock07:25:380
225629973,3cyclictest23593-21df11:40:230
225629973,1cyclictest141rcu_preempt07:45:230
225629973,1cyclictest0-21swapper/012:07:290
225629972,3cyclictest19982-21systemd-journal11:29:160
225629972,3cyclictest16969-21lldpd10:54:190
225629971,4cyclictest141rcu_preempt09:23:410
225629971,3cyclictest17586-21turbostat08:10:010
225629970,1cyclictest0-21swapper/010:16:540
225629970,1cyclictest0-21swapper/010:11:580
2254270,0sleep30-21swapper/310:49:193
1765270,0sleep30-21swapper/311:46:123
225749964,1cyclictest2790-21apt-get09:25:213
225749963,3cyclictest21593-21kworker/3:011:54:153
225749963,2cyclictest8468-21grep09:00:243
225749963,2cyclictest3758-21snmpd07:26:573
225749963,2cyclictest32682-21sed08:40:293
225749963,2cyclictest28850-21ssh11:05:153
225749963,2cyclictest26003-21sh11:22:103
225749963,2cyclictest22748-21cut08:20:203
225749963,2cyclictest19982-21systemd-journal07:54:323
225749963,2cyclictest18039-21taskset09:13:393
225749963,2cyclictest13659-21mailstats10:55:343
225749963,2cyclictest11309-21tr07:55:223
225749962,3cyclictest6712-21sensors_temp08:55:293
225749962,3cyclictest508-21dbus-daemon07:50:013
225749962,3cyclictest3843-21cut08:50:223
225749962,3cyclictest3758-21snmpd12:15:473
225749962,3cyclictest3758-21snmpd09:23:023
225749962,3cyclictest3758-21snmpd09:07:503
225749962,3cyclictest3758-21snmpd08:46:583
225749962,3cyclictest3758-21snmpd08:35:043
225749962,3cyclictest3758-21snmpd08:34:423
225749962,3cyclictest3758-21snmpd08:29:213
225749962,3cyclictest32034-21if_enp4s012:30:253
225749962,3cyclictest2678-21expr07:35:233
225749962,3cyclictest20753-21cat08:15:273
225749962,3cyclictest20609-21cat10:20:273
225749962,3cyclictest19116-21gdbus10:25:013
225749962,3cyclictest16473-21awk12:20:143
225749962,3cyclictest15992-21expr08:05:213
225749962,3cyclictest13383-21acpi08:00:193
225749962,3cyclictest1-21systemd12:05:013
225749962,3cyclictest11001-21fschecks_count09:30:243
225749962,3cyclictest0-21swapper/311:12:593
225749962,2cyclictest19982-21systemd-journal07:16:023
225749962,1cyclictest0-21swapper/312:01:103
225749962,1cyclictest0-21swapper/311:00:093
225749962,1cyclictest0-21swapper/309:42:213
225749961,2cyclictest22061-21ssh09:37:343
225749961,1cyclictest20786-21apt-get10:00:163
225749960,5cyclictest4642-21cat10:30:253
225749960,4cyclictest19982-21systemd-journal07:20:383
225709963,2cyclictest6241-21proc_pri11:10:332
225709963,2cyclictest31641-21ntp_states11:25:312
225709963,2cyclictest29741-11brltty09:57:052
225709963,2cyclictest27999-21ssh10:05:112
225709963,2cyclictest22895-21sshd11:40:092
225709963,2cyclictest20803-21sshd12:00:132
225709963,2cyclictest1883-21latency_hist07:35:012
225709963,2cyclictest16961-21fw_packets12:20:152
225709962,3cyclictest8761-21cut09:00:262
225709962,3cyclictest7172-21ssh11:30:232
225709962,3cyclictest7072-21ntpq12:35:302
225709962,3cyclictest508-21dbus-daemon09:00:012
225709962,3cyclictest3758-21snmpd10:31:082
225709962,3cyclictest3758-21snmpd10:00:162
225709962,3cyclictest3758-21snmpd08:13:292
225709962,3cyclictest3758-21snmpd08:08:062
225709962,3cyclictest3758-21snmpd07:16:022
225709962,3cyclictest3758-21snmpd07:11:502
225709962,3cyclictest3534-21cat09:25:252
225709962,3cyclictest29010-21ntp_kernel_pll_10:45:272
225709962,3cyclictest27024-21expr08:30:202
225709962,3cyclictest22434-21grep11:00:372
225709962,3cyclictest1605-21tr09:45:312
225709962,3cyclictest1-21systemd07:28:322
225709962,2cyclictest19982-21systemd-journal07:23:212
225709962,1cyclictest0-21swapper/212:12:452
225709962,1cyclictest0-21swapper/211:20:022
225709962,1cyclictest0-21swapper/210:55:232
225709962,1cyclictest0-21swapper/210:17:332
225709962,1cyclictest0-21swapper/209:36:232
225709961,3cyclictest19982-21systemd-journal12:05:102
225709961,3cyclictest19982-21systemd-journal07:42:192
225709960,4cyclictest19982-21systemd-journal11:45:582
225709960,2cyclictest31710-21df_inode12:30:232
225669964,1cyclictest8136-21df_inode09:50:271
225669964,1cyclictest21341-21grep08:15:331
225669963,2cyclictest5563-21fw_packets11:10:271
225669963,2cyclictest5560-21ssh09:48:451
225669963,2cyclictest25120-21cat12:25:271
225669963,2cyclictest22028-21ssh10:20:501
225669963,2cyclictest19802-21unixbench_singl09:35:341
225669963,2cyclictest12243-21diskmemload12:16:071
225669962,3cyclictest8391-21expr09:00:231
225669962,3cyclictest5308-21cat10:30:301
225669962,3cyclictest3758-21snmpd12:35:081
225669962,3cyclictest3758-21snmpd11:40:071
225669962,3cyclictest3758-21snmpd08:56:041
225669962,3cyclictest3758-21snmpd08:45:041
225669962,3cyclictest3758-21snmpd08:05:141
225669962,3cyclictest3758-21snmpd08:04:021
225669962,3cyclictest3758-21snmpd07:55:141
225669962,3cyclictest3758-21snmpd07:50:211
225669962,3cyclictest3758-21snmpd07:41:091
225669962,3cyclictest3758-21snmpd07:32:091
225669962,3cyclictest31922-21cat08:40:241
225669962,3cyclictest30792-21seq08:38:091
225669962,3cyclictest29379-21cat10:25:291
225669962,3cyclictest13534-21if_enp4s009:10:211
225669962,3cyclictest1-21systemd09:05:051
225669962,1cyclictest0-21swapper/111:52:451
225669962,1cyclictest0-21swapper/111:36:081
225669962,1cyclictest0-21swapper/110:55:261
225669962,1cyclictest0-21swapper/109:21:351
225669961,3cyclictest19982-21systemd-journal11:05:541
225669960,6cyclictest0-21swapper/112:10:131
225669960,6cyclictest0-21swapper/111:25:071
225669960,6cyclictest0-21swapper/110:11:021
225669960,5cyclictest23438-21smart_sda11:20:311
225669960,3cyclictest22723-21rm11:00:441
225669960,3cyclictest16225-21rm11:56:251
225669960,2cyclictest3255-21/usr/sbin/munin08:50:201
225669960,2cyclictest12422-21sshd10:35:031
225669960,2cyclictest12422-21sshd10:35:031
225629964,2cyclictest12-21ksoftirqd/012:10:220
225629963,2cyclictest9671-21taskset09:01:020
225629963,2cyclictest668-21runrttasks10:45:420
225629963,2cyclictest3758-21snmpd08:41:300
225629963,2cyclictest19982-21systemd-journal10:04:560
225629963,2cyclictest16372-21sh12:20:130
225629963,2cyclictest12243-21diskmemload09:37:200
225629963,2cyclictest1-21systemd09:06:090
225629963,2cyclictest11444-21sh12:15:540
225629962,4cyclictest141rcu_preempt09:40:150
225629962,3cyclictest508-21dbus-daemon08:56:590
225629962,3cyclictest3758-21snmpd12:00:480
225629962,3cyclictest3758-21snmpd09:55:570
225629962,3cyclictest3758-21snmpd08:48:390
225629962,3cyclictest3758-21snmpd08:30:160
225629962,3cyclictest3758-21snmpd07:52:500
225629962,3cyclictest3758-21snmpd07:43:180
225629962,3cyclictest22077-21chrt11:20:230
225629962,3cyclictest18359-21fw_packets08:10:250
225629962,3cyclictest1006-21cat09:45:230
225629962,1cyclictest12-21ksoftirqd/008:50:210
225629962,1cyclictest0-21swapper/012:36:190
225629962,1cyclictest0-21swapper/011:35:590
225629961,3cyclictest29741-11brltty09:10:460
225629960,4cyclictest19982-21systemd-journal10:35:530
225629960,4cyclictest19982-21systemd-journal10:35:520
225749952,2cyclictest18026-21cut08:10:243
225709953,1cyclictest7870-21find07:45:332
225709953,1cyclictest11481-21expr07:55:232
225709952,2cyclictest9197-21expr07:50:202
225709952,2cyclictest31978-21cut08:40:242
225709952,2cyclictest14135-21latency08:00:242
225709952,2cyclictest1177-21cpuspeed_turbos08:45:212
225709951,3cyclictest29741-11brltty08:35:182
225669952,2cyclictest3758-21snmpd08:25:041
225669952,2cyclictest28185-21ls07:20:221
225669952,2cyclictest27076-21cat08:30:201
225669952,2cyclictest22981-21cpuspeed_turbos07:10:241
225669952,2cyclictest18726-21needreboot08:10:281
225669951,3cyclictest5986-21ssh11:30:031
225669951,3cyclictest30517-21df_inode07:25:201
225669951,3cyclictest2212-21sort07:35:221
225629953,1cyclictest12-21ksoftirqd/008:25:170
225629953,1cyclictest12-21ksoftirqd/008:15:170
225629952,2cyclictest402-21perl07:30:240
225629951,3cyclictest13381-21/usr/sbin/munin08:00:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional