You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-10-04 - 07:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8.osadl.org (updated Wed Oct 04, 2023 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
135622305272,20sleep20-21swapper/219:09:542
133522275241,21sleep30-21swapper/319:07:103
132682274243,19sleep10-21swapper/119:06:191
132572256222,23sleep00-21swapper/019:06:100
23322630,0sleep30-21swapper/322:10:333
89982600,0sleep20-21swapper/222:35:192
245672570,0sleep30-21swapper/322:05:133
121842560,0sleep012-21ksoftirqd/023:55:220
232812550,0sleep10-21swapper/122:23:451
29502190,1sleep02953-21needreboot23:10:250
194552190,0sleep20-21swapper/222:41:042
5562170,0sleep2271rcuc/223:48:392
94422140,0sleep30-21swapper/300:15:193
244812140,0sleep00-21swapper/020:45:010
11932140,0sleep00-21swapper/023:09:570
254032130,0sleep30-21swapper/320:45:243
249102130,0sleep10-21swapper/122:45:111
188142120,0sleep30-21swapper/320:30:263
122412120,1chrt3539-21diskmemload21:15:233
324762100,0sleep20-21swapper/222:29:472
285472100,0sleep00-21swapper/021:46:200
247102100,0sleep00-21swapper/021:23:170
1368299105,3cyclictest19982-21systemd-journal19:34:361
28067290,0sleep20-21swapper/222:06:402
136909997,2cyclictest22261-21kworker/3:223:02:263
136909994,3cyclictest19982-21systemd-journal20:52:053
136869994,3cyclictest19982-21systemd-journal23:40:132
136829997,2cyclictest22-21ksoftirqd/123:00:011
136829994,3cyclictest19982-21systemd-journal00:38:121
136789996,2cyclictest12-21ksoftirqd/021:10:010
136909984,3cyclictest508-21dbus-daemon21:37:333
136909983,3cyclictest19982-21systemd-journal22:43:323
136909983,3cyclictest19982-21systemd-journal20:42:543
136909982,4cyclictest19982-21systemd-journal21:27:323
136869984,3cyclictest9359-21fw_packets22:15:202
136869984,3cyclictest3758-21snmpd23:04:472
136869984,3cyclictest19982-21systemd-journal21:16:172
136869983,3cyclictest19982-21systemd-journal21:07:152
136829984,3cyclictest19982-21systemd-journal00:11:051
136829983,3cyclictest19982-21systemd-journal19:46:351
136829983,3cyclictest19982-21systemd-journal00:08:101
136829983,3cyclictest16969-21lldpd21:20:031
136829981,4cyclictest28966-21ls20:55:011
136829981,3cyclictest19982-21systemd-journal23:17:411
9029270,0sleep20-21swapper/200:15:022
29669270,0sleep20-21swapper/219:42:322
2612270,1sleep129741-11brltty19:55:211
136909973,3cyclictest3758-21snmpd21:10:193
136909973,3cyclictest26488-21ntp_kernel_pll_22:25:253
136909973,3cyclictest22159-21sshd21:42:263
136909973,3cyclictest11736-21in:imuxsock19:15:013
136909973,3cyclictest0-21swapper/322:51:543
136909973,2cyclictest19982-21systemd-journal22:31:233
136909973,1cyclictest0-21swapper/323:44:023
136909973,1cyclictest0-21swapper/323:32:413
136909972,3cyclictest19982-21systemd-journal21:07:063
136909972,3cyclictest19982-21systemd-journal20:30:003
136909972,3cyclictest19982-21systemd-journal19:41:103
136909972,3cyclictest16969-21lldpd23:53:133
136909970,5cyclictest19982-21systemd-journal00:32:563
136909970,3cyclictest14582-21sed20:20:263
136869974,2cyclictest508-21dbus-daemon22:56:252
136869974,2cyclictest508-21dbus-daemon20:35:492
136869974,2cyclictest28-21ksoftirqd/219:25:002
136869974,2cyclictest19231-21ntpq21:40:282
136869973,3cyclictest508-21dbus-daemon21:25:002
136869973,3cyclictest5053-21sed21:10:262
136869973,3cyclictest1518-21cron00:35:002
136869973,3cyclictest1-21systemd22:33:032
136869973,1cyclictest0-21swapper/222:11:142
136869973,1cyclictest0-21swapper/221:55:392
136869972,3cyclictest16969-21lldpd23:27:072
136869972,1cyclictest0-21swapper/222:45:312
136869971,4cyclictest19982-21systemd-journal00:12:202
136829974,2cyclictest508-21dbus-daemon22:05:311
136829974,2cyclictest25405-21idleruntime20:45:241
136829973,3cyclictest508-21dbus-daemon23:05:351
136829973,3cyclictest508-21dbus-daemon21:55:051
136829973,3cyclictest508-21dbus-daemon21:54:151
136829973,3cyclictest3758-21snmpd00:17:251
136829973,3cyclictest29741-11brltty21:14:111
136829973,3cyclictest19982-21systemd-journal19:50:451
136829973,3cyclictest19982-21systemd-journal19:30:001
136829973,3cyclictest0-21swapper/123:00:541
136829973,2cyclictest16969-21lldpd22:03:051
136829973,1cyclictest0-21swapper/123:51:541
136829973,1cyclictest0-21swapper/100:04:501
136829972,3cyclictest20523-21systemd-udevd19:16:201
136829972,3cyclictest19982-21systemd-journal20:06:121
136829972,3cyclictest16969-21lldpd23:20:371
136829971,3cyclictest19982-21systemd-journal21:42:261
136829970,6cyclictest26632-21idleruntime-cro23:45:001
136789975,2cyclictest12-21ksoftirqd/021:00:010
136789974,2cyclictest11736-21in:imuxsock23:29:220
136789974,2cyclictest11736-21in:imuxsock22:13:010
136789973,3cyclictest508-21dbus-daemon21:31:430
136789973,3cyclictest3758-21snmpd21:36:560
136789973,3cyclictest28844-21entropy00:30:200
136789973,3cyclictest19982-21systemd-journal19:59:310
136789973,3cyclictest19982-21systemd-journal00:17:210
136789973,3cyclictest10143-21hddtemp_smartct22:55:230
136789973,3cyclictest0-21swapper/023:21:110
136789973,3cyclictest0-21swapper/022:44:590
136789973,3cyclictest0-21swapper/022:37:240
136789973,3cyclictest0-21swapper/000:12:190
136789973,2cyclictest19982-21systemd-journal00:26:310
136789973,1cyclictest0-21swapper/023:38:010
136789973,1cyclictest0-21swapper/022:54:460
136789973,1cyclictest0-21swapper/022:17:300
136789972,3cyclictest19982-21systemd-journal20:17:270
136789972,3cyclictest19982-21systemd-journal19:25:440
136789970,4cyclictest26395-21awk00:05:000
136909964,1cyclictest1949-21apt-get23:10:203
136909964,1cyclictest0-21swapper/323:55:233
136909963,2cyclictest7608-21ntp_states20:05:273
136909963,2cyclictest668-21runrttasks00:35:203
136909963,2cyclictest5240-21sh00:11:433
136909963,2cyclictest508-21dbus-daemon20:58:203
136909963,2cyclictest3758-21snmpd23:06:203
136909963,2cyclictest29741-11brltty22:00:053
136909963,2cyclictest25367-21sh00:27:003
136909963,2cyclictest18383-21ssh00:21:283
136909963,2cyclictest17817-21sh23:20:063
136909963,2cyclictest16756-21apt-get19:15:183
136909962,3cyclictest9953-21cat21:55:263
136909962,3cyclictest9881-21expr22:55:223
136909962,3cyclictest9544-21cat20:10:263
136909962,3cyclictest9352-21fschecks_time22:15:203
136909962,3cyclictest508-21dbus-daemon21:00:013
136909962,3cyclictest3758-21snmpd23:35:123
136909962,3cyclictest3758-21snmpd19:50:183
136909962,3cyclictest3758-21snmpd19:35:023
136909962,3cyclictest31102-21ntpq19:45:253
136909962,3cyclictest26866-21unixbench_multi22:45:323
136909962,3cyclictest26724-21cat23:25:203
136909962,3cyclictest26021-21grep21:45:213
136909962,3cyclictest20477-21cut20:35:263
136909962,3cyclictest20356-21cat21:20:233
136909962,3cyclictest19982-21systemd-journal20:19:333
136909962,3cyclictest18821-21awk22:20:283
136909962,1cyclictest0-21swapper/323:47:253
136909961,4cyclictest19832-21ssh00:00:213
136909961,3cyclictest19982-21systemd-journal19:30:193
136909961,2cyclictest11986-21ssh23:15:393
136909961,1cyclictest141rcu_preempt21:30:253
136909960,6cyclictest0-21swapper/321:50:253
136909960,6cyclictest0-21swapper/319:55:253
136909960,3cyclictest9083-21ssh22:35:223
136909960,3cyclictest19982-21systemd-journal00:05:403
136909960,2cyclictest19954-21ntp_states19:20:223
136869966,0cyclictest0-21swapper/221:50:182
136869964,1cyclictest11526-21tune2fs20:15:202
136869963,2cyclictest7881-21ssh23:52:532
136869963,2cyclictest3758-21snmpd00:25:202
136869963,2cyclictest30588-21ssh21:26:422
136869963,2cyclictest29550-21expr20:55:192
136869963,2cyclictest28217-21ssh00:05:242
136869963,2cyclictest2513-21cstates23:30:202
136869963,2cyclictest24723-21gdbus20:45:012
136869963,2cyclictest18207-21fschecks_count20:30:222
136869963,2cyclictest14460-21gltestperf19:10:282
136869963,2cyclictest1402-21unixbench_multi19:50:322
136869963,2cyclictest11736-21in:imuxsock20:20:232
136869963,2cyclictest11736-21in:imuxsock00:36:572
136869963,2cyclictest11402-21smart_sda23:15:292
136869962,3cyclictest508-21dbus-daemon23:10:112
136869962,3cyclictest3758-21snmpd23:55:162
136869962,3cyclictest3758-21snmpd20:51:492
136869962,3cyclictest3758-21snmpd20:27:092
136869962,3cyclictest3758-21snmpd19:57:212
136869962,3cyclictest3758-21snmpd00:21:172
136869962,3cyclictest2935-21expr21:30:202
136869962,3cyclictest2922-21ssh22:50:332
136869962,3cyclictest16582-21cat19:15:022
136869962,3cyclictest1-21systemd19:30:192
136869962,3cyclictest0-21swapper/222:20:032
136869962,1cyclictest52150irq/139-enp4s0-23:05:212
136869962,1cyclictest0-21swapper/223:35:072
136869962,1cyclictest0-21swapper/223:20:152
136869962,1cyclictest0-21swapper/222:00:042
136869962,1cyclictest0-21swapper/221:45:182
136869961,3cyclictest19982-21systemd-journal19:38:142
136869960,6cyclictest0-21swapper/200:01:182
136869960,4cyclictest19982-21systemd-journal20:10:222
136869960,3cyclictest10388-21/usr/sbin/munin21:35:222
136829966,0cyclictest0-21swapper/100:20:161
136829964,1cyclictest0-21swapper/122:10:061
136829963,2cyclictest508-21dbus-daemon20:00:211
136829963,2cyclictest508-21dbus-daemon19:40:201
136829963,2cyclictest508-21dbus-daemon00:25:411
136829963,2cyclictest30124-21ssh21:26:181
136829963,2cyclictest29373-21sh23:45:441
136829963,2cyclictest27555-21chrt19:39:111
136829963,2cyclictest26115-21ssh23:25:081
136829963,2cyclictest22777-21ssh22:43:431
136829963,2cyclictest16232-21cat20:25:251
136829963,2cyclictest15125-21sshd20:23:051
136829963,2cyclictest12534-21ssh22:16:571
136829963,2cyclictest11738-21rs:main1
136829963,2cyclictest0-21swapper/122:51:111
136829962,3cyclictest3758-21snmpd23:35:361
136829962,3cyclictest3758-21snmpd23:10:121
136829962,3cyclictest3758-21snmpd20:15:271
136829962,3cyclictest3758-21snmpd20:12:091
136829962,3cyclictest3758-21snmpd19:20:591
136829962,3cyclictest2923-21sed23:30:221
136829962,3cyclictest18254-21fschecks_time20:30:221
136829962,3cyclictest13198-21sed23:55:281
136829962,3cyclictest0-21swapper/122:28:231
136829962,1cyclictest0-21swapper/122:35:071
136829962,1cyclictest0-21swapper/121:46:481
136829962,1cyclictest0-21swapper/121:37:001
136829961,4cyclictest0-21swapper/121:15:131
136829961,4cyclictest0-21swapper/100:30:291
136829961,3cyclictest19982-21systemd-journal21:31:431
136829961,3cyclictest19982-21systemd-journal20:57:051
136829961,3cyclictest19982-21systemd-journal20:42:041
136829961,2cyclictest3817-21ssh22:31:231
136829960,3cyclictest16969-21lldpd19:13:431
136789964,1cyclictest5202-21needreboot21:10:270
136789963,2cyclictest966-21sh22:30:190
136789963,2cyclictest51950irq/137-enp4s0-00:20:190
136789963,2cyclictest508-21dbus-daemon22:05:060
136789963,2cyclictest32247-21uniq21:00:220
136789963,2cyclictest25872-21diskstats22:25:210
136789963,2cyclictest19448-21gunzip21:40:300
136789963,2cyclictest19265-21cstates23:40:230
136789963,2cyclictest17398-21gltestperf22:00:250
136789963,2cyclictest14534-21cut20:20:240
136789963,2cyclictest14159-21rm23:17:260
136789963,2cyclictest12-21ksoftirqd/000:05:190
136789963,2cyclictest11738-21rs:main0
136789963,2cyclictest0-21swapper/023:50:190
136789963,2cyclictest0-21swapper/021:27:350
136789962,3cyclictest6929-21cat20:05:210
136789962,3cyclictest4903-21rm21:52:010
136789962,3cyclictest3758-21snmpd20:50:130
136789962,3cyclictest3758-21snmpd20:37:100
136789962,3cyclictest3758-21snmpd20:04:450
136789962,3cyclictest3758-21snmpd19:50:240
136789962,3cyclictest3758-21snmpd19:45:020
136789962,3cyclictest3758-21snmpd19:41:410
136789962,3cyclictest3758-21snmpd19:30:050
136789962,3cyclictest3758-21snmpd19:18:170
136789962,3cyclictest29741-11brltty19:24:290
136789962,3cyclictest29741-11brltty00:36:320
136789962,3cyclictest14108-21cat19:10:260
136789962,3cyclictest0-21swapper/023:34:410
136789962,3cyclictest0-21swapper/023:00:240
136789962,3cyclictest0-21swapper/021:55:570
136789962,1cyclictest0-21swapper/022:45:250
136789961,3cyclictest19982-21systemd-journal20:32:280
136789960,6cyclictest0-21swapper/021:16:310
136789960,5cyclictest8841-21/usr/sbin/munin20:10:230
136789960,3cyclictest20492-21rm22:21:390
136789960,0cyclictest0-21swapper/023:47:360
136909952,2cyclictest21504-21expr19:25:213
136909951,3cyclictest5029-21/usr/sbin/munin20:00:233
136869952,2cyclictest7078-21fschecks_count20:05:222
136869952,2cyclictest31761-21cat21:00:192
136869952,2cyclictest30103-21date19:45:002
136869952,2cyclictest22368-21sort20:40:212
136869952,2cyclictest21726-21fschecks_time19:25:222
136869950,3cyclictest19982-21systemd-journal20:00:212
136829952,2cyclictest31717-21gdbus21:00:011
136829952,2cyclictest1748-21tr21:05:181
136789952,2cyclictest25637-21chrt19:35:060
136789950,3cyclictest19982-21systemd-journal20:45:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional