You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-05 - 01:20

x86 Intel Core i5-8265UE @1600 MHz, Linux 5.10.41-rt42 (Profile)

Latency plot of system in rack #a, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Sun Dec 04, 2022 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40562311282,19sleep10-21swapper/107:08:031
39232306277,19sleep00-21swapper/007:06:170
40322289258,19sleep20-21swapper/207:07:442
40132271245,16sleep30-21swapper/307:07:293
991321000,0sleep20-21swapper/212:00:182
67622700,0sleep30-21swapper/311:18:483
85922600,0sleep00-21swapper/009:59:160
231362600,0sleep00-21swapper/012:08:580
86852570,0sleep20-21swapper/209:18:452
176392560,1sleep017638-21sort08:50:010
146742560,0sleep20-21swapper/211:23:342
91052540,0sleep30-21swapper/309:39:213
43282520,0sleep10-21swapper/109:35:301
27852190,0sleep10-21swapper/110:35:211
86132170,0sleep10-21swapper/109:59:181
296512170,0sleep00-21swapper/012:12:310
234132150,0sleep20-21swapper/211:29:072
215732150,0sleep30-21swapper/309:46:293
45192130,0sleep20-21swapper/210:36:312
130762130,0sleep20-21swapper/209:20:492
43409996,2cyclictest28339-21sh09:50:293
43379996,2cyclictest19183-21ssh12:25:411
43329996,2cyclictest2527-21irqstats08:15:150
43409985,2cyclictest34-21ksoftirqd/307:45:013
43409985,2cyclictest24076-21ssh12:29:373
43389984,3cyclictest283-21systemd-journal11:55:152
43389981,4cyclictest1571-21basename09:55:022
43379983,2cyclictest9871-21cups-browsed12:00:001
43329984,2cyclictest283-21systemd-journal09:38:260
43329980,7cyclictest0-21swapper/009:15:380
43409976,1cyclictest34-21ksoftirqd/307:15:003
43409974,2cyclictest283-21systemd-journal12:31:203
43409974,2cyclictest1850-21awk11:35:203
43409973,3cyclictest283-21systemd-journal10:50:013
43409973,3cyclictest283-21systemd-journal09:34:013
43409973,3cyclictest283-21systemd-journal09:11:293
43409973,3cyclictest283-21systemd-journal07:50:003
43409973,1cyclictest0-21swapper/311:00:013
43409972,4cyclictest17867-21cat12:05:183
43409972,3cyclictest283-21systemd-journal12:39:103
43409972,3cyclictest283-21systemd-journal10:42:253
43409971,4cyclictest283-21systemd-journal11:40:483
43409971,4cyclictest283-21systemd-journal10:58:423
43389973,3cyclictest283-21systemd-journal11:39:592
43389973,3cyclictest283-21systemd-journal09:34:012
43389973,3cyclictest14659-21snmpd10:45:382
43389971,4cyclictest283-21systemd-journal12:20:002
43379976,1cyclictest30397-21ssh10:12:241
43379974,2cyclictest8479-21awk12:20:011
43379974,2cyclictest508-21dbus-daemon08:04:201
43379973,3cyclictest668-21runrttasks10:09:381
43379973,3cyclictest1-21systemd11:22:281
43379972,3cyclictest8755-21sendmail11:00:001
43379972,3cyclictest283-21systemd-journal07:24:421
43379971,3cyclictest25263-21munin-run10:30:021
43379970,2cyclictest27080-21ntp_states10:30:311
43329974,2cyclictest32644-21cron11:55:000
43329973,3cyclictest508-21dbus-daemon08:15:000
43329973,3cyclictest25712-21grep12:30:210
43329972,3cyclictest283-21systemd-journal10:34:480
43329971,3cyclictest8301-21ls12:39:590
43329971,3cyclictest26337-21latency_hist09:30:000
30349270,0sleep10-21swapper/109:11:311
23942270,0sleep00-21swapper/010:49:010
21078270,0sleep00-21swapper/011:47:150
43409966,0cyclictest0-21swapper/311:11:203
43409966,0cyclictest0-21swapper/310:16:003
43409964,1cyclictest10893-21df_inode09:40:153
43409963,2cyclictest9625-21ssh10:20:093
43409963,2cyclictest7369-21sh10:38:493
43409963,2cyclictest6607-21sh12:18:333
43409963,2cyclictest508-21dbus-daemon11:51:403
43409963,2cyclictest29741-11brltty09:19:503
43409963,2cyclictest28739-21rm10:11:053
43409963,2cyclictest283-21systemd-journal07:29:223
43409963,2cyclictest21693-21ssh09:26:183
43409963,2cyclictest19948-21ssh10:25:433
43409963,2cyclictest14169-21ssh12:03:143
43409963,2cyclictest0-21swapper/309:55:213
43409963,1cyclictest0-21swapper/311:25:463
43409962,3cyclictest508-21dbus-daemon08:20:013
43409962,3cyclictest30487-21cat08:05:163
43409962,3cyclictest283-21systemd-journal11:05:113
43409962,3cyclictest17551-21expr11:45:163
43409962,3cyclictest14659-21snmpd11:58:093
43409962,3cyclictest14659-21snmpd10:00:163
43409962,3cyclictest14659-21snmpd09:21:203
43409962,3cyclictest14659-21snmpd09:02:153
43409962,3cyclictest14659-21snmpd08:49:393
43409962,3cyclictest14659-21snmpd07:59:473
43409962,3cyclictest14659-21snmpd07:36:413
43409962,3cyclictest14659-21snmpd07:10:453
43409962,3cyclictest12212-21grep08:35:213
43409962,2cyclictest5539-1kworker/u9:210:05:573
43409962,2cyclictest283-21systemd-journal08:04:553
43409962,1cyclictest0-21swapper/311:21:083
43409960,6cyclictest0-21swapper/312:20:143
43409960,6cyclictest0-21swapper/312:12:323
43409960,4cyclictest283-21systemd-journal08:10:103
43409960,0cyclictest0-21swapper/310:54:593
43389966,0cyclictest0-21swapper/212:07:132
43389966,0cyclictest0-21swapper/211:00:432
43389963,2cyclictest29872-21sh10:12:002
43389963,2cyclictest28879-21sh12:32:022
43389963,2cyclictest28531-21ssh12:11:362
43389963,2cyclictest23205-21ssh09:47:492
43389963,2cyclictest1959-21awk11:15:262
43389963,2cyclictest18578-21cat10:25:202
43389963,2cyclictest17398-21chrt08:49:212
43389963,2cyclictest12212-21ssh10:00:442
43389963,2cyclictest1-21systemd09:00:132
43389963,2cyclictest11736-21in:imuxsock10:21:082
43389962,3cyclictest968-21cat08:10:222
43389962,3cyclictest508-21dbus-daemon10:45:012
43389962,3cyclictest508-21dbus-daemon07:29:432
43389962,3cyclictest2752-21cat09:35:132
43389962,3cyclictest26523-21cat10:50:212
43389962,3cyclictest25352-21cat09:05:152
43389962,3cyclictest17659-21cat11:45:172
43389962,3cyclictest17648-21expr11:05:172
43389962,3cyclictest14659-21snmpd11:41:242
43389962,3cyclictest14659-21snmpd10:31:342
43389962,3cyclictest14659-21snmpd09:25:222
43389962,3cyclictest14659-21snmpd09:12:502
43389962,3cyclictest14659-21snmpd08:57:042
43389962,3cyclictest14659-21snmpd08:50:492
43389962,3cyclictest14659-21snmpd08:38:302
43389962,3cyclictest14659-21snmpd08:20:002
43389962,3cyclictest14659-21snmpd08:07:302
43389962,3cyclictest13695-21cat08:40:162
43389962,3cyclictest1301-21tune2fs12:35:172
43389962,1cyclictest0-21swapper/209:40:292
43389961,3cyclictest283-21systemd-journal07:48:532
43389961,3cyclictest283-21systemd-journal07:33:112
43389961,3cyclictest19563-21rm12:25:592
43389960,6cyclictest0-21swapper/211:31:012
43389960,6cyclictest0-21swapper/211:11:242
43389960,4cyclictest283-21systemd-journal08:34:462
43389960,0cyclictest0-21swapper/210:18:512
43379966,0cyclictest0-21swapper/110:20:431
43379966,0cyclictest0-21swapper/109:28:151
43379963,2cyclictest668-21runrttasks08:46:121
43379963,2cyclictest508-21dbus-daemon12:10:511
43379963,2cyclictest29312-21ssh09:51:171
43379963,2cyclictest2918-21cat09:15:121
43379963,2cyclictest20966-21rm10:46:391
43379963,2cyclictest20630-21cron07:45:001
43379963,2cyclictest11736-21in:imuxsock11:48:111
43379963,2cyclictest11261-21sh12:00:531
43379963,1cyclictest283-21systemd-journal10:04:561
43379963,1cyclictest0-21swapper/111:04:041
43379962,3cyclictest8091-21cat07:15:161
43379962,3cyclictest28671-21seq10:51:211
43379962,3cyclictest283-21systemd-journal08:59:351
43379962,3cyclictest27305-21df09:30:171
43379962,3cyclictest18056-21cat11:05:181
43379962,3cyclictest17701-21cat11:25:161
43379962,3cyclictest14659-21snmpd12:06:181
43379962,3cyclictest14659-21snmpd11:52:201
43379962,3cyclictest14659-21snmpd10:17:181
43379962,3cyclictest14659-21snmpd09:42:481
43379962,3cyclictest14659-21snmpd08:27:171
43379962,3cyclictest14659-21snmpd08:15:171
43379962,3cyclictest14659-21snmpd07:52:411
43379962,3cyclictest14659-21snmpd07:49:381
43379962,3cyclictest14659-21snmpd07:35:101
43379962,3cyclictest14659-21snmpd07:30:191
43379962,3cyclictest12620-21sshd08:38:511
43379962,1cyclictest0-21swapper/111:37:331
43379962,1cyclictest0-21swapper/111:10:011
43379961,3cyclictest283-21systemd-journal09:23:091
43379961,3cyclictest283-21systemd-journal08:05:101
43379960,6cyclictest0-21swapper/112:37:431
43379960,6cyclictest0-21swapper/111:40:421
43379960,6cyclictest0-21swapper/108:50:011
43379960,0cyclictest0-21swapper/109:01:121
43329966,0cyclictest0-21swapper/011:44:260
43329963,2cyclictest9325-21ssh11:20:180
43329963,2cyclictest3843-21ssh11:16:280
43329963,2cyclictest28375-21sh11:31:370
43329963,2cyclictest283-21systemd-journal10:39:490
43329963,2cyclictest25042-21ssh11:10:080
43329963,2cyclictest23603-21tune2fs07:50:180
43329963,2cyclictest22015-21ssh09:46:520
43329963,2cyclictest17004-21ssh10:24:500
43329963,2cyclictest14659-21snmpd12:23:130
43329963,2cyclictest14659-21snmpd11:36:570
43329963,2cyclictest14659-21snmpd07:37:020
43329963,2cyclictest14176-21cat07:30:160
43329963,2cyclictest13820-21memory08:40:180
43329963,2cyclictest12691-21memory07:25:170
43329963,1cyclictest0-21swapper/009:24:000
43329962,3cyclictest9963-21sed12:00:200
43329962,3cyclictest6584-21date07:15:010
43329962,3cyclictest18122-21ntpq12:25:200
43329962,3cyclictest14659-21snmpd12:16:220
43329962,3cyclictest14659-21snmpd11:55:260
43329962,3cyclictest14659-21snmpd11:03:250
43329962,3cyclictest14659-21snmpd10:57:520
43329962,3cyclictest14659-21snmpd10:25:460
43329962,3cyclictest14659-21snmpd10:12:420
43329962,3cyclictest14659-21snmpd10:06:030
43329962,3cyclictest14659-21snmpd09:42:060
43329962,3cyclictest14659-21snmpd09:33:110
43329962,3cyclictest14659-21snmpd09:10:030
43329962,3cyclictest14659-21snmpd08:32:030
43329962,3cyclictest14659-21snmpd08:28:570
43329962,3cyclictest14659-21snmpd08:00:180
43329962,3cyclictest14659-21snmpd07:57:540
43329962,3cyclictest14659-21snmpd07:20:160
43329962,1cyclictest22187-21turbostat08:59:590
43329962,1cyclictest0-21swapper/010:01:140
43329961,1cyclictest0-21swapper/010:40:220
43329960,6cyclictest0-21swapper/011:26:100
43329960,6cyclictest0-21swapper/009:51:220
43329960,5cyclictest12682-21sshd08:39:310
43329960,0cyclictest0-21swapper/010:16:530
24760260,0chrt0-21swapper/210:09:202
43409955,0cyclictest29741-11brltty08:50:123
43409953,1cyclictest96682sleep307:20:163
43409953,1cyclictest9209-21if_enp4s008:30:183
43409952,2cyclictest24943-21expr09:05:123
43409952,2cyclictest24097-21awk07:50:203
43409952,2cyclictest14659-21snmpd08:40:003
43409952,2cyclictest14125-21cpu07:30:173
43409951,3cyclictest6407-21ls08:25:143
43409951,3cyclictest25660-21ssh10:30:093
43409951,3cyclictest14659-21snmpd08:20:063
43409951,3cyclictest0-21swapper/311:30:163
43409950,3cyclictest283-21systemd-journal08:56:273
43389953,1cyclictest10180-21tr07:20:172
43389952,2cyclictest865-21ssh12:15:132
43389952,2cyclictest6680-21ls08:25:132
43389952,2cyclictest4756-21cat08:20:202
43389952,2cyclictest25545-21cat07:55:132
43389952,2cyclictest25028-21ssh11:50:132
43389952,2cyclictest16951-21cat07:35:142
43389952,2cyclictest14659-21snmpd07:50:132
43389952,2cyclictest1404-21ssh10:55:202
43389952,2cyclictest0-21swapper/209:55:382
43389951,3cyclictest7090-21tune2fs07:15:022
43389951,3cyclictest18658-21/usr/sbin/munin07:40:162
43389950,3cyclictest283-21systemd-journal08:04:202
43389950,3cyclictest283-21systemd-journal07:10:052
43379952,2cyclictest8933-21df_inode08:30:151
43379952,2cyclictest655-21ssh11:15:091
43379952,2cyclictest4694-21grep07:10:131
43379952,2cyclictest25464-21sort07:55:141
43379952,2cyclictest24940-21expr09:05:121
43379952,2cyclictest18463-21cpu09:45:151
43379952,2cyclictest14659-21snmpd11:30:151
43379952,2cyclictest13251-21expr08:40:151
43379952,2cyclictest11857-21taskset07:25:021
43379951,3cyclictest8965-21/usr/sbin/munin12:20:151
43379951,3cyclictest4341-21expr08:20:161
43379951,3cyclictest32020-21/usr/sbin/munin08:10:151
43379951,1cyclictest0-21swapper/112:30:221
43379951,1cyclictest0-21swapper/110:40:111
43329953,2cyclictest508-21dbus-daemon09:08:080
43329953,1cyclictest29931-21cpuspeed_turbos08:05:130
43329952,2cyclictest7590-21lldpd10:50:070
43329952,2cyclictest7379-21chrt07:15:030
43329952,2cyclictest4771-21cat08:20:190
43329952,2cyclictest22735-21cat09:00:150
43329952,2cyclictest19557-21vmstat08:50:230
43329952,2cyclictest17096-21ssh11:05:050
43329951,3cyclictest14659-21snmpd07:40:140
43329950,2cyclictest20881-21/usr/sbin/munin07:45:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional