You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-09 - 17:01

x86 Intel Core i5-8265UE @1600 MHz, Linux 5.10.41-rt42 (Profile)

Latency plot of system in rack #a, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackaslot8.osadl.org (updated Wed Dec 07, 2022 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
146462331271,17sleep20-21swapper/207:09:142
146712294264,19sleep10-21swapper/107:09:331
127252275243,22sleep00-21swapper/007:05:060
127282267236,20sleep30-21swapper/307:05:093
103122650,0sleep30-21swapper/312:34:133
158992600,0sleep30-21swapper/310:36:243
128352600,0sleep00-21swapper/009:14:520
32312570,0sleep30-21swapper/312:30:003
307032570,0sleep00-21swapper/011:25:300
103192520,0sleep10-21swapper/111:13:341
147022170,0sleep10-21swapper/112:16:131
290432160,0sleep30-21swapper/310:45:133
230532160,0sleep00-21swapper/010:00:250
175242150,0sleep20-21swapper/209:37:242
22122140,0sleep00-21swapper/007:50:230
39482130,0sleep20-21swapper/211:30:012
317432120,0sleep30-21swapper/311:06:153
277142120,0sleep00-21swapper/008:45:250
14816991110,1cyclictest28-21ksoftirqd/211:00:012
79582100,1sleep10-21swapper/109:31:101
1481299100,9cyclictest8023-21ssh12:12:151
148219995,3cyclictest1-21systemd12:10:403
148219994,3cyclictest283-21systemd-journal08:37:513
148169983,3cyclictest283-21systemd-journal07:51:262
148129983,3cyclictest283-21systemd-journal10:59:461
148129983,2cyclictest29109-21dump-pmu-power09:25:001
148089984,3cyclictest283-21systemd-journal10:46:000
148089984,2cyclictest508-21dbus-daemon10:27:140
148089983,3cyclictest27468-21munin-run12:25:000
148089980,7cyclictest12703-21rm10:55:110
5070270,0sleep30-21swapper/309:50:113
3753270,0chrt0-21swapper/310:09:133
22640270,0sleep00-21swapper/011:40:440
148219976,1cyclictest34-21ksoftirqd/311:25:013
148219976,1cyclictest34-21ksoftirqd/307:15:013
148219975,2cyclictest34-21ksoftirqd/308:55:003
148219975,1cyclictest34-21ksoftirqd/312:40:003
148219974,2cyclictest11736-21in:imuxsock08:41:173
148219973,2cyclictest283-21systemd-journal10:40:353
148219973,2cyclictest283-21systemd-journal07:45:113
148219973,1cyclictest0-21swapper/311:18:183
148219973,1cyclictest0-21swapper/311:13:423
148219973,1cyclictest0-21swapper/309:58:513
148219972,3cyclictest283-21systemd-journal08:57:583
148169976,1cyclictest28-21ksoftirqd/211:10:012
148169975,2cyclictest28-21ksoftirqd/209:30:002
148169975,1cyclictest5785-21memory10:30:192
148169974,2cyclictest28-21ksoftirqd/209:40:022
148169974,2cyclictest28-21ksoftirqd/208:45:002
148169973,3cyclictest508-21dbus-daemon07:14:582
148169973,3cyclictest14659-21snmpd09:52:342
148169973,2cyclictest283-21systemd-journal08:39:372
148169973,1cyclictest0-21swapper/211:15:382
148169972,3cyclictest502-21systemd-logind10:14:182
148169972,3cyclictest283-21systemd-journal11:11:262
148169972,3cyclictest283-21systemd-journal07:30:482
148169970,7cyclictest28358-21ls07:35:252
148169970,4cyclictest28392-21munin-run11:05:012
148169970,4cyclictest21272-21munin-run08:35:002
148129976,1cyclictest6948-21ssh12:31:301
148129974,2cyclictest283-21systemd-journal08:22:311
148129973,3cyclictest283-21systemd-journal11:03:191
148129973,3cyclictest14659-21snmpd10:43:321
148129972,2cyclictest3453-21dump-pmu-power12:10:001
148129972,1cyclictest0-21swapper/110:50:191
148129970,4cyclictest26814-21awk07:35:011
148129970,4cyclictest20802-21latency_hist09:40:011
148089976,1cyclictest12-21ksoftirqd/012:00:000
148089975,1cyclictest9944-21ssh10:33:030
148089974,2cyclictest508-21dbus-daemon09:25:000
148089973,3cyclictest21997-21expr09:40:210
148089973,3cyclictest19116-21gdbus11:40:010
148089973,3cyclictest14659-21snmpd12:09:280
148089973,3cyclictest14659-21snmpd09:37:030
148089972,2cyclictest12-21ksoftirqd/008:37:550
10345270,0sleep10-21swapper/109:12:521
148219966,0cyclictest0-21swapper/309:26:143
148219965,1cyclictest34-21ksoftirqd/307:15:013
148219964,1cyclictest6521-21hddtemp_smartct08:00:203
148219964,1cyclictest4721-21diskmemload11:35:263
148219963,2cyclictest7265-21rm10:10:363
148219963,2cyclictest6397-21grep09:10:173
148219963,2cyclictest3945-21expr07:55:133
148219963,2cyclictest22981-21ssh12:01:103
148219963,2cyclictest21963-21ntp_states11:00:223
148219963,2cyclictest16750-21rm12:17:503
148219963,2cyclictest15588-21ssh10:15:523
148219963,2cyclictest14659-21snmpd10:28:443
148219963,2cyclictest11736-21in:imuxsock11:47:193
148219963,2cyclictest11736-21in:imuxsock11:27:173
148219963,2cyclictest10233-21ssh09:33:003
148219963,2cyclictest0-21swapper/310:00:553
148219963,1cyclictest0-21swapper/311:41:223
148219963,1cyclictest0-21swapper/310:20:063
148219962,3cyclictest9471-21ssh11:33:033
148219962,3cyclictest7680-21seq11:51:493
148219962,3cyclictest27673-21which07:35:203
148219962,3cyclictest26934-21cat08:45:193
148219962,3cyclictest17198-21expr08:25:163
148219962,3cyclictest14659-21snmpd12:20:233
148219962,3cyclictest14659-21snmpd11:57:553
148219962,3cyclictest14659-21snmpd10:54:243
148219962,3cyclictest14659-21snmpd09:45:073
148219962,3cyclictest14659-21snmpd09:17:203
148219962,3cyclictest14659-21snmpd09:06:443
148219962,3cyclictest14659-21snmpd08:23:403
148219962,3cyclictest14659-21snmpd08:17:463
148219962,3cyclictest14659-21snmpd08:10:193
148219962,3cyclictest14659-21snmpd07:50:393
148219962,1cyclictest0-21swapper/310:59:013
148219962,1cyclictest0-21swapper/310:31:403
148219962,1cyclictest0-21swapper/309:22:083
148219961,3cyclictest283-21systemd-journal08:30:513
148219961,3cyclictest283-21systemd-journal07:43:183
148219960,6cyclictest0-21swapper/312:09:213
148219960,2cyclictest24813-21/usr/sbin/munin07:30:153
148219960,0cyclictest0-21swapper/309:37:333
148169966,0cyclictest0-21swapper/211:23:082
148169965,1cyclictest28-21ksoftirqd/208:20:002
148169963,2cyclictest8787-21sh11:52:422
148169963,2cyclictest8098-21sh10:51:402
148169963,2cyclictest508-21dbus-daemon09:57:372
148169963,2cyclictest3802-21ssh12:30:152
148169963,2cyclictest283-21systemd-journal10:47:112
148169963,2cyclictest28226-21ssh10:24:372
148169963,2cyclictest14659-21snmpd12:01:042
148169963,2cyclictest14659-21snmpd09:48:132
148169963,2cyclictest11387-21ssh12:14:592
148169962,3cyclictest30382-21cat10:05:202
148169962,3cyclictest283-21systemd-journal07:48:442
148169962,3cyclictest22424-21basename10:40:312
148169962,3cyclictest14855-21apt-get08:20:192
148169962,3cyclictest14659-21snmpd12:23:532
148169962,3cyclictest14659-21snmpd12:18:172
148169962,3cyclictest14659-21snmpd11:44:282
148169962,3cyclictest14659-21snmpd10:15:382
148169962,3cyclictest14659-21snmpd10:04:492
148169962,3cyclictest14659-21snmpd09:13:202
148169962,3cyclictest14659-21snmpd08:52:232
148169962,3cyclictest14659-21snmpd08:41:562
148169962,3cyclictest14659-21snmpd08:28:252
148169962,3cyclictest14659-21snmpd08:12:372
148169962,3cyclictest14659-21snmpd08:06:012
148169962,3cyclictest14659-21snmpd08:00:462
148169962,3cyclictest14659-21snmpd07:23:182
148169962,3cyclictest13516-21find09:15:142
148169962,2cyclictest283-21systemd-journal07:45:012
148169962,1cyclictest141rcu_preempt11:31:362
148169962,1cyclictest0-21swapper/212:26:042
148169962,1cyclictest0-21swapper/212:05:502
148169962,1cyclictest0-21swapper/211:35:112
148169962,1cyclictest0-21swapper/210:35:202
148169962,1cyclictest0-21swapper/210:29:302
148169961,3cyclictest283-21systemd-journal08:59:132
148169960,6cyclictest0-21swapper/211:46:062
148129966,0cyclictest0-21swapper/111:26:331
148129963,2cyclictest847-21seq07:46:111
148129963,2cyclictest668-21runrttasks11:05:011
148129963,2cyclictest29953-21grep07:40:171
148129963,2cyclictest29505-21apt-get09:45:121
148129963,2cyclictest28903-21sed12:25:211
148129963,2cyclictest28036-21ntp_states07:35:231
148129963,2cyclictest24004-21ssh12:22:151
148129963,2cyclictest21198-21df_inode11:20:161
148129963,2cyclictest20674-21cat12:00:201
148129963,2cyclictest19579-21ssh11:39:451
148129963,2cyclictest15767-21cat07:10:171
148129963,2cyclictest14659-21snmpd09:05:321
148129963,2cyclictest11736-21in:imuxsock07:22:441
148129963,2cyclictest0-21swapper/110:31:001
148129963,1cyclictest0-21swapper/109:26:361
148129962,3cyclictest5996-21basename10:10:191
148129962,3cyclictest5885-21sshd08:00:101
148129962,3cyclictest4338-21cat07:55:171
148129962,3cyclictest29535-21basename10:45:161
148129962,3cyclictest29460-21awk08:50:201
148129962,3cyclictest283-21systemd-journal09:58:521
148129962,3cyclictest26619-21cat08:45:171
148129962,3cyclictest21699-21expr10:20:171
148129962,3cyclictest14659-21snmpd10:29:201
148129962,3cyclictest12616-21cat12:35:201
148129962,3cyclictest1-21systemd10:05:331
148129962,2cyclictest283-21systemd-journal11:55:141
148129962,1cyclictest0-21swapper/111:48:481
148129961,3cyclictest283-21systemd-journal11:15:371
148129960,6cyclictest0-21swapper/109:40:491
148129960,3cyclictest5477-21cstates09:50:161
148129960,0cyclictest0-21swapper/111:51:561
148129960,0cyclictest0-21swapper/110:00:471
148089964,2cyclictest11736-21in:imuxsock10:50:020
148089963,2cyclictest9005-21sh11:52:530
148089963,2cyclictest6584-21find10:10:240
148089963,2cyclictest5708-21proc_pri11:30:260
148089963,2cyclictest508-21dbus-daemon11:06:010
148089963,2cyclictest508-21dbus-daemon08:01:400
148089963,2cyclictest3487-21gdbus12:30:010
148089963,2cyclictest3311-21ssh10:08:520
148089963,2cyclictest283-21systemd-journal08:27:560
148089963,2cyclictest14659-21snmpd07:43:360
148089963,2cyclictest14312-21if_enp4s009:15:200
148089962,3cyclictest508-21dbus-daemon07:59:090
148089962,3cyclictest31075-21cat09:25:210
148089962,3cyclictest30714-21apt-get08:55:130
148089962,3cyclictest283-21systemd-journal08:23:110
148089962,3cyclictest24017-21sh10:21:110
148089962,3cyclictest23281-21cat07:25:170
148089962,3cyclictest2299-21sshd09:02:560
148089962,3cyclictest22386-21cat11:00:240
148089962,3cyclictest20670-21cat07:20:200
148089962,3cyclictest18371-21expr07:15:170
148089962,3cyclictest14659-21snmpd12:35:020
148089962,3cyclictest14659-21snmpd12:28:590
148089962,3cyclictest14659-21snmpd12:00:160
148089962,3cyclictest14659-21snmpd11:46:100
148089962,3cyclictest14659-21snmpd11:10:510
148089962,3cyclictest14659-21snmpd10:41:200
148089962,3cyclictest14659-21snmpd10:37:020
148089962,3cyclictest14659-21snmpd09:50:190
148089962,3cyclictest14659-21snmpd08:30:590
148089962,3cyclictest14659-21snmpd08:18:310
148089962,3cyclictest14659-21snmpd07:10:290
148089962,3cyclictest14477-21grep10:15:230
148089962,3cyclictest13532-21cat11:15:240
148089962,1cyclictest0-21swapper/012:10:160
148089962,1cyclictest0-21swapper/009:46:520
148089960,5cyclictest0-21swapper/009:10:000
148089960,2cyclictest13978-21uname09:55:190
148089960,2cyclictest12-21ksoftirqd/008:50:140
148089960,1cyclictest0-21swapper/007:30:130
148089960,0cyclictest0-21swapper/012:15:170
14490260,0sleep20-21swapper/212:36:082
148219953,1cyclictest34-21ksoftirqd/308:05:183
148219951,3cyclictest911-21df09:00:143
148219951,3cyclictest283-21systemd-journal09:40:023
148219951,3cyclictest21077-21cat07:20:233
148219951,3cyclictest14659-21snmpd07:25:093
148169955,0cyclictest0-21swapper/209:20:042
148169952,2cyclictest6352-21uname09:30:222
148169952,2cyclictest4341-21cat07:55:162
148169952,2cyclictest3347-21cut09:05:252
148169952,2cyclictest18776-21basename07:15:192
148169952,2cyclictest14659-21snmpd07:25:032
148169952,2cyclictest11943-21ssh11:55:032
148169951,2cyclictest29741-11brltty09:00:032
148129953,1cyclictest13463-21grep10:15:161
148129952,2cyclictest508-21dbus-daemon08:25:011
148129952,2cyclictest283-21systemd-journal11:30:011
148129952,2cyclictest22351-21sh11:40:301
148129952,2cyclictest21808-21expr08:35:171
148129952,2cyclictest19615-21expr08:30:141
148129952,2cyclictest17906-21threads07:15:021
148129952,2cyclictest14659-21snmpd08:40:081
148129952,2cyclictest1448-21dpkg07:50:171
148129952,2cyclictest1367-21cat09:00:181
148129952,2cyclictest13603-21sh09:15:171
148129952,2cyclictest13012-21ssh10:35:181
148129951,3cyclictest12881-21cat08:15:131
148129950,4cyclictest8515-21expr08:05:181
148129950,2cyclictest22591-21/usr/sbin/munin07:25:231
148089953,1cyclictest9095-21ntp_offset08:05:230
148089953,1cyclictest5083-21sh09:30:080
148089953,1cyclictest24174-21sort08:40:140
148089952,2cyclictest31697-21cpuspeed07:45:130
148089952,2cyclictest20771-21awk11:20:140
148089952,2cyclictest14659-21snmpd07:35:030
148089952,2cyclictest10408-21cpu08:10:120
148129942,1cyclictest10356-21acpi08:10:111
148129940,3cyclictest30674-21taskset08:55:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional