You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-31 - 01:29

x86 Intel Core i5-8265UE @1600 MHz, Linux 5.10.41-rt42 (Profile)

Latency plot of system in rack #a, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Thu Mar 30, 2023 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
181882336276,17sleep20-21swapper/207:08:152
182392310280,19sleep10-21swapper/107:08:561
180422283257,16sleep30-21swapper/307:06:273
183382279249,19sleep00-21swapper/007:09:590
233042630,4sleep122-21ksoftirqd/112:30:201
200582620,0sleep10-21swapper/109:42:471
90232580,0sleep10-21swapper/109:10:111
203682540,0sleep10-21swapper/110:05:211
219662530,0sleep20-21swapper/211:49:582
327662390,0sleep20-21swapper/210:14:402
137352190,0sleep00-21swapper/009:38:290
292472180,0sleep20-21swapper/212:14:322
90192170,0sleep20-21swapper/212:21:252
46652160,0sleep10-21swapper/109:31:131
79772140,0sleep20-21swapper/210:40:182
253012140,0sleep20-21swapper/211:31:122
92432130,0sleep10-21swapper/111:00:231
263292120,0sleep00-21swapper/010:30:430
151812120,0sleep00-21swapper/011:24:450
184479986,1cyclictest10455-21ssh09:59:353
184479984,3cyclictest3758-21snmpd09:43:373
184429986,1cyclictest20317-21ssh11:07:342
184429984,3cyclictest13259-21sshd11:03:202
184429984,2cyclictest283-21systemd-journal08:03:252
184429983,3cyclictest283-21systemd-journal08:58:542
184429982,2cyclictest21355-21fschecks_count08:25:202
184429982,2cyclictest21355-21fschecks_count08:25:192
184389985,2cyclictest3758-21snmpd09:22:121
184389985,2cyclictest22-21ksoftirqd/110:40:011
184389984,3cyclictest308-21systemd-udevd07:43:201
184389984,3cyclictest283-21systemd-journal10:30:411
184369985,2cyclictest508-21dbus-daemon09:01:490
184479976,1cyclictest34-21ksoftirqd/310:10:013
184479975,1cyclictest10944-21sh11:21:233
184479974,2cyclictest283-21systemd-journal10:59:403
184479974,2cyclictest283-21systemd-journal09:05:103
184479973,3cyclictest30219-21ntpq09:50:273
184479973,3cyclictest1-21systemd12:28:193
184479973,2cyclictest283-21systemd-journal11:35:203
184479972,3cyclictest283-21systemd-journal11:05:433
184479972,3cyclictest283-21systemd-journal09:15:013
184429973,3cyclictest3758-21snmpd07:17:252
184429973,3cyclictest339-21grep10:35:172
184429973,3cyclictest283-21systemd-journal11:29:552
184429973,2cyclictest283-21systemd-journal10:54:532
184429973,2cyclictest283-21systemd-journal08:15:312
184429973,1cyclictest0-21swapper/210:47:262
184429972,3cyclictest21506-21dump-pmu-power12:30:012
184429971,3cyclictest27372-21tr08:40:002
184389975,1cyclictest28634-21cron09:50:011
184389975,1cyclictest22-21ksoftirqd/108:19:411
184389973,3cyclictest3758-21snmpd07:35:271
184389973,3cyclictest283-21systemd-journal08:21:211
184389973,2cyclictest508-21dbus-daemon10:19:491
184389973,2cyclictest283-21systemd-journal11:49:051
184389970,4cyclictest23497-21sendmail07:20:011
184369974,2cyclictest11736-21in:imuxsock10:08:350
184369973,3cyclictest3758-21snmpd10:48:150
184369973,3cyclictest3758-21snmpd07:50:020
184369973,3cyclictest11736-21in:imuxsock10:11:300
184369972,3cyclictest29046-21turbostat09:25:000
184369972,3cyclictest283-21systemd-journal11:55:010
184369972,3cyclictest283-21systemd-journal10:39:400
184369972,3cyclictest283-21systemd-journal10:21:050
184369970,7cyclictest0-21swapper/010:56:560
184369970,6cyclictest668-21runrttasks09:31:510
184369970,6cyclictest15457-21smart_sda08:10:290
184369970,5cyclictest15345-21awk11:05:000
13417270,0sleep10-21swapper/111:44:431
184479966,0cyclictest0-21swapper/310:16:543
184479966,0cyclictest0-21swapper/309:25:243
184479966,0cyclictest0-21swapper/308:10:283
184479964,2cyclictest508-21dbus-daemon10:25:163
184479963,2cyclictest718-21ssh12:36:103
184479963,2cyclictest4319-21ssh11:58:563
184479963,2cyclictest3758-21snmpd07:33:493
184479963,2cyclictest26098-21ssh10:30:313
184479963,2cyclictest23777-21memory09:20:243
184479963,2cyclictest23073-21ntpq12:30:193
184479963,2cyclictest16799-21cat10:45:203
184479963,2cyclictest14237-21chrt09:13:533
184479963,2cyclictest13752-21rm10:00:453
184479963,2cyclictest10168-21sh11:00:543
184479962,3cyclictest3758-21snmpd11:18:363
184479962,3cyclictest3758-21snmpd08:58:443
184479962,3cyclictest3758-21snmpd08:36:513
184479962,3cyclictest3758-21snmpd08:26:503
184479962,3cyclictest3758-21snmpd08:26:503
184479962,3cyclictest24808-21seq12:10:583
184479962,3cyclictest23969-21cat07:20:183
184479962,3cyclictest22763-21cat09:45:183
184479962,3cyclictest11736-21in:imuxsock10:40:423
184479962,3cyclictest10481-21cat08:00:253
184479962,3cyclictest10258-21basename10:20:253
184479962,1cyclictest0-21swapper/311:40:463
184479962,1cyclictest0-21swapper/311:25:513
184479962,1cyclictest0-21swapper/309:30:123
184479961,3cyclictest283-21systemd-journal11:45:213
184479961,3cyclictest283-21systemd-journal07:40:293
184479960,6cyclictest0-21swapper/312:24:023
184479960,6cyclictest0-21swapper/311:54:053
184479960,6cyclictest0-21swapper/309:35:163
184479960,6cyclictest0-21swapper/308:45:163
184479960,2cyclictest3819-21ntp_states07:45:303
184479960,0cyclictest0-21swapper/312:17:043
184479960,0cyclictest0-21swapper/312:17:043
184479960,0cyclictest0-21swapper/311:13:323
184479960,0cyclictest0-21swapper/310:53:353
184479960,0cyclictest0-21swapper/310:38:033
184429966,0cyclictest0-21swapper/210:18:162
184429966,0cyclictest0-21swapper/209:16:472
184429963,2cyclictest508-21dbus-daemon08:25:012
184429963,2cyclictest508-21dbus-daemon08:10:562
184429963,2cyclictest3805-21ssh11:38:292
184429963,2cyclictest3758-21snmpd09:01:452
184429963,2cyclictest30355-21fschecks_time08:45:202
184429963,2cyclictest29076-21ssh11:54:122
184429963,2cyclictest21181-21ssh12:09:322
184429963,2cyclictest11948-21ssh10:21:012
184429963,2cyclictest11736-21in:imuxsock11:20:072
184429963,2cyclictest11736-21in:imuxsock09:53:402
184429962,3cyclictest9614-21ls08:00:012
184429962,3cyclictest508-21dbus-daemon07:25:012
184429962,3cyclictest3758-21snmpd12:36:152
184429962,3cyclictest3758-21snmpd11:55:072
184429962,3cyclictest3758-21snmpd10:01:592
184429962,3cyclictest3758-21snmpd09:55:112
184429962,3cyclictest3758-21snmpd09:42:232
184429962,3cyclictest3758-21snmpd09:29:282
184429962,3cyclictest3758-21snmpd09:20:042
184429962,3cyclictest3758-21snmpd09:05:182
184429962,3cyclictest3758-21snmpd07:47:192
184429962,3cyclictest3758-21snmpd07:32:432
184429962,3cyclictest3758-21snmpd07:10:252
184429962,3cyclictest29741-11brltty07:50:542
184429962,3cyclictest28199-21ssh12:33:542
184429962,3cyclictest27989-21cat07:30:012
184429962,3cyclictest23427-21cat09:45:232
184429962,3cyclictest11736-21in:imuxsock12:00:222
184429962,3cyclictest0-21swapper/212:17:482
184429962,3cyclictest0-21swapper/212:17:482
184429962,1cyclictest0-21swapper/211:40:542
184429962,1cyclictest0-21swapper/211:16:042
184429962,1cyclictest0-21swapper/210:59:442
184429962,1cyclictest0-21swapper/210:07:462
184429962,1cyclictest0-21swapper/209:36:012
184429961,3cyclictest283-21systemd-journal08:52:402
184429961,3cyclictest283-21systemd-journal08:31:212
184429960,6cyclictest0-21swapper/210:27:022
184429960,5cyclictest508-21dbus-daemon10:31:312
184389966,0cyclictest0-21swapper/112:14:081
184389965,1cyclictest22-21ksoftirqd/107:15:011
184389964,1cyclictest668-21runrttasks08:50:211
184389963,2cyclictest3758-21snmpd11:57:111
184389963,2cyclictest3758-21snmpd07:30:331
184389963,2cyclictest3289-21rm11:38:061
184389963,2cyclictest283-21systemd-journal09:06:271
184389963,2cyclictest24041-21tr08:30:261
184389963,2cyclictest21878-21awk08:25:231
184389963,2cyclictest21878-21awk08:25:221
184389963,2cyclictest18339-21ssh12:07:171
184389963,1cyclictest0-21swapper/111:26:341
184389962,3cyclictest7981-21/usr/sbin/munin10:40:301
184389962,3cyclictest508-21dbus-daemon09:55:391
184389962,3cyclictest3758-21snmpd12:35:021
184389962,3cyclictest3758-21snmpd12:23:321
184389962,3cyclictest3758-21snmpd10:45:371
184389962,3cyclictest3758-21snmpd10:26:571
184389962,3cyclictest3758-21snmpd10:12:021
184389962,3cyclictest3758-21snmpd10:04:271
184389962,3cyclictest3758-21snmpd09:52:591
184389962,3cyclictest3758-21snmpd09:15:141
184389962,3cyclictest3758-21snmpd08:42:001
184389962,3cyclictest3758-21snmpd08:02:501
184389962,3cyclictest3758-21snmpd07:56:291
184389962,3cyclictest3758-21snmpd07:20:041
184389962,3cyclictest32748-21expr11:15:211
184389962,3cyclictest308-21systemd-udevd07:47:201
184389962,3cyclictest30705-21ls08:45:221
184389962,1cyclictest0-21swapper/112:17:551
184389962,1cyclictest0-21swapper/112:17:551
184389962,1cyclictest0-21swapper/111:51:221
184389962,1cyclictest0-21swapper/111:21:021
184389962,1cyclictest0-21swapper/111:14:011
184389962,1cyclictest0-21swapper/110:54:121
184389960,6cyclictest0-21swapper/109:35:571
184389960,2cyclictest1727-21awk09:29:591
184369966,0cyclictest0-21swapper/012:36:090
184369964,1cyclictest29585-21cron08:45:010
184369964,1cyclictest10194-21df_inode08:00:230
184369963,2cyclictest7642-21ssh10:40:020
184369963,2cyclictest3758-21snmpd11:55:170
184369963,2cyclictest3758-21snmpd11:26:160
184369963,2cyclictest28390-21ssh10:52:220
184369963,2cyclictest26826-21sed08:35:290
184369963,2cyclictest25584-21rm09:46:420
184369963,2cyclictest24421-21expr11:10:180
184369963,2cyclictest23049-21grep07:15:300
184369963,2cyclictest11736-21in:imuxsock08:59:440
184369963,2cyclictest0-21swapper/012:20:230
184369962,3cyclictest8216-21cat07:55:200
184369962,3cyclictest738-21df07:40:220
184369962,3cyclictest6604-21expr12:00:170
184369962,3cyclictest3758-21snmpd12:15:230
184369962,3cyclictest3758-21snmpd12:15:230
184369962,3cyclictest3758-21snmpd12:12:270
184369962,3cyclictest3758-21snmpd12:07:180
184369962,3cyclictest3758-21snmpd11:46:080
184369962,3cyclictest3758-21snmpd11:41:470
184369962,3cyclictest3758-21snmpd11:32:280
184369962,3cyclictest3758-21snmpd11:08:580
184369962,3cyclictest3758-21snmpd09:15:270
184369962,3cyclictest3758-21snmpd09:11:440
184369962,3cyclictest3758-21snmpd08:54:480
184369962,3cyclictest3758-21snmpd08:29:230
184369962,3cyclictest3758-21snmpd08:29:230
184369962,3cyclictest3758-21snmpd08:05:370
184369962,3cyclictest3758-21snmpd07:35:490
184369962,3cyclictest3758-21snmpd07:26:360
184369962,3cyclictest31396-21smartctl11:35:230
184369962,3cyclictest283-21systemd-journal09:07:150
184369962,3cyclictest283-21systemd-journal08:34:370
184369962,3cyclictest283-21systemd-journal08:22:280
184369962,3cyclictest19155-21diskmemload09:41:530
184369962,3cyclictest1-21systemd08:15:300
184369962,1cyclictest0-21swapper/012:25:460
184369961,3cyclictest283-21systemd-journal11:17:490
184369961,3cyclictest283-21systemd-journal09:29:210
184369961,2cyclictest28560-21ssh12:34:100
184369960,6cyclictest0-21swapper/009:50:030
17247260,0sleep00-21swapper/010:25:070
184479953,1cyclictest34-21ksoftirqd/308:49:593
184479953,1cyclictest28454-21sed08:40:233
184479952,2cyclictest5664-21cat07:50:143
184479952,2cyclictest4442-21cat09:00:213
184479952,2cyclictest28091-21sh10:10:353
184479952,2cyclictest22621-21sort11:30:143
184479952,2cyclictest21493-21taskset07:15:153
184479952,2cyclictest19341-21cat07:10:173
184479952,2cyclictest18846-21expr08:20:213
184479952,2cyclictest16537-21cat08:15:193
184479952,2cyclictest15118-21cut12:05:263
184479951,3cyclictest6243-21/usr/sbin/munin12:00:153
184479951,3cyclictest3758-21snmpd08:30:153
184479951,3cyclictest3758-21snmpd08:05:143
184479951,3cyclictest30780-21expr07:35:223
184479950,3cyclictest283-21systemd-journal07:55:043
184479950,3cyclictest283-21systemd-journal07:25:033
184429952,2cyclictest31510-21ntp_states07:35:322
184429952,2cyclictest30526-1kworker/u9:011:10:062
184429952,2cyclictest27646-21cpu08:40:172
184429952,2cyclictest12686-21cat08:05:222
184429952,1cyclictest0-21swapper/209:30:022
184429951,3cyclictest9619-21/usr/sbin/munin09:10:192
184429951,3cyclictest508-21dbus-daemon07:40:032
184389955,0cyclictest0-21swapper/107:50:131
184389952,2cyclictest4333-21cat09:00:211
184389952,2cyclictest26649-21cat07:25:191
184389952,2cyclictest26056-21cat08:35:231
184389952,2cyclictest2558-21cat08:55:221
184389952,2cyclictest22512-21rm11:30:141
184389952,2cyclictest15779-21taskset11:05:111
184389952,2cyclictest14508-21expr08:10:221
184389952,2cyclictest14024-21rm12:25:151
184389952,2cyclictest12964-21ntp_kernel_pll_08:05:241
184389951,3cyclictest3758-21snmpd12:00:091
184389951,3cyclictest3758-21snmpd10:55:211
184389950,4cyclictest9169-21ssh10:20:041
184369953,1cyclictest11332-21sh10:00:040
184369952,2cyclictest29864-21apt-get08:45:170
184369952,2cyclictest29715-21unixbench_multi07:30:290
184369952,2cyclictest19325-21cat07:10:160
184369952,1cyclictest0-21swapper/009:55:080
184369951,3cyclictest24364-21/usr/sbin/munin07:20:190
184369951,1cyclictest1056-21sshd10:15:030
184369950,3cyclictest283-21systemd-journal07:45:040
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional