You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-03 - 07:40

x86 Intel Core i5-8265UE @1600 MHz, Linux 5.10.41-rt42 (Profile)

Latency plot of system in rack #a, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackaslot8.osadl.org (updated Sun Jul 03, 2022 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
63202289264,16sleep10-21swapper/119:08:401
63202289264,16sleep10-21swapper/119:08:391
63162288262,16sleep20-21swapper/219:08:372
63162288262,16sleep20-21swapper/219:08:372
61602269241,18sleep00-21swapper/019:06:380
61602269241,18sleep00-21swapper/019:06:370
60982265231,22sleep30-21swapper/319:06:043
60982265231,22sleep30-21swapper/319:06:033
115652640,0sleep20-21swapper/223:05:132
258382600,0sleep00-21swapper/022:53:210
81272560,0sleep10-21swapper/123:45:121
6062500,0sleep10-21swapper/121:12:181
101282220,1sleep30-21swapper/321:18:543
247682200,0sleep10-21swapper/123:13:561
115142190,0sleep10-21swapper/123:05:111
9582180,0sleep30-21swapper/322:15:333
46762180,0sleep10-21swapper/121:35:541
177872150,0sleep20-21swapper/222:48:152
299252140,0sleep00-21swapper/000:21:390
265672130,0sleep228-21ksoftirqd/222:54:002
97142120,0sleep30-21swapper/323:45:293
122722120,0sleep20-21swapper/220:30:192
292652100,0sleep10-21swapper/121:52:151
168062100,0sleep00-21swapper/021:44:420
65349995,3cyclictest16597-21snmpd21:51:303
65299994,3cyclictest283-21systemd-journal22:15:062
23885290,0sleep30-21swapper/323:34:243
7094280,0sleep20-21swapper/221:59:302
65349986,1cyclictest11901-21ssh21:20:123
65349985,2cyclictest11736-21in:imuxsock00:15:003
65349983,4cyclictest283-21systemd-journal19:50:113
65349983,3cyclictest283-21systemd-journal21:01:063
65299984,3cyclictest283-21systemd-journal19:52:172
65299980,2cyclictest27396-21cat21:30:122
65269984,3cyclictest283-21systemd-journal00:36:161
65269983,3cyclictest283-21systemd-journal21:06:551
65349974,2cyclictest508-21dbus-daemon23:44:083
65349973,3cyclictest283-21systemd-journal20:52:253
65349972,4cyclictest0-21swapper/300:22:073
65349972,3cyclictest283-21systemd-journal00:05:503
65299975,1cyclictest28-21ksoftirqd/223:20:012
65299974,2cyclictest19116-21gdbus20:20:002
65299973,3cyclictest508-21dbus-daemon21:14:262
65299973,3cyclictest508-21dbus-daemon19:25:012
65299973,3cyclictest283-21systemd-journal19:55:132
65299973,3cyclictest283-21systemd-journal00:11:342
65299973,3cyclictest16597-21snmpd20:44:302
65299973,3cyclictest16597-21snmpd20:10:202
65299973,3cyclictest0-21swapper/223:14:592
65299973,2cyclictest283-21systemd-journal22:02:492
65299973,2cyclictest283-21systemd-journal19:43:352
65299973,1cyclictest0-21swapper/222:34:452
65299972,3cyclictest283-21systemd-journal22:07:162
65269975,1cyclictest7713-21awk22:00:001
65269973,3cyclictest283-21systemd-journal23:42:531
65269973,3cyclictest283-21systemd-journal00:05:241
65269973,3cyclictest24855-21grep21:00:131
65269973,3cyclictest0-21swapper/122:30:111
65269972,3cyclictest283-21systemd-journal22:37:421
65269972,3cyclictest283-21systemd-journal20:17:431
65269970,1cyclictest0-21swapper/100:04:481
65239976,1cyclictest1247-21ssh21:34:310
65239974,2cyclictest26917-21df_inode21:05:120
65239974,2cyclictest1794-21ssh22:16:160
65239973,3cyclictest16597-21snmpd22:25:250
65239973,3cyclictest16597-21snmpd19:56:580
65239973,3cyclictest16597-21snmpd19:24:580
65239973,2cyclictest283-21systemd-journal19:28:310
65239971,3cyclictest508-21dbus-daemon23:59:590
65239970,1cyclictest17480-21ssh23:50:330
32723270,0sleep00-21swapper/021:55:030
14846270,0sleep20-21swapper/223:28:142
14464270,0sleep10-21swapper/122:25:121
13088270,0chrt0-21swapper/322:03:043
65349963,2cyclictest781-21ssh00:01:293
65349963,2cyclictest5135-21ssh23:21:343
65349963,2cyclictest356-21sh21:33:463
65349963,2cyclictest29908-21cut22:35:113
65349963,2cyclictest29045-21sed22:55:173
65349963,2cyclictest22134-21ssh23:54:333
65349963,2cyclictest19116-21gdbus20:05:013
65349963,2cyclictest17307-21taskset20:44:353
65349963,2cyclictest17271-21ssh22:26:203
65349963,2cyclictest16597-21snmpd23:07:073
65349963,2cyclictest16597-21snmpd23:00:403
65349963,2cyclictest16597-21snmpd19:30:253
65349963,2cyclictest16597-21snmpd00:36:033
65349963,2cyclictest16597-21snmpd00:25:273
65349963,1cyclictest0-21swapper/322:45:223
65349962,3cyclictest6913-21expr22:20:123
65349962,3cyclictest32329-21sshd21:11:423
65349962,3cyclictest26427-21sshd22:32:353
65349962,3cyclictest25818-21cat23:35:163
65349962,3cyclictest19587-21cat23:10:163
65349962,3cyclictest19046-21cat21:45:203
65349962,3cyclictest16597-21snmpd22:05:183
65349962,3cyclictest16597-21snmpd21:39:023
65349962,3cyclictest16597-21snmpd20:37:563
65349962,3cyclictest16597-21snmpd20:17:303
65349962,3cyclictest16597-21snmpd20:11:293
65349962,3cyclictest16597-21snmpd19:35:493
65349962,3cyclictest16597-21snmpd19:13:133
65349962,3cyclictest16597-21snmpd00:35:003
65349962,3cyclictest1601-21sed21:55:193
65349962,3cyclictest1-21systemd23:27:033
65349962,2cyclictest283-21systemd-journal19:40:003
65349962,1cyclictest0-21swapper/321:42:373
65349961,3cyclictest23391-21rm00:17:193
65349960,6cyclictest0-21swapper/321:25:083
65349960,6cyclictest0-21swapper/320:55:193
65349960,5cyclictest6331-21smart_sda22:40:193
65349960,5cyclictest11268-21expr20:30:123
65349960,3cyclictest26781-21cstates21:05:133
65349960,2cyclictest24218-21awk23:55:183
65299963,2cyclictest7534-21ssh00:29:192
65299963,2cyclictest5196-21awk23:00:212
65299963,2cyclictest508-21dbus-daemon00:06:142
65299963,2cyclictest283-21systemd-journal22:14:042
65299963,2cyclictest16597-21snmpd23:20:582
65299963,2cyclictest16597-21snmpd21:36:292
65299963,2cyclictest16033-21cron19:30:002
65299963,2cyclictest15270-21ssh21:21:472
65299963,2cyclictest11736-21in:imuxsock00:20:352
65299962,3cyclictest9891-21missed_timers00:30:162
65299962,3cyclictest28454-21ls22:55:142
65299962,3cyclictest16597-21snmpd23:30:042
65299962,3cyclictest16597-21snmpd22:38:222
65299962,3cyclictest16597-21snmpd21:26:392
65299962,3cyclictest16597-21snmpd21:17:062
65299962,3cyclictest16597-21snmpd20:47:002
65299962,3cyclictest16597-21snmpd19:34:582
65299962,3cyclictest16597-21snmpd19:17:002
65299962,3cyclictest1-21systemd19:46:522
65299962,3cyclictest10435-21sshd20:27:212
65299962,1cyclictest0-21swapper/223:58:502
65299962,1cyclictest0-21swapper/222:41:262
65299962,1cyclictest0-21swapper/221:50:052
65299962,1cyclictest0-21swapper/221:41:262
65299962,1cyclictest0-21swapper/200:37:342
65299961,3cyclictest283-21systemd-journal00:16:392
65299961,2cyclictest27622-21ssh23:36:032
65299960,6cyclictest0-21swapper/200:02:512
65299960,5cyclictest14102-21ntp_states20:35:162
65299960,0cyclictest0-21swapper/223:51:042
65269963,2cyclictest9628-21ntpq20:25:151
65269963,2cyclictest9626-21ssh22:21:241
65269963,2cyclictest8142-21ssh21:17:111
65269963,2cyclictest7991-21ssh23:23:581
65269963,2cyclictest18453-21ssh22:48:491
65269963,2cyclictest16597-21snmpd23:32:351
65269962,3cyclictest283-21systemd-journal22:57:021
65269962,3cyclictest283-21systemd-journal19:58:331
65269962,3cyclictest283-21systemd-journal19:49:221
65269962,3cyclictest24521-21sensors_temp23:55:201
65269962,3cyclictest18554-21ntpq20:45:171
65269962,3cyclictest17217-21seq20:43:271
65269962,3cyclictest16597-21snmpd23:35:051
65269962,3cyclictest16597-21snmpd23:15:281
65269962,3cyclictest16597-21snmpd22:19:501
65269962,3cyclictest16597-21snmpd22:12:351
65269962,3cyclictest16597-21snmpd21:40:451
65269962,3cyclictest16597-21snmpd20:33:421
65269962,3cyclictest16597-21snmpd20:23:441
65269962,3cyclictest16597-21snmpd20:02:171
65269962,3cyclictest16597-21snmpd19:50:591
65269962,3cyclictest16597-21snmpd19:41:561
65269962,3cyclictest16597-21snmpd19:26:281
65269962,3cyclictest16597-21snmpd19:15:391
65269962,3cyclictest16597-21snmpd00:26:341
65269962,3cyclictest16597-21snmpd00:22:571
65269962,3cyclictest16597-21snmpd00:18:031
65269962,3cyclictest16597-21snmpd00:12:051
65269962,3cyclictest13511-21sshd19:21:211
65269962,3cyclictest1-21systemd22:40:201
65269961,4cyclictest19860-21apt-get22:50:021
65269961,3cyclictest283-21systemd-journal23:01:111
65269961,3cyclictest12336-21rm22:02:241
65269961,2cyclictest9749-21rm23:25:121
65269961,2cyclictest283-21systemd-journal20:36:051
65269960,6cyclictest0-21swapper/121:33:221
65269960,6cyclictest0-21swapper/100:31:381
65269960,5cyclictest16572-21/usr/sbin/munin22:05:141
65269960,5cyclictest15524-21http23:50:121
65239966,0cyclictest0-21swapper/021:47:440
65239966,0cyclictest0-21swapper/020:20:180
65239964,1cyclictest6102-21cut22:40:180
65239963,2cyclictest8672-21ssh22:20:360
65239963,2cyclictest8252-21ssh21:38:560
65239963,2cyclictest7424-21cat19:10:140
65239963,2cyclictest508-21dbus-daemon22:07:000
65239963,2cyclictest508-21dbus-daemon19:50:110
65239963,2cyclictest4508-21ntpq23:00:160
65239963,2cyclictest401-21cstates23:40:130
65239963,2cyclictest283-21systemd-journal23:27:010
65239963,2cyclictest283-21systemd-journal19:36:220
65239963,2cyclictest28216-21awk22:55:110
65239963,2cyclictest23805-21sh21:27:320
65239963,2cyclictest22872-21ssh00:16:520
65239963,2cyclictest20720-21ssh23:10:300
65239963,2cyclictest14349-21sh22:45:200
65239963,2cyclictest11680-21perl23:05:130
65239962,3cyclictest6364-21date20:20:010
65239962,3cyclictest508-21dbus-daemon21:00:000
65239962,3cyclictest3780-21cron21:15:010
65239962,3cyclictest32421-21expr20:05:120
65239962,3cyclictest283-21systemd-journal21:20:350
65239962,3cyclictest27208-21cat23:15:150
65239962,3cyclictest16597-21snmpd23:35:410
65239962,3cyclictest16597-21snmpd23:34:020
65239962,3cyclictest16597-21snmpd22:30:490
65239962,3cyclictest16597-21snmpd22:10:120
65239962,3cyclictest16597-21snmpd22:02:020
65239962,3cyclictest16597-21snmpd20:41:480
65239962,3cyclictest16597-21snmpd20:26:390
65239962,3cyclictest16597-21snmpd19:47:260
65239962,3cyclictest16597-21snmpd19:30:500
65239962,3cyclictest16597-21snmpd00:25:400
65239962,3cyclictest13141-21awk20:35:110
65239962,3cyclictest1-21systemd19:41:020
65239962,3cyclictest11379-21cut20:30:140
65239961,4cyclictest4499-21apt-get00:05:020
65239961,3cyclictest3642-21sshd20:11:280
65239960,6cyclictest0-21swapper/022:37:260
65239960,5cyclictest26832-21timerwakeupswit21:50:200
65239960,4cyclictest283-21systemd-journal00:38:450
65239960,0cyclictest0-21swapper/023:45:010
65239960,0cyclictest0-21swapper/023:24:490
65239960,0cyclictest0-21swapper/000:11:440
2841260,0sleep00-21swapper/000:03:240
20205260,0sleep10-21swapper/121:46:031
18777260,0sleep20-21swapper/222:27:362
18182260,0sleep10-21swapper/121:24:171
6822250,0sleep30-21swapper/320:20:123
65349952,2cyclictest8878-21expr20:25:123
65349952,2cyclictest29897-21cstates20:00:133
65349952,2cyclictest283-21systemd-journal19:45:003
65349952,2cyclictest27790-21cat19:55:113
65349952,2cyclictest26755-21cat23:15:123
65349952,2cyclictest16597-21snmpd22:50:073
65349952,2cyclictest16597-21snmpd19:20:103
65349952,2cyclictest14335-21expr19:25:123
65349951,3cyclictest9088-21munin-node19:15:023
65349951,3cyclictest18628-21cut20:45:173
65349950,3cyclictest283-21systemd-journal22:10:023
65299953,1cyclictest16597-21snmpd23:40:022
65299952,2cyclictest6977-21grep20:20:132
65299952,2cyclictest283-21systemd-journal19:10:352
65299952,2cyclictest23416-21cat20:55:202
65299951,3cyclictest7747-21/usr/sbin/munin23:45:162
65299951,3cyclictest30664-21/usr/sbin/munin20:00:182
65299951,3cyclictest27193-21cat21:05:152
65299951,3cyclictest20714-21tr20:50:172
65299951,3cyclictest19103-21expr19:35:132
65299951,3cyclictest18412-21uname21:45:162
65299951,3cyclictest16597-21snmpd20:08:022
65299951,2cyclictest283-21systemd-journal21:01:562
65299951,1cyclictest0-21swapper/222:20:032
65269952,2cyclictest395-21cat20:05:141
65269952,2cyclictest22406-21expr20:55:131
65269952,2cyclictest20479-21if_enp4s020:50:161
65269952,2cyclictest19186-21cat19:35:141
65269952,2cyclictest16803-21fw_packets19:30:141
65269952,2cyclictest16597-21snmpd21:25:071
65269951,3cyclictest7228-21fschecks_time19:10:131
65269951,3cyclictest16597-21snmpd20:10:491
65239952,2cyclictest8994-21cpu00:30:120
65239952,2cyclictest29449-21fschecks_time21:10:140
65239952,2cyclictest19812-21/usr/sbin/munin20:50:200
65239952,2cyclictest18267-21cat20:45:150
65239951,3cyclictest9311-21munin-node19:15:020
65239951,3cyclictest25235-21ntp_kernel_pll_21:00:160
65239951,2cyclictest283-21systemd-journal20:00:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional