You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-06-27 - 05:22

x86 Intel Core i5-8265UE @1600 MHz, Linux 5.10.41-rt42 (Profile)

Latency plot of system in rack #a, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8.osadl.org (updated Mon Jun 27, 2022 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
219762309280,18sleep20-21swapper/219:06:142
220902283252,20sleep10-21swapper/119:07:301
221352273243,19sleep00-21swapper/019:08:050
221902260218,32sleep30-21swapper/319:08:493
173222570,0sleep20-21swapper/200:02:382
161342570,0sleep00-21swapper/021:54:390
178552560,0sleep00-21swapper/020:10:110
326762540,0sleep30-21swapper/300:13:243
52482530,0sleep10-21swapper/123:12:041
43062190,0sleep30-21swapper/300:38:353
21382190,0sleep00-21swapper/021:23:540
176542190,0sleep00-21swapper/000:25:140
298112180,0sleep00-21swapper/020:35:120
253532170,0sleep00-21swapper/019:15:020
28162140,0sleep00-21swapper/023:10:180
205102140,0sleep00-21swapper/023:43:300
161422140,0sleep334-21ksoftirqd/323:19:423
158502140,0sleep115855-21gltestperf22:15:261
314582130,0sleep00-21swapper/022:04:440
165792130,0sleep216571-21munin-run21:55:012
130592130,0sleep10-21swapper/121:30:201
2240799109,1cyclictest34-21ksoftirqd/323:15:013
6751290,1sleep30-21swapper/323:55:183
29291290,0sleep10-21swapper/100:33:241
224009990,8cyclictest29730-21/usr/sbin/munin22:45:191
224009990,8cyclictest14945-21ssh23:18:391
1430290,0sleep30-21swapper/323:30:203
32459280,3sleep012-21ksoftirqd/000:35:150
224079984,2cyclictest283-21systemd-journal19:54:163
224049985,2cyclictest28-21ksoftirqd/221:00:002
224049984,3cyclictest283-21systemd-journal21:34:182
224049984,2cyclictest283-21systemd-journal23:22:162
223969985,2cyclictest14300-21ssh22:56:320
223969985,2cyclictest12-21ksoftirqd/019:35:010
8789270,0sleep30-21swapper/322:53:203
30170270,0sleep10-21swapper/121:41:501
24129270,0sleep324134-21unixbench_singl19:10:203
224079976,1cyclictest34-21ksoftirqd/321:30:003
224079976,1cyclictest34-21ksoftirqd/300:25:003
224079974,2cyclictest508-21dbus-daemon23:00:013
224079973,3cyclictest29930-21cron19:25:013
224079973,3cyclictest16597-21snmpd22:47:553
224079973,3cyclictest1-21systemd19:16:163
224079972,4cyclictest0-21swapper/321:37:433
224079972,3cyclictest283-21systemd-journal21:49:443
224079970,1cyclictest23102-21ssh21:59:053
224049974,2cyclictest508-21dbus-daemon19:12:312
224049974,2cyclictest20865-21fschecks_time00:05:142
224049973,3cyclictest9980-21perf21:05:002
224049973,3cyclictest283-21systemd-journal23:09:222
224049973,3cyclictest283-21systemd-journal21:20:072
224049973,3cyclictest283-21systemd-journal20:12:172
224049973,3cyclictest16597-21snmpd23:32:112
224049973,1cyclictest0-21swapper/221:35:302
224049970,4cyclictest16536-21plymouth23:20:012
224049970,3cyclictest283-21systemd-journal00:24:002
224009973,3cyclictest9984-21sleep21:05:001
224009973,3cyclictest16597-21snmpd23:52:511
224009973,1cyclictest0-21swapper/122:31:561
224009972,3cyclictest283-21systemd-journal20:44:161
224009970,7cyclictest16597-21snmpd21:49:381
224009970,6cyclictest9283-21ls00:20:001
223969974,2cyclictest508-21dbus-daemon23:08:570
223969974,2cyclictest508-21dbus-daemon00:12:450
223969973,3cyclictest7718-21cron21:00:000
223969973,3cyclictest7641-21ntpq19:45:170
223969973,3cyclictest508-21dbus-daemon23:18:330
223969973,1cyclictest20371-21sendmail22:40:000
223969972,3cyclictest283-21systemd-journal20:45:310
223969972,3cyclictest283-21systemd-journal19:37:580
223969971,4cyclictest283-21systemd-journal22:27:410
20921270,0sleep00-21swapper/022:18:530
224079966,0cyclictest0-21swapper/323:39:023
224079966,0cyclictest0-21swapper/322:19:323
224079966,0cyclictest0-21swapper/321:12:093
224079964,1cyclictest13800-21grep22:35:143
224079963,2cyclictest8841-21sh22:10:193
224079963,2cyclictest6086-21chrt19:40:553
224079963,2cyclictest34-21ksoftirqd/320:53:013
224079963,2cyclictest27816-21sh00:32:023
224079963,2cyclictest26921-21rm23:25:503
224079963,2cyclictest16597-21snmpd21:31:073
224079963,2cyclictest16597-21snmpd00:05:273
224079963,2cyclictest15467-21ssh00:00:573
224079963,2cyclictest12200-21diskmemload21:15:093
224079963,1cyclictest0-21swapper/322:31:153
224079962,3cyclictest508-21dbus-daemon20:00:003
224079962,3cyclictest31426-21ntpq22:25:183
224079962,3cyclictest283-21systemd-journal20:05:003
224079962,3cyclictest25770-21ntpq20:25:173
224079962,3cyclictest23264-21expr23:45:143
224079962,3cyclictest23153-21cut22:20:133
224079962,3cyclictest22668-21expr20:20:133
224079962,3cyclictest20615-21sed20:15:143
224079962,3cyclictest17321-21cat23:20:123
224079962,3cyclictest16597-21snmpd23:00:193
224079962,3cyclictest16597-21snmpd22:07:213
224079962,3cyclictest16597-21snmpd22:02:333
224079962,3cyclictest16597-21snmpd20:59:393
224079962,3cyclictest16597-21snmpd20:06:053
224079962,3cyclictest16597-21snmpd19:48:083
224079962,3cyclictest16597-21snmpd19:36:383
224079962,3cyclictest16597-21snmpd19:25:223
224079962,3cyclictest16597-21snmpd00:15:213
224079962,3cyclictest1-21systemd21:05:073
224079962,1cyclictest0-21swapper/323:08:073
224079962,1cyclictest0-21swapper/321:20:493
224079962,1cyclictest0-21swapper/300:28:263
224079961,3cyclictest283-21systemd-journal21:40:093
224079961,2cyclictest1688-21sh20:45:133
224049966,0cyclictest0-21swapper/221:41:512
224049966,0cyclictest0-21swapper/200:34:132
224049966,0cyclictest0-21swapper/200:10:022
224049963,2cyclictest7845-21sh23:55:532
224049963,2cyclictest508-21dbus-daemon23:40:182
224049963,2cyclictest508-21dbus-daemon20:06:192
224049963,2cyclictest31789-21sed23:50:202
224049963,2cyclictest2862-21cut00:15:122
224049963,2cyclictest283-21systemd-journal22:40:362
224049963,2cyclictest283-21systemd-journal20:20:042
224049963,2cyclictest23843-21ssh23:03:092
224049963,2cyclictest16597-21snmpd21:48:322
224049963,2cyclictest16597-21snmpd20:42:002
224049963,2cyclictest14139-21grep21:10:182
224049963,2cyclictest11544-21sh22:55:112
224049963,2cyclictest10519-21ssh21:29:362
224049962,3cyclictest283-21systemd-journal23:35:172
224049962,3cyclictest283-21systemd-journal20:00:292
224049962,3cyclictest283-21systemd-journal19:48:232
224049962,3cyclictest28242-21smart_sda20:30:182
224049962,3cyclictest26239-21grep19:15:152
224049962,3cyclictest24600-21mktemp20:25:012
224049962,3cyclictest16597-21snmpd22:45:522
224049962,3cyclictest16597-21snmpd21:15:192
224049962,3cyclictest16597-21snmpd20:48:152
224049962,1cyclictest0-21swapper/223:10:152
224049962,1cyclictest0-21swapper/222:15:022
224049962,1cyclictest0-21swapper/221:55:082
224049962,1cyclictest0-21swapper/200:37:452
224049961,3cyclictest283-21systemd-journal22:25:112
224049961,3cyclictest283-21systemd-journal00:27:452
224049960,6cyclictest0-21swapper/223:29:022
224049960,0cyclictest0-21swapper/222:06:382
224049960,0cyclictest0-21swapper/222:01:282
224049960,0cyclictest0-21swapper/220:51:552
224009966,0cyclictest0-21swapper/120:35:241
224009963,2cyclictest8043-21head22:10:141
224009963,2cyclictest668-21runrttasks23:55:101
224009963,2cyclictest668-21runrttasks19:52:221
224009963,2cyclictest508-21dbus-daemon20:30:301
224009963,2cyclictest29387-21ssh23:06:241
224009963,2cyclictest28983-21cat21:20:111
224009963,2cyclictest25269-21ls19:15:021
224009963,2cyclictest22933-21ssh00:06:081
224009963,2cyclictest16597-21snmpd22:20:241
224009963,2cyclictest16597-21snmpd21:50:171
224009963,2cyclictest16597-21snmpd20:57:571
224009963,1cyclictest0-21swapper/121:10:291
224009962,3cyclictest28648-21cat19:20:151
224009962,3cyclictest283-21systemd-journal20:30:001
224009962,3cyclictest24895-21expr23:25:181
224009962,3cyclictest20207-21expr20:15:121
224009962,3cyclictest17758-21cat00:25:161
224009962,3cyclictest16597-21snmpd23:31:231
224009962,3cyclictest16597-21snmpd23:20:171
224009962,3cyclictest16597-21snmpd22:43:041
224009962,3cyclictest16597-21snmpd21:55:021
224009962,3cyclictest16597-21snmpd21:37:351
224009962,3cyclictest16597-21snmpd20:20:081
224009962,3cyclictest16597-21snmpd20:10:381
224009962,3cyclictest16597-21snmpd19:11:491
224009962,3cyclictest16597-21snmpd00:35:281
224009962,3cyclictest1532-21date22:05:271
224009962,1cyclictest0-21swapper/123:41:051
224009962,1cyclictest0-21swapper/100:20:211
224009962,1cyclictest0-21swapper/100:04:521
224009961,4cyclictest0-21swapper/123:45:111
224009961,3cyclictest283-21systemd-journal22:57:161
224009961,3cyclictest283-21systemd-journal22:38:311
224009961,3cyclictest283-21systemd-journal19:45:021
224009960,6cyclictest0-21swapper/121:15:201
224009960,6cyclictest0-21swapper/120:07:371
224009960,6cyclictest0-21swapper/120:03:171
224009960,5cyclictest8458-21sshd23:35:171
224009960,5cyclictest27614-21ssh00:10:101
224009960,0cyclictest0-21swapper/122:50:501
224009960,0cyclictest0-21swapper/119:36:141
223969964,1cyclictest12-21ksoftirqd/020:15:000
223969963,2cyclictest508-21dbus-daemon22:33:310
223969963,2cyclictest508-21dbus-daemon19:10:510
223969963,2cyclictest283-21systemd-journal20:21:190
223969963,2cyclictest24169-21sh00:30:090
223969963,2cyclictest23026-21rm23:24:020
223969963,2cyclictest17465-21awk21:55:140
223969963,2cyclictest16597-21snmpd23:30:350
223969963,2cyclictest10667-21ssh23:36:380
223969963,1cyclictest0-21swapper/022:06:090
223969962,3cyclictest508-21dbus-daemon21:30:180
223969962,3cyclictest283-21systemd-journal20:02:590
223969962,3cyclictest25251-21fschecks_time20:25:140
223969962,3cyclictest20909-21basename21:15:140
223969962,3cyclictest20197-21sed21:35:190
223969962,3cyclictest16597-21snmpd23:52:540
223969962,3cyclictest16597-21snmpd23:45:380
223969962,3cyclictest16597-21snmpd23:01:130
223969962,3cyclictest16597-21snmpd22:53:010
223969962,3cyclictest16597-21snmpd22:41:340
223969962,3cyclictest16597-21snmpd22:20:120
223969962,3cyclictest16597-21snmpd21:45:290
223969962,3cyclictest16597-21snmpd21:41:560
223969962,3cyclictest16597-21snmpd21:10:540
223969962,3cyclictest16597-21snmpd21:00:420
223969962,3cyclictest16597-21snmpd20:52:360
223969962,3cyclictest16597-21snmpd20:44:210
223969962,3cyclictest16597-21snmpd20:33:560
223969962,3cyclictest16597-21snmpd19:51:140
223969962,3cyclictest16597-21snmpd19:44:320
223969962,3cyclictest16597-21snmpd19:26:580
223969962,3cyclictest16597-21snmpd00:21:570
223969962,3cyclictest16597-21snmpd00:05:390
223969962,3cyclictest0-21swapper/021:07:370
223969962,1cyclictest0-21swapper/023:56:340
223969962,1cyclictest0-21swapper/022:47:560
13013260,0sleep30-21swapper/321:51:573
224079953,1cyclictest31506-21apt-get20:40:113
224079952,2cyclictest30400-21find20:35:183
224079952,2cyclictest11736-21in:imuxsock20:10:003
224079951,3cyclictest8678-21cat21:00:153
224079951,3cyclictest478-21grep19:30:153
224079951,3cyclictest30325-21cat23:50:123
224079951,3cyclictest16597-21snmpd23:40:053
224079950,4cyclictest27162-21grep20:30:123
224079950,4cyclictest20803-21/usr/sbin/munin22:40:203
224049953,1cyclictest4850-21df_inode19:40:122
224049953,1cyclictest324942sleep219:30:142
224049953,1cyclictest20563-21tr20:15:142
224049952,2cyclictest9631-21cut19:50:152
224049952,2cyclictest5468-21sh22:30:082
224049952,2cyclictest29764-21gltestperf20:35:132
224049952,2cyclictest28659-21interrupts19:20:162
224049952,2cyclictest2473-21cat19:35:122
224049952,2cyclictest22459-21sh23:45:002
224049952,2cyclictest12705-21latency_hist22:35:012
224049952,2cyclictest12071-21taskset21:08:332
224049952,1cyclictest0-21swapper/222:10:142
224049951,3cyclictest3977-21ssh22:50:132
224049951,3cyclictest31028-21/usr/sbin/munin19:25:162
224049951,3cyclictest23391-21basename22:20:142
224049951,3cyclictest11511-21/usr/sbin/munin19:55:122
224009952,2cyclictest5144-21cat19:40:141
224009952,2cyclictest508-21dbus-daemon19:25:271
224009952,2cyclictest31757-21switchtime22:25:201
224009952,2cyclictest283-21systemd-journal21:25:071
224009952,2cyclictest25034-21ls22:00:181
224009952,2cyclictest20872-21ssh23:00:391
224009952,2cyclictest1998-21irqstats20:45:161
224009952,2cyclictest16597-21snmpd21:05:091
224009952,2cyclictest16597-21snmpd19:56:321
224009951,3cyclictest4202-21cat20:50:141
224009950,3cyclictest502-21systemd-logind19:30:021
223969953,1cyclictest16597-21snmpd19:20:100
223969952,2cyclictest25291-21fw_packets23:25:200
223969952,2cyclictest2369-21ssh00:15:080
223969952,2cyclictest16276-21cat20:05:150
223969952,1cyclictest0-21swapper/021:25:110
223969951,3cyclictest13111-21ssh00:00:110
223969951,3cyclictest11942-21memory19:55:150
223969950,4cyclictest8288-21cat22:10:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional