You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-16 - 05:19

x86 Intel Core i5-8265UE @1600 MHz, Linux 5.10.41-rt42 (Profile)

Latency plot of system in rack #a, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Tue Aug 16, 2022 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
289572300270,19sleep20-21swapper/219:07:142
289662288257,19sleep10-21swapper/119:07:211
289482275246,19sleep00-21swapper/019:07:080
289212260228,21sleep30-21swapper/319:06:463
271862600,0sleep10-21swapper/123:33:101
316962550,1sleep30-21swapper/321:40:313
311112540,0sleep10-21swapper/122:04:121
266692190,0sleep30-21swapper/322:24:323
104002190,0sleep20-21swapper/219:35:212
219372180,0sleep00-21swapper/022:20:210
145632180,0sleep20-21swapper/223:24:492
5292150,0sleep00-21swapper/000:22:220
254342120,0sleep30-21swapper/323:31:353
181522120,0sleep30-21swapper/300:35:173
2927299105,3cyclictest283-21systemd-journal21:30:301
2927199101,3cyclictest1464-21idleruntime-cro00:00:000
292819998,1cyclictest34-21ksoftirqd/323:55:013
1322290,1sleep30-21swapper/323:37:273
4002280,0sleep30-21swapper/322:53:403
32247280,0sleep20-21swapper/222:27:592
292819984,3cyclictest508-21dbus-daemon22:34:433
292819980,7cyclictest32539-21sh00:21:543
292769984,3cyclictest508-21dbus-daemon00:06:522
292769983,3cyclictest283-21systemd-journal21:38:012
292719980,5cyclictest4230-21dump-pmu-power21:45:010
292819974,2cyclictest11646-21cron23:45:003
292819973,3cyclictest26953-21sshd20:11:503
292819973,3cyclictest18310-21apt-get19:55:143
292819973,3cyclictest17069-21sshd21:01:123
292819973,3cyclictest16597-21snmpd21:56:293
292819973,3cyclictest16597-21snmpd19:33:313
292819973,3cyclictest16597-21snmpd19:29:183
292819972,3cyclictest283-21systemd-journal21:35:543
292819972,3cyclictest283-21systemd-journal19:55:013
292769975,2cyclictest28-21ksoftirqd/200:00:002
292769975,1cyclictest28-21ksoftirqd/221:30:002
292769973,3cyclictest508-21dbus-daemon19:55:002
292769973,3cyclictest283-21systemd-journal22:11:042
292769973,3cyclictest16597-21snmpd20:43:482
292769973,3cyclictest16597-21snmpd19:56:152
292769973,3cyclictest0-21swapper/222:57:222
292769973,2cyclictest283-21systemd-journal23:20:002
292729974,2cyclictest502-21systemd-logind23:43:561
292729974,2cyclictest22-21ksoftirqd/119:40:001
292729974,2cyclictest19116-21gdbus21:40:021
292729974,2cyclictest11091-21ntpq22:35:201
292729973,2cyclictest283-21systemd-journal20:55:111
292729973,2cyclictest22-21ksoftirqd/100:00:001
292729972,3cyclictest283-21systemd-journal22:48:541
292729972,3cyclictest283-21systemd-journal20:05:001
292729972,2cyclictest3760-21idleruntime-cro20:35:011
292729970,7cyclictest31968-21ssh23:13:361
292729970,6cyclictest19626-21cron21:10:001
292729970,4cyclictest23806-21munin-run00:40:011
292719974,2cyclictest283-21systemd-journal21:15:010
292719973,3cyclictest508-21dbus-daemon23:00:340
292719973,3cyclictest16597-21snmpd00:33:560
292719973,1cyclictest0-21swapper/021:50:210
292719972,3cyclictest28819-21systemctl00:20:010
292719970,6cyclictest5660-21ssh21:22:150
27270270,0sleep30-21swapper/322:46:383
25500270,0sleep30-21swapper/322:00:103
23872270,6sleep224100-21unixbench_singl00:15:232
10160270,0sleep10-21swapper/121:49:021
5754260,1sleep20-21swapper/219:25:212
292819966,0cyclictest0-21swapper/322:16:413
292819966,0cyclictest0-21swapper/300:12:453
292819964,2cyclictest34-21ksoftirqd/300:10:003
292819963,2cyclictest508-21dbus-daemon19:20:003
292819963,2cyclictest34-21ksoftirqd/321:10:003
292819963,2cyclictest3226-21ssh22:06:553
292819963,2cyclictest3034-21awk23:15:303
292819963,2cyclictest29953-21expr20:20:143
292819963,2cyclictest283-21systemd-journal22:55:353
292819963,2cyclictest20621-21fschecks_time23:05:263
292819963,2cyclictest17399-21rm23:25:473
292819963,2cyclictest16597-21snmpd21:08:003
292819963,2cyclictest16597-21snmpd20:35:133
292819963,2cyclictest16597-21snmpd00:29:413
292819963,2cyclictest13367-21http19:45:013
292819963,2cyclictest11145-21ssh22:12:533
292819963,2cyclictest11013-21ssh21:25:383
292819963,2cyclictest10660-21ssh23:20:523
292819962,3cyclictest6596-21expr20:40:183
292819962,3cyclictest27725-21cat23:10:253
292819962,3cyclictest27664-21expr21:15:153
292819962,3cyclictest16597-21snmpd23:57:513
292819962,3cyclictest16597-21snmpd23:00:463
292819962,3cyclictest16597-21snmpd21:20:093
292819962,3cyclictest16597-21snmpd20:56:033
292819962,3cyclictest16597-21snmpd20:30:103
292819962,3cyclictest16597-21snmpd20:00:513
292819962,3cyclictest16597-21snmpd19:20:353
292819962,3cyclictest16597-21snmpd19:11:353
292819962,3cyclictest16597-21snmpd00:15:113
292819962,3cyclictest13050-21ssh23:45:183
292819962,3cyclictest12039-21cat21:50:203
292819962,1cyclictest0-21swapper/322:28:453
292819962,1cyclictest0-21swapper/321:30:533
292819961,4cyclictest8491-21apt-get20:45:133
292819961,4cyclictest11702-21unixbench_multi22:35:223
292819961,3cyclictest283-21systemd-journal20:51:583
292819961,3cyclictest283-21systemd-journal20:17:313
292819960,0cyclictest0-21swapper/322:41:213
292819960,0cyclictest0-21swapper/321:47:063
292769966,0cyclictest0-21swapper/222:51:022
292769966,0cyclictest0-21swapper/220:05:162
292769963,2cyclictest7124-21ssh00:03:472
292769963,2cyclictest7069-21ssh00:27:032
292769963,2cyclictest668-21runrttasks23:00:062
292769963,2cyclictest502-21systemd-logind22:30:082
292769963,2cyclictest283-21systemd-journal20:57:582
292769963,2cyclictest283-21systemd-journal20:02:552
292769963,2cyclictest26866-21sh23:10:132
292769963,2cyclictest25631-21ssh23:09:142
292769963,2cyclictest24316-21ssh23:30:332
292769963,2cyclictest16597-21snmpd21:04:562
292769963,2cyclictest16597-21snmpd19:45:212
292769963,2cyclictest16597-21snmpd00:30:262
292769963,2cyclictest14643-21cat22:15:222
292769963,2cyclictest1248-21taskset23:37:242
292769963,2cyclictest1-21systemd22:45:102
292769962,3cyclictest28695-21which21:15:222
292769962,3cyclictest19116-21gdbus21:10:002
292769962,3cyclictest17964-21cat22:40:192
292769962,3cyclictest16597-21snmpd23:52:142
292769962,3cyclictest16597-21snmpd23:40:592
292769962,3cyclictest16597-21snmpd22:21:232
292769962,3cyclictest16597-21snmpd21:55:442
292769962,3cyclictest16597-21snmpd21:45:112
292769962,3cyclictest16597-21snmpd20:52:292
292769962,3cyclictest16597-21snmpd20:36:142
292769962,3cyclictest16597-21snmpd20:22:102
292769962,3cyclictest16597-21snmpd19:10:322
292769962,3cyclictest16597-21snmpd00:13:262
292769962,3cyclictest16003-21cut23:25:162
292769962,1cyclictest0-21swapper/223:46:182
292769962,1cyclictest0-21swapper/222:39:332
292769962,1cyclictest0-21swapper/221:41:132
292769961,4cyclictest283-21systemd-journal22:00:572
292769960,6cyclictest0-21swapper/222:06:312
292769960,6cyclictest0-21swapper/221:33:542
292769960,6cyclictest0-21swapper/200:36:232
292769960,5cyclictest4401-21rm21:21:042
292769960,5cyclictest11655-21/usr/sbin/munin21:50:252
292729966,0cyclictest0-21swapper/121:27:521
292729963,2cyclictest283-21systemd-journal20:40:271
292729963,2cyclictest24252-21ssh21:59:131
292729963,2cyclictest22447-21ssh22:20:271
292729963,2cyclictest21152-21ntpq23:05:301
292729963,2cyclictest18168-21ssh00:11:081
292729963,2cyclictest16597-21snmpd20:17:141
292729963,2cyclictest16597-21snmpd19:53:001
292729963,2cyclictest16597-21snmpd00:16:441
292729963,2cyclictest1452-21rm22:51:161
292729963,2cyclictest10680-21rm22:58:161
292729963,2cyclictest10189-21ssh22:11:571
292729962,3cyclictest9610-21cat00:05:221
292729962,3cyclictest8854-21cat19:34:551
292729962,3cyclictest4230-21expr20:35:161
292729962,3cyclictest32362-21df20:25:151
292729962,3cyclictest3082-21expr22:30:151
292729962,3cyclictest30423-21cat20:20:171
292729962,3cyclictest283-21systemd-journal21:20:001
292729962,3cyclictest28048-21expr22:25:141
292729962,3cyclictest16597-21snmpd23:52:311
292729962,3cyclictest16597-21snmpd23:36:261
292729962,3cyclictest16597-21snmpd21:51:401
292729962,3cyclictest16597-21snmpd21:00:141
292729962,3cyclictest16597-21snmpd20:46:021
292729962,3cyclictest16597-21snmpd19:42:061
292729962,3cyclictest0-21swapper/121:12:061
292729962,1cyclictest0-21swapper/123:25:141
292729962,1cyclictest0-21swapper/100:20:211
292729960,6cyclictest0-21swapper/123:22:461
292729960,6cyclictest0-21swapper/123:01:271
292729960,6cyclictest0-21swapper/122:08:391
292729960,6cyclictest0-21swapper/121:21:511
292729960,6cyclictest0-21swapper/100:01:071
292729960,0cyclictest0-21swapper/122:41:521
292729960,0cyclictest0-21swapper/100:32:461
292719963,2cyclictest8809-21ssh21:47:440
292719963,2cyclictest7546-21tune2fs22:10:220
292719963,2cyclictest668-21runrttasks00:25:250
292719963,2cyclictest6234-21users19:25:220
292719963,2cyclictest508-21dbus-daemon23:16:260
292719963,2cyclictest28422-21expr23:10:310
292719963,2cyclictest25083-21ssh23:53:460
292719963,2cyclictest22586-21chrt23:06:170
292719963,2cyclictest1869-21sh22:51:400
292719963,2cyclictest16597-21snmpd23:24:050
292719963,2cyclictest16597-21snmpd21:37:460
292719963,2cyclictest11373-21fschecks_count20:50:130
292719962,3cyclictest32706-21expr19:15:150
292719962,3cyclictest283-21systemd-journal23:40:360
292719962,3cyclictest283-21systemd-journal21:56:120
292719962,3cyclictest283-21systemd-journal19:23:180
292719962,3cyclictest26134-21grep22:00:190
292719962,3cyclictest18254-21hwlatdetect00:35:180
292719962,3cyclictest16597-21snmpd23:35:460
292719962,3cyclictest16597-21snmpd22:56:120
292719962,3cyclictest16597-21snmpd22:36:350
292719962,3cyclictest16597-21snmpd22:34:440
292719962,3cyclictest16597-21snmpd22:26:240
292719962,3cyclictest16597-21snmpd22:15:570
292719962,3cyclictest16597-21snmpd21:32:240
292719962,3cyclictest16597-21snmpd20:26:370
292719962,3cyclictest16597-21snmpd19:50:420
292719962,3cyclictest16597-21snmpd19:47:030
292719962,3cyclictest16597-21snmpd19:37:570
292719962,3cyclictest16597-21snmpd19:30:060
292719962,3cyclictest16597-21snmpd00:03:260
292719962,3cyclictest13497-21basename23:45:220
292719962,2cyclictest283-21systemd-journal20:15:020
292719962,2cyclictest16597-21snmpd22:06:320
292719961,3cyclictest283-21systemd-journal20:06:410
292719960,2cyclictest29729-21/usr/sbin/munin20:20:210
292819955,0cyclictest0-21swapper/320:06:533
292819952,2cyclictest9212-21cpuspeed19:35:143
292819952,2cyclictest2425-21tune2fs00:00:173
292819952,2cyclictest15037-21sendmail_mailtr19:45:223
292819951,3cyclictest16597-21snmpd20:25:133
292819950,4cyclictest10536-21ssh00:30:153
292769952,2cyclictest7246-21ls19:30:142
292769952,2cyclictest29939-21expr00:20:172
292769952,2cyclictest29741-11brltty19:20:242
292769952,2cyclictest16597-21snmpd21:07:172
292769952,2cyclictest1610-21sshd19:16:412
292769951,3cyclictest8466-21/usr/sbin/munin20:45:162
292769951,3cyclictest16597-21snmpd20:30:012
292769951,3cyclictest16597-21snmpd20:10:012
292769951,3cyclictest11720-21cstates19:40:162
292769950,3cyclictest32687-21uname20:25:172
292769950,1cyclictest27471-21/usr/sbin/munin20:15:112
292729953,1cyclictest0-21swapper/100:25:021
292729952,2cyclictest4915-21expr19:25:141
292729952,2cyclictest3853-21cat19:20:231
292729952,2cyclictest32628-21expr19:15:161
292729952,2cyclictest30925-21cat21:40:251
292729952,2cyclictest25943-21cat20:10:201
292729952,2cyclictest16597-21snmpd20:50:081
292729952,2cyclictest14233-21cat19:45:151
292729952,2cyclictest13854-21ssh22:15:171
292729952,2cyclictest13803-21unixbench_multi23:45:231
292729951,3cyclictest23317-21expr20:05:171
292729951,3cyclictest18862-21expr19:55:161
292729951,3cyclictest16597-21snmpd23:15:031
292729950,4cyclictest29962-21cut19:10:211
292719953,1cyclictest8633-21sh00:05:050
292719953,1cyclictest21068-21cat20:00:140
292719952,2cyclictest9227-21expr21:25:160
292719952,2cyclictest4812-21wc20:35:190
292719952,2cyclictest24853-21ssh22:45:170
292719952,2cyclictest18699-21df_inode19:55:160
292719952,2cyclictest18445-21cat21:05:180
292719952,2cyclictest17981-21cat22:40:190
292719952,2cyclictest16597-21snmpd20:40:010
292719952,2cyclictest16597-21snmpd19:10:030
292719952,2cyclictest15410-21taskset21:00:100
292719952,2cyclictest12337-21ntp_offset19:40:200
292719952,1cyclictest0-21swapper/023:25:080
292719952,1cyclictest0-21swapper/000:10:070
292719951,3cyclictest283-21systemd-journal23:30:110
292719951,3cyclictest16597-21snmpd20:45:140
292719951,3cyclictest16597-21snmpd20:10:110
292719951,3cyclictest13782-21head20:55:160
292719950,4cyclictest3413-21sshd20:31:110
292719950,1cyclictest283-21systemd-journal21:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional