You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-30 - 09:13
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8.osadl.org (updated Sat Sep 30, 2023 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
248082316288,17sleep20-21swapper/219:07:412
230262284252,20sleep00-21swapper/019:05:170
249242282245,22sleep10-21swapper/119:09:131
246332258223,22sleep30-21swapper/319:05:403
304842580,0sleep10-21swapper/122:30:221
204282580,1sleep00-21swapper/022:45:300
208752540,0sleep00-21swapper/023:52:370
16622540,0sleep00-21swapper/021:23:130
238052530,0sleep20-21swapper/223:32:372
53092230,0sleep20-21swapper/222:58:372
236852190,0sleep00-21swapper/023:32:310
236362160,0sleep10-21swapper/123:32:291
306052150,0sleep20-21swapper/221:44:172
108012140,0sleep30-21swapper/323:01:483
247372130,0sleep10-21swapper/122:49:311
57662110,0sleep00-21swapper/023:42:240
2509299100,3cyclictest26775-21ssh23:35:031
2508799109,1cyclictest12-21ksoftirqd/000:00:000
937290,0sleep30-21swapper/300:25:273
251029994,3cyclictest19982-21systemd-journal20:38:303
250969994,3cyclictest19982-21systemd-journal23:22:282
250969994,3cyclictest19982-21systemd-journal20:06:232
250969990,8cyclictest4478-21expr20:45:232
250929994,3cyclictest19982-21systemd-journal00:22:331
24719290,0sleep30-21swapper/323:10:433
251029983,3cyclictest19982-21systemd-journal23:40:493
251029983,3cyclictest19982-21systemd-journal22:20:003
251029983,3cyclictest19982-21systemd-journal21:22:133
250969987,1cyclictest28-21ksoftirqd/223:20:002
250969985,2cyclictest508-21dbus-daemon23:42:552
250969985,2cyclictest43742sleep221:25:192
250969985,2cyclictest19982-21systemd-journal21:32:142
250969985,2cyclictest19982-21systemd-journal21:32:142
250969984,3cyclictest508-21dbus-daemon20:57:162
250969984,3cyclictest19982-21systemd-journal22:50:322
250969983,3cyclictest19982-21systemd-journal00:09:312
250969980,7cyclictest20412-21sh23:52:112
250929984,3cyclictest508-21dbus-daemon20:21:491
250929984,3cyclictest502-21systemd-logind23:27:291
250929983,3cyclictest19982-21systemd-journal20:04:431
250929983,3cyclictest19982-21systemd-journal19:56:221
250929980,1cyclictest13899-21strings21:05:211
250879984,3cyclictest508-21dbus-daemon22:40:200
250879984,3cyclictest508-21dbus-daemon21:51:560
250879984,3cyclictest19982-21systemd-journal19:53:270
250879984,2cyclictest19982-21systemd-journal19:25:310
250879983,3cyclictest19982-21systemd-journal23:38:190
250879980,7cyclictest18759-21ssh22:22:220
20734280,0sleep30-21swapper/322:45:453
251029974,2cyclictest508-21dbus-daemon20:03:293
251029973,3cyclictest3758-21snmpd23:34:583
251029973,3cyclictest3562-21memory19:30:253
251029973,3cyclictest30586-21diskstats23:15:213
251029973,3cyclictest29741-11brltty22:37:243
251029973,3cyclictest26240-21cron21:17:003
251029973,3cyclictest1-21systemd20:43:303
251029973,3cyclictest0-21swapper/323:29:563
251029973,3cyclictest0-21swapper/300:07:313
251029973,1cyclictest0-21swapper/323:37:403
251029972,3cyclictest19982-21systemd-journal22:28:143
251029972,3cyclictest19982-21systemd-journal20:17:393
251029971,4cyclictest19982-21systemd-journal23:47:303
251029971,2cyclictest502-21systemd-logind20:51:513
250969974,2cyclictest11738-21rs:main2
250969974,2cyclictest11736-21in:imuxsock19:58:282
250969973,3cyclictest508-21dbus-daemon23:38:442
250969973,3cyclictest30221-21df_inode20:30:212
250969973,3cyclictest29741-11brltty19:39:162
250969973,3cyclictest19982-21systemd-journal00:30:012
250969973,3cyclictest16969-21lldpd22:04:092
250969973,3cyclictest0-21swapper/223:49:562
250969973,1cyclictest0-21swapper/222:31:102
250969973,1cyclictest0-21swapper/222:05:062
250969973,1cyclictest0-21swapper/221:49:162
250969973,1cyclictest0-21swapper/200:13:492
250969972,3cyclictest3758-21snmpd22:43:012
250969972,3cyclictest19982-21systemd-journal22:39:052
250969972,3cyclictest19982-21systemd-journal21:07:112
250969972,3cyclictest19982-21systemd-journal20:11:322
250969972,3cyclictest19982-21systemd-journal00:30:042
250969971,4cyclictest19982-21systemd-journal21:38:342
250969971,4cyclictest19982-21systemd-journal21:10:372
250969970,4cyclictest9193-21munin-run19:45:012
250929974,2cyclictest11736-21in:imuxsock00:29:391
250929973,3cyclictest31310-21cron21:45:021
250929973,3cyclictest29741-11brltty19:33:501
250929973,3cyclictest19982-21systemd-journal22:50:451
250929973,3cyclictest19982-21systemd-journal21:13:071
250929973,3cyclictest0-21swapper/121:22:431
250929973,1cyclictest0-21swapper/122:28:201
250929973,1cyclictest0-21swapper/122:16:381
250929973,1cyclictest0-21swapper/121:15:011
250929972,3cyclictest19982-21systemd-journal21:31:041
250929972,3cyclictest19982-21systemd-journal21:31:041
250929972,3cyclictest16969-21lldpd19:14:021
250929971,5cyclictest1856-21/usr/sbin/munin20:40:361
250929970,4cyclictest6948-21systemctl23:00:001
250879976,1cyclictest0-21swapper/022:36:320
250879974,2cyclictest508-21dbus-daemon23:14:570
250879974,2cyclictest508-21dbus-daemon19:44:160
250879974,2cyclictest508-21dbus-daemon19:37:360
250879974,2cyclictest11736-21in:imuxsock22:59:310
250879973,3cyclictest7534-21/usr/sbin/munin23:00:250
250879973,3cyclictest3758-21snmpd20:49:590
250879973,3cyclictest3758-21snmpd20:19:590
250879973,3cyclictest3758-21snmpd19:55:450
250879973,3cyclictest29793-21sort22:30:180
250879973,3cyclictest26826-21grep20:20:300
250879973,3cyclictest19982-21systemd-journal22:10:000
250879973,3cyclictest19982-21systemd-journal21:50:010
250879973,3cyclictest19982-21systemd-journal21:27:130
250879973,3cyclictest17605-21systemctl20:00:300
250879973,3cyclictest0-21swapper/023:15:020
250879973,3cyclictest0-21swapper/022:15:240
250879973,3cyclictest0-21swapper/000:32:270
250879973,3cyclictest0-21swapper/000:21:460
250879973,3cyclictest0-21swapper/000:07:310
250879973,2cyclictest19982-21systemd-journal00:16:130
250879973,1cyclictest0-21swapper/023:24:110
250879973,1cyclictest0-21swapper/021:40:210
250879973,1cyclictest0-21swapper/021:16:240
250879971,4cyclictest668-21runrttasks00:13:080
251029966,0cyclictest0-21swapper/322:50:403
251029966,0cyclictest0-21swapper/319:40:293
251029964,1cyclictest5444-21grep19:35:213
251029963,2cyclictest668-21runrttasks00:35:403
251029963,2cyclictest29927-21chrt19:17:273
251029963,2cyclictest29741-11brltty20:05:593
251029963,2cyclictest27854-21ssh23:57:193
251029963,2cyclictest25865-21cut20:20:223
251029963,2cyclictest19012-21rm21:35:203
251029963,2cyclictest14986-21latency_hist21:10:013
251029963,2cyclictest14826-21sh22:20:133
251029963,2cyclictest12118-21ssh00:10:103
251029963,2cyclictest11736-21in:imuxsock23:50:503
251029963,2cyclictest0-21swapper/322:00:303
251029963,2cyclictest0-21swapper/321:34:243
251029963,2cyclictest0-21swapper/321:34:243
251029962,3cyclictest668-21runrttasks23:21:003
251029962,3cyclictest5795-21expr00:30:203
251029962,3cyclictest508-21dbus-daemon21:25:003
251029962,3cyclictest508-21dbus-daemon20:56:523
251029962,3cyclictest3758-21snmpd22:55:043
251029962,3cyclictest3758-21snmpd21:55:083
251029962,3cyclictest3758-21snmpd21:01:093
251029962,3cyclictest3758-21snmpd20:10:103
251029962,3cyclictest3758-21snmpd19:20:143
251029962,3cyclictest29741-11brltty22:14:533
251029962,3cyclictest27989-21df_inode20:25:193
251029962,3cyclictest27777-21cat22:05:293
251029962,3cyclictest26302-21cut00:20:263
251029962,3cyclictest22487-21latency_hist21:15:003
251029962,3cyclictest19982-21systemd-journal21:47:213
251029962,3cyclictest19982-21systemd-journal20:45:353
251029962,3cyclictest18862-21grep00:15:223
251029962,3cyclictest14562-21cat19:55:223
251029962,3cyclictest12205-21cat22:40:193
251029962,3cyclictest10121-21cat19:45:233
251029962,3cyclictest0-21swapper/321:50:503
251029962,1cyclictest0-21swapper/321:40:513
251029961,2cyclictest26289-21sh19:10:273
251029960,5cyclictest30762-21ntpq20:30:253
251029960,5cyclictest15482-21expr23:05:223
251029960,4cyclictest31047-40systemctl00:00:013
251029960,0cyclictest0-21swapper/322:30:423
250969966,0cyclictest0-21swapper/223:26:412
250969964,1cyclictest668-21runrttasks21:57:072
250969963,2cyclictest8475-21wc20:55:012
250969963,2cyclictest668-21runrttasks00:16:122
250969963,2cyclictest508-21dbus-daemon22:15:182
250969963,2cyclictest508-21dbus-daemon19:25:302
250969963,2cyclictest3758-21snmpd22:25:082
250969962,3cyclictest8822-21cat23:00:262
250969962,3cyclictest3758-21snmpd20:40:322
250969962,3cyclictest3758-21snmpd20:37:432
250969962,3cyclictest3758-21snmpd20:21:162
250969962,3cyclictest3758-21snmpd19:45:492
250969962,3cyclictest3758-21snmpd00:35:292
250969962,3cyclictest31794-21expr00:00:192
250969962,3cyclictest28283-21cat20:25:212
250969962,3cyclictest25215-21grep20:20:002
250969962,3cyclictest16829-21df20:00:212
250969962,3cyclictest16536-21sed22:20:292
250969962,3cyclictest13191-21unixbench_singl19:50:322
250969962,3cyclictest0-21swapper/223:13:452
250969962,3cyclictest0-21swapper/222:45:112
250969962,3cyclictest0-21swapper/200:21:522
250969962,2cyclictest16969-21lldpd19:20:172
250969962,1cyclictest0-21swapper/221:15:442
250969961,4cyclictest25300-21/usr/sbin/munin19:10:272
250969961,3cyclictest19982-21systemd-journal21:53:122
250969961,3cyclictest19982-21systemd-journal19:16:192
250969961,3cyclictest1-21systemd21:20:382
250969961,1cyclictest1444-21irqstats22:10:252
250969960,2cyclictest15899-21irqstats23:05:242
250929963,2cyclictest5116-21idleruntime21:25:231
250929963,2cyclictest508-21dbus-daemon19:20:551
250929963,2cyclictest4922-21sh23:41:351
250929963,2cyclictest3758-21snmpd20:05:571
250929963,2cyclictest29331-21systemctl19:15:301
250929963,2cyclictest19317-21ssh22:22:461
250929963,2cyclictest19300-21ssh23:07:361
250929963,2cyclictest17135-21ssh21:58:011
250929963,2cyclictest13527-21sh00:35:321
250929962,4cyclictest26485-21kworker/1:222:07:311
250929962,3cyclictest8602-21ntpq23:00:251
250929962,3cyclictest3758-21snmpd23:21:211
250929962,3cyclictest3758-21snmpd22:10:201
250929962,3cyclictest3758-21snmpd21:47:521
250929962,3cyclictest3758-21snmpd21:03:081
250929962,3cyclictest3758-21snmpd20:55:201
250929962,3cyclictest3758-21snmpd20:51:021
250929962,3cyclictest3758-21snmpd19:47:131
250929962,3cyclictest3758-21snmpd19:36:151
250929962,3cyclictest3758-21snmpd00:15:241
250929962,3cyclictest3758-21snmpd00:10:201
250929962,3cyclictest32195-21cat00:00:211
250929962,3cyclictest28808-21sensors_temp20:25:271
250929962,3cyclictest23035-21df23:10:171
250929962,3cyclictest21274-21unixbench_singl22:00:301
250929962,3cyclictest21128-21cat20:10:211
250929962,3cyclictest16767-21cat23:50:201
250929962,3cyclictest12630-21ntpq19:50:281
250929962,3cyclictest0-21swapper/123:56:141
250929962,3cyclictest0-21swapper/120:45:021
250929962,3cyclictest0-21swapper/100:30:101
250929962,1cyclictest0-21swapper/123:17:121
250929962,1cyclictest0-21swapper/122:40:191
250929962,1cyclictest0-21swapper/100:08:301
250929961,5cyclictest26485-21kworker/1:219:42:121
250929961,4cyclictest0-21swapper/119:30:011
250929961,2cyclictest6019-21ssh21:50:101
250929960,3cyclictest5199-21diskstats22:35:241
250929960,3cyclictest21019-21ssh21:36:321
250929960,2cyclictest16136-21latency_hist23:50:011
250879966,0cyclictest0-21swapper/021:32:470
250879966,0cyclictest0-21swapper/021:32:470
250879963,2cyclictest19321-21ssh22:00:080
250879963,2cyclictest14985-21diskmemload23:06:360
250879963,2cyclictest11016-21cat21:00:210
250879963,2cyclictest0-21swapper/022:10:140
250879963,1cyclictest0-21swapper/022:50:030
250879962,3cyclictest9243-21awk20:55:240
250879962,3cyclictest7268-21ntpq20:50:250
250879962,3cyclictest508-21dbus-daemon19:25:020
250879962,3cyclictest508-21dbus-daemon00:00:000
250879962,3cyclictest3758-21snmpd21:05:560
250879962,3cyclictest3758-21snmpd20:35:190
250879962,3cyclictest3758-21snmpd20:25:020
250879962,3cyclictest3758-21snmpd20:10:480
250879962,3cyclictest3422-21cat19:30:240
250879962,3cyclictest28727-21grep19:15:250
250879962,3cyclictest26268-21ntpq19:10:260
250879962,3cyclictest22672-21expr22:25:210
250879962,3cyclictest2017-21expr20:40:210
250879962,3cyclictest13278-21if_err_enp4s023:25:240
250879962,3cyclictest12804-21ssh21:55:160
250879962,3cyclictest1-21systemd20:31:500
250879962,3cyclictest12104-21cat00:35:170
250879962,3cyclictest10038-21cat23:45:220
250879962,1cyclictest0-21swapper/021:10:420
250879960,6cyclictest0-21swapper/019:45:070
250879960,6cyclictest0-21swapper/000:25:020
250879960,5cyclictest18672-21diskstats21:35:190
251029953,1cyclictest3758-21snmpd19:50:063
251029952,2cyclictest3758-21snmpd19:25:023
250969951,3cyclictest3758-21snmpd19:30:112
250969951,3cyclictest29741-11brltty21:00:112
250929952,2cyclictest414-21expr20:35:261
250929952,2cyclictest30091-21expr20:30:211
250929952,2cyclictest23553-21fschecks_count20:15:231
250879952,2cyclictest18881-21cat20:05:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional