You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-17 - 03:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackaslot8.osadl.org (updated Sun Jun 16, 2024 00:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
84942319286,21sleep20-21swapper/219:09:422
84942319286,21sleep20-21swapper/219:09:422
83532276212,21sleep00-21swapper/019:07:510
83532276212,21sleep00-21swapper/019:07:510
81672265233,20sleep30-21swapper/319:05:413
81672265233,20sleep30-21swapper/319:05:413
82002261227,22sleep10-21swapper/119:06:071
82002261227,22sleep10-21swapper/119:06:061
139372700,0sleep30-21swapper/300:05:543
235642570,1sleep00-21swapper/022:11:590
116882550,0sleep10-21swapper/121:40:291
268512540,0sleep30-21swapper/321:50:563
171502540,0sleep30-21swapper/322:07:333
211842210,0sleep00-21swapper/021:47:230
133742190,0sleep20-21swapper/200:05:292
81782180,1sleep37997-21latency_hist00:30:003
269962180,0sleep20-21swapper/221:29:472
39922170,0sleep20-21swapper/221:58:032
121732130,0sleep20-21swapper/200:32:592
862699103,2cyclictest4441-21sh21:35:320
862699103,2cyclictest1894-21sh21:56:050
862699103,1cyclictest16519-21ntp_kernel_pll_22:55:210
862699102,3cyclictest29741-11brltty20:10:160
862699102,2cyclictest21860-21cat19:35:230
862699100,4cyclictest19982-21systemd-journal21:03:400
862699100,3cyclictest502-21systemd-logind20:05:010
862699100,3cyclictest19982-21systemd-journal22:15:260
862699100,3cyclictest19982-21systemd-journal20:30:010
862699100,3cyclictest19982-21systemd-journal20:16:570
862699100,3cyclictest19982-21systemd-journal00:40:000
862699100,3cyclictest19982-21systemd-journal00:20:100
86379994,3cyclictest19982-21systemd-journal22:27:363
86349998,1cyclictest28-21ksoftirqd/222:50:012
86349993,5cyclictest0-21swapper/222:02:052
86319992,5cyclictest502-21systemd-logind23:25:311
86269997,1cyclictest25236-21tr19:45:130
86269996,0cyclictest0-21swapper/020:30:280
86269993,2cyclictest11736-21in:imuxsock23:41:460
86269993,1cyclictest51950irq/137-enp4s0-21:30:110
86269993,1cyclictest30706-21diskmemload21:10:340
86269993,1cyclictest2142-21missed_timers22:20:230
86269993,1cyclictest11738-21rs:main0
86269992,3cyclictest31815-21apt-get20:00:110
86269992,3cyclictest29424-21fschecks_time23:05:230
86269992,3cyclictest13388-21snmpd19:21:020
86269992,2cyclictest26862-21rm21:29:390
86269992,2cyclictest24534-21taskset23:01:040
86269992,2cyclictest24534-21taskset23:01:040
86269992,2cyclictest10728-21ssh22:26:370
86269992,1cyclictest0-21swapper/022:42:400
86269991,2cyclictest11137-21ssh22:50:520
86269990,3cyclictest9497-21ntp_states00:30:220
86269990,3cyclictest27088-21rm21:51:080
86269990,3cyclictest19982-21systemd-journal23:26:460
86269990,3cyclictest19982-21systemd-journal20:22:390
86269990,3cyclictest19982-21systemd-journal19:59:250
17195290,0sleep30-21swapper/321:22:283
86379986,1cyclictest0-21swapper/323:36:363
86379983,3cyclictest19982-21systemd-journal23:06:063
86379983,3cyclictest19982-21systemd-journal21:57:593
86349987,1cyclictest28-21ksoftirqd/219:20:012
86349984,1cyclictest508-21dbus-daemon20:20:002
86349980,5cyclictest28718-21ls00:20:002
86319985,2cyclictest508-21dbus-daemon23:56:231
86319983,3cyclictest19982-21systemd-journal23:05:011
86319983,3cyclictest19982-21systemd-journal23:05:001
86269982,2cyclictest26929-21ssh23:51:030
86269982,2cyclictest20306-21ssh22:34:360
86269982,2cyclictest13388-21snmpd19:50:470
86269982,1cyclictest14350-21ssh21:42:540
86269982,1cyclictest0-21swapper/021:21:500
86269981,1cyclictest0-21swapper/000:08:580
86269980,3cyclictest6948-21diskstats22:00:170
86269980,3cyclictest18101-21df_inode23:45:180
86269980,0cyclictest0-21swapper/021:18:490
29601280,0sleep00-21swapper/023:30:070
24965280,0sleep10-21swapper/120:55:191
20034280,0sleep30-21swapper/320:45:123
18467280,0sleep10-21swapper/123:45:221
86379973,3cyclictest9812-21sensors_temp00:30:263
86379973,3cyclictest508-21dbus-daemon23:20:013
86379973,3cyclictest508-21dbus-daemon21:18:163
86379973,3cyclictest508-21dbus-daemon19:26:523
86379973,3cyclictest508-21dbus-daemon19:20:003
86379973,3cyclictest29741-11brltty22:13:213
86379973,3cyclictest19982-21systemd-journal23:32:043
86379973,3cyclictest19982-21systemd-journal20:58:063
86379973,3cyclictest19982-21systemd-journal20:11:313
86379973,3cyclictest13388-21snmpd22:59:233
86379973,3cyclictest13388-21snmpd20:39:163
86379973,3cyclictest13388-21snmpd20:32:323
86379973,3cyclictest13388-21snmpd20:20:243
86379973,3cyclictest0-21swapper/323:03:493
86379973,3cyclictest0-21swapper/323:03:493
86379973,3cyclictest0-21swapper/300:14:543
86379972,3cyclictest19982-21systemd-journal22:30:023
86379972,3cyclictest19982-21systemd-journal22:17:363
86379972,3cyclictest19982-21systemd-journal21:34:593
86349974,2cyclictest502-21systemd-logind20:52:242
86349973,3cyclictest502-21systemd-logind00:00:342
86349973,3cyclictest19982-21systemd-journal22:14:562
86349973,3cyclictest19116-21gdbus21:10:012
86349973,3cyclictest16969-21lldpd21:45:232
86349973,3cyclictest13388-21snmpd23:02:322
86349973,3cyclictest13388-21snmpd23:02:322
86349973,3cyclictest0-21swapper/221:36:042
86349973,1cyclictest0-21swapper/223:49:122
86349973,1cyclictest0-21swapper/223:41:022
86349973,1cyclictest0-21swapper/223:39:022
86349973,1cyclictest0-21swapper/222:30:512
86349973,1cyclictest0-21swapper/221:50:072
86349972,3cyclictest19982-21systemd-journal22:55:542
86349972,3cyclictest19982-21systemd-journal20:14:322
86349971,3cyclictest16344-21rm21:21:402
86349970,7cyclictest0-21swapper/220:25:232
86319976,1cyclictest21394-21ssh22:59:431
86319976,1cyclictest17112-21ssh22:07:311
86319975,1cyclictest22-21ksoftirqd/123:10:011
86319973,3cyclictest508-21dbus-daemon23:34:421
86319973,3cyclictest508-21dbus-daemon20:26:321
86319973,3cyclictest19982-21systemd-journal00:04:441
86319973,1cyclictest0-21swapper/123:17:161
86319973,1cyclictest0-21swapper/122:01:201
86319972,4cyclictest0-21swapper/121:55:011
86319972,3cyclictest19982-21systemd-journal00:14:591
86319971,4cyclictest19982-21systemd-journal21:49:341
86319971,3cyclictest13669-21cron19:20:011
86319970,2cyclictest19982-21systemd-journal20:40:011
86269973,1cyclictest0-21swapper/023:58:560
86269972,3cyclictest19982-21systemd-journal23:24:410
86269972,2cyclictest15292-21ssh22:05:340
86269970,4cyclictest24831-21perf19:45:000
86269970,2cyclictest26705-21sh22:38:530
17537270,0sleep10-21swapper/121:22:461
86379964,2cyclictest24559-21kworker/3:119:40:153
86379963,2cyclictest9938-21missed_timers22:50:213
86379963,2cyclictest508-21dbus-daemon22:00:273
86379963,2cyclictest3541-21chrt20:06:043
86379963,2cyclictest31153-21rm21:10:123
86379963,2cyclictest29741-11brltty21:25:463
86379963,2cyclictest29604-21fschecks_time00:20:163
86379963,2cyclictest19982-21systemd-journal22:21:173
86379963,2cyclictest19579-21ssh21:45:593
86379963,2cyclictest13388-21snmpd00:00:063
86379963,2cyclictest11736-21in:imuxsock20:45:013
86379962,3cyclictest9206-21expr19:10:153
86379962,3cyclictest29088-21expr21:05:163
86379962,3cyclictest2906-21expr22:45:143
86379962,3cyclictest28748-21expr22:40:133
86379962,3cyclictest27023-21fw_packets21:00:143
86379962,3cyclictest23878-21expr00:15:163
86379962,3cyclictest22855-21apt-get23:25:113
86379962,3cyclictest19982-21systemd-journal19:54:253
86379962,3cyclictest17578-21ntpq23:20:203
86379962,3cyclictest15156-21basename00:35:183
86379962,3cyclictest13388-21snmpd23:46:393
86379962,3cyclictest13388-21snmpd23:13:443
86379962,3cyclictest13388-21snmpd20:25:023
86379962,3cyclictest13388-21snmpd19:55:063
86379962,3cyclictest13388-21snmpd19:36:113
86379962,3cyclictest1-21systemd19:31:033
86379962,3cyclictest11081-21cat23:40:113
86379962,1cyclictest0-21swapper/323:52:123
86379962,1cyclictest0-21swapper/322:36:243
86379961,3cyclictest19982-21systemd-journal23:58:323
86379961,3cyclictest19982-21systemd-journal21:41:393
86379961,3cyclictest19982-21systemd-journal20:00:153
86379960,4cyclictest19982-21systemd-journal21:32:263
86349964,2cyclictest28-21ksoftirqd/220:35:432
86349963,2cyclictest949-21ssh23:55:372
86349963,2cyclictest508-21dbus-daemon22:22:062
86349963,2cyclictest30080-21df23:30:122
86349963,2cyclictest29250-21cut00:20:132
86349963,2cyclictest27678-21expr19:50:132
86349963,2cyclictest20713-21tail20:45:192
86349963,2cyclictest19757-21ssh23:21:592
86349963,2cyclictest19523-21grep19:30:222
86349963,2cyclictest14612-21cat19:20:192
86349963,2cyclictest14339-21strings22:05:182
86349963,1cyclictest0-21swapper/221:17:042
86349962,3cyclictest9601-21ntpq20:20:212
86349962,3cyclictest31257-21sshd19:57:232
86349962,3cyclictest28686-21expr22:40:142
86349962,3cyclictest28128-21tr22:15:232
86349962,3cyclictest2426-21fschecks_time20:05:172
86349962,3cyclictest19982-21systemd-journal23:50:052
86349962,3cyclictest19982-21systemd-journal21:00:052
86349962,3cyclictest16161-21grep00:35:282
86349962,3cyclictest13388-21snmpd22:53:472
86349962,3cyclictest13388-21snmpd21:32:532
86349962,3cyclictest13388-21snmpd19:12:162
86349962,3cyclictest0-21swapper/221:10:232
86349962,1cyclictest0-21swapper/222:36:362
86349962,1cyclictest0-21swapper/222:25:152
86349962,1cyclictest0-21swapper/200:25:242
86349961,3cyclictest19982-21systemd-journal23:28:012
86349961,3cyclictest19982-21systemd-journal20:56:342
86349961,2cyclictest9603-21ssh23:15:042
86349960,6cyclictest0-21swapper/223:14:222
86349960,4cyclictest19982-21systemd-journal00:10:092
86349960,3cyclictest19982-21systemd-journal23:05:292
86349960,3cyclictest13597-21rm21:42:142
86319966,0cyclictest0-21swapper/119:50:161
86319964,1cyclictest28329-21cut21:30:161
86319963,2cyclictest8059-21cstates22:25:141
86319963,2cyclictest508-21dbus-daemon00:05:331
86319963,2cyclictest4732-21hddtemp_smartct23:35:181
86319963,2cyclictest32422-21sh21:55:181
86319963,2cyclictest32286-21sh21:10:221
86319963,2cyclictest30676-21ssh22:40:591
86319963,2cyclictest29741-11brltty00:20:091
86319963,2cyclictest20981-21expr19:35:121
86319963,2cyclictest14834-21chrt20:32:011
86319963,2cyclictest13388-21snmpd21:36:201
86319963,2cyclictest13388-21snmpd00:36:311
86319963,2cyclictest0-21swapper/123:10:051
86319963,2cyclictest0-21swapper/100:16:061
86319963,1cyclictest0-21swapper/122:30:421
86319962,3cyclictest6756-21cat21:15:171
86319962,3cyclictest30210-21cat19:55:151
86319962,3cyclictest25426-21cat19:45:151
86319962,3cyclictest22181-21expr20:50:121
86319962,3cyclictest20832-21sed20:45:221
86319962,3cyclictest20494-21cat22:10:101
86319962,3cyclictest19982-21systemd-journal22:15:261
86319962,3cyclictest13388-21snmpd21:25:231
86319962,3cyclictest13388-21snmpd21:05:001
86319962,3cyclictest13388-21snmpd20:41:251
86319962,3cyclictest13388-21snmpd20:10:181
86319962,3cyclictest13388-21snmpd19:44:231
86319962,3cyclictest13388-21snmpd19:30:081
86319962,3cyclictest13388-21snmpd19:25:381
86319962,3cyclictest13388-21snmpd19:11:471
86319962,3cyclictest0-21swapper/123:50:541
86319962,3cyclictest0-21swapper/122:20:331
86319962,3cyclictest0-21swapper/100:25:051
86319962,1cyclictest0-21swapper/123:40:061
86319962,1cyclictest0-21swapper/123:21:221
86319961,3cyclictest19982-21systemd-journal22:45:031
86319961,2cyclictest10741-21ssh00:31:121
86319960,6cyclictest0-21swapper/122:35:181
86319960,5cyclictest31790-21/usr/sbin/munin20:00:131
86319960,4cyclictest19982-21systemd-journal20:05:161
86319960,3cyclictest11986-21rm22:51:431
86269966,0cyclictest0-21swapper/019:30:120
86269964,1cyclictest10973-21latency_hist19:15:010
86269963,2cyclictest668-21runrttasks00:12:200
86269963,2cyclictest12-21ksoftirqd/020:50:140
86269962,3cyclictest4838-21cat23:35:200
86269962,3cyclictest3161-21smartctl00:25:130
86269962,3cyclictest28982-21expr21:05:160
86269962,3cyclictest20084-21expr20:45:110
86269962,3cyclictest17907-21ls20:40:140
86269962,3cyclictest13388-21snmpd23:15:050
86269962,3cyclictest13388-21snmpd23:11:560
86269962,3cyclictest13388-21snmpd00:15:160
86269962,3cyclictest12087-21df_inode19:15:150
86269962,3cyclictest0-21swapper/022:46:020
86269961,4cyclictest25448-21cut20:55:240
86269960,2cyclictest19982-21systemd-journal19:29:060
86379952,2cyclictest25793-21cut19:45:213
86379952,2cyclictest14450-21tune2fs19:20:163
86379952,2cyclictest13388-21snmpd20:15:183
86379951,3cyclictest22473-21tune2fs20:50:153
86349952,2cyclictest19752-21wc20:45:012
86349952,2cyclictest16773-21which19:25:192
86349952,2cyclictest13388-21snmpd19:35:022
86349951,3cyclictest23239-21/usr/sbin/munin19:40:142
86349950,4cyclictest31790-21/usr/sbin/munin20:00:112
86349950,3cyclictest19982-21systemd-journal19:45:142
86349950,1cyclictest13138-21apt-get20:30:122
86319953,1cyclictest668-21runrttasks20:19:581
86319952,2cyclictest26721-21cat21:00:111
86319952,2cyclictest14097-21cut19:20:141
86319950,3cyclictest19982-21systemd-journal20:20:061
86269952,2cyclictest15637-21cat20:35:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional