You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-09 - 09:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackaslot8.osadl.org (updated Sun Feb 09, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
33372311283,18sleep20-21swapper/219:09:302
32442282248,22sleep10-21swapper/119:08:171
31142281247,21sleep30-21swapper/319:06:503
30832279208,22sleep00-21swapper/019:06:270
301572720,0sleep00-21swapper/023:42:120
54962560,0sleep30-21swapper/322:58:193
101212200,0sleep10-21swapper/123:27:201
153592150,1sleep20-21swapper/220:45:152
263542140,0sleep20-21swapper/221:34:402
45522110,0sleep20-21swapper/222:32:072
30702100,0sleep20-21swapper/221:15:502
3889290,1sleep13492-21cyclictest20:20:151
35139997,1cyclictest34-21ksoftirqd/320:30:013
35139993,5cyclictest0-21swapper/322:20:013
35089994,3cyclictest19982-21systemd-journal19:45:182
35089994,3cyclictest19982-21systemd-journal19:42:062
35049994,3cyclictest19982-21systemd-journal21:45:281
34989994,3cyclictest19982-21systemd-journal21:31:400
34989994,3cyclictest19982-21systemd-journal19:29:270
23966290,0sleep20-21swapper/222:47:572
23966290,0sleep20-21swapper/222:47:572
35139984,3cyclictest30654-21snmpd22:14:123
35139984,3cyclictest19982-21systemd-journal19:54:293
35139980,7cyclictest0-21swapper/323:20:583
35089986,1cyclictest0-21swapper/222:40:242
35089984,3cyclictest508-21dbus-daemon22:02:092
35089984,3cyclictest508-21dbus-daemon20:19:302
35089984,3cyclictest19982-21systemd-journal23:44:462
35089983,3cyclictest19982-21systemd-journal23:08:042
35089983,3cyclictest19982-21systemd-journal21:08:532
35089983,3cyclictest19982-21systemd-journal00:33:542
35089980,7cyclictest0-21swapper/200:28:532
35049983,3cyclictest19982-21systemd-journal20:46:541
35049981,5cyclictest502-21systemd-logind21:38:211
34989984,3cyclictest19982-21systemd-journal20:15:010
34989984,3cyclictest13787-21cat20:40:220
30255280,0sleep10-21swapper/122:27:131
35139976,1cyclictest34-21ksoftirqd/300:20:003
35139975,1cyclictest34-21ksoftirqd/323:10:013
35139975,1cyclictest34-21ksoftirqd/321:20:003
35139975,1cyclictest34-21ksoftirqd/319:40:003
35139974,2cyclictest32727-21sshd00:37:593
35139974,2cyclictest21542-21tr21:00:133
35139974,1cyclictest19982-21systemd-journal20:49:323
35139973,3cyclictest508-21dbus-daemon20:42:023
35139973,3cyclictest508-21dbus-daemon20:17:013
35139973,3cyclictest29292-21ls20:05:143
35139973,3cyclictest19982-21systemd-journal22:42:383
35139973,3cyclictest19982-21systemd-journal21:25:493
35139973,3cyclictest16088-21seq21:51:053
35139973,1cyclictest0-21swapper/322:38:203
35139973,1cyclictest0-21swapper/321:35:083
35139973,1cyclictest0-21swapper/300:10:233
35139972,3cyclictest19982-21systemd-journal20:31:363
35139971,4cyclictest19982-21systemd-journal23:36:253
35089974,2cyclictest508-21dbus-daemon23:40:002
35089974,2cyclictest30654-21snmpd21:59:362
35089974,2cyclictest11736-21in:imuxsock21:20:502
35089973,2cyclictest19982-21systemd-journal19:35:002
35089973,2cyclictest19982-21systemd-journal00:17:182
35089973,1cyclictest0-21swapper/200:22:182
35089972,3cyclictest19982-21systemd-journal23:05:012
35089972,3cyclictest19982-21systemd-journal22:55:582
35089972,3cyclictest19982-21systemd-journal22:07:302
35089970,4cyclictest12163-21munin-run20:40:012
35049976,1cyclictest22-21ksoftirqd/121:15:011
35049974,2cyclictest7325-21cron21:20:001
35049974,2cyclictest22-21ksoftirqd/121:05:001
35049973,3cyclictest508-21dbus-daemon20:06:591
35049973,3cyclictest19982-21systemd-journal19:22:211
35049973,3cyclictest19982-21systemd-journal00:02:561
35049973,1cyclictest0-21swapper/123:47:531
35049973,1cyclictest0-21swapper/123:02:351
35049973,1cyclictest0-21swapper/122:52:431
35049973,1cyclictest0-21swapper/122:05:441
35049973,1cyclictest0-21swapper/100:17:571
35049973,1cyclictest0-21swapper/100:13:341
35049972,3cyclictest7661-21lldpd21:05:161
35049972,3cyclictest19982-21systemd-journal20:52:531
35049971,4cyclictest19982-21systemd-journal20:29:561
35049970,3cyclictest22149-21munin-run00:30:001
34989976,1cyclictest0-21swapper/023:23:350
34989974,2cyclictest508-21dbus-daemon21:04:340
34989974,2cyclictest23015-21tr00:30:100
34989973,3cyclictest508-21dbus-daemon00:09:480
34989973,3cyclictest30654-21snmpd19:20:210
34989973,3cyclictest19982-21systemd-journal23:00:540
34989973,3cyclictest17335-21sshd23:32:500
34989973,3cyclictest1-21systemd19:41:330
34989973,3cyclictest0-21swapper/023:45:270
34989973,3cyclictest0-21swapper/022:42:460
34989973,3cyclictest0-21swapper/021:53:490
34989973,1cyclictest0-21swapper/023:17:390
34989973,1cyclictest0-21swapper/022:48:380
34989973,1cyclictest0-21swapper/022:48:380
34989973,1cyclictest0-21swapper/022:05:320
34989972,3cyclictest502-21systemd-logind23:27:400
34989972,3cyclictest19982-21systemd-journal00:18:340
34989971,4cyclictest502-21systemd-logind22:31:460
34989970,6cyclictest2168-21wc00:40:000
34989970,4cyclictest27844-21munin-run22:25:010
34989970,4cyclictest13744-21turbostat22:15:010
35139965,1cyclictest6426-21kworker/3:221:55:033
35139963,2cyclictest9650-21tail19:20:153
35139963,2cyclictest3594-21rm21:40:413
35139963,2cyclictest30654-21snmpd19:55:043
35139963,2cyclictest30419-21sh23:17:163
35139963,2cyclictest29086-21sh22:51:243
35139963,2cyclictest28551-21rm22:25:303
35139963,2cyclictest28392-21ssh00:07:163
35139963,2cyclictest2488-21timerwakeupswit22:30:163
35139963,2cyclictest19708-21hddtemp_smartct20:55:163
35139963,2cyclictest1-21systemd00:00:003
35139963,2cyclictest11736-21in:imuxsock20:52:203
35139962,3cyclictest8217-21date21:45:013
35139962,3cyclictest7659-21lldpd23:46:413
35139962,3cyclictest508-21dbus-daemon21:21:403
35139962,3cyclictest508-21dbus-daemon20:05:013
35139962,3cyclictest30654-21snmpd23:51:573
35139962,3cyclictest30654-21snmpd23:44:113
35139962,3cyclictest30654-21snmpd23:25:443
35139962,3cyclictest30654-21snmpd22:21:363
35139962,3cyclictest30654-21snmpd22:07:003
35139962,3cyclictest30654-21snmpd21:12:483
35139962,3cyclictest30654-21snmpd19:28:273
35139962,3cyclictest30654-21snmpd19:15:333
35139962,3cyclictest30654-21snmpd19:15:003
35139962,3cyclictest28666-21systemctl22:00:203
35139962,3cyclictest24141-21expr21:05:133
35139962,3cyclictest20852-21cat21:30:153
35139962,3cyclictest20510-21cat23:10:123
35139962,3cyclictest19116-21gdbus20:15:013
35139962,3cyclictest13625-21/usr/sbin/munin23:55:173
35139962,1cyclictest0-21swapper/300:32:323
35139961,4cyclictest21487-21smart_sda19:45:203
35139961,3cyclictest19982-21systemd-journal00:21:543
35139961,1cyclictest141rcu_preempt23:30:093
35139960,6cyclictest0-21swapper/322:46:563
35139960,6cyclictest0-21swapper/322:46:563
35139960,6cyclictest0-21swapper/300:27:223
35139960,4cyclictest19982-21systemd-journal19:31:063
35139960,3cyclictest8512-21sed23:00:163
35139960,2cyclictest5234-21sshd20:22:183
35089963,2cyclictest9856-21ssh23:52:332
35089963,2cyclictest668-21runrttasks00:12:262
35089963,2cyclictest30654-21snmpd21:35:102
35089963,2cyclictest30654-21snmpd19:39:092
35089963,2cyclictest28138-21xargs23:15:192
35089963,2cyclictest19431-21cron23:14:562
35089963,2cyclictest13357-21sh22:39:062
35089963,2cyclictest11736-21in:imuxsock21:25:252
35089963,2cyclictest11736-21in:imuxsock20:40:472
35089962,3cyclictest4401-21forks19:10:142
35089962,3cyclictest31721-21expr20:10:102
35089962,3cyclictest30654-21snmpd23:59:122
35089962,3cyclictest30654-21snmpd22:50:582
35089962,3cyclictest30654-21snmpd22:10:302
35089962,3cyclictest30654-21snmpd20:30:052
35089962,3cyclictest30654-21snmpd20:23:172
35089962,3cyclictest30654-21snmpd20:06:342
35089962,3cyclictest30654-21snmpd19:51:072
35089962,3cyclictest30654-21snmpd19:17:472
35089962,3cyclictest29741-11brltty19:26:572
35089962,3cyclictest25688-21ntpq19:55:202
35089962,3cyclictest22289-21ntp_kernel_pll_21:00:182
35089962,3cyclictest20045-21ntp_kernel_pll_20:55:192
35089962,3cyclictest15825-21grep22:15:222
35089962,3cyclictest13742-21sed23:30:102
35089962,1cyclictest0-21swapper/223:49:212
35089962,1cyclictest0-21swapper/222:29:322
35089962,1cyclictest0-21swapper/221:52:192
35089962,1cyclictest0-21swapper/221:40:062
35089962,1cyclictest0-21swapper/221:10:382
35089962,1cyclictest0-21swapper/200:36:372
35089962,1cyclictest0-21swapper/200:06:512
35089962,1cyclictest0-21swapper/200:01:262
35089961,3cyclictest19982-21systemd-journal21:48:232
35089961,2cyclictest8378-21ssh23:25:322
35089961,2cyclictest24054-21ssh22:21:192
35089960,3cyclictest18855-21awk20:55:012
35089960,0cyclictest0-21swapper/223:20:412
35049966,0cyclictest0-21swapper/122:23:091
35049965,1cyclictest22-21ksoftirqd/100:20:011
35049963,2cyclictest30654-21snmpd20:38:141
35049963,2cyclictest29155-21ssh22:00:251
35049963,2cyclictest27733-21ntp_states23:40:201
35049963,2cyclictest23495-21open_inodes19:50:251
35049963,2cyclictest2286-21cat22:55:211
35049963,2cyclictest21217-21hddtemp_smartct21:55:161
35049963,2cyclictest18152-21ssh21:53:051
35049963,2cyclictest1-21systemd22:10:541
35049963,1cyclictest0-21swapper/123:50:331
35049962,3cyclictest8534-21head20:30:181
35049962,3cyclictest508-21dbus-daemon19:11:551
35049962,3cyclictest30654-21snmpd23:39:441
35049962,3cyclictest30654-21snmpd23:20:141
35049962,3cyclictest30654-21snmpd20:56:061
35049962,3cyclictest30654-21snmpd20:11:241
35049962,3cyclictest30654-21snmpd19:48:401
35049962,3cyclictest30654-21snmpd19:30:481
35049962,3cyclictest25389-21cat19:55:181
35049962,3cyclictest19982-21systemd-journal23:55:121
35049962,3cyclictest18344-21expr19:40:161
35049962,3cyclictest16743-21ntpq19:35:171
35049962,3cyclictest1498-21expr20:15:121
35049962,3cyclictest1-21systemd21:41:421
35049962,3cyclictest1-21systemd20:04:541
35049962,3cyclictest0-21swapper/123:16:251
35049962,3cyclictest0-21swapper/122:20:001
35049962,3cyclictest0-21swapper/121:28:101
35049962,1cyclictest0-21swapper/123:32:391
35049962,1cyclictest0-21swapper/122:45:381
35049962,1cyclictest0-21swapper/122:45:381
35049962,1cyclictest0-21swapper/122:43:151
35049962,1cyclictest0-21swapper/122:31:031
35049962,1cyclictest0-21swapper/121:30:111
35049962,1cyclictest0-21swapper/121:21:331
35049962,1cyclictest0-21swapper/100:31:091
35049962,1cyclictest0-21swapper/100:06:581
35049961,4cyclictest0-21swapper/123:10:121
35049961,3cyclictest19982-21systemd-journal19:17:091
35049960,6cyclictest0-21swapper/120:40:181
35049960,6cyclictest0-21swapper/100:36:401
35049960,5cyclictest7879-21/usr/sbin/munin22:35:151
35049960,5cyclictest16356-21sh23:06:581
34989963,2cyclictest668-21runrttasks21:05:260
34989963,2cyclictest5454-21cron20:25:010
34989963,2cyclictest3391-21sh19:10:020
34989963,2cyclictest29833-21sh21:36:200
34989963,2cyclictest19771-21cat20:55:170
34989963,2cyclictest16282-21sh23:06:560
34989963,2cyclictest11736-21in:imuxsock20:07:500
34989963,2cyclictest11736-21in:imuxsock19:39:530
34989962,3cyclictest9101-21find20:30:220
34989962,3cyclictest7661-21lldpd19:53:570
34989962,3cyclictest508-21dbus-daemon23:36:010
34989962,3cyclictest32064-21basename00:10:110
34989962,3cyclictest30654-21snmpd22:15:180
34989962,3cyclictest30654-21snmpd21:25:250
34989962,3cyclictest30654-21snmpd20:02:190
34989962,3cyclictest30654-21snmpd19:19:540
34989962,3cyclictest30654-21snmpd00:26:190
34989962,3cyclictest25086-21cat19:55:150
34989962,3cyclictest1466-21cat22:55:150
34989962,3cyclictest0-21swapper/023:55:020
34989962,3cyclictest0-21swapper/023:50:130
34989962,3cyclictest0-21swapper/022:35:480
34989962,3cyclictest0-21swapper/021:20:310
34989962,1cyclictest0-21swapper/022:51:260
34989962,1cyclictest0-21swapper/021:46:550
34989962,1cyclictest0-21swapper/000:20:070
34989962,1cyclictest0-21swapper/000:00:210
34989961,3cyclictest19982-21systemd-journal23:10:090
34989961,2cyclictest28435-21ssh22:25:240
34989960,6cyclictest0-21swapper/021:10:170
34989960,5cyclictest14636-21/usr/sbin/munin20:45:120
34989960,4cyclictest19982-21systemd-journal21:15:240
34989960,3cyclictest4567-21rm21:41:380
34989960,3cyclictest29710-21ssh22:00:580
34989960,3cyclictest23058-21ssh21:56:090
35139953,1cyclictest18102-21sort19:40:163
35139951,3cyclictest19982-21systemd-journal20:37:213
35089952,2cyclictest9094-21chrt19:20:112
35089952,2cyclictest27165-21tr20:00:142
35089951,3cyclictest6399-21cat20:25:152
35049952,2cyclictest11486-21grep19:25:141
34989953,1cyclictest1284-21cpu20:15:110
34989952,2cyclictest30654-21snmpd20:35:080
34989952,2cyclictest30654-21snmpd20:25:140
34989952,2cyclictest30654-21snmpd19:30:030
34989952,2cyclictest17015-21cpuspeed_turbos20:50:150
34989950,1cyclictest20252-21/usr/sbin/munin19:45:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional