You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-17 - 21:15
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Mon Jun 17, 2024 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
270142284249,22sleep20-21swapper/207:09:082
271102283250,19sleep30-21swapper/307:09:453
267902281246,23sleep10-21swapper/107:06:121
271022259223,22sleep00-21swapper/007:09:370
273932600,0sleep20-21swapper/210:15:312
47842560,0sleep10-21swapper/107:30:161
280312560,0sleep00-21swapper/009:19:090
321532530,0sleep20-21swapper/210:20:162
86582190,1sleep38655-21fschecks_count11:00:163
2725399108,1cyclictest28-21ksoftirqd/212:39:592
2725399108,1cyclictest28-21ksoftirqd/212:29:592
2725399108,1cyclictest28-21ksoftirqd/212:19:592
2725399108,1cyclictest28-21ksoftirqd/210:55:002
2725399108,1cyclictest28-21ksoftirqd/208:45:012
30007290,0sleep20-21swapper/209:47:112
272539997,2cyclictest28-21ksoftirqd/211:15:012
272539997,2cyclictest271rcuc/210:00:102
272539997,1cyclictest28-21ksoftirqd/211:25:002
272539996,2cyclictest28-21ksoftirqd/211:20:002
272539995,3cyclictest28-21ksoftirqd/209:30:012
272539994,3cyclictest19982-21systemd-journal12:11:102
272539992,5cyclictest141rcu_preempt10:25:452
272539992,2cyclictest141rcu_preempt09:50:132
272499994,3cyclictest19982-21systemd-journal09:46:001
272499990,8cyclictest21635-21sh09:13:401
272449990,7cyclictest19917-21rm10:08:230
13510290,1sleep30-21swapper/311:35:153
8560280,0sleep00-21swapper/010:28:570
272579985,2cyclictest34-21ksoftirqd/311:25:093
272579984,3cyclictest508-21dbus-daemon09:53:553
272579984,2cyclictest19982-21systemd-journal07:35:253
272579983,3cyclictest19982-21systemd-journal09:58:063
272579981,3cyclictest19982-21systemd-journal09:11:463
272539987,1cyclictest28-21ksoftirqd/211:45:012
272539987,1cyclictest28-21ksoftirqd/210:40:002
272539987,1cyclictest28-21ksoftirqd/210:35:002
272539987,1cyclictest28-21ksoftirqd/210:00:012
272539987,1cyclictest28-21ksoftirqd/209:35:002
272539987,1cyclictest28-21ksoftirqd/209:25:012
272539986,1cyclictest28-21ksoftirqd/211:30:002
272539984,2cyclictest19982-21systemd-journal11:31:322
272539982,5cyclictest141rcu_preempt11:35:122
272539982,1cyclictest141rcu_preempt07:35:162
272539980,5cyclictest20985-21munin-run10:10:022
272539980,5cyclictest17799-21readlink08:00:012
272499984,3cyclictest19982-21systemd-journal12:26:361
272499983,3cyclictest19982-21systemd-journal09:40:251
272499983,1cyclictest31484-21awk10:20:001
272449983,3cyclictest19982-21systemd-journal09:40:080
272449983,3cyclictest19982-21systemd-journal08:38:280
272449982,3cyclictest2110-21turbostat07:25:010
272579976,1cyclictest0-21swapper/312:06:253
272579974,2cyclictest19982-21systemd-journal10:20:283
272579973,3cyclictest3490-21tune2fs09:25:163
272579973,3cyclictest29741-11brltty08:39:403
272579973,3cyclictest28721-21sed11:20:013
272579973,3cyclictest20512-21df_inode10:40:153
272579973,3cyclictest19982-21systemd-journal10:50:223
272579973,3cyclictest19982-21systemd-journal09:47:313
272579973,3cyclictest16053-21ntpq09:05:263
272579973,3cyclictest13388-21snmpd10:11:253
272579973,3cyclictest1-21systemd11:51:593
272579973,3cyclictest11289-21cat08:55:213
272579973,3cyclictest0-21swapper/312:23:003
272579973,3cyclictest0-21swapper/310:06:003
272579973,2cyclictest19982-21systemd-journal08:09:573
272579973,1cyclictest0-21swapper/312:39:163
272579972,3cyclictest19982-21systemd-journal08:21:433
272579972,3cyclictest19982-21systemd-journal07:54:483
272539976,1cyclictest28-21ksoftirqd/209:40:012
272539976,1cyclictest28-21ksoftirqd/208:30:292
272539976,1cyclictest28-21ksoftirqd/208:30:002
272539976,1cyclictest28-21ksoftirqd/208:25:002
272539976,1cyclictest28-21ksoftirqd/207:15:002
272539975,1cyclictest28-21ksoftirqd/210:40:012
272539974,2cyclictest32569-21apt-get07:20:102
272539974,2cyclictest28-21ksoftirqd/207:50:022
272539973,3cyclictest508-21dbus-daemon10:47:192
272539973,3cyclictest508-21dbus-daemon08:37:412
272539972,3cyclictest19982-21systemd-journal07:16:132
272539970,4cyclictest18560-21munin-run11:10:012
272539970,3cyclictest502-21systemd-logind08:08:142
272499974,2cyclictest668-21runrttasks10:05:131
272499974,2cyclictest21894-21sh10:10:171
272499973,3cyclictest8791-21systemctl11:30:251
272499973,3cyclictest508-21dbus-daemon12:09:591
272499973,3cyclictest508-21dbus-daemon10:00:371
272499973,3cyclictest32022-21gdbus08:08:141
272499973,3cyclictest19982-21systemd-journal11:26:071
272499973,3cyclictest19982-21systemd-journal08:26:451
272499973,3cyclictest19982-21systemd-journal08:24:501
272499973,3cyclictest13388-21snmpd07:29:051
272499973,1cyclictest0-21swapper/112:35:351
272499973,1cyclictest0-21swapper/112:15:191
272499972,3cyclictest19982-21systemd-journal11:07:211
272499972,3cyclictest16969-21lldpd11:20:431
272499970,6cyclictest32403-21hddtemp_smartct12:20:151
272499970,3cyclictest19982-21systemd-journal11:14:551
272449975,1cyclictest25913-21uniq10:45:190
272449974,2cyclictest508-21dbus-daemon08:46:200
272449974,2cyclictest11736-21in:imuxsock11:15:000
272449973,3cyclictest5456-21cut09:55:130
272449973,3cyclictest508-21dbus-daemon11:28:380
272449973,3cyclictest508-21dbus-daemon08:33:490
272449973,3cyclictest508-21dbus-daemon07:19:590
272449973,3cyclictest19982-21systemd-journal11:58:290
272449973,3cyclictest19982-21systemd-journal08:50:150
272449973,3cyclictest19982-21systemd-journal08:08:140
272449973,3cyclictest16969-21lldpd09:47:220
272449973,3cyclictest14559-21latency_hist09:35:000
272449973,3cyclictest13388-21snmpd08:57:500
272449973,3cyclictest13388-21snmpd07:43:560
272449973,3cyclictest0-21swapper/012:06:090
272449973,3cyclictest0-21swapper/011:06:030
272449973,1cyclictest0-21swapper/010:16:580
272449972,3cyclictest19982-21systemd-journal11:37:480
272449972,3cyclictest19982-21systemd-journal08:20:040
272449972,3cyclictest16969-21lldpd12:28:350
272449971,3cyclictest25260-21rm11:46:380
272449971,3cyclictest22516-21turbostat09:15:000
272449970,7cyclictest0-21swapper/012:00:580
272449970,4cyclictest15566-21munin-run07:55:010
20041270,0sleep10-21swapper/110:40:081
13243270,0chrt0-21swapper/112:04:531
12613270,0sleep30-21swapper/312:31:253
1087270,0sleep30-21swapper/308:35:003
272579963,2cyclictest668-21runrttasks11:40:163
272579963,2cyclictest5418-21sh11:57:253
272579963,2cyclictest508-21dbus-daemon09:20:073
272579963,2cyclictest32619-21cpu07:20:113
272579963,2cyclictest27455-21sh10:46:073
272579963,2cyclictest26087-21ssh10:15:073
272579963,2cyclictest20981-21systemctl12:10:253
272579963,2cyclictest19982-21systemd-journal11:32:293
272579963,2cyclictest15792-21/usr/sbin/munin07:55:333
272579963,2cyclictest15602-21sshd10:35:193
272579963,2cyclictest13388-21snmpd08:48:503
272579963,2cyclictest10229-21head10:30:173
272579963,2cyclictest0-21swapper/311:07:193
272579963,2cyclictest0-21swapper/309:38:113
272579963,1cyclictest0-21swapper/312:15:203
272579963,1cyclictest0-21swapper/309:16:583
272579962,3cyclictest29887-21basename08:25:173
272579962,3cyclictest26003-21smart_sda08:15:323
272579962,3cyclictest14302-21sed09:00:283
272579962,3cyclictest13388-21snmpd12:25:133
272579962,3cyclictest13388-21snmpd11:13:483
272579962,3cyclictest13388-21snmpd10:58:323
272579962,3cyclictest13388-21snmpd08:11:053
272579962,3cyclictest13388-21snmpd07:30:403
272579962,3cyclictest13388-21snmpd07:13:043
272579962,3cyclictest10426-21grep07:40:343
272579961,3cyclictest3857-21sshd08:40:093
272579961,3cyclictest19982-21systemd-journal12:04:313
272579961,3cyclictest19982-21systemd-journal11:45:413
272579960,5cyclictest31083-21sshd11:20:443
272579960,3cyclictest12117-21cat07:45:223
272579960,2cyclictest5997-21sh10:25:273
272579960,2cyclictest10730-21/usr/sbin/munin10:00:183
272579960,0cyclictest0-21swapper/309:40:313
272539965,1cyclictest28-21ksoftirqd/207:45:022
272539964,2cyclictest52150irq/139-enp4s0-09:40:152
272539964,1cyclictest28-21ksoftirqd/209:10:002
272539963,2cyclictest9477-21readlink11:00:252
272539963,2cyclictest28-21ksoftirqd/211:55:092
272539963,2cyclictest28-21ksoftirqd/210:55:022
272539963,2cyclictest13388-21snmpd07:50:512
272539963,2cyclictest13388-21snmpd07:30:202
272539962,3cyclictest508-21dbus-daemon08:55:302
272539962,3cyclictest32462-21if_enp4s012:20:152
272539962,3cyclictest30073-21awk11:50:252
272539962,3cyclictest16969-21lldpd10:10:302
272539962,3cyclictest13388-21snmpd09:10:372
272539962,3cyclictest13388-21snmpd08:46:282
272539962,3cyclictest13388-21snmpd08:14:092
272539962,3cyclictest11212-21sed12:30:202
272539962,1cyclictest141rcu_preempt07:30:002
272539961,2cyclictest23340-21ssh09:15:142
272539961,2cyclictest15510-21ssh12:05:282
272539960,6cyclictest0-21swapper/211:45:482
272539960,5cyclictest7926-21ssh12:00:112
272499966,0cyclictest0-21swapper/111:46:261
272499963,2cyclictest9565-21ssh09:30:191
272499963,2cyclictest421-21expr07:20:121
272499963,2cyclictest4152-21sh09:25:251
272499963,2cyclictest30372-21cpuspeed07:15:141
272499963,2cyclictest21331-21sh11:43:461
272499963,2cyclictest16268-21awk09:05:301
272499963,2cyclictest13388-21snmpd10:51:001
272499963,2cyclictest13388-21snmpd08:54:041
272499963,2cyclictest11736-21in:imuxsock07:38:461
272499962,3cyclictest6795-21cat08:45:181
272499962,3cyclictest508-21dbus-daemon09:05:001
272499962,3cyclictest3407-21ntp_kernel_pll_11:55:231
272499962,3cyclictest19982-21systemd-journal11:35:391
272499962,3cyclictest13388-21snmpd12:11:071
272499962,3cyclictest13388-21snmpd11:00:391
272499962,3cyclictest13388-21snmpd09:35:051
272499962,3cyclictest13388-21snmpd08:41:481
272499962,3cyclictest13388-21snmpd08:30:041
272499962,3cyclictest13388-21snmpd08:17:461
272499962,3cyclictest13388-21snmpd07:56:441
272499962,3cyclictest13388-21snmpd07:46:141
272499962,3cyclictest1-21systemd09:55:361
272499962,3cyclictest10713-21which12:30:151
272499962,2cyclictest19982-21systemd-journal10:48:301
272499962,1cyclictest0-21swapper/110:38:231
272499962,1cyclictest0-21swapper/110:25:221
272499962,1cyclictest0-21swapper/110:20:041
272499962,1cyclictest0-21swapper/109:21:221
272499962,1cyclictest0-21swapper/109:15:051
272499961,3cyclictest19982-21systemd-journal11:17:291
272499961,3cyclictest19982-21systemd-journal08:03:221
272499961,3cyclictest19982-21systemd-journal07:42:051
272499960,5cyclictest28733-21/usr/sbin/munin11:50:131
272499960,3cyclictest7872-21cron11:00:011
272499960,0cyclictest0-21swapper/110:30:311
272499960,0cyclictest0-21swapper/109:50:081
272499960,0cyclictest0-21swapper/108:55:001
272449963,2cyclictest8032-21smart_sda07:35:290
272449963,2cyclictest490-21wc09:50:230
272449963,2cyclictest3457-21cat10:55:150
272449963,2cyclictest31690-21ssh09:21:090
272449963,2cyclictest28533-21proc_pri07:10:290
272449963,2cyclictest21560-21cut10:10:130
272449963,2cyclictest13388-21snmpd09:01:230
272449963,2cyclictest13388-21snmpd08:40:510
272449963,2cyclictest0-21swapper/012:18:480
272449962,3cyclictest3606-21ls07:25:330
272449962,3cyclictest30528-21cat10:50:130
272449962,3cyclictest30139-21sensors_temp11:50:240
272449962,3cyclictest29576-21ls08:25:150
272449962,3cyclictest20058-21expr12:10:150
272449962,3cyclictest18061-21apt-get08:00:130
272449962,3cyclictest16969-21lldpd08:15:200
272449962,3cyclictest16969-21lldpd08:12:200
272449962,3cyclictest13388-21snmpd12:20:110
272449962,3cyclictest13388-21snmpd10:40:060
272449962,3cyclictest13388-21snmpd10:35:530
272449962,3cyclictest13388-21snmpd10:22:520
272449962,3cyclictest13388-21snmpd09:25:310
272449962,3cyclictest13388-21snmpd09:06:020
272449962,3cyclictest13388-21snmpd07:55:470
272449962,3cyclictest11823-21ntpq10:00:180
272449962,3cyclictest0-21swapper/012:30:180
272449962,3cyclictest0-21swapper/011:00:200
272449962,3cyclictest0-21swapper/009:35:250
272449961,2cyclictest12-21ksoftirqd/007:30:130
272449960,6cyclictest0-21swapper/011:30:040
272449960,6cyclictest0-21swapper/011:16:200
272449960,5cyclictest18135-21expr11:40:120
272449960,4cyclictest13342-21turbostat07:50:010
272449960,3cyclictest29172-21rm11:20:120
272449960,3cyclictest15960-21diskstats12:35:110
272449960,3cyclictest11403-21rm10:30:330
272579952,2cyclictest9272-21sort09:30:173
272579952,2cyclictest30400-21cat07:15:133
272579952,2cyclictest13388-21snmpd08:50:053
272579951,3cyclictest2452-21ls07:25:143
272579951,3cyclictest18032-21/usr/sbin/munin08:00:153
272539952,2cyclictest8718-21expr08:50:152
272539952,2cyclictest25602-21cat08:15:202
272539952,2cyclictest13533-21cat09:00:182
272539951,3cyclictest18256-21cat08:00:132
272499953,1cyclictest11736-21in:imuxsock08:35:071
272499952,2cyclictest27814-21expr07:10:141
272499952,2cyclictest13601-21apt-get07:50:151
272499951,3cyclictest23213-21cat08:10:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional