You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 09:50
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8.osadl.org (updated Fri May 09, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
208462305271,21sleep20-21swapper/219:09:032
206362279246,20sleep30-21swapper/319:06:233
207662274239,23sleep10-21swapper/119:08:011
208342258224,22sleep00-21swapper/019:08:520
42992620,0sleep30-21swapper/323:08:483
129662590,1sleep20-21swapper/200:34:242
296842580,0sleep20-21swapper/221:49:042
39622390,0sleep00-21swapper/021:29:470
17452200,0sleep30-21swapper/321:51:523
116872190,0sleep10-21swapper/122:00:161
141292160,0sleep00-21swapper/000:05:190
293092150,0sleep00-21swapper/019:25:170
171272150,0sleep1211rcuc/121:40:021
123282150,0sleep30-21swapper/321:10:323
58412130,0sleep20-21swapper/219:45:172
21045991310,2cyclictest28-21ksoftirqd/221:40:012
2103799131,3cyclictest19982-21systemd-journal00:05:000
2103799121,3cyclictest19982-21systemd-journal22:13:140
2103799120,3cyclictest11968-21df22:00:150
2104199111,4cyclictest6018-21df21:00:141
2103799113,7cyclictest0-21swapper/021:39:150
2103799112,2cyclictest24548-21munin-run22:35:000
2103799111,2cyclictest19982-21systemd-journal20:05:100
2104999106,3cyclictest27371-21df20:35:113
2104199103,5cyclictest5149-21apt-get21:55:111
2103799103,2cyclictest889-21rm22:15:560
2103799103,2cyclictest508-21dbus-daemon20:59:220
2103799103,2cyclictest1340-21kworker/u8:223:12:530
2103799103,1cyclictest29555-21sh22:38:340
2103799102,2cyclictest863-21rm21:50:580
2103799102,2cyclictest26937-21basename19:20:130
2103799102,2cyclictest21678-21expr20:20:170
2103799102,1cyclictest23442-21grep19:15:010
2103799102,1cyclictest0-21swapper/023:41:380
2103799101,3cyclictest2714-21grep20:50:180
2103799101,2cyclictest19081-21kworker/u8:021:20:010
2103799100,9cyclictest30173-21expr00:20:130
2103799100,3cyclictest6265-40systemctl00:00:010
2103799100,3cyclictest31286-21latency_hist20:45:000
2103799100,3cyclictest19982-21systemd-journal23:21:410
2103799100,3cyclictest19982-21systemd-journal20:39:540
2103799100,3cyclictest19476-21/usr/sbin/munin22:55:180
2103799100,3cyclictest18375-21run-parts00:40:000
2103799100,2cyclictest19982-21systemd-journal23:50:010
2103799100,2cyclictest19982-21systemd-journal21:25:010
32018290,0sleep20-21swapper/200:21:002
210499995,3cyclictest19982-21systemd-journal21:23:343
210459995,3cyclictest11736-21in:imuxsock22:00:192
210419995,3cyclictest30654-21snmpd21:51:061
210419990,6cyclictest5278-21ssh22:45:051
210419990,6cyclictest26822-21ssh00:16:571
210419990,6cyclictest26709-21ssh22:35:271
210379993,1cyclictest27043-21sh21:46:190
210379992,3cyclictest6180-21ls19:45:200
210379992,3cyclictest30654-21snmpd20:26:150
210379992,2cyclictest8962-21cut00:30:130
210379992,2cyclictest671-21rm23:05:200
210379992,2cyclictest3851-21wc19:40:170
210379992,2cyclictest30654-21snmpd23:52:300
210379992,2cyclictest30654-21snmpd23:30:190
210379992,2cyclictest30654-21snmpd23:04:560
210379992,2cyclictest30654-21snmpd20:33:030
210379992,2cyclictest30654-21snmpd20:10:150
210379992,2cyclictest30654-21snmpd19:54:560
210379992,2cyclictest17136-21sh22:29:190
210379992,2cyclictest11789-21ssh23:39:300
210379992,1cyclictest9771-21cut19:55:130
210379992,1cyclictest5408-21ssh22:45:140
210379992,1cyclictest4075-21gltestperf00:25:140
210379992,1cyclictest12402-21fschecks_count22:50:150
210379992,1cyclictest0-21swapper/023:17:220
210379991,2cyclictest502-21systemd-logind23:25:220
210379990,8cyclictest8520-21df_inode21:05:130
210379990,4cyclictest26031-21rm00:16:010
210379990,3cyclictest7317-21sh21:55:550
210379990,3cyclictest4866-21cstates21:30:140
210379990,3cyclictest31518-21wc22:40:140
210379990,3cyclictest22910-21ssh00:14:210
210379990,3cyclictest19625-21grep20:15:200
210379990,3cyclictest1-21systemd20:45:360
210379990,2cyclictest19982-21systemd-journal21:04:480
210379990,2cyclictest19982-21systemd-journal20:00:420
210499986,1cyclictest31537-21/usr/sbin/munin21:25:213
210499984,3cyclictest19982-21systemd-journal22:18:403
210459986,1cyclictest28-21ksoftirqd/221:55:022
210459986,1cyclictest28-21ksoftirqd/221:45:012
210459986,1cyclictest12666-21ps20:00:172
210459983,3cyclictest19982-21systemd-journal23:22:302
210459983,3cyclictest19982-21systemd-journal23:15:422
210459983,3cyclictest19982-21systemd-journal22:55:002
210419982,1cyclictest29741-11brltty19:18:011
210419981,3cyclictest23207-21ssh23:23:081
210419981,2cyclictest27210-21kworker/u8:222:26:441
210419981,1cyclictest0-21swapper/120:58:321
210419980,5cyclictest3387-21ssh00:25:051
210419980,5cyclictest32425-21taskset23:55:111
210419980,5cyclictest31696-21ssh00:20:361
210419980,5cyclictest3159-21idleruntime-cro20:55:001
210419980,5cyclictest29284-21sh23:52:071
210419980,5cyclictest26984-21ssh22:11:111
210419980,5cyclictest24195-21ssh23:48:371
210419980,5cyclictest23879-21sh21:20:091
210419980,5cyclictest23344-21ssh22:33:481
210419980,5cyclictest20685-21ssh22:06:241
210419980,5cyclictest19838-21ssh21:40:351
210419980,5cyclictest1721-21ssh22:41:301
210419980,5cyclictest1646-21ssh21:27:211
210419980,5cyclictest13143-21ssh00:34:381
210419980,5cyclictest11378-21ssh00:03:591
210379982,2cyclictest2302-21chrt19:38:270
210379982,1cyclictest0-21swapper/022:05:140
210379980,3cyclictest5521-21expr22:20:110
210379980,3cyclictest31016-21expr19:30:130
210379980,3cyclictest21617-21df19:10:130
210379980,3cyclictest19608-21ssh21:40:220
210379980,3cyclictest11624-21/usr/sbin/munin21:10:190
210499974,2cyclictest670-21grep23:55:153
210499974,2cyclictest19982-21systemd-journal21:37:453
210499973,3cyclictest19982-21systemd-journal00:26:203
210499973,3cyclictest17312-21apt21:15:123
210499973,3cyclictest0-21swapper/323:41:333
210499973,3cyclictest0-21swapper/300:19:223
210499973,1cyclictest0-21swapper/323:28:383
210499973,1cyclictest0-21swapper/321:57:563
210499973,1cyclictest0-21swapper/300:05:023
210499972,3cyclictest7661-21lldpd22:31:183
210499972,3cyclictest19982-21systemd-journal19:40:083
210499971,2cyclictest6196-21ssh21:30:203
210459976,1cyclictest28-21ksoftirqd/200:20:002
210459975,2cyclictest28-21ksoftirqd/220:35:002
210459975,1cyclictest30746-21grep22:15:022
210459975,1cyclictest28-21ksoftirqd/221:25:002
210459973,3cyclictest7661-21lldpd22:53:182
210459973,3cyclictest30654-21snmpd23:25:442
210459973,3cyclictest24992-21awk22:35:142
210459973,3cyclictest19982-21systemd-journal21:05:012
210459973,3cyclictest19982-21systemd-journal20:59:482
210459973,3cyclictest19116-21gdbus20:50:012
210459973,1cyclictest0-21swapper/221:32:272
210459973,1cyclictest0-21swapper/200:00:062
210459972,3cyclictest19982-21systemd-journal20:39:472
210459972,3cyclictest19982-21systemd-journal19:34:172
210419976,1cyclictest211rcuc/122:55:341
210419976,1cyclictest211rcuc/121:34:501
210419975,1cyclictest29217-21apt-get20:40:111
210419973,3cyclictest30654-21snmpd19:45:581
210419973,3cyclictest29741-11brltty19:32:371
210419973,3cyclictest11593-21ssh21:10:181
210419973,1cyclictest0-21swapper/100:05:401
210419972,4cyclictest3404-21uname19:40:141
210419972,4cyclictest23269-21wc19:15:001
210419972,4cyclictest1-21systemd21:46:081
210419972,3cyclictest19982-21systemd-journal20:50:001
210419972,3cyclictest19982-21systemd-journal00:11:191
210419971,6cyclictest0-21swapper/123:35:141
210419971,3cyclictest13494-21apt20:05:001
210419971,2cyclictest4038-21sh22:19:061
210419971,2cyclictest1881-21wc19:35:211
210419971,2cyclictest18709-21ssh23:44:551
210419971,2cyclictest10554-21ssh23:13:401
210419970,7cyclictest0-21swapper/120:25:141
210419970,5cyclictest7365-21ssh22:20:501
210419970,5cyclictest28892-21ssh23:02:291
210419970,3cyclictest31897-21grep23:05:151
210419970,3cyclictest26404-21/usr/sbin/munin23:25:191
210419970,3cyclictest19425-21ssh21:15:431
210419970,3cyclictest14580-21ssh23:16:021
210419970,3cyclictest12369-21df_inode22:50:151
210419970,2cyclictest22-21ksoftirqd/123:30:011
16009270,0sleep10-21swapper/100:36:231
8516260,0sleep20-21swapper/223:36:062
210499964,1cyclictest25336-21df_inode22:35:163
210499963,2cyclictest8116-21/usr/sbin/munin21:05:163
210499963,2cyclictest7966-21cat19:50:153
210499963,2cyclictest30654-21snmpd22:27:493
210499963,2cyclictest30654-21snmpd20:09:333
210499963,2cyclictest30654-21snmpd00:11:053
210499963,2cyclictest28699-21wc19:25:143
210499963,2cyclictest27480-21ssh22:11:423
210499963,2cyclictest25729-21if_enp4s023:00:153
210499963,2cyclictest2293-21rm23:31:263
210499963,2cyclictest11736-21in:imuxsock23:55:003
210499963,1cyclictest0-21swapper/323:21:203
210499962,3cyclictest7048-21cat23:35:173
210499962,3cyclictest5164-21ntpd22:55:173
210499962,3cyclictest508-21dbus-daemon20:15:343
210499962,3cyclictest508-21dbus-daemon00:00:153
210499962,3cyclictest30654-21snmpd22:20:163
210499962,3cyclictest30654-21snmpd22:07:443
210499962,3cyclictest30654-21snmpd21:40:023
210499962,3cyclictest30654-21snmpd20:47:513
210499962,3cyclictest30654-21snmpd20:10:123
210499962,3cyclictest30654-21snmpd20:03:323
210499962,3cyclictest30654-21snmpd19:10:153
210499962,3cyclictest30654-21snmpd00:36:583
210499962,3cyclictest30654-21snmpd00:31:583
210499962,3cyclictest30180-21cat20:40:163
210499962,3cyclictest26973-21cat19:20:123
210499962,3cyclictest19116-21gdbus23:45:013
210499962,3cyclictest13200-21which22:50:193
210499962,3cyclictest12276-21if_err_enp4s022:00:173
210499962,3cyclictest10388-21which19:55:183
210499962,3cyclictest0-21swapper/321:46:443
210499962,2cyclictest19982-21systemd-journal20:55:133
210499962,2cyclictest19982-21systemd-journal19:19:153
210499962,1cyclictest0-21swapper/322:46:353
210499962,1cyclictest0-21swapper/300:22:203
210499961,3cyclictest19982-21systemd-journal23:10:233
210499961,3cyclictest19982-21systemd-journal21:01:293
210499961,3cyclictest19982-21systemd-journal20:25:103
210499961,2cyclictest3273-21ssh22:43:113
210499960,6cyclictest0-21swapper/323:17:143
210499960,6cyclictest0-21swapper/319:30:153
210499960,5cyclictest5521-21hddtemp_smartct19:45:163
210499960,5cyclictest20900-21gltestperf20:20:113
210499960,5cyclictest1175-21/usr/sbin/munin20:50:133
210459963,2cyclictest835-21sh23:55:162
210459963,2cyclictest7501-21sh21:56:072
210459963,2cyclictest6883-21seq23:10:182
210459963,2cyclictest31901-21sh23:54:482
210459963,2cyclictest3019-21ssh22:18:012
210459963,2cyclictest23063-21sort20:25:152
210459963,2cyclictest1691-21sh23:06:052
210459963,2cyclictest14531-21ssh23:40:302
210459963,2cyclictest10455-21ssh21:10:072
210459962,3cyclictest32123-21if_err_enp4s022:40:162
210459962,3cyclictest30654-21snmpd23:32:462
210459962,3cyclictest30654-21snmpd22:32:392
210459962,3cyclictest30654-21snmpd20:20:142
210459962,3cyclictest30654-21snmpd20:06:292
210459962,3cyclictest30654-21snmpd19:40:142
210459962,3cyclictest30654-21snmpd19:22:032
210459962,3cyclictest30654-21snmpd19:15:452
210459962,3cyclictest30654-21snmpd19:12:392
210459962,3cyclictest16153-21ls20:10:142
210459962,3cyclictest1-21systemd20:50:132
210459962,3cyclictest0-21swapper/221:18:012
210459962,3cyclictest0-21swapper/200:08:142
210459962,1cyclictest0-21swapper/221:26:442
210459962,1cyclictest0-21swapper/200:35:082
210459962,1cyclictest0-21swapper/200:11:042
210459961,3cyclictest7661-21lldpd22:45:332
210459961,3cyclictest19982-21systemd-journal20:43:322
210459961,3cyclictest19982-21systemd-journal20:17:142
210459960,3cyclictest18453-21cstates22:05:162
210419965,1cyclictest22-21ksoftirqd/119:55:011
210419965,1cyclictest211rcuc/120:16:521
210419963,2cyclictest30654-21snmpd20:11:081
210419963,1cyclictest19982-21systemd-journal19:52:381
210419962,3cyclictest8087-21gdbus21:05:001
210419962,3cyclictest508-21dbus-daemon20:30:361
210419962,3cyclictest30654-21snmpd20:38:361
210419962,3cyclictest29261-21grep19:25:181
210419962,3cyclictest28425-21cut19:25:011
210419962,3cyclictest14426-21cat20:05:161
210419961,2cyclictest22300-21seq20:24:561
210499952,2cyclictest575-21tr19:35:153
210499952,2cyclictest24998-21cat20:30:133
210459952,2cyclictest9906-21head19:55:142
210459952,2cyclictest7394-21cat19:50:112
210459952,2cyclictest30654-21snmpd21:00:012
210459952,2cyclictest30654-21snmpd19:25:042
210459952,2cyclictest24856-21rm23:00:022
210459952,2cyclictest19320-21gdbus23:45:032
210459952,2cyclictest11642-21ssh22:25:082
210459951,3cyclictest1088-21hddtemp_smartct19:35:172
210459951,2cyclictest19982-21systemd-journal00:24:592
210459950,0cyclictest0-21swapper/222:20:052
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional