You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 04:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8.osadl.org (updated Sat Jul 27, 2024 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
225092278252,15sleep20-21swapper/219:09:032
225842276240,20sleep30-21swapper/319:10:003
223642272243,17sleep10-21swapper/119:07:081
222812269220,39sleep00-21swapper/019:06:030
65992610,0sleep10-21swapper/123:23:121
63142610,0sleep30-21swapper/322:24:473
211372580,0sleep30-21swapper/300:12:273
161042580,0sleep00-21swapper/023:48:420
188912560,1sleep10-21swapper/122:12:131
166832560,0sleep00-21swapper/023:10:100
149142550,0sleep10-21swapper/121:10:491
221072170,0sleep00-21swapper/023:52:010
215602170,0sleep221557-21sh23:12:442
268922160,0sleep30-21swapper/300:16:083
66142150,0sleep00-21swapper/021:45:170
172752140,0sleep30-21swapper/323:10:133
161912140,0sleep20-21swapper/221:50:522
22726991210,2cyclictest34-21ksoftirqd/319:20:013
2272699119,1cyclictest34-21ksoftirqd/300:00:013
2271199118,2cyclictest12-21ksoftirqd/000:20:000
21942110,1sleep20-21swapper/221:42:472
146192110,0sleep30-21swapper/322:49:113
2272699108,2cyclictest34-21ksoftirqd/323:00:013
227269996,2cyclictest34-21ksoftirqd/320:15:013
227229996,2cyclictest28-21ksoftirqd/220:25:002
227229990,8cyclictest6602-21munin-run22:25:012
227139994,3cyclictest19982-21systemd-journal23:45:011
227139994,3cyclictest19982-21systemd-journal22:52:461
227119998,1cyclictest12-21ksoftirqd/021:45:000
11684290,0sleep10-21swapper/121:29:111
4733280,0chrt0-21swapper/021:25:020
227269984,3cyclictest19982-21systemd-journal19:34:363
227269983,3cyclictest7661-21lldpd23:45:393
227269983,3cyclictest19982-21systemd-journal21:35:003
227229984,3cyclictest508-21dbus-daemon22:42:452
227229983,3cyclictest19982-21systemd-journal22:35:152
227229982,3cyclictest12089-21munin-run21:10:002
227229980,5cyclictest1-21systemd21:40:012
227139987,1cyclictest28459-1kworker/u9:023:10:431
227139984,3cyclictest19982-21systemd-journal22:05:011
227139984,3cyclictest19982-21systemd-journal19:58:231
227119983,4cyclictest19982-21systemd-journal21:30:590
227119983,3cyclictest19982-21systemd-journal23:00:010
227119983,3cyclictest19982-21systemd-journal21:00:000
8800270,0sleep10-21swapper/100:25:351
227269976,1cyclictest32029-21kworker/3:121:49:403
227269974,2cyclictest7827-21/usr/sbin/munin00:25:173
227269974,2cyclictest11736-21in:imuxsock21:08:533
227269973,3cyclictest7656-21snmpd20:59:083
227269973,3cyclictest20801-21apt-get21:15:103
227269973,3cyclictest0-21swapper/323:36:313
227269973,3cyclictest0-21swapper/323:25:083
227269973,3cyclictest0-21swapper/323:18:383
227269973,3cyclictest0-21swapper/322:43:443
227269973,3cyclictest0-21swapper/322:15:453
227269973,3cyclictest0-21swapper/321:20:133
227269973,3cyclictest0-21swapper/321:12:203
227269973,2cyclictest7661-21lldpd22:13:403
227269973,1cyclictest0-21swapper/323:04:293
227269973,1cyclictest0-21swapper/321:25:043
227269972,3cyclictest7659-21lldpd20:43:083
227269972,3cyclictest19982-21systemd-journal00:38:443
227269970,6cyclictest24194-21expr22:35:153
227229976,1cyclictest28-21ksoftirqd/223:10:012
227229975,1cyclictest6774-21apt-get00:25:102
227229975,1cyclictest0-21swapper/222:15:162
227229974,2cyclictest52150irq/139-enp4s0-19:50:242
227229974,2cyclictest508-21dbus-daemon23:04:022
227229974,2cyclictest24949-21/usr/sbin/munin23:15:142
227229973,3cyclictest7656-21snmpd20:42:042
227229973,3cyclictest6644-21ntpq20:55:182
227229973,3cyclictest30129-21ntpq21:20:162
227229973,3cyclictest25521-21ntpq19:15:012
227229973,3cyclictest0-21swapper/223:47:122
227229973,3cyclictest0-21swapper/221:49:082
227229972,3cyclictest19982-21systemd-journal22:57:572
227139974,2cyclictest508-21dbus-daemon23:57:011
227139974,2cyclictest502-21systemd-logind22:39:501
227139974,2cyclictest11736-21in:imuxsock19:18:451
227139973,3cyclictest19982-21systemd-journal19:55:001
227139973,3cyclictest18165-21ntpq00:10:211
227139973,3cyclictest0-21swapper/122:19:261
227139973,3cyclictest0-21swapper/121:53:561
227139973,1cyclictest0-21swapper/121:41:051
227139970,7cyclictest12091-21diskmemload22:57:001
227139970,4cyclictest15485-21basename20:05:011
227139970,3cyclictest19982-21systemd-journal23:54:061
227119976,1cyclictest12-21ksoftirqd/022:00:010
227119976,1cyclictest12-21ksoftirqd/022:00:010
227119975,1cyclictest12-21ksoftirqd/020:20:010
227119973,3cyclictest7656-21snmpd22:41:080
227119973,3cyclictest7656-21snmpd20:21:160
227119973,3cyclictest7656-21snmpd19:56:420
227119973,3cyclictest7656-21snmpd19:38:590
227119973,3cyclictest508-21dbus-daemon22:20:140
227119973,3cyclictest508-21dbus-daemon21:09:180
227119973,3cyclictest26979-21ntpq19:15:160
227119973,3cyclictest1625-21diskstats23:20:160
227119973,2cyclictest19982-21systemd-journal20:50:310
227119973,1cyclictest0-21swapper/023:39:040
227119973,1cyclictest0-21swapper/023:09:490
227119973,1cyclictest0-21swapper/022:07:120
227119973,1cyclictest0-21swapper/021:37:460
227119973,1cyclictest0-21swapper/021:18:240
227119972,3cyclictest7661-21lldpd22:14:550
227119972,3cyclictest19982-21systemd-journal22:18:340
227119971,3cyclictest4493-21grep21:25:000
227119971,3cyclictest20338-21awk21:15:000
227119971,2cyclictest16595-21munin-run00:10:010
227119970,1cyclictest0-21swapper/020:05:110
11969270,0sleep00-21swapper/022:27:330
227269965,1cyclictest34-21ksoftirqd/319:30:013
227269964,1cyclictest5248-21grep19:40:183
227269964,1cyclictest166242sleep320:05:183
227269963,2cyclictest9441-21sh22:25:353
227269963,2cyclictest9407-21cat19:50:133
227269963,2cyclictest7661-21lldpd21:56:213
227269963,2cyclictest7661-21lldpd21:56:213
227269963,2cyclictest508-21dbus-daemon21:00:073
227269963,2cyclictest32700-21sh00:20:183
227269963,2cyclictest31245-21expr22:00:143
227269963,2cyclictest2560-21fschecks_count00:00:153
227269963,2cyclictest20794-21cat20:15:153
227269963,2cyclictest16271-21ssh21:50:553
227269963,2cyclictest13723-21sh22:09:373
227269963,2cyclictest11736-21in:imuxsock19:37:313
227269963,1cyclictest0-21swapper/322:50:263
227269962,3cyclictest9818-21cut00:05:133
227269962,3cyclictest7656-21snmpd23:31:173
227269962,3cyclictest7656-21snmpd23:21:093
227269962,3cyclictest7656-21snmpd19:56:063
227269962,3cyclictest7656-21snmpd19:10:223
227269962,3cyclictest29259-21expr20:35:113
227269962,3cyclictest19116-21gdbus20:05:013
227269962,3cyclictest18414-21cat23:50:113
227269962,3cyclictest0-21swapper/300:30:053
227269961,4cyclictest29741-11brltty22:31:053
227269961,2cyclictest8495-21ssh23:05:113
227269960,3cyclictest4686-21rm21:44:433
227269960,3cyclictest28729-21munin-run20:35:003
227269960,3cyclictest24958-21ssh21:36:423
227269960,3cyclictest1497-21rm23:40:013
227229964,1cyclictest28-21ksoftirqd/223:40:012
227229964,1cyclictest24946-21grep00:15:152
227229963,2cyclictest8677-21seq22:45:152
227229963,2cyclictest508-21dbus-daemon22:05:062
227229963,2cyclictest508-21dbus-daemon20:54:172
227229963,2cyclictest31589-21apt-get00:20:092
227229963,2cyclictest28-21ksoftirqd/223:29:562
227229963,2cyclictest26516-21turbostat.cron20:34:552
227229963,2cyclictest21661-21wc21:15:172
227229963,2cyclictest19894-21sh21:34:092
227229963,2cyclictest16972-21ssh22:10:432
227229963,2cyclictest16963-21ssh00:10:092
227229963,1cyclictest0-21swapper/221:13:142
227229962,3cyclictest7656-21snmpd23:30:392
227229962,3cyclictest7656-21snmpd20:14:282
227229962,3cyclictest7656-21snmpd20:01:442
227229962,3cyclictest7656-21snmpd19:58:122
227229962,3cyclictest7656-21snmpd19:25:322
227229962,3cyclictest7280-21df_inode19:45:132
227229962,3cyclictest7162-21seq21:25:422
227229962,3cyclictest508-21dbus-daemon00:00:012
227229962,3cyclictest28276-21apt-get19:20:122
227229962,3cyclictest1963-21cat23:20:172
227229962,3cyclictest17147-21netstat22:50:152
227229962,3cyclictest10518-21sed00:05:192
227229962,3cyclictest0-21swapper/222:31:012
227229962,2cyclictest19982-21systemd-journal19:12:552
227229962,1cyclictest0-21swapper/223:55:212
227229962,1cyclictest0-21swapper/223:35:032
227229962,1cyclictest0-21swapper/222:25:122
227229962,1cyclictest0-21swapper/222:02:152
227229961,3cyclictest19982-21systemd-journal19:43:222
227229960,6cyclictest0-21swapper/221:55:182
227229960,6cyclictest0-21swapper/221:55:182
227229960,6cyclictest0-21swapper/200:30:132
227229960,0cyclictest0-21swapper/223:50:102
227229960,0cyclictest0-21swapper/200:35:152
227139966,0cyclictest0-21swapper/122:25:151
227139966,0cyclictest0-21swapper/120:30:151
227139964,1cyclictest7649-21munin-run21:00:001
227139963,2cyclictest9929-21ssh21:47:231
227139963,2cyclictest7656-21snmpd19:31:381
227139963,2cyclictest508-21dbus-daemon00:00:211
227139963,2cyclictest24542-21rm00:37:441
227139963,2cyclictest19982-21systemd-journal23:45:201
227139963,2cyclictest16956-21mailstats22:30:211
227139963,2cyclictest11736-21in:imuxsock20:13:241
227139963,2cyclictest11651-21sh23:25:441
227139963,2cyclictest11094-21sh22:07:351
227139963,2cyclictest10096-21taskset21:05:031
227139963,2cyclictest0-21swapper/100:05:211
227139963,1cyclictest0-21swapper/122:45:421
227139962,3cyclictest7656-21snmpd20:51:491
227139962,3cyclictest7656-21snmpd19:45:111
227139962,3cyclictest5367-21cat19:40:191
227139962,3cyclictest508-21dbus-daemon20:20:011
227139962,3cyclictest508-21dbus-daemon00:35:001
227139962,3cyclictest29827-21cat20:35:151
227139962,3cyclictest25348-21expr23:15:121
227139962,3cyclictest24144-21ntpq19:10:181
227139962,3cyclictest22295-21threads21:15:221
227139962,3cyclictest1149-21load23:00:161
227139962,1cyclictest0-21swapper/123:38:511
227139962,1cyclictest0-21swapper/123:33:371
227139962,1cyclictest0-21swapper/121:56:401
227139962,1cyclictest0-21swapper/121:56:391
227139962,1cyclictest0-21swapper/121:35:051
227139962,1cyclictest0-21swapper/121:30:171
227139962,1cyclictest0-21swapper/100:17:201
227139961,3cyclictest19982-21systemd-journal23:06:061
227139961,3cyclictest19982-21systemd-journal20:25:051
227139961,2cyclictest32409-21ntp_states22:20:201
227139960,6cyclictest0-21swapper/122:41:221
227119963,2cyclictest4285-21sh23:40:570
227119963,2cyclictest4251-21ssh22:03:390
227119963,2cyclictest26049-21ntp_kernel_err23:15:180
227119963,2cyclictest23078-21ssh00:36:010
227119963,2cyclictest18524-21hddtemp_smartct20:10:140
227119963,1cyclictest0-21swapper/023:02:190
227119962,3cyclictest7972-21awk22:45:100
227119962,3cyclictest7661-21lldpd20:26:090
227119962,3cyclictest7656-21snmpd21:00:410
227119962,3cyclictest7656-21snmpd20:45:550
227119962,3cyclictest7656-21snmpd19:50:060
227119962,3cyclictest7656-21snmpd19:48:420
227119962,3cyclictest7656-21snmpd19:25:100
227119962,3cyclictest7656-21snmpd00:32:530
227119962,3cyclictest508-21dbus-daemon22:51:560
227119962,3cyclictest32460-21wc00:20:160
227119962,3cyclictest30321-21cat20:35:200
227119962,3cyclictest2933-21cat00:00:180
227119962,3cyclictest27391-21cat20:30:140
227119962,3cyclictest1-21systemd20:04:130
227119962,1cyclictest0-21swapper/000:26:050
227119962,1cyclictest0-21swapper/000:11:580
227119961,4cyclictest12439-21ssh23:26:200
227119961,3cyclictest7656-21snmpd23:31:570
227119961,3cyclictest502-21systemd-logind21:52:170
227119961,3cyclictest29741-11brltty19:10:240
227119961,3cyclictest19982-21systemd-journal19:21:320
227119961,2cyclictest30278-21ssh23:57:030
227119960,6cyclictest0-21swapper/022:31:420
227119960,4cyclictest19982-21systemd-journal22:35:400
227269952,2cyclictest7656-21snmpd19:20:143
227269952,2cyclictest6872-21/usr/sbin/munin19:45:143
227269952,2cyclictest22738-21sh20:20:113
227269951,3cyclictest3857-21cat20:50:143
227269951,3cyclictest24775-21expr20:25:123
227269950,3cyclictest19982-21systemd-journal20:45:313
227229952,2cyclictest9279-21cat21:00:242
227229952,2cyclictest2664-21cat19:35:132
227229952,2cyclictest25250-21irqstats20:25:162
227229952,2cyclictest20497-21cat20:15:132
227229952,2cyclictest1874-21taskset19:31:002
227229952,2cyclictest1156-21tr20:45:112
227229950,4cyclictest28953-21/usr/sbin/munin20:35:202
227229950,3cyclictest19982-21systemd-journal20:05:022
227139952,2cyclictest8324-21entropy21:00:151
227139952,2cyclictest31793-21cat19:25:231
227139952,2cyclictest31512-21expr20:40:131
227139952,2cyclictest3124-21if_err_enp4s019:35:151
227139952,2cyclictest28946-21cat19:20:161
227139952,2cyclictest28901-21ssh21:20:031
227139952,2cyclictest15727-21/usr/sbin/munin20:05:111
227139952,2cyclictest1342-21expr20:45:111
227139951,3cyclictest22809-21tune2fs20:20:131
227139951,1cyclictest0-21swapper/100:20:061
227119951,3cyclictest32697-21apt-get19:30:100
227119950,4cyclictest31185-21/usr/sbin/munin20:40:120
227119950,3cyclictest19982-21systemd-journal19:40:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional