You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-26 - 18:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackaslot8.osadl.org (updated Tue Aug 26, 2025 12:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
307442322290,20sleep20-21swapper/207:08:272
308492307279,18sleep10-21swapper/107:09:461
308142286229,43sleep30-21swapper/307:09:193
308012262228,22sleep00-21swapper/007:09:080
240182600,0sleep30-21swapper/309:37:583
219842600,0sleep00-21swapper/011:24:430
325962570,0sleep10-21swapper/111:32:291
90402530,0sleep30-21swapper/307:30:133
242732530,0sleep10-21swapper/110:32:221
121432190,0sleep3331rcuc/311:15:223
28642170,0sleep228-21ksoftirqd/209:47:152
28642170,0sleep228-21ksoftirqd/209:47:152
56822160,0sleep10-21swapper/109:22:461
40952150,0sleep00-21swapper/011:35:210
229622150,0sleep30-21swapper/309:10:203
134182150,0sleep10-21swapper/112:07:081
125102140,0sleep00-21swapper/009:28:430
3101799129,2cyclictest34-21ksoftirqd/307:45:013
310109992,4cyclictest29522-21munin-run10:10:012
310019994,3cyclictest19982-21systemd-journal10:55:000
310019994,3cyclictest19982-21systemd-journal07:32:200
1736290,2sleep1221ksoftirqd/107:15:011
310179985,2cyclictest34-21ksoftirqd/312:20:013
310179985,2cyclictest13490-21ssh11:42:393
310179984,3cyclictest508-21dbus-daemon07:37:203
310179983,3cyclictest7661-21lldpd10:56:463
310179983,3cyclictest19982-21systemd-journal08:29:543
310109984,3cyclictest508-21dbus-daemon11:25:332
310109984,3cyclictest19982-21systemd-journal08:01:312
310109983,3cyclictest19982-21systemd-journal12:20:112
310109983,3cyclictest19982-21systemd-journal11:50:592
310109983,3cyclictest19982-21systemd-journal08:30:182
310109981,4cyclictest3396-21cat12:00:012
310069984,3cyclictest508-21dbus-daemon10:00:511
310069984,3cyclictest26143-21ntpq11:00:211
310069982,3cyclictest24422-21munin-run08:05:011
310019983,3cyclictest7661-21lldpd10:44:250
5986270,1sleep10-21swapper/108:35:131
310179976,1cyclictest8943-21ssh12:04:183
310179974,2cyclictest508-21dbus-daemon07:30:013
310179974,2cyclictest26957-21/usr/sbin/munin10:35:203
310179973,3cyclictest508-21dbus-daemon07:46:303
310179973,3cyclictest30654-21snmpd09:06:573
310179973,3cyclictest19982-21systemd-journal11:38:033
310179973,3cyclictest19982-21systemd-journal10:05:333
310179973,3cyclictest0-21swapper/312:07:463
310179972,3cyclictest7661-21lldpd07:53:553
310179972,3cyclictest7661-21lldpd07:53:553
310179972,3cyclictest19982-21systemd-journal10:34:293
310179972,3cyclictest19982-21systemd-journal07:20:083
310179972,2cyclictest32636-1kworker/u9:011:30:333
310179971,4cyclictest19982-21systemd-journal09:19:333
310179971,3cyclictest24563-21rm11:25:313
310109974,2cyclictest30654-21snmpd10:14:322
310109973,3cyclictest24691-21apt-get08:05:152
310109973,3cyclictest19982-21systemd-journal12:30:002
310109973,3cyclictest19982-21systemd-journal08:15:142
310109973,3cyclictest19982-21systemd-journal07:39:292
310109973,3cyclictest19982-21systemd-journal07:31:292
310109973,3cyclictest15324-21diskstats10:25:142
310109973,1cyclictest0-21swapper/212:02:582
310109972,3cyclictest19982-21systemd-journal12:15:332
310109972,3cyclictest19982-21systemd-journal10:15:002
310109972,3cyclictest19982-21systemd-journal08:41:102
310109971,2cyclictest19982-21systemd-journal11:20:322
310069976,1cyclictest22-21ksoftirqd/110:00:011
310069974,2cyclictest508-21dbus-daemon12:33:571
310069974,2cyclictest30654-21snmpd10:38:061
310069974,2cyclictest29741-11brltty11:42:391
310069974,2cyclictest22628-21munin-run12:15:001
310069973,3cyclictest30654-21snmpd08:57:241
310069973,3cyclictest30654-21snmpd07:59:521
310069973,3cyclictest29741-11brltty11:55:111
310069973,3cyclictest19982-21systemd-journal11:18:271
310069973,3cyclictest19982-21systemd-journal09:07:281
310069973,3cyclictest19982-21systemd-journal08:07:221
310069973,3cyclictest0-21swapper/112:19:291
310069973,1cyclictest17678-21tail12:35:141
310069973,1cyclictest0-21swapper/110:22:591
310069972,4cyclictest29501-21timerandwakeup09:15:201
310069971,4cyclictest19982-21systemd-journal09:02:011
310069971,3cyclictest19948-21apt07:55:021
310069971,3cyclictest19948-21apt07:55:011
310019974,2cyclictest508-21dbus-daemon09:18:170
310019974,2cyclictest19982-21systemd-journal07:15:010
310019973,3cyclictest508-21dbus-daemon12:30:000
310019973,3cyclictest508-21dbus-daemon08:20:000
310019973,3cyclictest30806-21memory10:10:170
310019973,3cyclictest26109-21ntpq11:00:210
310019973,3cyclictest23147-21cat08:00:160
310019973,3cyclictest21424-21diskstats10:30:210
310019973,3cyclictest19982-21systemd-journal08:50:000
310019973,3cyclictest11736-21in:imuxsock12:32:170
310019973,3cyclictest11736-21in:imuxsock10:35:050
310019973,3cyclictest0-21swapper/012:10:450
310019972,3cyclictest19982-21systemd-journal07:44:000
17363270,0sleep00-21swapper/011:45:220
17363270,0sleep00-21swapper/011:45:210
16112270,6sleep021677-21diskmemload09:58:510
8025260,0sleep30-21swapper/309:25:083
310179966,0cyclictest0-21swapper/311:01:243
310179963,2cyclictest508-21dbus-daemon09:44:103
310179963,2cyclictest31406-21apt-get08:20:133
310179963,2cyclictest31361-21awk11:55:223
310179963,2cyclictest31260-21sh11:05:123
310179963,2cyclictest30657-21ssh10:10:173
310179963,2cyclictest29150-21apt-get08:15:163
310179963,2cyclictest22849-21ssh11:50:123
310179963,2cyclictest22734-21expr08:00:143
310179963,2cyclictest17659-21ls12:10:223
310179963,2cyclictest17575-21cut09:00:133
310179963,2cyclictest14933-21ssh10:50:513
310179963,2cyclictest11736-21in:imuxsock12:21:533
310179963,2cyclictest0-21swapper/310:20:163
310179963,2cyclictest0-21swapper/309:30:213
310179962,3cyclictest6202-21tune2fs09:50:143
310179962,3cyclictest4036-21which08:30:153
310179962,3cyclictest32709-21if_enp4s009:45:153
310179962,3cyclictest32709-21if_enp4s009:45:153
310179962,3cyclictest30654-21snmpd12:30:153
310179962,3cyclictest30654-21snmpd12:25:033
310179962,3cyclictest30654-21snmpd10:41:393
310179962,3cyclictest30654-21snmpd10:16:513
310179962,3cyclictest30654-21snmpd08:46:303
310179962,3cyclictest30654-21snmpd07:11:453
310179962,3cyclictest19982-21systemd-journal08:45:003
310179962,3cyclictest16088-21sh11:45:013
310179962,3cyclictest16088-21sh11:45:013
310179962,3cyclictest15143-21cat10:25:133
310179962,3cyclictest13348-21sed09:55:253
310179962,2cyclictest7661-21lldpd09:20:423
310179962,1cyclictest0-21swapper/310:46:083
310179961,3cyclictest19982-21systemd-journal07:55:013
310179961,3cyclictest19982-21systemd-journal07:15:053
310179961,2cyclictest28186-21kworker/u8:111:20:143
310179961,1cyclictest0-21swapper/311:10:173
310179960,4cyclictest19982-21systemd-journal08:55:463
310179960,0cyclictest0-21swapper/310:00:033
310109966,0cyclictest0-21swapper/209:15:272
310109963,2cyclictest8052-21/usr/sbin/munin09:25:202
310109963,2cyclictest6141-21cron07:25:012
310109963,2cyclictest30654-21snmpd12:30:572
310109963,2cyclictest30654-21snmpd08:59:062
310109963,2cyclictest29779-21sh10:36:432
310109963,2cyclictest21677-21diskmemload10:02:352
310109963,2cyclictest16897-21sh10:53:112
310109962,3cyclictest31758-21expr07:10:142
310109962,3cyclictest30654-21snmpd12:36:332
310109962,3cyclictest30654-21snmpd12:10:482
310109962,3cyclictest30654-21snmpd10:56:482
310109962,3cyclictest30654-21snmpd07:40:462
310109962,3cyclictest29206-21cat11:30:142
310109962,3cyclictest27602-21cat08:10:172
310109962,3cyclictest21723-21cat10:30:212
310109962,3cyclictest19982-21systemd-journal07:58:362
310109962,3cyclictest1901-21sshd07:15:072
310109962,3cyclictest16520-21grep11:45:172
310109962,3cyclictest16520-21grep11:45:172
310109962,3cyclictest15655-21cat07:45:132
310109962,3cyclictest15036-21date09:30:212
310109962,3cyclictest10648-21ntp_kernel_pll_11:40:172
310109962,1cyclictest0-21swapper/212:05:372
310109962,1cyclictest0-21swapper/211:36:462
310109962,1cyclictest0-21swapper/211:05:072
310109962,1cyclictest0-21swapper/211:03:292
310109962,1cyclictest0-21swapper/210:40:202
310109962,1cyclictest0-21swapper/209:35:342
310109962,1cyclictest0-21swapper/209:21:282
310109961,3cyclictest19982-21systemd-journal10:22:582
310109961,3cyclictest19982-21systemd-journal08:23:382
310109961,2cyclictest28-21ksoftirqd/209:50:142
310109960,6cyclictest0-21swapper/211:16:212
310109960,6cyclictest0-21swapper/211:10:202
310109960,5cyclictest12384-21gltestperf09:55:192
310109960,4cyclictest19982-21systemd-journal10:48:252
310109960,3cyclictest22512-21/usr/sbin/munin09:10:162
310069963,2cyclictest8962-21sh10:45:591
310069963,2cyclictest6598-21ssh11:37:251
310069963,2cyclictest4941-21unixbench_multi08:30:211
310069963,2cyclictest31248-21sh12:20:241
310069963,2cyclictest30654-21snmpd11:47:031
310069963,2cyclictest30654-21snmpd11:47:021
310069963,2cyclictest30654-21snmpd07:10:091
310069963,2cyclictest21381-21systemctl09:35:181
310069963,2cyclictest19982-21systemd-journal11:50:291
310069963,2cyclictest14918-21apt-get10:25:121
310069962,3cyclictest9333-21tr09:25:181
310069962,3cyclictest8422-21date07:30:011
310069962,3cyclictest508-21dbus-daemon08:52:261
310069962,3cyclictest31777-21expr08:20:131
310069962,3cyclictest30654-21snmpd12:25:381
310069962,3cyclictest30654-21snmpd10:55:031
310069962,3cyclictest30654-21snmpd08:49:191
310069962,3cyclictest14539-21diskstats09:30:191
310069962,3cyclictest1002-21ls10:40:141
310069962,3cyclictest0-21swapper/109:12:371
310069962,1cyclictest0-21swapper/111:25:191
310069962,1cyclictest0-21swapper/110:07:471
310069962,1cyclictest0-21swapper/109:50:081
310069962,1cyclictest0-21swapper/109:48:491
310069962,1cyclictest0-21swapper/109:48:491
310069962,1cyclictest0-21swapper/109:41:151
310069961,4cyclictest0-21swapper/110:52:251
310069961,3cyclictest19982-21systemd-journal11:22:371
310069961,2cyclictest5863-21ssh12:00:531
310069960,6cyclictest0-21swapper/111:10:511
310069960,6cyclictest0-21swapper/110:18:251
310069960,5cyclictest8635-21/usr/sbin/munin07:30:121
310069960,5cyclictest14030-21memory07:40:161
310069960,3cyclictest1124-21ssh10:12:381
310069960,2cyclictest29125-21/usr/sbin/munin08:15:161
310069960,2cyclictest2787-21sshd08:25:211
310069960,0cyclictest0-21swapper/111:07:291
310019966,0cyclictest0-21swapper/012:20:080
310019964,1cyclictest4906-21smart_sda10:15:230
310019964,1cyclictest11736-21in:imuxsock08:30:120
310019963,2cyclictest668-21runrttasks12:05:380
310019963,2cyclictest668-21runrttasks10:26:480
310019963,2cyclictest6084-21expr09:50:130
310019963,2cyclictest3543-21taskset07:15:440
310019963,2cyclictest31612-21wc08:20:130
310019963,2cyclictest30926-21sendmail_mailtr11:55:190
310019963,2cyclictest30654-21snmpd09:40:200
310019963,2cyclictest29741-11brltty09:00:210
310019963,2cyclictest23815-21sshd09:10:250
310019963,2cyclictest2232-21cstates09:20:140
310019963,2cyclictest21677-21diskmemload10:45:120
310019963,2cyclictest19944-21turbostat.cron07:59:560
310019963,2cyclictest1663-21sh09:45:490
310019963,2cyclictest1663-21sh09:45:490
310019963,2cyclictest1468-21cat08:25:120
310019963,2cyclictest1-21systemd08:56:110
310019963,2cyclictest10879-21awk11:15:140
310019963,2cyclictest0-21swapper/007:50:010
310019963,2cyclictest0-21swapper/007:50:010
310019963,1cyclictest0-21swapper/011:50:090
310019962,3cyclictest508-21dbus-daemon08:43:400
310019962,3cyclictest30654-21snmpd12:37:520
310019962,3cyclictest30654-21snmpd11:07:010
310019962,3cyclictest30654-21snmpd10:55:540
310019962,3cyclictest30654-21snmpd08:13:320
310019962,3cyclictest24659-21sshd08:05:130
310019962,3cyclictest19971-21expr09:05:170
310019962,3cyclictest13884-21mailstats08:50:220
310019962,3cyclictest0-21swapper/011:30:080
310019962,3cyclictest0-21swapper/009:32:250
310019962,1cyclictest0-21swapper/012:00:480
310019962,1cyclictest0-21swapper/011:44:130
310019962,1cyclictest0-21swapper/011:11:250
310019961,4cyclictest18156-21cut10:00:160
310019961,4cyclictest0-21swapper/010:05:150
310019961,4cyclictest0-21swapper/007:20:070
310019961,3cyclictest19982-21systemd-journal11:25:430
310019961,3cyclictest19982-21systemd-journal09:36:150
310019960,5cyclictest9233-21grep10:20:130
310019960,3cyclictest23652-21tail12:15:140
310019960,2cyclictest15432-21/usr/sbin/munin07:45:210
310179952,2cyclictest5783-21tr08:35:133
310179952,2cyclictest19982-21systemd-journal08:05:143
310179952,2cyclictest17731-21idleruntime12:35:153
310179952,2cyclictest13496-21cat08:50:173
310179951,2cyclictest19982-21systemd-journal08:10:013
310109952,2cyclictest6235-21head08:35:152
310109952,2cyclictest30654-21snmpd09:05:182
310109952,2cyclictest26672-21grep09:40:152
310109952,2cyclictest18136-21expr07:50:122
310109952,2cyclictest18136-21expr07:50:122
310109952,2cyclictest1360-21cpuspeed_turbos08:25:122
310109952,2cyclictest10473-21expr08:45:122
310109951,3cyclictest6687-21expr07:25:142
310109951,3cyclictest18130-21cat09:00:172
310109951,3cyclictest13350-21cat08:50:162
310069952,2cyclictest30654-21snmpd08:40:021
310069952,2cyclictest26917-21apt-get08:10:141
310069952,2cyclictest11172-21expr07:35:141
310069951,2cyclictest19982-21systemd-journal07:20:081
310069950,3cyclictest19982-21systemd-journal07:45:101
310019952,2cyclictest6496-21cpuspeed07:25:140
310019951,3cyclictest19982-21systemd-journal08:35:130
310019951,3cyclictest11070-21grep07:35:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional