You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 02:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Sat Nov 22, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
695523929,0sleep30-21swapper/318:19:473
672523727,0sleep20-21swapper/218:17:102
658723627,0sleep10-21swapper/118:15:331
690223423,0sleep00-21swapper/018:19:120
329980,0watchdog/30-21swapper/322:46:383
329970,0watchdog/30-21swapper/318:25:153
29925070,0irq/32-eth0-rx-18291-21latency_hist22:29:543
29925070,0irq/32-eth0-rx-0-21swapper/322:30:073
149970,0migration/012867-21sh23:01:090
8315060,0irq/26-0000:00:4970-21perf21:44:542
70349965,0cyclictest24394-21find18:50:201
70339960,0cyclictest0-21swapper/023:24:550
35160,0ktimersoftd/30-21swapper/318:20:183
339960,0migration/36899-21sh22:04:043
339960,0migration/318424-21sh21:30:393
329960,0watchdog/324624-21kthreadcore20:00:103
329960,0watchdog/323121-21ssh22:11:023
329960,0watchdog/319690-21kthreadcore19:10:153
329960,0watchdog/30-21swapper/323:30:583
329960,0watchdog/30-21swapper/323:26:463
329960,0watchdog/30-21swapper/323:17:303
329960,0watchdog/30-21swapper/323:07:063
329960,0watchdog/30-21swapper/323:04:503
329960,0watchdog/30-21swapper/322:40:503
329960,0watchdog/30-21swapper/322:39:423
329960,0watchdog/30-21swapper/322:20:263
329960,0watchdog/30-21swapper/322:05:503
329960,0watchdog/30-21swapper/321:28:263
329960,0watchdog/30-21swapper/321:15:263
329960,0watchdog/30-21swapper/321:05:063
329960,0watchdog/30-21swapper/320:56:583
329960,0watchdog/30-21swapper/320:54:103
329960,0watchdog/30-21swapper/320:49:063
329960,0watchdog/30-21swapper/320:37:383
329960,0watchdog/30-21swapper/320:24:543
29935060,0irq/33-eth0-tx-0-21swapper/019:00:200
29925060,0irq/32-eth0-rx-0-21swapper/323:45:133
29925060,0irq/32-eth0-rx-0-21swapper/323:41:043
29925060,0irq/32-eth0-rx-0-21swapper/323:10:003
29925060,0irq/32-eth0-rx-0-21swapper/322:59:003
29925060,0irq/32-eth0-rx-0-21swapper/322:50:093
29925060,0irq/32-eth0-rx-0-21swapper/321:55:153
29925060,0irq/32-eth0-rx-0-21swapper/321:50:553
29925060,0irq/32-eth0-rx-0-21swapper/321:42:093
29925060,0irq/32-eth0-rx-0-21swapper/320:40:073
29925060,0irq/32-eth0-rx-0-21swapper/320:21:333
29925060,0irq/32-eth0-rx-0-21swapper/319:00:123
149960,0migration/03544-21sh21:41:070
149960,0migration/031254-21aten_r1power_fr22:00:080
149960,0migration/026432-21kthreadcore22:50:150
149960,0migration/011798-21sh20:33:200
8315050,0irq/26-0000:00:7229-21munin-run22:04:542
70359954,0cyclictest23871-21snmpd19:52:402
70349955,0cyclictest0-21swapper/123:30:041
70349955,0cyclictest0-21swapper/123:22:401
70349955,0cyclictest0-21swapper/121:57:511
70349955,0cyclictest0-21swapper/121:36:241
70349955,0cyclictest0-21swapper/121:23:281
70349955,0cyclictest0-21swapper/120:05:071
70349955,0cyclictest0-21swapper/119:49:591
70349954,0cyclictest26797-21sh22:34:251
70349950,0cyclictest0-21swapper/123:36:231
70349950,0cyclictest0-21swapper/123:09:351
70349950,0cyclictest0-21swapper/121:51:041
70349950,0cyclictest0-21swapper/120:23:061
70339954,0cyclictest32683-21perl23:15:080
339950,0migration/332416-21kthreadcore19:20:103
339950,0migration/33231-21kthreadcore19:45:143
339950,0migration/329018-21kthreadcore18:55:113
339950,0migration/327393-21kthreadcore22:15:113
339950,0migration/320226-21diskmemload21:34:583
339950,0migration/319083-21munin-node18:50:123
329950,0watchdog/35396-21if_eth1.220:10:103
329950,0watchdog/332727-21ssh21:20:143
329950,0watchdog/326005-21df19:40:063
329950,0watchdog/318894-21diskstats19:35:063
329950,0watchdog/313985-21python19:50:143
329950,0watchdog/30-21swapper/323:19:543
329950,0watchdog/30-21swapper/321:45:143
329950,0watchdog/30-21swapper/321:10:103
329950,0watchdog/30-21swapper/320:05:183
329950,0watchdog/30-21swapper/319:55:303
329950,0watchdog/30-21swapper/319:31:303
329950,0watchdog/30-21swapper/319:24:543
329950,0watchdog/30-21swapper/319:16:353
329950,0watchdog/30-21swapper/319:06:353
329950,0watchdog/30-21swapper/318:46:273
329950,0watchdog/30-21swapper/318:40:313
329950,0watchdog/30-21swapper/318:37:513
29935050,0irq/33-eth0-tx-6526-21ssh20:49:312
29935050,0irq/33-eth0-tx-410-21munin-node19:45:170
29935050,0irq/33-eth0-tx-21658-21munin-node18:30:190
29935050,0irq/33-eth0-tx-18802-21munin-node22:45:192
29935050,0irq/33-eth0-tx-1722-21ssh22:35:472
29935050,0irq/33-eth0-tx-0-21swapper/221:21:252
29935050,0irq/33-eth0-tx-0-21swapper/020:20:000
29935050,0irq/33-eth0-tx-0-21swapper/020:19:420
29935050,0irq/33-eth0-tx-0-21swapper/019:20:090
29935050,0irq/33-eth0-tx-0-21swapper/018:38:130
29925050,0irq/32-eth0-rx-10191-21which20:30:173
29925050,0irq/32-eth0-rx-0-21swapper/323:34:573
29925050,0irq/32-eth0-rx-0-21swapper/321:00:023
259950,0migration/28862-21fschecks_time23:20:082
259950,0migration/210218-21sh21:26:512
249950,0watchdog/211735-21fschecks_count19:30:102
179950,0migration/19977-21kthreadcore19:05:111
179950,0migration/14946-21sh22:58:121
179950,0migration/14943-21kthreadcore19:00:131
179950,0migration/13244-21sh22:39:481
179950,0migration/129793-21fschecks_time21:40:071
179950,0migration/129136-21kthreadcore20:25:141
179950,0migration/124195-21taskset23:28:031
179950,0migration/116493-21taskset22:25:191
179950,0migration/116493-21gltestperf20:55:071
179950,0migration/11407-21cpuspeed_turbos22:20:111
179950,0migration/113965-21taskset20:35:121
179950,0migration/112888-21sh23:01:141
149950,0migration/09870-21sh22:25:000
149950,0migration/08618-21kthreadcore20:10:120
149950,0migration/08595-21fschecks_time19:50:060
149950,0migration/08409-21sh23:37:150
149950,0migration/03529-21sh22:55:170
149950,0migration/032061-21taskset18:56:200
149950,0migration/031235-21switchtime22:15:190
149950,0migration/027993-21runrttasks21:38:040
149950,0migration/026428-21kthreadcore23:10:110
149950,0migration/024870-21fschecks_count21:00:080
149950,0migration/024459-21timerwakeupswit23:45:210
149950,0migration/024179-21kthreadcore20:40:130
149950,0migration/022160-21aten_r1power_cu20:00:040
149950,0migration/021758-21taskset19:58:080
149950,0migration/021561-21sh23:06:090
149950,0migration/021461-21latency_hist18:29:540
149950,0migration/020694-21kthreadcore19:35:110
149950,0migration/020658-21sh20:38:100
149950,0migration/019022-21idleruntime21:15:090
149950,0migration/018844-21sh21:31:480
149950,0migration/018184-21taskset18:50:070
149950,0migration/017783-21kthreadcore19:10:140
149950,0migration/017185-21smart_nvme0n118:45:160
149950,0migration/01699-21fschecks_count22:20:120
149950,0migration/016187-21aten_r1power_en22:10:070
149950,0migration/016185-21kthreadcore19:30:150
149950,0migration/014335-21sh20:52:160
149950,0migration/012964-21sh21:48:110
8315040,0irq/26-0000:00:10340-21forks22:25:062
8315040,0irq/26-0000:00:0-21swapper/221:50:072
70359943,0cyclictest4792-21awk18:40:112
70359943,0cyclictest3599-21lldpd23:45:312
70359943,0cyclictest3599-21lldpd20:01:132
70359943,0cyclictest3599-21lldpd19:47:422
70359943,0cyclictest3599-21lldpd19:39:432
70359943,0cyclictest3599-21lldpd18:25:132
70359943,0cyclictest22041-21awk22:50:122
70359943,0cyclictest18414-21sadc20:19:542
70359943,0cyclictest17949-21sadc18:49:542
70359943,0cyclictest15586-21awk23:05:072
70349944,0cyclictest0-21swapper/122:49:431
70349943,0cyclictest560-21awk23:15:101
70349943,0cyclictest5103-21awk19:25:091
70349943,0cyclictest28919-21sadc19:19:541
70349943,0cyclictest26113-21sadc23:49:541
70349943,0cyclictest23871-21snmpd21:29:551
70349943,0cyclictest23871-21snmpd21:17:551
70349943,0cyclictest23871-21snmpd20:09:551
70349943,0cyclictest23871-21snmpd19:44:101
70349943,0cyclictest23871-21snmpd18:58:391
70349943,0cyclictest23871-21snmpd18:45:091
70349943,0cyclictest23871-21snmpd18:40:541
70349943,0cyclictest17245-21awk22:10:111
70339944,0cyclictest0-21swapper/021:55:310
70339943,0cyclictest8358-21awk18:20:080
70339943,0cyclictest4735-21awk22:40:100
70339943,0cyclictest3599-21lldpd21:53:150
70339943,0cyclictest31753-21awk20:45:130
70339940,0cyclictest26851-21ethtool19:40:100
329940,0watchdog/30-21swapper/320:18:263
29935040,0irq/33-eth0-tx-9493-21munin-node23:40:062
29935040,0irq/33-eth0-tx-31820-21ssh23:15:002
29935040,0irq/33-eth0-tx-29359-21munin-node20:05:180
29935040,0irq/33-eth0-tx-25932-21ssh22:32:042
29935040,0irq/33-eth0-tx-22571-21munin-node21:55:162
29935040,0irq/33-eth0-tx-21646-21kthreadcore22:10:142
29935040,0irq/33-eth0-tx-17979-21munin-node21:15:142
29935040,0irq/33-eth0-tx-0-21swapper/223:30:012
29935040,0irq/33-eth0-tx-0-21swapper/223:27:592
29935040,0irq/33-eth0-tx-0-21swapper/223:00:002
29935040,0irq/33-eth0-tx-0-21swapper/222:19:552
29935040,0irq/33-eth0-tx-0-21swapper/222:14:562
29935040,0irq/33-eth0-tx-0-21swapper/221:34:572
29935040,0irq/33-eth0-tx-0-21swapper/221:09:582
29935040,0irq/33-eth0-tx-0-21swapper/221:05:142
29935040,0irq/33-eth0-tx-0-21swapper/220:59:572
29935040,0irq/33-eth0-tx-0-21swapper/220:55:072
29935040,0irq/33-eth0-tx-0-21swapper/220:50:042
29935040,0irq/33-eth0-tx-0-21swapper/220:35:012
29935040,0irq/33-eth0-tx-0-21swapper/220:20:522
29935040,0irq/33-eth0-tx-0-21swapper/019:25:100
29935040,0irq/33-eth0-tx-0-21swapper/018:40:070
29925040,0irq/32-eth0-rx-0-21swapper/318:30:143
259940,0migration/25967-21fschecks_time21:45:082
259940,0migration/2341-21kthreadcore22:55:122
259940,0migration/222091-21munin-node20:40:092
259940,0migration/221692-21runrttasks18:30:082
259940,0migration/219032-21kthreadcore19:10:142
249940,0watchdog/29881-21sed20:30:142
249940,0watchdog/25572-21irqstats19:25:102
249940,0watchdog/231243-21cut20:25:142
249940,0watchdog/223393-21cpu23:10:062
249940,0watchdog/214105-21cat21:30:102
249940,0watchdog/213603-21ntp_states22:05:142
249940,0watchdog/212668-21kthreadcore18:20:112
249940,0watchdog/20-21swapper/223:35:182
249940,0watchdog/20-21swapper/222:41:502
249940,0watchdog/20-21swapper/218:55:112
179940,0migration/17750-21cpuspeed_turbos22:05:061
179940,0migration/13782-21taskset19:24:421
179940,0migration/124904-21fschecks_count21:00:091
179940,0migration/12395-21kthreadcore22:00:151
179940,0migration/121799-21aten_r1power_vo18:30:081
179940,0migration/120036-21kernelversion19:35:111
179940,0migration/117473-21kthreadcore19:55:131
179940,0migration/117010-21kthreadcore19:10:141
179940,0migration/114721-21kthreadcore20:15:111
179940,0migration/114574-21aten_r1power_vo18:25:081
179940,0migration/114197-21kthreadcore23:40:121
179940,0migration/111329-21aten_r1power_en19:30:081
179940,0migration/111044-21kthreadcore21:45:131
169940,0watchdog/19363-21munin-node21:10:221
169940,0watchdog/13902-21aten_r1power_cu20:30:061
169940,0watchdog/131085-21ssh23:13:341
169940,0watchdog/124623-21kthreadcore20:00:101
169940,0watchdog/11356-21cat20:45:141
169940,0watchdog/112667-21cut18:20:111
169940,0watchdog/10-21swapper/122:50:101
169940,0watchdog/10-21swapper/121:05:101
169940,0watchdog/10-21swapper/120:39:581
169940,0watchdog/10-21swapper/119:45:141
149940,0migration/03560-21kthreadcore21:05:140
149940,0migration/032048-21kthreadcore20:25:150
149940,0migration/026414-21irqcore23:30:120
149940,0migration/020148-21kthreadcore20:55:110
149940,0migration/011737-21kthreadcore23:40:120
149940,0migration/011484-21kthreadcore22:05:120
149940,0migration/010093-21kthreadcore23:20:110
70359933,0cyclictest3599-21lldpd19:05:132
70359932,0cyclictest3599-21lldpd19:15:422
70359932,0cyclictest3206-21sensors20:05:172
70359932,0cyclictest30184-21awk19:20:072
70359932,0cyclictest21167-21sensors19:55:172
70359932,0cyclictest20380-21cut18:50:162
70359932,0cyclictest1243-21awk19:00:102
70359930,0cyclictest0-21swapper/218:39:122
70349930,0cyclictest29676-21perl18:35:101
70349930,0cyclictest25682-21perl22:15:071
70339933,0cyclictest31622-21kworker/0:019:15:110
70339933,0cyclictest15514-21kworker/0:222:46:200
70339933,0cyclictest15514-21kworker/0:222:35:080
70339933,0cyclictest15514-21kworker/0:222:30:040
70339931,0cyclictest12713-21kthreadcore21:10:130
70339930,0cyclictest3682-21perl21:25:070
70339930,0cyclictest0-21swapper/021:20:080
29935030,0irq/33-eth0-tx-0-21swapper/019:05:180
249930,0watchdog/20-21swapper/220:10:102
169930,0watchdog/14210-21irqcore21:25:101
169930,0watchdog/10-21swapper/122:40:181
169930,0watchdog/10-21swapper/120:50:061
70359921,0cyclictest0-21swapper/219:39:542
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional