You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-12 - 20:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Wed Nov 12, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1133324435,0sleep10-21swapper/106:17:591
1126524025,0sleep00-21swapper/006:17:100
1127923728,0sleep30-21swapper/306:17:173
1125423627,0sleep20-21swapper/206:17:042
116229988,0cyclictest0-21swapper/008:20:530
329970,0watchdog/30-21swapper/309:29:013
29935070,0irq/33-eth0-tx-0-21swapper/008:05:420
29925070,0irq/32-eth0-rx-0-21swapper/310:05:453
29925070,0irq/32-eth0-rx-0-21swapper/309:11:113
8315060,0irq/26-0000:00:5506-21turbostat.cron10:00:282
339960,0migration/33943-21sh10:52:413
339960,0migration/317658-21df_abs10:25:433
339960,0migration/313272-21sh10:02:493
339960,0migration/31114-21timerandwakeup08:00:553
329960,0watchdog/38314-21ls11:15:403
329960,0watchdog/30-21swapper/311:20:443
329960,0watchdog/30-21swapper/311:11:083
329960,0watchdog/30-21swapper/311:01:003
329960,0watchdog/30-21swapper/310:50:243
329960,0watchdog/30-21swapper/310:45:163
329960,0watchdog/30-21swapper/310:31:083
329960,0watchdog/30-21swapper/310:22:203
329960,0watchdog/30-21swapper/310:16:043
329960,0watchdog/30-21swapper/309:56:243
329960,0watchdog/30-21swapper/309:52:283
329960,0watchdog/30-21swapper/309:41:003
329960,0watchdog/30-21swapper/309:35:453
329960,0watchdog/30-21swapper/309:16:093
329960,0watchdog/30-21swapper/308:45:493
329960,0watchdog/30-21swapper/308:43:333
329960,0watchdog/30-21swapper/308:39:373
329960,0watchdog/30-21swapper/308:31:133
329960,0watchdog/30-21swapper/308:25:293
329960,0watchdog/30-21swapper/308:22:013
329960,0watchdog/30-21swapper/308:10:453
329960,0watchdog/30-21swapper/306:45:413
29925060,0irq/32-eth0-rx-0-21swapper/311:47:453
29925060,0irq/32-eth0-rx-0-21swapper/311:47:453
29925060,0irq/32-eth0-rx-0-21swapper/311:26:273
29925060,0irq/32-eth0-rx-0-21swapper/311:05:473
29925060,0irq/32-eth0-rx-0-21swapper/309:30:343
29925060,0irq/32-eth0-rx-0-21swapper/309:20:493
29925060,0irq/32-eth0-rx-0-21swapper/309:05:483
29925060,0irq/32-eth0-rx-0-21swapper/308:56:233
29925060,0irq/32-eth0-rx-0-21swapper/308:52:033
116249965,0cyclictest20739-21cat08:35:472
116249960,0cyclictest0-21swapper/209:53:402
116249960,0cyclictest0-21swapper/209:41:502
116239960,0cyclictest0-21swapper/108:30:461
339950,0migration/35997-21fschecks_count07:45:423
339950,0migration/330096-21taskset06:55:373
339950,0migration/32217-21fschecks_time07:20:463
339950,0migration/321166-21kthreadcore08:15:503
339950,0migration/320067-21fschecks_count11:40:413
339950,0migration/312990-21if_eth1.206:20:463
339950,0migration/312174-21kthreadcore06:40:463
329950,0watchdog/32850-21diskstats10:35:403
329950,0watchdog/30-21swapper/311:35:363
329950,0watchdog/30-21swapper/311:30:323
329950,0watchdog/30-21swapper/310:10:363
329950,0watchdog/30-21swapper/309:45:443
329950,0watchdog/30-21swapper/309:00:453
329950,0watchdog/30-21swapper/307:41:293
329950,0watchdog/30-21swapper/307:35:413
329950,0watchdog/30-21swapper/307:34:573
329950,0watchdog/30-21swapper/307:25:293
329950,0watchdog/30-21swapper/307:15:293
329950,0watchdog/30-21swapper/307:05:293
329950,0watchdog/30-21swapper/307:03:413
329950,0watchdog/30-21swapper/306:50:293
329950,0watchdog/30-21swapper/306:36:013
329950,0watchdog/30-21swapper/306:33:333
329950,0watchdog/30-21swapper/306:26:133
29935050,0irq/33-eth0-tx-8849-21munin-node07:25:570
29935050,0irq/33-eth0-tx-8512-21munin-node10:20:472
29935050,0irq/33-eth0-tx-11838-21munin-node06:20:540
29935050,0irq/33-eth0-tx-0-21swapper/007:56:070
29935050,0irq/33-eth0-tx-0-21swapper/007:45:400
29935050,0irq/33-eth0-tx-0-21swapper/007:40:410
29935050,0irq/33-eth0-tx-0-21swapper/007:20:450
29925050,0irq/32-eth0-rx-8922-21kthreadcore10:55:493
29925050,0irq/32-eth0-rx-0-21swapper/308:05:433
29925050,0irq/32-eth0-rx-0-21swapper/307:56:073
29925050,0irq/32-eth0-rx-0-21swapper/307:50:393
29925050,0irq/32-eth0-rx-0-21swapper/307:10:413
27150,0ktimersoftd/20-21swapper/208:54:042
249950,0watchdog/217029-21ssh09:28:532
249950,0watchdog/210982-21needreboot07:00:492
179950,0migration/18025-21missed_timers11:30:471
179950,0migration/17365-21kthreadcore07:00:471
179950,0migration/16171-21fschecks_time10:55:451
179950,0migration/14629-21sh09:01:361
179950,0migration/1313-21munin-node10:30:541
179950,0migration/12985-21kthreadcore06:35:481
179950,0migration/12951-21fschecks_count10:35:411
179950,0migration/126809-21munin-node10:45:541
179950,0migration/126106-21latency_hist06:30:281
179950,0migration/124926-21sh10:28:161
179950,0migration/123307-21kthreadcore07:55:451
179950,0migration/122037-21kthreadcore08:15:501
179950,0migration/121057-21sh11:01:291
179950,0migration/112273-21hddtemp_smartct10:40:431
179950,0migration/111167-21kthreadcore06:40:461
179950,0migration/110369-21munin-node08:45:551
169950,0watchdog/16386-21runrttasks08:10:411
149950,0migration/09903-21sh08:30:440
149950,0migration/09878-21irqrtprio06:40:450
149950,0migration/08754-21taskset09:25:070
149950,0migration/07952-21irqrtprio09:05:450
149950,0migration/05177-21kthreadcore10:15:500
149950,0migration/04679-21kernelversion09:40:460
149950,0migration/030640-21unixbench_multi09:15:540
149950,0migration/030299-21kthreadcore07:15:490
149950,0migration/029233-21kthreadcore10:30:460
149950,0migration/029188-21kthreadcore09:35:470
149950,0migration/028451-21kthreadcore11:05:500
149950,0migration/028103-21gltestperf08:00:440
149950,0migration/027224-21gltestperf06:30:430
149950,0migration/024420-21irqrtprio06:50:450
149950,0migration/022587-21kthreadcore07:10:470
149950,0migration/022376-21sh09:12:000
149950,0migration/02023-21taskset08:42:300
149950,0migration/019864-21kthreadcore07:30:470
149950,0migration/018871-21kthreadcore08:15:490
149950,0migration/018106-21kthreadcore07:50:460
149950,0migration/016854-21aten_r1power_cu11:20:400
149950,0migration/015548-21taskset10:05:420
149950,0migration/015335-21kthreadcore11:35:460
149950,0migration/014216-21kthreadcore07:05:490
149950,0migration/013828-21kthreadcore11:15:460
149950,0migration/011437-21kthreadcore08:10:470
116249955,0cyclictest0-21swapper/211:16:262
116249955,0cyclictest0-21swapper/211:11:042
116249955,0cyclictest0-21swapper/210:51:502
116249955,0cyclictest0-21swapper/210:27:152
116249950,0cyclictest0-21swapper/209:31:542
116249950,0cyclictest0-21swapper/209:14:152
116239955,0cyclictest19221-21crond11:40:281
116239954,0cyclictest21715-21awk09:50:431
8315040,0irq/26-0000:00:21125-21cut08:55:282
29935040,0irq/33-eth0-tx-8805-21ssh11:31:022
29935040,0irq/33-eth0-tx-7814-21ssh09:22:362
29935040,0irq/33-eth0-tx-7416-21hddtemp_smartct09:05:432
29935040,0irq/33-eth0-tx-4876-21ssh09:02:152
29935040,0irq/33-eth0-tx-32380-21grep11:45:462
29935040,0irq/33-eth0-tx-32380-21grep11:45:462
29935040,0irq/33-eth0-tx-26890-21ssh10:46:052
29935040,0irq/33-eth0-tx-23614-21diskmemload11:29:342
29935040,0irq/33-eth0-tx-16713-21tr10:40:472
29935040,0irq/33-eth0-tx-0-21swapper/211:40:302
29935040,0irq/33-eth0-tx-0-21swapper/211:39:122
29935040,0irq/33-eth0-tx-0-21swapper/211:22:102
29935040,0irq/33-eth0-tx-0-21swapper/211:07:502
29935040,0irq/33-eth0-tx-0-21swapper/211:00:542
29935040,0irq/33-eth0-tx-0-21swapper/210:37:302
29935040,0irq/33-eth0-tx-0-21swapper/210:30:462
29935040,0irq/33-eth0-tx-0-21swapper/210:16:032
29935040,0irq/33-eth0-tx-0-21swapper/210:10:402
29935040,0irq/33-eth0-tx-0-21swapper/210:05:462
29935040,0irq/33-eth0-tx-0-21swapper/210:01:262
29935040,0irq/33-eth0-tx-0-21swapper/209:15:432
29935040,0irq/33-eth0-tx-0-21swapper/208:48:452
29551240,0sleep10-21swapper/111:45:461
29551240,0sleep10-21swapper/111:45:461
259940,0migration/29381-21gltestperf06:40:422
259940,0migration/227576-21kthreadcore06:50:472
259940,0migration/226612-21sh09:35:302
259940,0migration/225281-21munin-node08:20:462
259940,0migration/21664-21aten_r1power_vo07:20:452
259940,0migration/216008-21kthreadcore09:45:472
259940,0migration/213802-21taskset11:00:232
249940,0watchdog/29079-21cpuspeed_turbos07:25:452
249940,0watchdog/229411-21cut07:15:492
249940,0watchdog/217332-21kthreadcore06:20:492
179940,0migration/18529-21aten_r1power_cu10:20:391
179940,0migration/17204-21kthreadcore07:20:511
179940,0migration/16884-21timerandwakeup08:25:551
179940,0migration/14521-21munin-node09:40:461
179940,0migration/13142-21kthreadcore09:20:471
179940,0migration/130846-21sh09:16:121
179940,0migration/129774-21timerandwakeup10:10:541
179940,0migration/129694-21kthreadcore09:35:471
179940,0migration/128634-21kthreadcore08:20:481
179940,0migration/128521-21kthreadcore11:05:501
179940,0migration/126867-21kthreadcore06:50:461
179940,0migration/12310-21kthreadcore10:50:461
179940,0migration/121878-21kthreadcore07:10:461
179940,0migration/121601-21taskset06:45:491
179940,0migration/120978-21kthreadcore11:20:471
179940,0migration/119987-21fschecks_count11:40:411
179940,0migration/119652-21kthreadcore09:10:481
179940,0migration/118435-21users06:20:571
179940,0migration/117485-21kthreadcore10:05:451
179940,0migration/114335-21kthreadcore07:50:451
179940,0migration/111925-21munin-node07:45:541
169940,0watchdog/128692-21kthreadcore07:35:491
169940,0watchdog/124985-21ssh08:36:051
169940,0watchdog/12117-21ssh08:42:411
169940,0watchdog/119472-21ssh09:48:041
169940,0watchdog/112827-21cut07:05:451
169940,0watchdog/110133-21gltestperf09:25:451
169940,0watchdog/10-21swapper/111:10:401
169940,0watchdog/10-21swapper/107:40:571
149940,0migration/027776-21kthreadcore10:10:470
149940,0migration/022834-21kthreadcore09:30:490
149940,0migration/020222-21sh08:53:350
149940,0migration/015093-21kthreadcore10:40:460
116249943,0cyclictest3599-21lldpd08:26:212
116249943,0cyclictest3599-21lldpd08:16:212
116249943,0cyclictest3599-21lldpd07:45:512
116249943,0cyclictest3599-21lldpd07:30:512
116249943,0cyclictest3599-21lldpd06:34:212
116249943,0cyclictest28217-21awk08:00:452
116249943,0cyclictest28190-21awk08:40:442
116249943,0cyclictest23871-21snmpd08:12:012
116249943,0cyclictest23871-21snmpd07:38:152
116249943,0cyclictest23871-21snmpd07:10:302
116249943,0cyclictest23871-21snmpd07:05:302
116249943,0cyclictest23871-21snmpd06:48:302
116249943,0cyclictest10416-21awk08:30:462
116239943,0cyclictest6888-21awk10:00:441
116239943,0cyclictest3599-21lldpd10:16:241
116239943,0cyclictest31276-21awk06:55:451
116239943,0cyclictest26638-21awk11:25:431
116239943,0cyclictest23871-21snmpd09:55:321
116239943,0cyclictest17193-21awk07:30:451
116229944,0cyclictest1911-21grep06:35:440
116229943,0cyclictest6309-21kworker/0:111:28:200
116229943,0cyclictest6309-21kworker/0:111:10:420
116229943,0cyclictest6073-21awk07:00:450
116229943,0cyclictest3262-21irqbalance09:52:520
116229943,0cyclictest29401-21awk11:45:440
116229943,0cyclictest29401-21awk11:45:440
116229943,0cyclictest24525-21awk07:35:470
116229943,0cyclictest23871-21snmpd08:57:310
116229943,0cyclictest21245-21awk10:45:440
116229943,0cyclictest20240-21awk06:25:460
116229943,0cyclictest1334-21awk08:25:450
116229940,0cyclictest30101-21munin-node06:55:430
116229940,0cyclictest0-21swapper/010:35:280
8315030,0irq/26-0000:00:0-21swapper/207:50:522
259930,0migration/219339-21cpuspeed_turbos06:25:422
249930,0watchdog/26645-21kthreadcore06:35:492
249930,0watchdog/20-21swapper/208:05:452
249930,0watchdog/20-21swapper/207:40:452
179930,0migration/11661-21runrttasks08:04:121
179930,0migration/114636-21timerwakeupswit11:15:531
169930,0watchdog/10-21swapper/109:30:491
169930,0watchdog/10-21swapper/109:08:091
169930,0watchdog/10-21swapper/108:55:451
169930,0watchdog/10-21swapper/108:50:491
169930,0watchdog/10-21swapper/108:05:451
169930,0watchdog/10-21swapper/107:25:451
116249933,0cyclictest0-21swapper/206:55:282
116249932,0cyclictest21041-21awk07:55:432
116239931,0cyclictest0-21swapper/106:30:471
116229933,0cyclictest6309-21kworker/0:111:40:280
116229933,0cyclictest6309-21kworker/0:111:30:440
116229933,0cyclictest32026-21kworker/0:209:00:490
116229933,0cyclictest32026-21kworker/0:208:45:350
116229933,0cyclictest32026-21kworker/0:208:35:310
116229933,0cyclictest21028-21kworker/0:010:55:410
116229933,0cyclictest21028-21kworker/0:010:50:410
116229933,0cyclictest12110-21kworker/0:110:25:290
116229933,0cyclictest12110-21kworker/0:110:20:290
116229933,0cyclictest12110-21kworker/0:110:00:330
116229933,0cyclictest12110-21kworker/0:109:55:410
116229933,0cyclictest12110-21kworker/0:109:45:350
116229933,0cyclictest12110-21kworker/0:109:25:430
116229932,0cyclictest3599-21lldpd06:45:510
116229930,0cyclictest14786-21perl11:00:420
116239921,0cyclictest26610-21ls07:15:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional