You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-03 - 19:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Thu Jul 03, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3265524027,0sleep20-21swapper/206:25:122
3264623930,0sleep30-21swapper/306:25:053
43123625,0sleep00-21swapper/006:28:020
3272423525,0sleep10-21swapper/106:25:591
329970,0watchdog/324088-21irqcore08:38:473
5549960,0cyclictest0-21swapper/211:50:382
5549960,0cyclictest0-21swapper/209:33:472
5549960,0cyclictest0-21swapper/208:40:392
35160,0ktimersoftd/30-21swapper/309:20:023
339960,0migration/327774-21irqrtprio07:53:493
339960,0migration/321849-21sh09:58:133
339960,0migration/319241-21rt-features11:28:533
339960,0migration/315378-21sh10:31:243
339960,0migration/310579-21sh09:13:333
329960,0watchdog/323182-21grep10:53:503
329960,0watchdog/312259-21cut11:08:463
329960,0watchdog/30-21swapper/311:49:423
329960,0watchdog/30-21swapper/311:44:143
329960,0watchdog/30-21swapper/311:38:303
329960,0watchdog/30-21swapper/311:24:223
329960,0watchdog/30-21swapper/311:20:503
329960,0watchdog/30-21swapper/310:51:023
329960,0watchdog/30-21swapper/310:34:423
329960,0watchdog/30-21swapper/310:25:103
329960,0watchdog/30-21swapper/310:20:343
329960,0watchdog/30-21swapper/310:04:343
329960,0watchdog/30-21swapper/310:02:503
329960,0watchdog/30-21swapper/309:48:503
329960,0watchdog/30-21swapper/309:46:543
329960,0watchdog/30-21swapper/309:38:303
329960,0watchdog/30-21swapper/309:35:503
329960,0watchdog/30-21swapper/308:56:353
329960,0watchdog/30-21swapper/308:45:073
329960,0watchdog/30-21swapper/308:33:433
329960,0watchdog/30-21swapper/308:31:153
329960,0watchdog/30-21swapper/307:58:513
329960,0watchdog/30-21swapper/307:48:513
329960,0watchdog/30-21swapper/307:33:513
329960,0watchdog/30-21swapper/307:18:433
329960,0watchdog/30-21swapper/307:08:233
329960,0watchdog/30-21swapper/306:48:473
29925060,0irq/32-eth0-rx-9378-21cut10:08:513
29925060,0irq/32-eth0-rx-6386-21runrttasks09:04:043
29925060,0irq/32-eth0-rx-0-21swapper/311:55:533
29925060,0irq/32-eth0-rx-0-21swapper/311:33:443
29925060,0irq/32-eth0-rx-0-21swapper/311:03:493
29925060,0irq/32-eth0-rx-0-21swapper/310:46:213
29925060,0irq/32-eth0-rx-0-21swapper/309:32:553
29925060,0irq/32-eth0-rx-0-21swapper/309:00:083
149960,0migration/07565-21kthreadcore07:38:500
8315050,0irq/26-0000:00:21945-21perf10:03:252
5549955,0cyclictest0-21swapper/210:16:502
5549950,0cyclictest0-21swapper/209:53:012
5429955,0cyclictest0-21swapper/011:40:280
5429955,0cyclictest0-21swapper/011:27:040
5429955,0cyclictest0-21swapper/009:58:370
5429955,0cyclictest0-21swapper/009:28:400
5429950,0cyclictest0-21swapper/011:44:530
5429950,0cyclictest0-21swapper/011:20:360
5429950,0cyclictest0-21swapper/009:55:090
339950,0migration/39946-21kthreadcore07:38:503
339950,0migration/38497-21munin-node06:33:443
339950,0migration/37006-21kthreadcore06:53:503
339950,0migration/326673-21fschecks_count07:08:443
339950,0migration/31988-21taskset09:08:333
339950,0migration/313917-21kthreadcore07:43:503
329950,0watchdog/30-21swapper/311:15:063
329950,0watchdog/30-21swapper/310:58:543
329950,0watchdog/30-21swapper/310:13:303
329950,0watchdog/30-21swapper/308:23:393
329950,0watchdog/30-21swapper/308:18:353
329950,0watchdog/30-21swapper/308:14:593
329950,0watchdog/30-21swapper/308:08:393
329950,0watchdog/30-21swapper/308:03:393
329950,0watchdog/30-21swapper/307:31:353
329950,0watchdog/30-21swapper/307:23:313
329950,0watchdog/30-21swapper/307:14:513
329950,0watchdog/30-21swapper/306:58:513
329950,0watchdog/30-21swapper/306:43:433
329950,0watchdog/30-21swapper/306:38:393
329950,0watchdog/30-21swapper/306:29:313
30768250,0sleep030845-21sort08:23:300
29935050,0irq/33-eth0-tx-6304-21df08:28:460
29935050,0irq/33-eth0-tx-28048-21ssh09:41:032
29935050,0irq/33-eth0-tx-22461-21munin-node06:43:500
29935050,0irq/33-eth0-tx-20118-21cut08:33:492
29935050,0irq/33-eth0-tx-19084-21munin-node09:18:472
29935050,0irq/33-eth0-tx-11674-21grep10:08:512
29935050,0irq/33-eth0-tx-0-21swapper/211:56:032
29935050,0irq/33-eth0-tx-0-21swapper/211:41:042
29935050,0irq/33-eth0-tx-0-21swapper/210:58:062
29925050,0irq/32-eth0-rx-0-21swapper/310:38:303
29925050,0irq/32-eth0-rx-0-21swapper/309:23:343
29925050,0irq/32-eth0-rx-0-21swapper/308:49:113
27150,0ktimersoftd/20-21swapper/210:43:572
259950,0migration/29597-21cpuspeed_turbos10:48:442
259950,0migration/29304-21sh06:33:482
259950,0migration/29231-21irqcore10:28:482
259950,0migration/26829-21kthreadcore06:53:502
259950,0migration/26059-21kthreadcore11:03:492
259950,0migration/232256-21needreboot10:58:512
259950,0migration/231475-21munin-node08:43:442
259950,0migration/227440-21latency_hist09:23:302
259950,0migration/222457-21latency_hist07:28:302
259950,0migration/218952-21kthreadcore10:33:492
259950,0migration/217719-21sh09:15:012
179950,0migration/131807-21fschecks_time08:23:421
179950,0migration/130536-21taskset08:20:481
179950,0migration/113197-21kthreadcore08:03:491
169950,0watchdog/19877-21kthreadcore07:18:471
149950,0migration/08717-21kthreadcore10:08:500
149950,0migration/07575-21timerwakeupswit10:43:570
149950,0migration/05055-21kthreadcore11:03:480
149950,0migration/04205-21runrttasks09:46:050
149950,0migration/030992-21munin-node07:33:460
149950,0migration/029490-21kthreadcore10:38:500
149950,0migration/028571-21munin-node07:28:500
149950,0migration/027790-21munin-node11:33:530
149950,0migration/027704-21irqrtprio10:58:480
149950,0migration/025667-21ksmtuned09:02:580
149950,0migration/022750-21kthreadcore10:53:500
149950,0migration/022592-21kthreadcore09:38:490
149950,0migration/020071-21taskset09:37:460
149950,0migration/018970-21latency_hist07:03:300
149950,0migration/01830-21kthreadcore10:23:510
149950,0migration/017388-21kthreadcore06:58:500
149950,0migration/013725-21kthreadcore11:08:470
149950,0migration/013560-21kthreadcore08:03:490
149950,0migration/010282-21kthreadcore08:48:500
8315040,0irq/26-0000:00:0-21swapper/208:53:362
5549943,0cyclictest9669-21awk08:03:472
5549943,0cyclictest3599-21lldpd11:15:592
5549943,0cyclictest3599-21lldpd08:28:482
5549943,0cyclictest3599-21lldpd07:16:162
5549943,0cyclictest23871-21snmpd07:12:232
5549943,0cyclictest23871-21snmpd06:48:532
5549943,0cyclictest2228-21awk07:58:492
5549943,0cyclictest16411-21awk06:38:472
5479943,0cyclictest3599-21lldpd10:02:121
5479943,0cyclictest3599-21lldpd09:28:381
5479943,0cyclictest3599-21lldpd08:50:171
5479943,0cyclictest3599-21lldpd08:41:251
5479943,0cyclictest3599-21lldpd07:54:041
5479943,0cyclictest3262-21irqbalance10:58:541
5479943,0cyclictest3262-21irqbalance10:53:441
5479943,0cyclictest3262-21irqbalance08:59:341
5479943,0cyclictest3262-21irqbalance08:53:441
5479943,0cyclictest3262-21irqbalance08:29:151
5479943,0cyclictest3262-21irqbalance08:14:551
5479943,0cyclictest3262-21irqbalance07:28:351
5479943,0cyclictest3262-21irqbalance07:14:051
5479943,0cyclictest3262-21irqbalance07:10:151
5479943,0cyclictest3262-21irqbalance07:04:551
5479943,0cyclictest3262-21irqbalance06:58:351
5479943,0cyclictest3262-21irqbalance06:53:351
5479943,0cyclictest3262-21irqbalance06:49:051
5479943,0cyclictest3157-21awk09:08:481
5479943,0cyclictest23871-21snmpd11:04:231
5479943,0cyclictest23871-21snmpd07:49:231
5479943,0cyclictest23871-21snmpd07:44:381
5479943,0cyclictest23871-21snmpd06:33:531
5479943,0cyclictest23495-21awk06:43:461
5479943,0cyclictest2323-21awk07:58:501
5479943,0cyclictest1978-21awk10:43:471
5479943,0cyclictest1760-21sadc09:08:301
5479943,0cyclictest14159-21awk11:28:461
5479943,0cyclictest13260-21awk09:33:551
5429944,0cyclictest0-21swapper/011:54:590
5429944,0cyclictest0-21swapper/009:48:420
5429943,0cyclictest23871-21snmpd09:08:380
5429943,0cyclictest20196-21awk07:48:450
5429943,0cyclictest16634-21awk11:48:510
5429943,0cyclictest14732-21kworker/0:009:15:590
5429943,0cyclictest12995-21awk07:43:480
5429940,0cyclictest0-21swapper/009:06:280
29935040,0irq/33-eth0-tx-4199-21ssh11:23:332
29935040,0irq/33-eth0-tx-23483-21munin-node10:18:582
29935040,0irq/33-eth0-tx-21596-21ssh11:33:332
29935040,0irq/33-eth0-tx-2010-21ssh09:08:392
29935040,0irq/33-eth0-tx-0-21swapper/211:43:472
29935040,0irq/33-eth0-tx-0-21swapper/211:29:352
29935040,0irq/33-eth0-tx-0-21swapper/211:18:422
29935040,0irq/33-eth0-tx-0-21swapper/211:08:432
29935040,0irq/33-eth0-tx-0-21swapper/210:38:342
29935040,0irq/33-eth0-tx-0-21swapper/210:23:572
29935040,0irq/33-eth0-tx-0-21swapper/209:53:392
29935040,0irq/33-eth0-tx-0-21swapper/209:43:312
29935040,0irq/33-eth0-tx-0-21swapper/209:30:532
29935040,0irq/33-eth0-tx-0-21swapper/209:03:512
29935040,0irq/33-eth0-tx-0-21swapper/209:00:562
29935040,0irq/33-eth0-tx-0-21swapper/208:48:312
29935040,0irq/33-eth0-tx-0-21swapper/007:13:580
259940,0migration/27950-21latency_hist06:33:312
259940,0migration/27304-21kthreadcore07:38:502
259940,0migration/23001-21kthreadcore07:33:492
259940,0migration/229985-21sendmail_mailtr08:18:532
259940,0migration/226514-21kthreadcore07:28:492
259940,0migration/223052-21kthreadcore07:48:482
259940,0migration/218709-21kthreadcore08:08:502
259940,0migration/214519-21kthreadcore07:43:502
259940,0migration/211456-21kthreadcore07:18:472
249940,0watchdog/223623-21if_eth1.306:43:472
249940,0watchdog/20-21swapper/210:03:502
249940,0watchdog/20-21swapper/208:23:472
179940,0migration/19729-21kthreadcore07:38:501
179940,0migration/17754-21kthreadcore11:23:531
179940,0migration/15947-21kthreadcore06:28:501
179940,0migration/131340-21irqrtprio07:33:471
179940,0migration/130786-21cpuspeed_turbos10:03:451
179940,0migration/129809-21fschecks_time09:43:461
179940,0migration/124215-21kthreadcore11:33:491
179940,0migration/122633-21kthreadcore09:38:491
179940,0migration/121532-21switchtime06:38:561
179940,0migration/117507-21kthreadcore08:33:491
179940,0migration/116207-21fschecks_count08:08:461
169940,0watchdog/17913-21aten_r1power_cu10:28:421
169940,0watchdog/16615-21ssh11:43:301
169940,0watchdog/15993-21fschecks_time09:48:461
169940,0watchdog/131215-21aten_r1power_fr08:43:431
169940,0watchdog/127835-21kthreadcore10:18:501
169940,0watchdog/124722-21fschecks_count11:53:421
169940,0watchdog/122951-21cut09:18:511
169940,0watchdog/121426-21switchtime10:13:541
169940,0watchdog/116841-21cut07:23:511
169940,0watchdog/116386-21ssh10:33:421
169940,0watchdog/11036-21kernelversion10:23:501
169940,0watchdog/10-21swapper/111:38:461
169940,0watchdog/10-21swapper/111:18:461
169940,0watchdog/10-21swapper/111:08:461
169940,0watchdog/10-21swapper/110:48:501
169940,0watchdog/10-21swapper/109:23:511
169940,0watchdog/10-21swapper/109:13:471
149940,0migration/07859-21runrttasks06:33:250
149940,0migration/028284-21kthreadcore08:38:480
149940,0migration/023159-21kthreadcore11:13:560
149940,0migration/021891-21sh08:53:590
149940,0migration/019393-21kthreadcore10:33:490
149940,0migration/016276-21kthreadcore08:33:480
5549932,0cyclictest3599-21lldpd07:01:162
5549932,0cyclictest25176-21sensors07:03:572
5549932,0cyclictest23871-21snmpd08:16:532
5479932,0cyclictest25937-21awk10:38:471
5479932,0cyclictest20709-21awk11:13:541
5429933,0cyclictest5726-21kworker/0:007:23:410
5429933,0cyclictest5726-21kworker/0:007:18:410
5429933,0cyclictest5726-21kworker/0:007:08:430
5429933,0cyclictest14732-21kworker/0:011:28:300
5429933,0cyclictest14732-21kworker/0:010:48:550
5429933,0cyclictest14732-21kworker/0:010:28:350
5429933,0cyclictest14732-21kworker/0:010:18:410
5429933,0cyclictest14732-21kworker/0:010:13:330
5429933,0cyclictest14732-21kworker/0:009:23:370
5429933,0cyclictest14732-21kworker/0:009:18:430
5429933,0cyclictest14732-21kworker/0:008:43:310
5429933,0cyclictest14732-21kworker/0:008:13:350
5429933,0cyclictest14732-21kworker/0:008:08:450
5429933,0cyclictest10966-21kworker/0:007:58:310
5429933,0cyclictest10966-21kworker/0:007:53:310
5429932,0cyclictest5716-21awk06:53:490
5429932,0cyclictest31594-21awk10:03:480
5429932,0cyclictest31279-21cut08:23:410
5429930,0cyclictest0-21swapper/006:38:480
5429930,0cyclictest0-21swapper/006:33:430
29935030,0irq/33-eth0-tx-0-21swapper/006:48:440
249930,0watchdog/20-21swapper/207:53:512
169930,0watchdog/15673-21diskmemload11:48:421
169930,0watchdog/15673-21diskmemload09:53:381
169930,0watchdog/10-21swapper/110:08:461
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional