You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-10-09 - 05:15
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Wed Oct 09, 2024 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
416323627,0sleep10-21swapper/118:40:261
436823526,0sleep30-21swapper/318:42:483
425523526,0sleep20-21swapper/218:41:312
436523329,0sleep00-21swapper/018:42:450
45839970,0cyclictest23944-21tune2fs20:04:440
339970,0migration/331139-21switchtime20:49:563
329970,0watchdog/30-21swapper/321:55:043
29925070,0irq/32-eth0-rx-0-21swapper/321:09:543
45969960,0cyclictest0-21swapper/220:53:272
45879966,0cyclictest15933-21diskmemload22:40:341
45879960,0cyclictest0-21swapper/123:14:471
339960,0migration/310605-21sh22:30:403
329960,0watchdog/326709-21cpuspeed_turbos18:59:443
329960,0watchdog/324098-21cut20:04:443
329960,0watchdog/30-21swapper/323:44:323
329960,0watchdog/30-21swapper/323:31:443
329960,0watchdog/30-21swapper/322:44:523
329960,0watchdog/30-21swapper/322:37:003
329960,0watchdog/30-21swapper/322:28:123
329960,0watchdog/30-21swapper/322:17:363
329960,0watchdog/30-21swapper/322:12:523
329960,0watchdog/30-21swapper/322:04:363
329960,0watchdog/30-21swapper/321:49:323
329960,0watchdog/30-21swapper/321:45:323
329960,0watchdog/30-21swapper/321:34:363
329960,0watchdog/30-21swapper/321:29:403
329960,0watchdog/30-21swapper/321:25:323
329960,0watchdog/30-21swapper/321:21:403
329960,0watchdog/30-21swapper/320:59:323
329960,0watchdog/30-21swapper/320:55:243
329960,0watchdog/30-21swapper/320:14:523
329960,0watchdog/30-21swapper/319:09:403
329960,0watchdog/30-21swapper/300:05:323
329960,0watchdog/30-21swapper/300:00:003
329960,0watchdog/30-21swapper/300:00:003
29935060,0irq/33-eth0-tx-1490-21munin-node19:49:450
29925060,0irq/32-eth0-rx-5253-21kthreadcore20:34:483
29925060,0irq/32-eth0-rx-504-21port_sshpriv20:29:543
29925060,0irq/32-eth0-rx-19493-21kthreadcore20:44:503
29925060,0irq/32-eth0-rx-19031-21munin-node22:19:433
29925060,0irq/32-eth0-rx-0-21swapper/323:19:483
29925060,0irq/32-eth0-rx-0-21swapper/323:13:403
29925060,0irq/32-eth0-rx-0-21swapper/323:04:353
29925060,0irq/32-eth0-rx-0-21swapper/322:56:103
29925060,0irq/32-eth0-rx-0-21swapper/322:01:163
29925060,0irq/32-eth0-rx-0-21swapper/321:05:273
149960,0migration/027681-21sh22:40:250
149960,0migration/027552-21fschecks_time20:29:460
149960,0migration/024830-21taskset23:19:350
149960,0migration/011062-21ntp_offset19:29:530
8315050,0irq/26-0000:00:24568-21ssh22:03:142
45969955,0cyclictest0-21swapper/223:53:292
45969955,0cyclictest0-21swapper/222:19:032
45969955,0cyclictest0-21swapper/221:11:442
45969950,0cyclictest0-21swapper/221:25:462
45969950,0cyclictest0-21swapper/220:46:592
45969950,0cyclictest0-21swapper/219:44:422
45879955,0cyclictest0-21swapper/123:54:031
45879950,0cyclictest0-21swapper/122:32:511
45879950,0cyclictest0-21swapper/122:17:521
45879950,0cyclictest0-21swapper/121:29:091
339950,0migration/34428-21taskset19:25:193
339950,0migration/331878-21kthreadcore19:44:523
339950,0migration/328669-21kthreadcore19:19:523
339950,0migration/318237-21latency_hist23:34:323
329950,0watchdog/331489-21if_eth120:09:483
329950,0watchdog/320579-21if_err_eth1.319:39:483
329950,0watchdog/313232-21perl20:19:443
329950,0watchdog/310338-21ntpq21:14:523
329950,0watchdog/30-21swapper/323:49:323
329950,0watchdog/30-21swapper/323:39:563
329950,0watchdog/30-21swapper/323:26:083
329950,0watchdog/30-21swapper/322:59:523
329950,0watchdog/30-21swapper/322:39:403
329950,0watchdog/30-21swapper/321:39:403
329950,0watchdog/30-21swapper/320:44:283
329950,0watchdog/30-21swapper/320:24:403
329950,0watchdog/30-21swapper/319:59:523
329950,0watchdog/30-21swapper/319:56:163
329950,0watchdog/30-21swapper/319:50:443
329950,0watchdog/30-21swapper/319:34:523
329950,0watchdog/30-21swapper/319:29:443
329950,0watchdog/30-21swapper/319:06:363
329950,0watchdog/30-21swapper/318:54:373
329950,0watchdog/30-21swapper/318:51:493
329950,0watchdog/30-21swapper/300:09:403
29935050,0irq/33-eth0-tx-9791-21ssh23:29:402
29935050,0irq/33-eth0-tx-8611-21munin-node19:09:450
29935050,0irq/33-eth0-tx-30789-21fschecks_time21:29:462
29935050,0irq/33-eth0-tx-25305-21munin-node22:04:552
29935050,0irq/33-eth0-tx-24897-21kthreadcore22:59:492
29935050,0irq/33-eth0-tx-16160-21ssh20:44:340
29935050,0irq/33-eth0-tx-14416-21ssh22:54:452
29935050,0irq/33-eth0-tx-0-21swapper/223:28:592
29935050,0irq/33-eth0-tx-0-21swapper/223:15:302
29935050,0irq/33-eth0-tx-0-21swapper/222:22:452
29935050,0irq/33-eth0-tx-0-21swapper/221:56:272
29935050,0irq/33-eth0-tx-0-21swapper/018:47:360
29925050,0irq/32-eth0-rx-6386-21runrttasks18:44:433
29925050,0irq/32-eth0-rx-0-21swapper/323:54:423
29925050,0irq/32-eth0-rx-0-21swapper/323:16:173
29925050,0irq/32-eth0-rx-0-21swapper/322:49:553
259950,0migration/27210-21kthreadcore00:04:492
259950,0migration/26162-21kthreadcore19:49:502
259950,0migration/232516-21fschecks_count21:49:452
259950,0migration/230899-21kthreadcore23:39:502
259950,0migration/227352-21timerwakeupswit22:39:582
259950,0migration/216067-21kthreadcore20:19:492
259950,0migration/215601-21kthreadcore22:34:492
259950,0migration/210689-21sh23:48:082
259950,0migration/210095-21munin-node20:39:472
249950,0watchdog/224318-21hwlatdetect21:44:482
249950,0watchdog/224097-21sed20:04:442
179950,0migration/18723-21taskset23:27:171
179950,0migration/1385-21kthreadcore19:24:501
179950,0migration/13085-21kthreadcore20:54:471
179950,0migration/130592-21taskset23:02:501
179950,0migration/127151-21sh21:08:461
179950,0migration/121233-21kthreadcore19:59:521
179950,0migration/120522-21kthreadcore21:39:501
179950,0migration/117483-21kthreadcore22:34:491
179950,0migration/117162-21munin-node20:44:461
179950,0migration/111926-21sh00:06:421
169950,0watchdog/127784-21aten_r1power_en21:09:441
169950,0watchdog/119214-21diskstats22:19:441
169950,0watchdog/115938-21ssh21:58:081
149950,0migration/08395-21sh20:56:040
149950,0migration/07877-21kthreadcore00:04:490
149950,0migration/06507-21kthreadcore23:44:490
149950,0migration/06143-21kthreadcore23:24:500
149950,0migration/05572-21munin-node21:49:500
149950,0migration/05428-21munin-node23:04:540
149950,0migration/05089-21irqrtprio22:29:490
149950,0migration/03521-21kthreadcore22:09:500
149950,0migration/032658-21kthreadcore22:24:510
149950,0migration/029881-21taskset23:00:450
149950,0migration/029573-21kthreadcore21:09:500
149950,0migration/027786-21kthreadcore19:19:520
149950,0migration/024540-21taskset22:03:100
149950,0migration/021332-21kthreadcore21:04:500
149950,0migration/020000-21munin-node22:19:470
149950,0migration/018779-21missed_timers21:19:510
149950,0migration/01751-21kthreadcore22:44:490
149950,0migration/015339-21idleruntime22:54:490
149950,0migration/014825-21taskset23:11:020
149950,0migration/013668-21irqrtprio19:34:500
149950,0migration/010875-21kthreadcore21:54:500
8315040,0irq/26-0000:00:0-21swapper/222:44:322
8315040,0irq/26-0000:00:0-21swapper/222:09:492
8315040,0irq/26-0000:00:0-21swapper/221:34:442
45969944,0cyclictest0-21swapper/219:04:492
45969943,0cyclictest5853-21awk18:44:472
45969943,0cyclictest3599-21lldpd20:25:302
45969943,0cyclictest3599-21lldpd19:42:002
45969943,0cyclictest3599-21lldpd19:13:292
45969943,0cyclictest3599-21lldpd19:01:252
45969943,0cyclictest3599-21lldpd18:59:252
45969943,0cyclictest2886-21awk20:34:452
45969943,0cyclictest20010-21awk21:04:482
45879943,0cyclictest500-21awk21:49:461
45879943,0cyclictest3599-21lldpd22:45:321
45879943,0cyclictest3599-21lldpd19:12:281
45879943,0cyclictest27859-21awk19:44:491
45879943,0cyclictest26407-21awk22:04:481
45879943,0cyclictest2589-21awk19:49:471
45879943,0cyclictest23871-21snmpd20:41:141
45879943,0cyclictest23871-21snmpd19:17:441
45879943,0cyclictest23871-21snmpd00:02:301
45879943,0cyclictest23871-21snmpd00:02:301
45879943,0cyclictest2325-21awk19:04:491
45879943,0cyclictest20660-21awk19:39:491
45879943,0cyclictest13914-21awk21:19:471
45879943,0cyclictest10853-21awk23:29:451
45839943,0cyclictest3599-21lldpd22:34:320
45839943,0cyclictest3599-21lldpd22:15:020
45839943,0cyclictest3262-21irqbalance23:55:060
45839943,0cyclictest31429-21awk20:09:480
45839943,0cyclictest23871-21snmpd23:44:000
45839943,0cyclictest20355-21awk18:54:480
45839943,0cyclictest19461-21awk23:34:480
45839943,0cyclictest10869-21awk23:29:450
45839940,0cyclictest0-21swapper/000:09:350
29935040,0irq/33-eth0-tx-24833-21ssh23:19:352
29935040,0irq/33-eth0-tx-18360-21munin-node23:34:502
29935040,0irq/33-eth0-tx-0-21swapper/223:59:382
29935040,0irq/33-eth0-tx-0-21swapper/223:59:382
29935040,0irq/33-eth0-tx-0-21swapper/223:09:352
29935040,0irq/33-eth0-tx-0-21swapper/223:04:342
29935040,0irq/33-eth0-tx-0-21swapper/222:50:132
29935040,0irq/33-eth0-tx-0-21swapper/222:30:462
29935040,0irq/33-eth0-tx-0-21swapper/222:24:432
29935040,0irq/33-eth0-tx-0-21swapper/221:39:482
29935040,0irq/33-eth0-tx-0-21swapper/221:19:522
29935040,0irq/33-eth0-tx-0-21swapper/200:10:222
29935040,0irq/33-eth0-tx-0-21swapper/020:39:420
29935040,0irq/33-eth0-tx-0-21swapper/018:59:490
259940,0migration/24185-21munin-node21:14:422
259940,0migration/230365-21kthreadcore20:29:512
259940,0migration/223263-21kthreadcore23:54:482
259940,0migration/214759-21kthreadcore19:34:512
259940,0migration/21403-21cpuspeed_turbos20:54:422
249940,0watchdog/28332-21grep20:14:522
249940,0watchdog/230562-21cpu20:09:442
249940,0watchdog/20-21swapper/220:59:522
249940,0watchdog/20-21swapper/219:19:482
179940,0migration/12491-21runrttasks22:27:241
179940,0migration/117225-21kthreadcore20:19:491
179940,0migration/115720-21taskset19:59:121
179940,0migration/114958-21kthreadcore18:49:501
179940,0migration/113110-21taskset00:09:421
179940,0migration/112616-21taskset22:50:331
169940,0watchdog/132611-21iwlist23:04:481
169940,0watchdog/127961-21ethtool23:39:481
169940,0watchdog/126256-21ssh23:38:401
169940,0watchdog/125380-21grep20:49:441
169940,0watchdog/124096-21grep20:04:441
169940,0watchdog/123009-21munin-node19:19:481
169940,0watchdog/122764-21kthreadcore23:54:481
169940,0watchdog/122096-21kthreadcore21:59:521
169940,0watchdog/110169-21ssh23:46:361
169940,0watchdog/10-21swapper/123:19:481
169940,0watchdog/10-21swapper/122:09:321
169940,0watchdog/10-21swapper/121:46:161
169940,0watchdog/10-21swapper/120:34:401
169940,0watchdog/10-21swapper/120:24:521
149940,0migration/08866-21kthreadcore21:34:480
149940,0migration/032673-21kthreadcore21:29:500
149940,0migration/021298-21kthreadcore19:14:520
149940,0migration/014506-21kthreadcore23:49:510
149940,0migration/010784-21fschecks_time20:59:470
8315030,0irq/26-0000:00:0-21swapper/219:54:322
45969932,0cyclictest5746-21perl19:29:472
45969932,0cyclictest3599-21lldpd19:26:592
45969932,0cyclictest22780-21perf19:19:312
45969932,0cyclictest16006-21basename19:59:442
45879932,0cyclictest6823-21grep21:34:441
45879932,0cyclictest6645-21cut21:14:491
45879932,0cyclictest6229-21awk19:29:491
45879932,0cyclictest19718-21fschecks_count18:54:461
45879932,0cyclictest13325-21awk19:34:491
45839933,0cyclictest28732-21kworker/0:021:39:440
45839933,0cyclictest23704-21kworker/0:019:04:330
45839933,0cyclictest23246-21kworker/0:223:59:400
45839933,0cyclictest23246-21kworker/0:223:59:400
45839933,0cyclictest23246-21kworker/0:223:15:160
45839933,0cyclictest23246-21kworker/0:222:49:320
45839933,0cyclictest23246-21kworker/0:222:05:160
45839933,0cyclictest23246-21kworker/0:221:45:200
45839933,0cyclictest12250-21kworker/0:120:49:520
45839933,0cyclictest12250-21kworker/0:120:24:440
45839933,0cyclictest12250-21kworker/0:119:54:430
45839933,0cyclictest12250-21kworker/0:119:44:350
45839933,0cyclictest0-21swapper/019:39:400
45839932,0cyclictest6249-21cut20:34:470
45839932,0cyclictest5263-21awk21:14:470
45839932,0cyclictest28732-21kworker/0:021:24:440
45839932,0cyclictest16584-21grep20:19:490
45839932,0cyclictest16068-21aten_r1power_fr19:59:440
45839930,0cyclictest5249-21munin-node20:14:540
329930,0watchdog/30-21swapper/319:15:003
29935030,0irq/33-eth0-tx-0-21swapper/019:24:430
29935030,0irq/33-eth0-tx-0-21swapper/018:49:440
249930,0watchdog/212055-21munin-node18:49:452
179930,0migration/131224-21kthreadcore20:29:511
179930,0migration/129962-21sh21:29:391
169930,0watchdog/10-21swapper/123:09:441
169930,0watchdog/10-21swapper/122:54:481
169930,0watchdog/10-21swapper/120:59:521
169930,0watchdog/10-21swapper/120:14:521
169930,0watchdog/10-21swapper/120:09:481
169930,0watchdog/10-21swapper/118:59:441
45879921,0cyclictest0-21swapper/118:44:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional