You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-10-04 - 02:14

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.229-rt200 (Profile)

Latency plot of system in rack #1, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -n -a -t3 -p99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack1slot2.osadl.org (updated Tue Oct 04, 2022 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
335724334,0sleep10-21swapper/118:38:301
348424231,0sleep30-21swapper/318:39:563
335323624,0sleep20-21swapper/218:38:272
336123026,0sleep00-21swapper/018:38:330
36839988,0cyclictest14075-21diskmemload20:41:422
8315060,0irq/26-0000:00:25192-21ls18:56:143
36839963,0cyclictest22815-21awk19:16:372
36729966,0cyclictest0-21swapper/123:45:531
36729960,0cyclictest0-21swapper/122:06:151
36729960,0cyclictest0-21swapper/121:18:481
36719960,0cyclictest27678-21chrt21:24:200
36719960,0cyclictest0-21swapper/019:46:350
36719960,0cyclictest0-21swapper/018:41:380
35160,0ktimersoftd/30-21swapper/323:26:143
339960,0migration/3430-21sh22:02:473
339960,0migration/332624-21munin-node20:31:293
339960,0migration/32913-21munin-node20:06:423
339960,0migration/324716-21sh23:50:363
339960,0migration/322646-21kthreadcore00:06:343
339960,0migration/319298-21sh23:10:583
28015060,0irq/29-eth0-tx-28938-21munin-node19:21:430
28005060,0irq/28-eth0-rx-8950-21awk22:46:353
28005060,0irq/28-eth0-rx-8657-21df_inode20:56:313
28005060,0irq/28-eth0-rx-2977-21perl21:11:303
28005060,0irq/28-eth0-rx-14278-21awk21:51:363
28005060,0irq/28-eth0-rx-0-21swapper/322:07:143
149960,0migration/08758-21timerandwakeup21:11:420
149960,0migration/01481-21sh22:05:220
149960,0migration/013887-21ntp_offset22:46:410
149960,0migration/013473-21sh21:35:480
36839954,0cyclictest15496-21kworker/2:220:59:112
36839950,0cyclictest14075-21diskmemload23:37:422
36729950,0cyclictest0-21swapper/121:40:431
36729950,0cyclictest0-21swapper/121:34:341
36729950,0cyclictest0-21swapper/120:52:001
36729950,0cyclictest0-21swapper/119:26:421
36719955,0cyclictest0-21swapper/022:41:350
36719950,0cyclictest0-21swapper/023:25:350
339950,0migration/38981-21kthreadcore19:06:373
339950,0migration/35436-21kthreadcore18:41:353
339950,0migration/33379-21kthreadcore19:46:343
339950,0migration/331551-21unixbench_multi20:26:443
339950,0migration/327802-21smart_sdb20:01:413
339950,0migration/326892-21munin-node21:06:313
339950,0migration/322118-21kthreadcore22:51:353
339950,0migration/319990-21munin-node22:31:413
339950,0migration/319841-21aten2_r1power_v23:11:303
339950,0migration/317893-21latency_hist20:21:143
329950,0watchdog/331272-21wc23:36:133
329950,0watchdog/331149-21memory19:41:383
329950,0watchdog/330010-21ssh21:43:013
329950,0watchdog/322591-21latency_hist23:31:133
329950,0watchdog/30-21swapper/322:41:013
329950,0watchdog/30-21swapper/321:46:373
28015050,0irq/29-eth0-tx-9265-21if_eth021:51:322
28015050,0irq/29-eth0-tx-5555-21kthreadcore22:06:412
28015050,0irq/29-eth0-tx-32606-21hddtemp_smartct21:46:332
28015050,0irq/29-eth0-tx-25751-21ssh00:07:402
28015050,0irq/29-eth0-tx-25243-21munin-node23:51:352
28015050,0irq/29-eth0-tx-23971-21ssh23:48:422
28015050,0irq/29-eth0-tx-23033-21munin-node20:46:402
28015050,0irq/29-eth0-tx-2281-21pmu-power22:21:352
28015050,0irq/29-eth0-tx-21631-21munin-node20:01:390
28015050,0irq/29-eth0-tx-12076-21ssh22:27:562
28015050,0irq/29-eth0-tx-0-21swapper/019:26:290
28005050,0irq/28-eth0-rx-32212-21idleruntime22:41:343
28005050,0irq/28-eth0-rx-23154-21egrep20:46:303
28005050,0irq/28-eth0-rx-14757-21diskstats20:41:353
28005050,0irq/28-eth0-rx-11158-21munin-node18:46:383
28005050,0irq/28-eth0-rx-0-21swapper/323:56:133
28005050,0irq/28-eth0-rx-0-21swapper/323:44:573
28005050,0irq/28-eth0-rx-0-21swapper/323:21:443
28005050,0irq/28-eth0-rx-0-21swapper/323:19:483
28005050,0irq/28-eth0-rx-0-21swapper/322:57:443
28005050,0irq/28-eth0-rx-0-21swapper/322:21:523
28005050,0irq/28-eth0-rx-0-21swapper/322:17:033
28005050,0irq/28-eth0-rx-0-21swapper/322:14:103
28005050,0irq/28-eth0-rx-0-21swapper/321:37:273
28005050,0irq/28-eth0-rx-0-21swapper/321:21:563
28005050,0irq/28-eth0-rx-0-21swapper/321:17:373
28005050,0irq/28-eth0-rx-0-21swapper/321:01:413
28005050,0irq/28-eth0-rx-0-21swapper/320:51:373
259950,0migration/28420-21kthreadcore20:11:372
259950,0migration/2828-21kthreadcore23:16:362
259950,0migration/26941-21sh20:53:442
259950,0migration/230298-21taskset21:43:412
259950,0migration/224379-21sh21:58:252
259950,0migration/211104-21aten2_r1power_c22:11:322
249950,0watchdog/23485-21munin-node21:11:342
179950,0migration/19379-21kthreadcore22:26:351
179950,0migration/128446-21runrttasks19:18:221
179950,0migration/126837-21sh23:12:461
179950,0migration/125438-21sh00:06:521
179950,0migration/120239-21kthreadcore23:46:401
179950,0migration/117329-21timerwakeupswit19:31:491
169950,0watchdog/130593-21kthreadcore21:26:331
169950,0watchdog/113436-21port_sshpriv19:06:421
149950,0migration/08767-21fschecks_count21:51:300
149950,0migration/05269-21kthreadcore22:06:400
149950,0migration/03750-21kthreadcore23:56:360
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional