You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-07-29 - 04:23

Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz, Linux 4.4.229-rt200 (Profile)

Latency plot of system in rack #1, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 100 highest latencies:
System rack1slot2.osadl.org (updated Thu Jul 29, 2021 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291450580,0irq/31-eth10-21swapper/119:20:251
291450580,0irq/31-eth10-21swapper/119:20:251
1528523723,0sleep00-21swapper/019:18:390
1528523723,0sleep00-21swapper/019:18:390
1527823627,0sleep30-21swapper/319:18:343
1527823627,0sleep30-21swapper/319:18:343
1530923526,0sleep20-21swapper/219:18:572
1530923526,0sleep20-21swapper/219:18:572
15724993333,0cyclictest18390-21kworker/0:222:32:390
157299976,0cyclictest0-21swapper/122:42:311
28425060,0irq/33-eth0-tx-6274-21munin-node20:08:230
157439966,0cyclictest0-21swapper/321:48:153
157439965,0cyclictest3111-21irqbalance21:58:093
157379965,0cyclictest21960-21ssh21:33:512
157379960,0cyclictest0-21swapper/200:40:122
157299960,0cyclictest0-21swapper/123:42:541
157249960,0cyclictest0-21swapper/022:07:010
8555250,0sleep08777-21cut20:12:590
4150,0ktimersoftd/00-21swapper/023:19:320
4150,0ktimersoftd/00-21swapper/022:19:490
4150,0ktimersoftd/00-21swapper/000:10:580
35150,0ktimersoftd/30-21swapper/322:19:493
339950,0migration/3998-21switchtime23:03:283
329950,0watchdog/30-21swapper/323:58:143
28425050,0irq/33-eth0-tx-32100-21munin-node21:48:122
28425050,0irq/33-eth0-tx-28501-21munin-node19:48:170
28425050,0irq/33-eth0-tx-27008-21chrt00:43:212
28425050,0irq/33-eth0-tx-23301-21perl22:53:142
28425050,0irq/33-eth0-tx-19049-21ssh21:31:442
28425050,0irq/33-eth0-tx-18451-21munin-node19:28:190
28425050,0irq/33-eth0-tx-1736-21ssh23:40:592
28425050,0irq/33-eth0-tx-16361-21munin-node20:28:170
28425050,0irq/33-eth0-tx-11524-21diskmemload23:07:402
28425050,0irq/33-eth0-tx-0-21swapper/223:43:482
28425050,0irq/33-eth0-tx-0-21swapper/222:05:272
28425050,0irq/33-eth0-tx-0-21swapper/200:26:452
28425050,0irq/33-eth0-tx-0-21swapper/200:03:182
28425050,0irq/33-eth0-tx-0-21swapper/019:58:290
28425050,0irq/33-eth0-tx-0-21swapper/019:34:590
28415050,0irq/32-eth0-rx-5471-21tune2fs22:33:233
28415050,0irq/32-eth0-rx-27510-21interrupts23:33:163
28415050,0irq/32-eth0-rx-13574-21perl23:18:133
259950,0migration/229866-21munin-node20:53:162
259950,0migration/218661-21unixbench_multi22:08:342
259950,0migration/218572-21unixbench-2d23:58:252
259950,0migration/217950-21switchtime20:28:242
259950,0migration/214575-21timerandwakeup23:18:232
259950,0migration/211588-21sh22:39:122
249950,0watchdog/214091-21df_inode22:43:132
19150,0ktimersoftd/10-21swapper/100:10:581
179950,0migration/132646-21munin-node19:53:231
179950,0migration/124229-21sh00:05:271
179950,0migration/123058-21sh22:13:561
179950,0migration/11787-21taskset21:48:411
157439950,0cyclictest0-21swapper/321:56:463
157379955,0cyclictest0-21swapper/221:58:172
157379955,0cyclictest0-21swapper/200:33:262
157379955,0cyclictest0-21swapper/200:14:402
157379954,0cyclictest5395-21snmpd00:49:392
157379950,0cyclictest0-21swapper/223:11:362
157379950,0cyclictest0-21swapper/200:08:402
157299955,0cyclictest0-21swapper/121:30:561
157299955,0cyclictest0-21swapper/100:44:141
157299955,0cyclictest0-21swapper/100:31:271
157299950,0cyclictest0-21swapper/123:14:411
157299950,0cyclictest0-21swapper/121:47:301
157249955,0cyclictest5395-21snmpd21:58:090
157249955,0cyclictest0-21swapper/023:05:020
157249955,0cyclictest0-21swapper/022:51:190
157249955,0cyclictest0-21swapper/021:38:380
157249950,0cyclictest0-21swapper/023:54:520
157249950,0cyclictest0-21swapper/021:56:180
157249950,0cyclictest0-21swapper/021:11:410
149950,0migration/07636-21sh22:35:220
149950,0migration/0665-21latency_hist22:28:000
149950,0migration/05692-21sh23:44:370
149950,0migration/04953-21munin-node23:08:150
149950,0migration/031725-21fschecks_time19:53:130
149950,0migration/023417-21taskset21:37:360
149950,0migration/023074-21sh00:38:410
149950,0migration/021054-21munin-node22:13:130
149950,0migration/01875-21fschecks_count21:03:140
149950,0migration/018013-21latency_hist22:48:000
149950,0migration/016810-21cpu23:58:130
149950,0migration/015519-21timerwakeupswit20:23:230
149950,0migration/014373-21taskset00:29:350
149950,0migration/011250-21latency_hist20:17:590
8305040,0irq/26-0000:00:0-21swapper/222:58:202
8305040,0irq/26-0000:00:0-21swapper/220:58:262
339940,0migration/34409-21fschecks_time20:03:143
339940,0migration/331690-21gltestperf23:38:143
339940,0migration/33085-21taskset19:59:023
339940,0migration/329515-21munin-node21:43:243
339940,0migration/32405-21munin-node21:03:163
339940,0migration/323940-21cpu21:38:143
339940,0migration/310278-21munin-node20:13:253
329940,0watchdog/326100-21perl00:43:143
329940,0watchdog/3222672chrt00:38:223
329940,0watchdog/318329-21munin-node22:48:173
329940,0watchdog/317398-21iostat20:28:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional