You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-02-06 - 13:37

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.229-rt200 (Profile)

Latency plot of system in rack #1, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -n -a -t3 -p99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack1slot2.osadl.org (updated Mon Feb 06, 2023 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1487124028,0sleep30-21swapper/306:25:283
805723934,0sleep20-21swapper/206:21:022
1468923927,0sleep10-21swapper/106:23:181
1489123625,0sleep00-21swapper/006:25:410
8315060,0irq/26-0000:00:29499-21turbostat10:50:582
339960,0migration/37338-21sh09:44:123
339960,0migration/332630-21sh08:27:433
339960,0migration/320696-21taskset10:46:083
339960,0migration/319756-21sh11:21:073
339960,0migration/316610-21gltestperf11:01:173
339960,0migration/313830-21sh10:57:403
339960,0migration/312681-21sh09:29:573
329960,0watchdog/35296-21perl08:36:153
329960,0watchdog/313972-21ssh09:07:193
28015060,0irq/29-eth0-tx-0-21swapper/210:11:132
28005060,0irq/28-eth0-rx-8990-21python10:36:413
28005060,0irq/28-eth0-rx-3346-21snmpd08:52:273
28005060,0irq/28-eth0-rx-29066-21chrt06:33:153
28005060,0irq/28-eth0-rx-28082-21python10:11:253
28005060,0irq/28-eth0-rx-2336-21ssh08:32:003
28005060,0irq/28-eth0-rx-18547-21cut10:26:223
28005060,0irq/28-eth0-rx-16429-21users11:51:363
28005060,0irq/28-eth0-rx-14710-21iostat_ios09:31:213
149939965,0cyclictest5513-21pluginstate09:41:260
149939960,0cyclictest0-21swapper/010:04:090
149960,0migration/032622-21latency_hist11:45:580
149960,0migration/032529-21sh10:31:390
149960,0migration/023356-21sh11:40:390
8315050,0irq/26-0000:00:24601-21pmu-power09:16:212
8315050,0irq/26-0000:00:18104-21crond08:21:002
339950,0migration/34288-21kthreadcore11:11:213
339950,0migration/330868-21runrttasks09:39:523
339950,0migration/330045-21cpuspeed_turbos10:51:153
339950,0migration/329791-21missed_timers11:41:243
339950,0migration/328000-21munin-node08:56:133
339950,0migration/326671-21kthreadcore07:16:263
339950,0migration/32516-21sh09:59:353
339950,0migration/323551-21kthreadcore09:16:183
339950,0migration/321475-21latency_hist10:10:583
339950,0migration/318777-21kthreadcore07:11:223
339950,0migration/318442-21kthreadcore08:46:183
339950,0migration/31625-21kthreadcore07:21:233
339950,0migration/316016-21sh09:11:163
339950,0migration/313261-21kthreadcore07:51:213
339950,0migration/312424-21sh10:21:233
339950,0migration/310814-21kthreadcore07:06:243
329950,0watchdog/35460-21ssh09:03:353
329950,0watchdog/34358-21diskstats08:11:193
329950,0watchdog/328310-21kthreadcore06:56:203
329950,0watchdog/323614-21needreboot09:51:233
329950,0watchdog/313902-21ssh08:42:113
329950,0watchdog/31351-21open_inodes09:21:273
329950,0watchdog/30-21swapper/310:01:233
28015050,0irq/29-eth0-tx-4444-21load10:16:252
28015050,0irq/29-eth0-tx-4402-21munin-node09:26:192
28015050,0irq/29-eth0-tx-3380-21ssh11:26:512
28015050,0irq/29-eth0-tx-32158-21grep08:56:192
28015050,0irq/29-eth0-tx-23543-21ssh08:48:192
28015050,0irq/29-eth0-tx-18773-21ntp_states10:06:242
28015050,0irq/29-eth0-tx-14712-21munin-node09:46:302
28015050,0irq/29-eth0-tx-11381-21munin-node07:31:192
28015050,0irq/29-eth0-tx-0-21swapper/211:47:152
28015050,0irq/29-eth0-tx-0-21swapper/211:44:012
28015050,0irq/29-eth0-tx-0-21swapper/211:32:202
28015050,0irq/29-eth0-tx-0-21swapper/211:16:232
28015050,0irq/29-eth0-tx-0-21swapper/210:52:352
28015050,0irq/29-eth0-tx-0-21swapper/210:45:452
28015050,0irq/29-eth0-tx-0-21swapper/210:36:192
28015050,0irq/29-eth0-tx-0-21swapper/210:31:402
28015050,0irq/29-eth0-tx-0-21swapper/210:01:272
28015050,0irq/29-eth0-tx-0-21swapper/210:00:442
28005050,0irq/28-eth0-rx-8798-21tr09:46:203
28005050,0irq/28-eth0-rx-561-21latency_hist07:01:003
28005050,0irq/28-eth0-rx-19551-21sh10:44:073
28005050,0irq/28-eth0-rx-0-21swapper/311:46:273
28005050,0irq/28-eth0-rx-0-21swapper/311:31:233
28005050,0irq/28-eth0-rx-0-21swapper/311:16:203
28005050,0irq/28-eth0-rx-0-21swapper/310:31:203
28005050,0irq/28-eth0-rx-0-21swapper/306:52:323
259950,0migration/27458-21kthreadcore06:41:232
259950,0migration/222489-21latency_hist09:35:592
259950,0migration/219483-21kthreadcore09:51:212
259950,0migration/218520-21runrttasks07:36:082
249950,0watchdog/231772-21ssh11:07:022
249950,0watchdog/230974-21kthreadcore07:41:232
249950,0watchdog/228311-21kthreadcore06:56:202
19150,0ktimersoftd/10-21swapper/110:48:241
179950,0migration/18622-21sh09:46:191
179950,0migration/1819-21kthreadcore10:51:211
179950,0migration/15450-21kthreadcore07:01:201
179950,0migration/13468-21kthreadcore11:11:211
179950,0migration/1300-21kthreadcore08:56:191
179950,0migration/126814-21kthreadcore07:16:261
179950,0migration/125002-21kthreadcore10:11:201
179950,0migration/113926-21python10:21:271
169950,0watchdog/15441-21kthreadcore10:36:351
169950,0watchdog/116905-21kthreadcore09:31:231
149959955,0cyclictest0-21swapper/209:33:462
149959950,0cyclictest0-21swapper/211:23:582
149959950,0cyclictest0-21swapper/211:03:382
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional