You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-13 - 04:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Tue May 13, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2849123828,0sleep10-21swapper/118:29:481
2828123827,0sleep30-21swapper/318:27:173
2853423727,0sleep20-21swapper/218:30:172
2855623521,0sleep00-21swapper/018:30:310
28722991414,0cyclictest2777-21sensors18:32:011
35170,0ktimersoftd/318051-21grep21:51:543
329970,0watchdog/327869-21hostname22:16:503
329970,0watchdog/30-21swapper/322:41:583
329970,0watchdog/30-21swapper/322:40:423
329970,0watchdog/30-21swapper/322:14:383
29925070,0irq/32-eth0-rx-0-21swapper/323:07:013
29925070,0irq/32-eth0-rx-0-21swapper/320:53:523
29925070,0irq/32-eth0-rx-0-21swapper/320:26:583
8315060,0irq/26-0000:00:23390-21kthreadcore18:46:573
339960,0migration/326410-21kthreadcore22:51:573
339960,0migration/326170-21unixbench_multi20:57:063
339960,0migration/31630-21munin-node23:16:503
339960,0migration/316157-21sh21:31:553
339960,0migration/315120-21sh23:02:533
329960,0watchdog/326369-21sort23:11:543
329960,0watchdog/3141832sleep319:03:273
329960,0watchdog/311887-21grep21:46:583
329960,0watchdog/30-21swapper/323:57:343
329960,0watchdog/30-21swapper/323:43:383
329960,0watchdog/30-21swapper/323:22:503
329960,0watchdog/30-21swapper/322:47:223
329960,0watchdog/30-21swapper/322:32:023
329960,0watchdog/30-21swapper/321:41:033
329960,0watchdog/30-21swapper/321:13:393
329960,0watchdog/30-21swapper/321:01:513
329960,0watchdog/30-21swapper/320:50:353
329960,0watchdog/30-21swapper/320:42:273
329960,0watchdog/30-21swapper/320:11:593
329960,0watchdog/30-21swapper/319:07:233
29935060,0irq/33-eth0-tx-18415-21munin-node19:32:070
29925060,0irq/32-eth0-rx-4505-21diskstats22:21:563
29925060,0irq/32-eth0-rx-17432-21python19:27:023
29925060,0irq/32-eth0-rx-0-21swapper/323:32:333
29925060,0irq/32-eth0-rx-0-21swapper/323:29:393
29925060,0irq/32-eth0-rx-0-21swapper/322:56:533
29925060,0irq/32-eth0-rx-0-21swapper/322:28:373
29925060,0irq/32-eth0-rx-0-21swapper/322:10:543
29925060,0irq/32-eth0-rx-0-21swapper/321:59:353
29925060,0irq/32-eth0-rx-0-21swapper/321:25:183
29925060,0irq/32-eth0-rx-0-21swapper/320:37:003
29925060,0irq/32-eth0-rx-0-21swapper/320:37:003
29925060,0irq/32-eth0-rx-0-21swapper/320:31:563
287229960,0cyclictest0-21swapper/123:01:551
287219960,0cyclictest0-21swapper/023:57:490
149960,0migration/0912-21sh23:15:370
149960,0migration/02027-21munin-node22:01:520
4150,0ktimersoftd/00-21swapper/023:17:110
339950,0migration/33196-21taskset18:32:393
339950,0migration/330831-21kthreadcore19:36:553
339950,0migration/326532-21sh23:47:383
339950,0migration/325720-21kthreadcore21:16:593
339950,0migration/323787-21kthreadcore19:56:593
339950,0migration/310845-21kthreadcore21:26:583
329950,0watchdog/30-21swapper/323:53:103
329950,0watchdog/30-21swapper/321:06:473
329950,0watchdog/30-21swapper/320:01:393
329950,0watchdog/30-21swapper/319:51:553
329950,0watchdog/30-21swapper/319:47:033
329950,0watchdog/30-21swapper/319:41:513
329950,0watchdog/30-21swapper/319:31:393
329950,0watchdog/30-21swapper/319:26:113
329950,0watchdog/30-21swapper/319:16:433
329950,0watchdog/30-21swapper/319:12:593
329950,0watchdog/30-21swapper/318:57:153
329950,0watchdog/30-21swapper/318:52:033
329950,0watchdog/30-21swapper/318:37:353
29935050,0irq/33-eth0-tx-32632-21ssh21:57:412
29935050,0irq/33-eth0-tx-23507-21ssh20:37:092
29935050,0irq/33-eth0-tx-23507-21ssh20:37:092
29935050,0irq/33-eth0-tx-22056-21munin-node19:56:570
29935050,0irq/33-eth0-tx-11032-21munin-node18:42:040
29935050,0irq/33-eth0-tx-0-21swapper/223:13:202
29935050,0irq/33-eth0-tx-0-21swapper/223:06:572
29935050,0irq/33-eth0-tx-0-21swapper/221:49:172
29935050,0irq/33-eth0-tx-0-21swapper/220:33:212
29935050,0irq/33-eth0-tx-0-21swapper/020:32:000
29935050,0irq/33-eth0-tx-0-21swapper/020:22:020
29935050,0irq/33-eth0-tx-0-21swapper/019:06:540
29925050,0irq/32-eth0-rx-29300-21kthreadcore20:21:563
29925050,0irq/32-eth0-rx-13816-21kthreadcore18:42:003
29925050,0irq/32-eth0-rx-0-21swapper/323:36:523
29925050,0irq/32-eth0-rx-0-21swapper/322:01:593
29925050,0irq/32-eth0-rx-0-21swapper/321:41:463
29925050,0irq/32-eth0-rx-0-21swapper/320:16:493
29925050,0irq/32-eth0-rx-0-21swapper/320:07:003
287239955,0cyclictest0-21swapper/221:12:582
287239955,0cyclictest0-21swapper/220:59:342
287239955,0cyclictest0-21swapper/220:16:512
287239954,0cyclictest26798-21sed20:41:572
287239954,0cyclictest0-21swapper/223:26:592
287239950,0cyclictest0-21swapper/221:11:142
287229955,0cyclictest0-21swapper/120:11:581
287229950,0cyclictest0-21swapper/122:51:591
287219955,0cyclictest0-21swapper/023:30:230
287219955,0cyclictest0-21swapper/023:22:300
287219955,0cyclictest0-21swapper/021:01:230
287219955,0cyclictest0-21swapper/020:37:070
287219955,0cyclictest0-21swapper/020:37:070
287219955,0cyclictest0-21swapper/018:36:400
287219950,0cyclictest0-21swapper/023:38:160
287219950,0cyclictest0-21swapper/023:09:060
287219950,0cyclictest0-21swapper/022:00:590
287219950,0cyclictest0-21swapper/021:31:210
259950,0migration/223517-21kthreadcore22:31:592
259950,0migration/212486-21sh22:42:062
249950,0watchdog/28385-21users23:17:062
249950,0watchdog/28277-21fschecks_count19:46:552
249950,0watchdog/27130-21cut23:36:582
249950,0watchdog/230070-21python21:37:032
249950,0watchdog/224105-21tr22:11:582
19150,0ktimersoftd/10-21swapper/123:17:111
179950,0migration/132322-21munin-node21:57:071
179950,0migration/120911-21taskset21:15:181
179950,0migration/120586-21runrttasks22:30:391
179950,0migration/11662-21pmu-power23:52:031
179950,0migration/115172-21munin-node21:31:521
169950,0watchdog/114183-21unixbench_singl19:47:071
169950,0watchdog/111376-40yum-updatesd-he21:07:311
149950,0migration/09362-21fschecks_time21:46:540
149950,0migration/06172-21sh21:26:230
149950,0migration/05916-21kthreadcore19:21:570
149950,0migration/05096-21taskset22:39:380
149950,0migration/04978-21fschecks_time20:06:550
149950,0migration/032400-21fschecks_count22:56:550
149950,0migration/032351-21kthreadcore21:01:570
149950,0migration/032291-21munin-node20:42:070
149950,0migration/032017-21kthreadcore20:01:580
149950,0migration/028871-21kthreadcore19:36:540
149950,0migration/026101-21fschecks_time18:51:530
149950,0migration/025216-21kthreadcore23:47:000
149950,0migration/02486-21munin-node18:31:580
149950,0migration/024148-21munin-node22:51:530
149950,0migration/023500-21kthreadcore20:16:560
149950,0migration/02142-21sh23:34:320
149950,0migration/017500-21df_abs21:51:520
149950,0migration/016847-21processes22:07:010
149950,0migration/016427-21irqcore21:31:580
149950,0migration/016100-21kthreadcore20:51:580
149950,0migration/015685-21fschecks_time19:51:550
149950,0migration/015602-21sh22:46:560
149950,0migration/013683-21kthreadcore19:26:570
149950,0migration/010603-21kthreadcore23:01:570
149950,0migration/010186-21munin-node22:22:030
8315040,0irq/26-0000:00:0-21swapper/221:21:382
8315040,0irq/26-0000:00:0-21swapper/220:51:392
29935040,0irq/33-eth0-tx-9181-21idleruntime23:01:542
29935040,0irq/33-eth0-tx-7708-21diskmemload22:16:432
29935040,0irq/33-eth0-tx-26360-21ssh23:47:072
29935040,0irq/33-eth0-tx-0-21swapper/223:57:002
29935040,0irq/33-eth0-tx-0-21swapper/223:22:322
29935040,0irq/33-eth0-tx-0-21swapper/222:56:432
29935040,0irq/33-eth0-tx-0-21swapper/222:51:552
29935040,0irq/33-eth0-tx-0-21swapper/222:27:422
29935040,0irq/33-eth0-tx-0-21swapper/222:01:522
29935040,0irq/33-eth0-tx-0-21swapper/221:41:422
29935040,0irq/33-eth0-tx-0-21swapper/221:01:422
29935040,0irq/33-eth0-tx-0-21swapper/019:42:040
29935040,0irq/33-eth0-tx-0-21swapper/019:16:590
29935040,0irq/33-eth0-tx-0-21swapper/019:02:000
29935040,0irq/33-eth0-tx-0-21swapper/018:56:550
287239943,0cyclictest7712-21awk21:26:562
287239943,0cyclictest3599-21lldpd23:52:522
287239943,0cyclictest3599-21lldpd18:41:582
287239943,0cyclictest3599-21lldpd18:40:582
287239943,0cyclictest3599-21lldpd18:31:582
287239943,0cyclictest3262-21irqbalance23:41:522
287239943,0cyclictest3262-21irqbalance21:16:522
287239943,0cyclictest3262-21irqbalance20:27:132
287239943,0cyclictest3262-21irqbalance20:08:532
287239943,0cyclictest3262-21irqbalance20:03:332
287239943,0cyclictest3262-21irqbalance19:59:032
287239943,0cyclictest3262-21irqbalance19:51:532
287239943,0cyclictest3262-21irqbalance19:41:432
287239943,0cyclictest3262-21irqbalance19:38:432
287239943,0cyclictest3262-21irqbalance19:31:532
287239943,0cyclictest3262-21irqbalance19:12:532
287239943,0cyclictest3262-21irqbalance19:06:432
287239943,0cyclictest3262-21irqbalance19:02:032
287239943,0cyclictest3262-21irqbalance18:46:432
287239943,0cyclictest30878-21awk22:36:582
287239943,0cyclictest15761-21awk22:46:572
287239943,0cyclictest1346-21awk18:56:552
287239943,0cyclictest11706-21awk22:06:552
287239943,0cyclictest0-21swapper/220:46:522
287229944,0cyclictest0-21swapper/122:35:071
287229943,0cyclictest3599-21lldpd22:11:381
287229943,0cyclictest23871-21snmpd23:59:131
287229943,0cyclictest23871-21snmpd22:48:131
287229943,0cyclictest23871-21snmpd22:45:131
287229943,0cyclictest23871-21snmpd20:46:111
287229943,0cyclictest23871-21snmpd20:34:261
287229943,0cyclictest23104-21awk19:56:571
287229943,0cyclictest18208-21awk21:51:551
287229943,0cyclictest12437-21awk23:41:561
287229943,0cyclictest12248-21awk19:26:541
287229940,0cyclictest0-21swapper/122:25:591
287219944,0cyclictest0-21swapper/023:41:530
287219943,0cyclictest28954-21awk23:51:560
287219943,0cyclictest28937-21awk22:16:550
287219943,0cyclictest14139-21awk21:11:530
287219940,0cyclictest14407-21munin-run22:46:370
259940,0migration/230963-21kthreadcore19:16:562
259940,0migration/229119-21kthreadcore18:51:582
259940,0migration/228916-21kthreadcore23:31:592
259940,0migration/228904-21kthreadcore20:21:562
259940,0migration/215642-21kthreadcore19:26:572
249940,0watchdog/211954-21fschecks_count20:11:552
249940,0watchdog/20-21swapper/222:21:382
249940,0watchdog/20-21swapper/221:51:542
249940,0watchdog/20-21swapper/221:31:552
249940,0watchdog/20-21swapper/219:21:552
179940,0migration/1872-21kthreadcore21:21:561
179940,0migration/1599-21entropy18:56:521
179940,0migration/1458-21aten_r1power_vo19:41:521
179940,0migration/13952-21kthreadcore20:46:561
179940,0migration/13721-21kthreadcore22:01:571
179940,0migration/13330-21kthreadcore20:26:581
179940,0migration/132247-21taskset21:41:391
179940,0migration/131160-21kthreadcore21:01:571
179940,0migration/129306-21aten_r1power_cu20:01:521
179940,0migration/126519-21kthreadcore21:17:001
179940,0migration/125799-21munin-node19:36:491
179940,0migration/125041-21taskset23:11:341
179940,0migration/118839-21kthreadcore20:36:591
179940,0migration/118839-21kthreadcore20:36:591
179940,0migration/118596-21taskset20:16:481
179940,0migration/118448-21aten_r1power_cu19:31:531
179940,0migration/117476-21kthreadcore19:06:571
179940,0migration/115381-21kthreadcore18:42:011
179940,0migration/111953-21runrttasks20:51:561
169940,0watchdog/15685-21kthreadcore18:36:591
169940,0watchdog/14945-21idleruntime19:21:551
169940,0watchdog/123444-21irqstats19:11:591
169940,0watchdog/118733-21tr19:51:591
169940,0watchdog/114069-21yum19:02:071
169940,0watchdog/10-21swapper/123:46:541
169940,0watchdog/10-21swapper/123:36:581
169940,0watchdog/10-21swapper/123:31:461
169940,0watchdog/10-21swapper/123:14:581
169940,0watchdog/10-21swapper/122:56:541
169940,0watchdog/10-21swapper/122:18:581
169940,0watchdog/10-21swapper/121:46:541
149940,0migration/09100-21irqrtprio19:46:580
149940,0migration/0695-21fschecks_time21:41:530
149940,0migration/02775-21fschecks_time20:46:520
149940,0migration/025798-21kthreadcore19:12:000
149940,0migration/025604-21kthreadcore21:16:590
149940,0migration/020365-21kthreadcore18:46:560
149940,0migration/014181-21irqrtprio22:26:580
149940,0migration/013646-21kthreadcore20:11:590
287229932,0cyclictest5184-21awk20:06:571
287229932,0cyclictest30390-21grep20:21:561
287229932,0cyclictest26459-21awk18:51:551
287229932,0cyclictest13636-21grep23:21:581
287229930,0cyclictest8662-21grep21:26:581
287229930,0cyclictest19080-21perl18:46:521
287219933,0cyclictest19567-21kworker/0:122:32:010
287219933,0cyclictest19567-21kworker/0:122:11:390
287219933,0cyclictest19567-21kworker/0:121:36:470
287219933,0cyclictest19567-21kworker/0:121:06:430
287219933,0cyclictest19567-21kworker/0:120:26:390
169930,0watchdog/17708-21diskmemload22:37:021
169930,0watchdog/17708-21diskmemload21:37:031
169930,0watchdog/10-21swapper/123:26:541
169930,0watchdog/10-21swapper/122:07:301
169930,0watchdog/10-21swapper/120:56:591
169920,0watchdog/10-21swapper/119:16:471
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional