You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 06:02
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Mon Apr 06, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3257424132,0sleep20-21swapper/219:17:132
3261223624,0sleep00-21swapper/019:17:430
3260823525,0sleep10-21swapper/119:17:401
3256523324,0sleep30-21swapper/319:17:063
373991010,0cyclictest0-21swapper/221:20:262
8305060,0irq/26-0000:00:6379-21awk22:49:302
3739960,0cyclictest0-21swapper/223:23:282
3689960,0cyclictest0-21swapper/122:16:331
339960,0migration/34774-21sh21:31:313
339960,0migration/33635-21sh23:42:263
339960,0migration/326409-21sh23:34:593
339960,0migration/325185-21sh00:31:553
339960,0migration/321990-21timerandwakeup00:09:543
339960,0migration/321197-21missed_timers22:54:463
339960,0migration/320450-21sh23:51:083
29805060,0irq/28-eth0-rx-3440-21fschecks_time00:39:433
29805060,0irq/28-eth0-rx-2797-21diskstats23:24:413
19160,0ktimersoftd/10-21swapper/100:34:591
149960,0migration/05677-21irqcore19:44:450
149960,0migration/029662-21munin-node22:24:460
149960,0migration/027227-21sh23:37:150
149960,0migration/019355-21sh23:34:260
149960,0migration/016414-21sh21:58:130
149960,0migration/013575-21sh22:50:380
8305050,0irq/26-0000:00:22285-21cat19:34:293
8305050,0irq/26-0000:00:0-21swapper/319:39:293
3739955,0cyclictest0-21swapper/223:46:572
3739955,0cyclictest0-21swapper/223:33:052
3739955,0cyclictest0-21swapper/222:57:272
3739955,0cyclictest0-21swapper/220:14:452
3739950,0cyclictest0-21swapper/200:30:002
3689955,0cyclictest0-21swapper/123:29:051
3689955,0cyclictest0-21swapper/122:56:551
3689955,0cyclictest0-21swapper/122:38:571
3689955,0cyclictest0-21swapper/121:20:451
3689950,0cyclictest0-21swapper/122:46:121
3689950,0cyclictest0-21swapper/120:59:561
3689950,0cyclictest0-21swapper/119:34:481
3679955,0cyclictest0-21swapper/023:59:500
3679954,0cyclictest12434-21grep20:34:430
35150,0ktimersoftd/30-21swapper/300:34:593
339950,0migration/39506-21gltestperf21:54:443
339950,0migration/38688-21kthreadcore22:49:463
339950,0migration/35327-21fschecks_time20:29:413
339950,0migration/3462-21fschecks_count23:04:423
339950,0migration/332376-21kthreadcore22:44:463
339950,0migration/331914-21kthreadcore21:09:483
339950,0migration/331523-21taskset21:48:223
339950,0migration/329150-21kthreadcore19:59:453
339950,0migration/328547-21diskmemload22:40:253
339950,0migration/328547-21diskmemload22:40:253
339950,0migration/327932-21latency_hist22:24:303
339950,0migration/324618-21sh00:14:423
339950,0migration/322635-21fschecks_time23:54:433
339950,0migration/322543-21kthreadcore22:19:463
339950,0migration/321257-21sh21:39:533
339950,0migration/320123-21munin-node20:39:443
339950,0migration/319571-21taskset21:22:113
339950,0migration/317035-21latency_hist21:59:293
339950,0migration/316509-21if_eth1.320:14:443
339950,0migration/311054-21sshd19:45:073
339950,0migration/310354-21kthreadcore20:54:443
329950,0watchdog/38933-21ssh22:10:043
329950,0watchdog/3298050irq/28-eth0-rx-22:34:323
329950,0watchdog/326214-21cut22:04:403
329950,0watchdog/32504-21grep20:24:483
329950,0watchdog/315234-21munin-node19:29:403
329950,0watchdog/30-21swapper/320:44:443
29815050,0irq/29-eth0-tx-9219-21perl21:54:432
29815050,0irq/29-eth0-tx-5581-21chrt00:01:392
29815050,0irq/29-eth0-tx-3524-21snmpd21:52:112
29815050,0irq/29-eth0-tx-339-21munin-node00:19:452
29815050,0irq/29-eth0-tx-29714-21munin-node20:24:430
29815050,0irq/29-eth0-tx-22003-21ssh21:40:552
29815050,0irq/29-eth0-tx-21936-21munin-node23:54:412
29815050,0irq/29-eth0-tx-0-21swapper/222:14:352
29815050,0irq/29-eth0-tx-0-21swapper/221:37:022
29815050,0irq/29-eth0-tx-0-21swapper/221:30:292
29815050,0irq/29-eth0-tx-0-21swapper/021:19:410
29815050,0irq/29-eth0-tx-0-21swapper/020:59:080
29815050,0irq/29-eth0-tx-0-21swapper/020:14:500
29815050,0irq/29-eth0-tx-0-21swapper/019:39:410
29805050,0irq/28-eth0-rx-8579-21ssh00:23:483
29805050,0irq/28-eth0-rx-6977-21tail00:04:413
29805050,0irq/28-eth0-rx-31923-21kthreadcore23:19:483
29805050,0irq/28-eth0-rx-29985-21open_inodes22:59:493
29805050,0irq/28-eth0-rx-10341-21load22:29:473
29805050,0irq/28-eth0-rx-0-21swapper/323:59:473
29805050,0irq/28-eth0-rx-0-21swapper/323:45:373
29805050,0irq/28-eth0-rx-0-21swapper/323:09:423
29805050,0irq/28-eth0-rx-0-21swapper/321:24:433
29805050,0irq/28-eth0-rx-0-21swapper/320:09:293
29805050,0irq/28-eth0-rx-0-21swapper/300:49:293
29805050,0irq/28-eth0-rx-0-21swapper/300:24:453
259950,0migration/26453-21munin-node23:04:502
259950,0migration/25789-21kthreadcore00:39:482
259950,0migration/23455-21runrttasks19:39:512
259950,0migration/229628-21needreboot21:44:492
259950,0migration/228429-21kthreadcore21:04:462
259950,0migration/228183-21latency_hist23:39:292
259950,0migration/225765-21kthreadcore22:59:462
259950,0migration/225305-21kthreadcore23:34:482
259950,0migration/223080-21kthreadcore23:14:482
259950,0migration/216199-21sh23:11:432
259950,0migration/214749-21ksmtuned22:53:562
259950,0migration/212942-21missed_timers00:04:492
249950,0watchdog/223120-21cat21:59:482
19150,0ktimersoftd/10-21swapper/121:52:261
19150,0ktimersoftd/10-21swapper/100:46:291
179950,0migration/16394-21kthreadcore19:19:461
179950,0migration/15942-21munin-node21:14:441
179950,0migration/15534-21kthreadcore22:09:501
179950,0migration/15473-21munin-node22:29:441
179950,0migration/131526-21fschecks_time23:59:431
179950,0migration/127063-21kthreadcore00:14:461
179950,0migration/126512-21kthreadcore22:39:471
179950,0migration/126512-21kthreadcore22:39:471
179950,0migration/117357-21kthreadcore00:09:461
179950,0migration/115665-21df_abs20:14:411
179950,0migration/114996-21latency_hist19:29:291
169950,0watchdog/16314-21runrttasks00:22:171
149950,0migration/09859-21sh00:41:030
149950,0migration/08128-21kthreadcore21:14:460
149950,0migration/04519-21latency_hist23:44:290
149950,0migration/04413-21taskset21:12:040
149950,0migration/03383-21needreboot21:29:490
149950,0migration/03092-21kthreadcore00:19:500
149950,0migration/030467-21munin-node00:14:500
149950,0migration/029582-21kthreadcore23:19:470
149950,0migration/029278-21taskset23:56:340
149950,0migration/026463-21kthreadcore21:04:460
149950,0migration/026243-21fschecks_count22:04:400
149950,0migration/024850-21hddtemp_smartct22:59:420
149950,0migration/024554-21kthreadcore20:39:460
149950,0migration/022947-21fschecks_count19:34:420
149950,0migration/022638-21sh22:39:430
149950,0migration/022638-21sh22:39:430
149950,0migration/022379-21kthreadcore00:29:470
149950,0migration/022314-21kthreadcore22:19:460
149950,0migration/021045-21kthreadcore19:54:460
149950,0migration/020854-21taskset22:38:020
149950,0migration/019057-21kthreadcore21:39:470
149950,0migration/01858-21fschecks_time20:49:420
149950,0migration/017377-21latency_hist23:14:290
149950,0migration/016866-21kthreadcore22:54:440
149950,0migration/013015-21kthreadcore00:44:490
149950,0migration/010904-21proc_pri23:44:510
149950,0migration/010152-21munin-node00:24:440
3739943,0cyclictest5684-21ps19:44:452
3739943,0cyclictest5622-21awk20:29:432
3739943,0cyclictest5438-21awk22:29:442
3739943,0cyclictest3571-21lldpd19:33:502
3739943,0cyclictest3524-21snmpd20:55:552
3739943,0cyclictest3524-21snmpd20:21:552
3739943,0cyclictest3524-21snmpd20:11:242
3739943,0cyclictest3524-21snmpd20:00:542
3739943,0cyclictest3524-21snmpd19:57:242
3739943,0cyclictest3524-21snmpd19:24:392
3739943,0cyclictest3524-21snmpd19:20:092
3739943,0cyclictest12716-21awk19:49:432
3689943,0cyclictest9365-21awk20:54:421
3689943,0cyclictest9154-21awk20:09:431
3689943,0cyclictest3571-21lldpd22:50:531
3689943,0cyclictest3524-21snmpd23:59:121
3689943,0cyclictest3524-21snmpd21:42:111
3689943,0cyclictest3524-21snmpd19:49:541
3689943,0cyclictest30651-21awk19:39:431
3689943,0cyclictest29112-21awk22:24:441
3689943,0cyclictest28033-21sadc23:39:291
3689943,0cyclictest24673-21awk21:44:451
3689943,0cyclictest23779-21awk21:04:441
3689943,0cyclictest23550-21awk20:19:441
3689943,0cyclictest22071-21awk21:24:471
3689943,0cyclictest20555-21awk22:19:441
3689943,0cyclictest2018-21awk20:49:431
3679944,0cyclictest14951-21turbostat19:29:280
3679944,0cyclictest0-21swapper/019:19:400
3679944,0cyclictest0-21swapper/000:39:050
3679943,0cyclictest918-21awk21:49:440
3679943,0cyclictest3524-21snmpd21:00:250
3679940,0cyclictest0-21swapper/021:38:440
339940,0migration/39810-21kthreadcore21:34:483
339940,0migration/3971-21taskset20:49:323
339940,0migration/37545-21taskset19:22:183
329940,0watchdog/323519-21idleruntime20:19:443
329940,0watchdog/30-21swapper/319:49:403
329940,0watchdog/30-21swapper/319:24:443
29815040,0irq/29-eth0-tx-688-21ssh00:34:582
29815040,0irq/29-eth0-tx-25123-21gltestperf00:14:432
29815040,0irq/29-eth0-tx-13326-21munin-node23:49:442
29815040,0irq/29-eth0-tx-0-21swapper/223:24:402
29815040,0irq/29-eth0-tx-0-21swapper/222:40:102
29815040,0irq/29-eth0-tx-0-21swapper/222:40:102
29815040,0irq/29-eth0-tx-0-21swapper/222:38:282
29815040,0irq/29-eth0-tx-0-21swapper/222:20:202
29815040,0irq/29-eth0-tx-0-21swapper/222:09:352
29815040,0irq/29-eth0-tx-0-21swapper/221:25:092
29815040,0irq/29-eth0-tx-0-21swapper/200:24:412
29815040,0irq/29-eth0-tx-0-21swapper/200:09:462
29815040,0irq/29-eth0-tx-0-21swapper/020:09:470
29805040,0irq/28-eth0-rx-0-21swapper/323:32:563
29805040,0irq/28-eth0-rx-0-21swapper/323:15:033
29805040,0irq/28-eth0-rx-0-21swapper/322:17:523
29805040,0irq/28-eth0-rx-0-21swapper/321:49:453
259940,0migration/2740-21latency_hist20:04:292
259940,0migration/24946-21kthreadcore20:49:462
259940,0migration/230351-21munin-node20:24:432
259940,0migration/229702-21kthreadcore22:04:452
259940,0migration/229124-21kthreadcore20:44:492
259940,0migration/228714-21timerandwakeup19:34:542
259940,0migration/219070-21aten_r1power_fr20:39:402
259940,0migration/217241-21kthreadcore20:34:472
259940,0migration/216620-21munin-node20:59:452
259940,0migration/212830-21irqrtprio00:44:482
259940,0migration/211404-21unixbench_singl21:14:542
249940,0watchdog/20-21swapper/222:24:482
19140,0ktimersoftd/10-21swapper/123:41:331
179940,0migration/19565-21kthreadcore19:44:461
179940,0migration/19230-21kthreadcore21:34:471
179940,0migration/17803-21taskset20:08:361
179940,0migration/13995-21unixbench_multi21:29:551
179940,0migration/1352-21kthreadcore21:09:481
179940,0migration/132695-21kthreadcore20:24:471
179940,0migration/129970-21munin-node22:59:491
179940,0migration/128934-21kthreadcore23:19:471
179940,0migration/119809-21fschecks_time20:39:421
179940,0migration/119093-21kthreadcore23:49:481
179940,0migration/117198-21kthreadcore20:34:471
179940,0migration/11594-21irqrtprio23:04:451
179940,0migration/114038-21kthreadcore00:24:471
179940,0migration/112039-21hwlatdetect23:29:431
179940,0migration/111833-21kthreadcore00:04:481
169940,0watchdog/128263-21kthreadcore22:04:441
169940,0watchdog/126769-21grep20:44:441
169940,0watchdog/112172-21ssh23:47:051
169940,0watchdog/10-21swapper/123:09:441
169940,0watchdog/10-21swapper/121:59:481
169940,0watchdog/10-21swapper/121:54:481
149940,0migration/06619-21kthreadcore23:24:460
149940,0migration/015567-21runrttasks23:09:580
8305030,0irq/26-0000:00:0-21swapper/320:04:293
4130,0ktimersoftd/00-21swapper/020:22:540
3739932,0cyclictest3524-21snmpd21:09:402
3689932,0cyclictest3524-21snmpd23:14:351
3689932,0cyclictest27074-21awk19:59:431
3689932,0cyclictest19383-21tune2fs19:54:421
3689932,0cyclictest18016-21cat00:29:411
3689930,0cyclictest3620-21perl00:39:441
3689930,0cyclictest0-21swapper/119:29:361
3679933,0cyclictest11886-21diskmemload22:46:220
3679932,0cyclictest3571-21lldpd00:10:540
3679932,0cyclictest26192-21munin-node20:44:440
3679932,0cyclictest1982-21awk20:04:430
3679932,0cyclictest18147-21hwlatdetect21:59:440
3679932,0cyclictest14387-21awk23:49:440
3679930,0cyclictest13080-21ssh00:04:510
3679930,0cyclictest11668-21munin-node19:49:470
3679930,0cyclictest11172-21diskmemload22:30:020
3679930,0cyclictest0-21swapper/020:29:530
3679930,0cyclictest0-21swapper/019:59:410
329930,0watchdog/30-21swapper/321:14:403
29815030,0irq/29-eth0-tx-0-21swapper/019:29:400
29805030,0irq/28-eth0-rx-0-21swapper/320:59:483
29805030,0irq/28-eth0-rx-0-21swapper/320:34:403
29805030,0irq/28-eth0-rx-0-21swapper/319:54:403
3679921,0cyclictest3524-21snmpd23:04:340
3679921,0cyclictest20777-21date21:24:290
3679921,0cyclictest11886-21diskmemload22:14:310
3679921,0cyclictest0-21swapper/021:44:420
3679920,0cyclictest0-21swapper/022:09:300
329920,0watchdog/30-21swapper/321:04:403
169920,0watchdog/10-21swapper/120:29:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional