You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-07 - 06:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Sun Jun 07, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1689924940,0sleep10-21swapper/119:24:311
1674623626,0sleep20-21swapper/219:22:472
1667823430,0sleep30-21swapper/319:22:003
1669923026,0sleep00-21swapper/019:22:090
16976991010,0cyclictest0-21swapper/021:25:020
339970,0migration/315720-21kthreadcore21:34:553
29805070,0irq/28-eth0-rx-3524-21snmpd23:25:403
8305060,0irq/26-0000:00:31331-21munin-run19:34:343
8305060,0irq/26-0000:00:0-21swapper/320:39:353
339960,0migration/39068-21kthreadcore22:24:543
339960,0migration/38830-21aten_r1power_cu00:19:493
339960,0migration/38721-21sh22:06:053
339960,0migration/332483-21sh22:01:243
339960,0migration/329410-21irqrtprio19:54:523
339960,0migration/316580-21kthreadcore23:44:543
29805060,0irq/28-eth0-rx-9514-21ssh23:21:523
29805060,0irq/28-eth0-rx-27035-21iostat23:14:523
29805060,0irq/28-eth0-rx-16419-21iostat_ios00:04:523
29805060,0irq/28-eth0-rx-12674-21python19:39:593
29805060,0irq/28-eth0-rx-0-21swapper/300:25:203
169879964,0cyclictest28235-21diskmemload21:33:502
169769964,0cyclictest32737-21awk19:34:510
339950,0migration/35178-21sh22:42:313
339950,0migration/331285-21kthreadcore20:39:533
339950,0migration/327213-21runrttasks23:33:083
339950,0migration/32607-21timerwakeupswit21:25:003
339950,0migration/325345-21fschecks_time20:14:493
339950,0migration/323852-21df_abs22:54:503
339950,0migration/322451-21kthreadcore20:09:543
339950,0migration/320507-21fschecks_time00:44:493
339950,0migration/314466-21kthreadcore22:09:563
339950,0migration/312327-21kthreadcore20:04:523
339950,0migration/31213-21kthreadcore20:19:533
329950,0watchdog/38522-21cut20:44:543
329950,0watchdog/35833-21yum19:35:023
329950,0watchdog/329343-21users21:40:033
329950,0watchdog/30-21swapper/320:29:543
329950,0watchdog/30-21swapper/300:11:553
29815050,0irq/29-eth0-tx-9862-21ssh22:09:152
29815050,0irq/29-eth0-tx-7637-21kthreadcore23:19:552
29815050,0irq/29-eth0-tx-3082-21chrt22:23:062
29815050,0irq/29-eth0-tx-29061-21ssh22:38:062
29815050,0irq/29-eth0-tx-2678-21ssh23:35:542
29815050,0irq/29-eth0-tx-23837-21munin-node00:09:562
29815050,0irq/29-eth0-tx-20631-21ssh21:35:142
29815050,0irq/29-eth0-tx-19699-21munin-node00:44:552
29815050,0irq/29-eth0-tx-17943-21fschecks_time23:09:492
29815050,0irq/29-eth0-tx-14330-21ssh00:03:002
29815050,0irq/29-eth0-tx-0-21swapper/223:45:302
29815050,0irq/29-eth0-tx-0-21swapper/223:16:512
29815050,0irq/29-eth0-tx-0-21swapper/222:58:582
29815050,0irq/29-eth0-tx-0-21swapper/222:44:522
29815050,0irq/29-eth0-tx-0-21swapper/200:41:062
29805050,0irq/28-eth0-rx-9140-21grep23:04:513
29805050,0irq/28-eth0-rx-26728-21cut23:49:553
29805050,0irq/28-eth0-rx-2315-21munin-node00:34:583
29805050,0irq/28-eth0-rx-21099-21tr19:49:483
29805050,0irq/28-eth0-rx-14854-21ssh22:49:443
29805050,0irq/28-eth0-rx-13681-21cut20:49:533
29805050,0irq/28-eth0-rx-13316-21fschecks_time22:29:503
29805050,0irq/28-eth0-rx-12303-21taskset21:31:283
29805050,0irq/28-eth0-rx-0-21swapper/323:59:513
29805050,0irq/28-eth0-rx-0-21swapper/323:09:533
29805050,0irq/28-eth0-rx-0-21swapper/322:59:543
29805050,0irq/28-eth0-rx-0-21swapper/322:34:543
29805050,0irq/28-eth0-rx-0-21swapper/322:20:083
29805050,0irq/28-eth0-rx-0-21swapper/321:58:193
29805050,0irq/28-eth0-rx-0-21swapper/321:51:103
29805050,0irq/28-eth0-rx-0-21swapper/321:44:483
29805050,0irq/28-eth0-rx-0-21swapper/319:45:003
29805050,0irq/28-eth0-rx-0-21swapper/300:50:163
29805050,0irq/28-eth0-rx-0-21swapper/300:39:533
29805050,0irq/28-eth0-rx-0-21swapper/300:31:093
259950,0migration/29601-21kernelversion21:49:512
259950,0migration/29497-21sh23:04:512
259950,0migration/2484-21idleruntime21:04:512
259950,0migration/24208-21latency_hist23:39:362
259950,0migration/223025-21sh00:08:082
259950,0migration/217279-21latency_hist20:09:352
259950,0migration/213198-21kthreadcore22:09:562
249950,0watchdog/212097-21ssh22:28:592
179950,0migration/18608-21kthreadcore19:39:521
179950,0migration/131895-21dump-pmu-power22:59:361
179950,0migration/126778-21sh23:31:571
179950,0migration/124812-21kthreadcore00:44:551
179950,0migration/123118-21needreboot19:24:541
179950,0migration/11674-21kthreadcore21:44:551
169879955,0cyclictest0-21swapper/200:49:572
169879950,0cyclictest0-21swapper/223:57:182
169879950,0cyclictest0-21swapper/222:42:532
169799954,0cyclictest32131-21grep00:29:551
169799954,0cyclictest29967-21sadc23:54:361
169799950,0cyclictest0-21swapper/122:04:531
169769955,0cyclictest41ktimersoftd/023:19:010
169769955,0cyclictest0-21swapper/022:49:420
169769953,0cyclictest9302-21sh23:21:170
169769953,0cyclictest8845-21df23:04:500
169769953,0cyclictest8207-21sh00:18:420
169769953,0cyclictest7717-21perl21:09:490
169769953,0cyclictest7686-21ps19:39:520
169769953,0cyclictest6196-21sh22:44:420
169769953,0cyclictest6129-21rt-features22:59:570
169769953,0cyclictest5547-21cut19:59:530
169769953,0cyclictest4790-21perl22:24:490
169769953,0cyclictest454-21sed21:44:540
169769953,0cyclictest3524-21snmpd20:44:540
169769953,0cyclictest31073-21sh00:11:300
169769953,0cyclictest30673-21diskstats23:54:490
169769953,0cyclictest28618-21wc22:19:510
169769953,0cyclictest28595-21memory21:39:550
169769953,0cyclictest28001-21sh22:35:090
169769953,0cyclictest2659-21df00:34:490
169769953,0cyclictest23377-21port_sshpriv23:09:570
169769953,0cyclictest23371-21users21:55:030
169769953,0cyclictest21980-21gltestperf20:34:490
169769953,0cyclictest21913-21sh00:05:040
169769953,0cyclictest19438-21aten2.4-expect23:29:470
169769953,0cyclictest18460-21sh22:14:090
169769953,0cyclictest18023-21tune2fs00:24:510
169769953,0cyclictest14985-21if_eth120:29:520
169769953,0cyclictest14858-21iostat_ios19:44:510
169769953,0cyclictest14581-21kthreadcore22:29:540
169769953,0cyclictest13979-21sh00:02:010
169769953,0cyclictest13884-21perl23:44:500
169769953,0cyclictest10677-21python23:40:000
169769953,0cyclictest1015-21ssh00:31:450
169769952,0cyclictest6644-21munin-node20:24:500
169769952,0cyclictest32368-21diskstats21:04:490
169769952,0cyclictest32199-21diskstats20:19:490
169769952,0cyclictest30638-21perl22:39:500
169769952,0cyclictest26042-21python00:45:000
169769952,0cyclictest25726-21df_inode21:59:470
169769952,0cyclictest25191-21diskstats20:59:480
169769952,0cyclictest22857-21iostat_ios23:49:530
169769952,0cyclictest1958-21diskstats22:04:510
169769952,0cyclictest19452-21cut22:14:480
169769952,0cyclictest15575-21ssh00:20:060
169769952,0cyclictest10557-21diskstats20:04:470
169769952,0cyclictest10333-21cut21:29:550
169769950,0cyclictest41ktimersoftd/022:55:320
169769950,0cyclictest0-21swapper/000:51:530
169950,0watchdog/111697-21kthreadcore23:04:551
339940,0migration/37603-21fschecks_time21:09:493
339940,0migration/34459-21kernelversion19:59:523
339940,0migration/32738-21kthreadcore21:04:543
329940,0watchdog/330246-21ssh23:54:473
329940,0watchdog/314043-21munin-node21:15:023
329940,0watchdog/30-21swapper/323:34:473
329940,0watchdog/30-21swapper/321:19:543
329940,0watchdog/30-21swapper/320:24:543
29815040,0irq/29-eth0-tx-19454-21ssh22:30:172
29815040,0irq/29-eth0-tx-17528-21ssh23:25:082
29815040,0irq/29-eth0-tx-17439-21ssh00:24:392
29815040,0irq/29-eth0-tx-0-21swapper/223:49:462
29815040,0irq/29-eth0-tx-0-21swapper/223:29:472
29815040,0irq/29-eth0-tx-0-21swapper/222:14:552
29815040,0irq/29-eth0-tx-0-21swapper/221:59:412
29815040,0irq/29-eth0-tx-0-21swapper/221:54:402
29815040,0irq/29-eth0-tx-0-21swapper/221:39:522
29815040,0irq/29-eth0-tx-0-21swapper/200:34:482
29815040,0irq/29-eth0-tx-0-21swapper/200:19:502
29815040,0irq/29-eth0-tx-0-21swapper/200:14:522
29805040,0irq/28-eth0-rx-9529-21kthreadcore23:39:543
29805040,0irq/28-eth0-rx-23216-21kthreadcore20:54:543
29805040,0irq/28-eth0-rx-0-21swapper/322:45:153
29805040,0irq/28-eth0-rx-0-21swapper/322:16:353
29805040,0irq/28-eth0-rx-0-21swapper/320:59:543
29805040,0irq/28-eth0-rx-0-21swapper/319:24:463
29805040,0irq/28-eth0-rx-0-21swapper/300:14:473
259940,0migration/23777-21fschecks_count20:44:502
259940,0migration/232317-21kthreadcore20:39:542
259940,0migration/230102-21kthreadcore00:29:542
259940,0migration/229182-21fschecks_time21:24:482
259940,0migration/221509-21kthreadcore20:54:532
259940,0migration/220876-21aten_r1power_en19:49:472
259940,0migration/214764-21latency_hist22:49:362
259940,0migration/210608-21cpuspeed_turbos20:49:472
249940,0watchdog/213405-21yum21:10:022
249940,0watchdog/21273-21kthreadcore21:44:552
249940,0watchdog/20-21swapper/221:19:542
249940,0watchdog/20-21swapper/221:15:022
179940,0migration/15514-21kthreadcore22:59:541
179940,0migration/15008-21fschecks_time23:39:491
179940,0migration/132096-21kthreadcore22:19:541
179940,0migration/13126-21idleruntime23:19:511
179940,0migration/129032-21munin-node19:54:501
179940,0migration/127453-21kthreadcore00:09:541
179940,0migration/126998-21kthreadcore22:34:551
179940,0migration/125558-21fschecks_time20:59:491
179940,0migration/124241-21taskset20:56:101
179940,0migration/123272-21taskset00:08:501
179940,0migration/122554-21latency_hist21:39:351
179940,0migration/119721-21kthreadcore21:54:541
179940,0migration/11774-21kthreadcore22:39:541
179940,0migration/117363-21kthreadcore19:44:531
179940,0migration/11725-21kthreadcore00:14:541
179940,0migration/115370-21irqcore21:14:541
179940,0migration/114726-21kthreadcore23:44:531
179940,0migration/113859-21kthreadcore20:04:521
179940,0migration/113269-21kthreadcore21:49:531
179940,0migration/110096-21sh00:37:441
169879943,0cyclictest769-21awk22:59:512
169879943,0cyclictest7636-21awk19:39:512
169879943,0cyclictest3524-21snmpd19:38:222
169879943,0cyclictest3250-21irqbalance19:34:012
169879943,0cyclictest14929-21awk20:29:512
169879943,0cyclictest14693-21awk19:44:502
169799943,0cyclictest9935-21awk00:19:541
169799943,0cyclictest7796-21awk23:59:521
169799943,0cyclictest5015-21awk22:24:501
169799943,0cyclictest4341-21awk20:44:521
169799943,0cyclictest3571-21lldpd22:11:381
169799943,0cyclictest3571-21lldpd21:07:361
169799943,0cyclictest3571-21lldpd20:24:361
169799943,0cyclictest3571-21lldpd00:49:391
169799943,0cyclictest31316-21awk23:54:521
169799943,0cyclictest29319-21awk20:39:511
169799943,0cyclictest26917-21awk23:14:511
169799943,0cyclictest20030-21awk22:14:511
169799943,0cyclictest18614-21awk00:24:531
169799943,0cyclictest13669-21awk22:29:521
169799943,0cyclictest11448-21awk20:49:501
169769943,0cyclictest28430-21df_inode19:54:480
169769942,0cyclictest7986-21sh21:49:420
169769942,0cyclictest3524-21snmpd20:09:470
169769942,0cyclictest3524-21snmpd19:49:430
169769942,0cyclictest3524-21snmpd19:29:400
169769942,0cyclictest29153-21if_eth023:34:510
169769942,0cyclictest28259-21taskset20:39:380
169769942,0cyclictest24446-21date20:14:350
169769942,0cyclictest21031-21latency_hist21:19:350
169769942,0cyclictest17833-21tr20:54:470
169769942,0cyclictest17169-21latency_hist19:24:360
169769942,0cyclictest14132-21aten2.4-expect21:14:480
169769942,0cyclictest13666-21cut21:34:350
169769942,0cyclictest10998-21ssh00:39:370
169769942,0cyclictest10757-21sh23:24:400
169769942,0cyclictest10401-21awk20:49:460
169940,0watchdog/16314-21runrttasks21:24:501
169940,0watchdog/130203-21memory19:29:541
169940,0watchdog/126868-21kthreadcore21:19:541
169940,0watchdog/126806-21cut21:59:511
169940,0watchdog/120284-21timerandwakeup21:35:021
169940,0watchdog/112080-21ssh21:30:581
169940,0watchdog/111021-21sort23:24:471
169940,0watchdog/10-21swapper/122:53:151
249930,0watchdog/20-21swapper/221:00:022
249930,0watchdog/20-21swapper/220:24:542
249930,0watchdog/20-21swapper/220:14:542
169879932,0cyclictest3524-21snmpd20:21:532
169879932,0cyclictest23383-21sensors19:24:572
169879932,0cyclictest22213-21awk20:34:512
169879930,0cyclictest0-21swapper/219:54:522
169799933,0cyclictest3571-21lldpd20:20:361
169799933,0cyclictest3571-21lldpd19:52:361
169799933,0cyclictest3571-21lldpd00:39:391
169799932,0cyclictest3571-21lldpd19:59:361
169799932,0cyclictest3524-21snmpd23:35:371
169799932,0cyclictest32594-21awk19:34:501
169799932,0cyclictest29232-21grep20:14:531
169799932,0cyclictest25103-21tr20:34:531
169799932,0cyclictest14984-21awk20:29:521
169799930,0cyclictest7679-21perl21:09:491
169799930,0cyclictest6977-21perl22:44:481
169930,0watchdog/10-21swapper/123:09:591
169930,0watchdog/10-21swapper/120:09:541
169879920,0cyclictest0-21swapper/220:04:352
169879920,0cyclictest0-21swapper/219:59:352
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional