You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 11:50
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Thu Jan 01, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1023926463,0sleep20-21swapper/219:09:262
1029223727,0sleep30-21swapper/319:10:073
1034923626,0sleep10-21swapper/119:10:451
1005923228,0sleep00-21swapper/019:07:190
339960,0migration/35063-21irqrtprio19:51:513
339960,0migration/34543-21munin-node21:51:593
339960,0migration/332083-21sh22:08:023
339960,0migration/331797-21munin-node21:16:533
339960,0migration/331674-21sh00:19:263
339960,0migration/329248-21sh21:13:233
339960,0migration/326705-21sh22:23:193
339960,0migration/323074-21sh23:35:133
339960,0migration/319490-21munin-node00:31:543
29805060,0irq/28-eth0-rx-9868-21diskstats23:46:503
29805060,0irq/28-eth0-rx-22034-21kthreadcore21:26:523
29805060,0irq/28-eth0-rx-15091-21diskstats20:21:503
19160,0ktimersoftd/10-21swapper/121:46:191
149960,0migration/030989-21sh00:17:470
149960,0migration/024011-21unixbench_multi22:57:030
104949965,0cyclictest22573-21sh23:34:000
8305050,0irq/26-0000:00:17973-21crond00:31:372
8305050,0irq/26-0000:00:12690-21latency_hist00:06:372
339950,0migration/37917-21fschecks_time21:56:513
339950,0migration/37565-21latency_hist21:01:363
339950,0migration/332689-21latency_hist20:56:363
339950,0migration/330076-21kthreadcore22:26:533
339950,0migration/32971-21kthreadcore00:21:553
339950,0migration/329663-21fschecks_count20:31:503
339950,0migration/328792-21kthreadcore23:01:563
339950,0migration/322919-21munin-node20:26:533
339950,0migration/321655-21aten_r1power_fr21:46:493
339950,0migration/320015-21kthreadcore19:16:523
339950,0migration/315282-21sh21:23:183
339950,0migration/314001-21kthreadcore00:26:563
339950,0migration/312935-21df_abs21:41:503
339950,0migration/312839-21kthreadcore19:56:543
339950,0migration/312116-21kthreadcore19:31:543
339950,0migration/311430-21cpuspeed_turbos20:41:493
329950,0watchdog/35854-21perl22:31:523
329950,0watchdog/332621-21tune2fs22:46:523
29815050,0irq/29-eth0-tx-25792-21python22:22:002
29815050,0irq/29-eth0-tx-17903-21munin-node19:16:480
29815050,0irq/29-eth0-tx-10924-21munin-node19:57:000
29815050,0irq/29-eth0-tx-0-21swapper/020:56:490
29815050,0irq/29-eth0-tx-0-21swapper/020:46:590
29815050,0irq/29-eth0-tx-0-21swapper/020:11:540
29805050,0irq/28-eth0-rx-6118-21kthreadcore00:01:573
29805050,0irq/28-eth0-rx-26811-21df_abs21:31:503
29805050,0irq/28-eth0-rx-24184-21cpuspeed_turbos23:36:483
29805050,0irq/28-eth0-rx-16898-21grep23:12:113
29805050,0irq/28-eth0-rx-16211-21awk22:01:493
29805050,0irq/28-eth0-rx-14644-21ssh23:31:213
29805050,0irq/28-eth0-rx-14357-21ssh00:10:413
29805050,0irq/28-eth0-rx-0-21swapper/323:56:423
29805050,0irq/28-eth0-rx-0-21swapper/323:51:513
29805050,0irq/28-eth0-rx-0-21swapper/323:21:193
29805050,0irq/28-eth0-rx-0-21swapper/322:51:543
29805050,0irq/28-eth0-rx-0-21swapper/322:20:583
29805050,0irq/28-eth0-rx-0-21swapper/322:12:003
29805050,0irq/28-eth0-rx-0-21swapper/321:36:553
29805050,0irq/28-eth0-rx-0-21swapper/320:36:573
29805050,0irq/28-eth0-rx-0-21swapper/320:16:543
29805050,0irq/28-eth0-rx-0-21swapper/300:36:563
27150,0ktimersoftd/20-21swapper/221:46:192
249950,0watchdog/21554-21grep19:26:552
249950,0watchdog/20-21swapper/200:17:322
179950,0migration/15884-21sh22:47:011
179950,0migration/13330-21sh22:28:311
179950,0migration/132348-21aten_r1power_fr19:26:481
179950,0migration/132132-21taskset20:09:221
179950,0migration/124728-21switchtime00:32:031
179950,0migration/121298-21kthreadcore20:46:551
179950,0migration/118077-21kthreadcore00:11:531
179950,0migration/116969-21munin-node19:12:011
179950,0migration/113877-21kthreadcore22:51:551
179950,0migration/111614-21kthreadcore20:16:551
169950,0watchdog/14660-21cat21:52:001
169950,0watchdog/121954-21diskmemload22:11:561
169950,0watchdog/117426-21python20:41:591
149950,0migration/07768-21sh22:50:370
149950,0migration/07310-21munin-node23:42:070
149950,0migration/06761-21fschecks_count23:26:510
149950,0migration/06633-21kthreadcore20:36:570
149950,0migration/05774-21latency19:26:560
149950,0migration/05719-21kthreadcore19:51:520
149950,0migration/0492-21kthreadcore21:51:520
149950,0migration/04817-21kthreadcore00:21:560
149950,0migration/031271-21kthreadcore23:21:540
149950,0migration/029913-21irqrtprio22:26:520
149950,0migration/029819-21kthreadcore22:06:550
149950,0migration/027024-21latency_hist00:36:370
149950,0migration/025275-21kthreadcore20:26:560
149950,0migration/019954-21kthreadcore19:36:570
149950,0migration/019114-21fschecks_time00:31:510
149950,0migration/018648-21needreboot21:41:570
149950,0migration/017004-21taskset23:49:030
149950,0migration/016732-21kthreadcore21:06:560
149950,0migration/015591-21kthreadcore20:41:550
149950,0migration/015214-21taskset22:52:090
149950,0migration/014589-21latency_hist20:21:360
149950,0migration/013986-21kthreadcore19:11:540
149950,0migration/011298-21munin-node22:32:000
149950,0migration/011004-21munin-node22:16:500
149950,0migration/010360-21kthreadcore21:56:550
104969955,0cyclictest0-21swapper/222:16:542
104959955,0cyclictest3524-21snmpd19:35:151
104949955,0cyclictest0-21swapper/021:32:550
104949955,0cyclictest0-21swapper/021:16:220
104949955,0cyclictest0-21swapper/021:01:510
104949950,0cyclictest0-21swapper/023:41:240
104949950,0cyclictest0-21swapper/021:24:150
8305040,0irq/26-0000:00:6614-21kthreadcore22:11:562
339940,0migration/32924-21kthreadcore23:41:563
339940,0migration/321212-21kthreadcore20:01:563
339940,0migration/320217-21kthreadcore20:46:543
339940,0migration/315818-21fschecks_time00:11:483
339940,0migration/313651-21kthreadcore19:11:543
329940,0watchdog/328951-21kthreadcore19:21:553
329940,0watchdog/322646-21if_err_eth1.319:42:033
329940,0watchdog/311152-21kthreadcore21:01:553
329940,0watchdog/30-21swapper/319:28:473
29815040,0irq/29-eth0-tx-4897-21ssh22:31:402
29815040,0irq/29-eth0-tx-32488-21tune2fs22:46:522
29815040,0irq/29-eth0-tx-2856-21chrt23:06:472
29815040,0irq/29-eth0-tx-24567-21ssh22:58:102
29815040,0irq/29-eth0-tx-21954-21diskmemload22:06:382
29815040,0irq/29-eth0-tx-1511-21uptime23:57:022
29815040,0irq/29-eth0-tx-14295-21sensors_volt21:22:022
29815040,0irq/29-eth0-tx-10943-21irqrtprio23:46:542
29815040,0irq/29-eth0-tx-0-21swapper/223:41:402
29815040,0irq/29-eth0-tx-0-21swapper/223:16:432
29815040,0irq/29-eth0-tx-0-21swapper/223:01:552
29815040,0irq/29-eth0-tx-0-21swapper/222:36:372
29815040,0irq/29-eth0-tx-0-21swapper/222:01:532
29815040,0irq/29-eth0-tx-0-21swapper/221:56:532
29815040,0irq/29-eth0-tx-0-21swapper/221:46:442
29815040,0irq/29-eth0-tx-0-21swapper/221:32:222
29815040,0irq/29-eth0-tx-0-21swapper/200:21:512
29815040,0irq/29-eth0-tx-0-21swapper/200:11:522
29815040,0irq/29-eth0-tx-0-21swapper/019:46:580
29815040,0irq/29-eth0-tx-0-21swapper/019:41:580
29815040,0irq/29-eth0-tx-0-21swapper/019:31:580
29805040,0irq/28-eth0-rx-0-21swapper/323:22:313
29805040,0irq/28-eth0-rx-0-21swapper/322:56:553
29805040,0irq/28-eth0-rx-0-21swapper/322:41:413
29805040,0irq/28-eth0-rx-0-21swapper/322:37:033
2895240,0sleep30-21swapper/323:06:533
259940,0migration/28342-21latency_hist22:51:372
259940,0migration/24586-21fschecks_time00:01:522
259940,0migration/231622-21kthreadcore22:26:532
259940,0migration/22238-21kthreadcore20:56:542
259940,0migration/220109-21kthreadcore23:51:562
259940,0migration/218612-21unixbench_singl23:12:202
259940,0migration/218461-21munin-node20:46:482
259940,0migration/218370-21kthreadcore23:31:572
259940,0migration/212717-21irqrtprio20:41:542
259940,0migration/210192-21kthreadcore20:16:542
249940,0watchdog/232517-21kthreadcore21:51:522
249940,0watchdog/230869-21kthreadcore20:51:552
249940,0watchdog/222382-21grep20:26:512
249940,0watchdog/20-21swapper/222:41:562
249940,0watchdog/20-21swapper/221:01:552
249940,0watchdog/20-21swapper/220:06:512
249940,0watchdog/20-21swapper/219:42:072
179940,0migration/18275-21sh00:24:311
179940,0migration/132632-21munin-node21:31:581
179940,0migration/130088-21kthreadcore23:01:561
179940,0migration/129818-21taskset21:50:431
179940,0migration/128828-21kthreadcore20:51:541
179940,0migration/126081-21kthreadcore23:36:541
179940,0migration/125140-21taskset23:52:151
179940,0migration/121825-21kthreadcore19:16:531
179940,0migration/120390-21kthreadcore21:06:571
179940,0migration/120048-21sh23:15:421
179940,0migration/115046-21kthreadcore23:46:561
179940,0migration/111379-21kthreadcore23:26:561
179940,0migration/111089-21latency_hist20:41:361
169940,0watchdog/19123-21ntpq23:07:041
169940,0watchdog/15082-21irqrtprio19:51:511
169940,0watchdog/13493-21rm00:01:361
169940,0watchdog/1286792sleep123:20:081
169940,0watchdog/128334-21grep20:27:031
169940,0watchdog/127146-21ssh00:36:481
169940,0watchdog/125790-21python22:22:001
169940,0watchdog/125048-21ssh21:29:191
169940,0watchdog/123408-21cut19:42:071
169940,0watchdog/122517-21ssh23:33:521
169940,0watchdog/116340-21cut19:56:551
169940,0watchdog/10-21swapper/122:41:561
169940,0watchdog/10-21swapper/122:03:481
169940,0watchdog/10-21swapper/121:36:561
169940,0watchdog/10-21swapper/121:21:551
149940,0migration/04560-21kthreadcore22:11:560
149940,0migration/04223-21df_abs00:01:510
149940,0migration/023978-21kthreadcore22:21:550
149940,0migration/018656-21kthreadcore00:11:530
149940,0migration/018201-21aten_r1power_fr20:01:490
149940,0migration/01410-21kthreadcore20:31:550
104969943,0cyclictest7255-21awk23:26:532
104969943,0cyclictest4514-21awk21:36:532
104969943,0cyclictest3571-21lldpd21:17:182
104969943,0cyclictest3571-21lldpd21:06:472
104969943,0cyclictest3571-21lldpd20:35:162
104969943,0cyclictest3571-21lldpd19:53:152
104969943,0cyclictest3571-21lldpd19:31:442
104969943,0cyclictest3524-21snmpd23:23:582
104969943,0cyclictest3524-21snmpd21:12:122
104969943,0cyclictest3524-21snmpd20:40:272
104969943,0cyclictest3524-21snmpd20:22:122
104969943,0cyclictest3524-21snmpd20:11:422
104969943,0cyclictest3524-21snmpd20:02:422
104969943,0cyclictest3524-21snmpd19:47:112
104969943,0cyclictest3524-21snmpd19:40:412
104969943,0cyclictest28246-21awk00:36:552
104969943,0cyclictest26207-21awk19:21:532
104969943,0cyclictest24994-21awk23:36:512
104969943,0cyclictest19422-21awk00:31:532
104969943,0cyclictest19007-21awk19:16:502
104969943,0cyclictest18306-21awk21:26:502
104969943,0cyclictest11988-21awk19:56:522
104959943,0cyclictest8206-21awk21:56:531
104959943,0cyclictest6038-21awk22:31:531
104959943,0cyclictest4976-21awk00:01:551
104959943,0cyclictest3571-21lldpd23:21:501
104959943,0cyclictest3571-21lldpd22:17:491
104959943,0cyclictest3571-21lldpd21:15:471
104959943,0cyclictest3571-21lldpd21:00:171
104959943,0cyclictest3524-21snmpd21:18:571
104959943,0cyclictest3524-21snmpd20:22:421
104959943,0cyclictest25104-21awk00:16:541
104959943,0cyclictest18455-21awk22:56:521
104959943,0cyclictest15511-21awk19:36:551
104959943,0cyclictest1330-21awk20:11:551
104959943,0cyclictest1168-21sadc22:11:361
104959943,0cyclictest10550-21awk00:26:541
104949944,0cyclictest0-21swapper/022:42:170
104949943,0cyclictest8529-21awk20:16:520
104949943,0cyclictest3524-21snmpd23:12:580
104949943,0cyclictest3524-21snmpd23:04:280
104949943,0cyclictest31852-21kworker/0:021:26:540
104949943,0cyclictest31852-21kworker/0:021:19:240
104949943,0cyclictest25637-21munin-node20:51:490
104949943,0cyclictest22616-21awk21:46:520
104949943,0cyclictest19453-21awk23:51:540
104949943,0cyclictest13846-21kworker/0:122:40:540
104949940,0cyclictest25368-21munin-node20:06:490
339930,0migration/315108-21aten_r1power_po21:06:503
329930,0watchdog/310493-21cyclictest19:40:473
329930,0watchdog/30-21swapper/320:11:593
329930,0watchdog/30-21swapper/320:06:473
29805030,0irq/28-eth0-rx-0-21swapper/319:46:483
169930,0watchdog/10-21swapper/121:01:511
104969932,0cyclictest11740-21awk19:11:512
104959932,0cyclictest3571-21lldpd00:08:201
104959932,0cyclictest3524-21snmpd20:06:121
104959932,0cyclictest3397-21sadc19:51:361
104959932,0cyclictest26137-21awk19:21:521
104959932,0cyclictest1588-21awk23:41:531
104959930,0cyclictest20647-21rm22:37:031
104949933,0cyclictest31852-21kworker/0:022:01:440
104949933,0cyclictest31852-21kworker/0:021:36:520
104949933,0cyclictest1449-21kworker/0:223:56:380
104949933,0cyclictest1449-21kworker/0:200:26:380
104949933,0cyclictest1449-21kworker/0:200:06:420
104949933,0cyclictest13846-21kworker/0:123:16:440
104949933,0cyclictest13846-21kworker/0:123:06:400
104949933,0cyclictest13324-21kworker/0:219:21:490
104959921,0cyclictest6314-21runrttasks20:31:421
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional