You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 22:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Tue Dec 23, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2396828886,0sleep00-21swapper/007:07:150
2400326665,0sleep20-21swapper/207:07:412
2407625857,0sleep10-21swapper/107:08:281
24358994141,0cyclictest1080-21md0_resync09:02:191
2410823928,0sleep30-21swapper/307:08:543
24357993131,0cyclictest1079-21md1_resync09:02:190
243599988,0cyclictest13243-21sensors09:46:152
243589970,0cyclictest0-21swapper/108:16:111
8305060,0irq/26-0000:00:8127-21munin-node12:26:113
8305060,0irq/26-0000:00:6023-21if_err_eth1.310:56:123
8305060,0irq/26-0000:00:29854-21tune2fs10:16:083
8305060,0irq/26-0000:00:28479-21kthreadcore12:01:123
8305060,0irq/26-0000:00:2762-21tr09:41:113
8305060,0irq/26-0000:00:23496-21ls11:56:193
8305060,0irq/26-0000:00:20989-21vmstat07:26:183
8305060,0irq/26-0000:00:18040-21load12:11:133
8305060,0irq/26-0000:00:1484-21sh12:40:543
8305060,0irq/26-0000:00:13782-21perl11:36:123
8305060,0irq/26-0000:00:11782-21python08:26:183
8305060,0irq/26-0000:00:11379-21grep09:11:123
339960,0migration/324577-21latency07:51:113
339960,0migration/322424-21sh11:05:083
339960,0migration/314391-21kthreadcore08:31:113
339960,0migration/311131-21sh12:09:353
29815060,0irq/29-eth0-tx-6809-21cat09:10:540
29805060,0irq/28-eth0-rx-31283-21ssh11:46:063
29805060,0irq/28-eth0-rx-24986-21ssh11:21:323
29805060,0irq/28-eth0-rx-24986-21ssh11:21:323
29805060,0irq/28-eth0-rx-16866-21ssh10:41:273
29805060,0irq/28-eth0-rx-14803-21yum11:51:233
27160,0ktimersoftd/20-21swapper/212:40:412
243599966,0cyclictest0-21swapper/208:28:392
243599960,0cyclictest0-21swapper/212:13:092
243599960,0cyclictest0-21swapper/210:19:382
243599960,0cyclictest0-21swapper/208:21:522
243599960,0cyclictest0-21swapper/207:19:072
243589960,0cyclictest0-21swapper/108:50:231
8305050,0irq/26-0000:00:7265-21kthreadcore07:41:113
8305050,0irq/26-0000:00:4124-21kthreadcore08:01:103
8305050,0irq/26-0000:00:3783-21crond10:30:543
8305050,0irq/26-0000:00:32105-21ntp_kernel_pll_07:56:143
8305050,0irq/26-0000:00:31804-21switchtime09:21:183
8305050,0irq/26-0000:00:29904-21tr10:51:153
8305050,0irq/26-0000:00:2645-21ssh09:26:093
8305050,0irq/26-0000:00:25096-21kthreadcore08:16:113
8305050,0irq/26-0000:00:24992-21df_inode07:11:093
8305050,0irq/26-0000:00:24238-21ssh09:19:183
8305050,0irq/26-0000:00:23235-21kthreadcore07:31:133
8305050,0irq/26-0000:00:21229-21if_eth008:36:103
8305050,0irq/26-0000:00:20432-21tr08:11:123
8305050,0irq/26-0000:00:20012-21munin-node10:11:113
8305050,0irq/26-0000:00:19946-21tune2fs10:46:073
8305050,0irq/26-0000:00:1908-21runrttasks08:42:403
8305050,0irq/26-0000:00:17818-21grep08:56:083
8305050,0irq/26-0000:00:10469-21kthreadcore07:21:133
8305050,0irq/26-0000:00:10167-21irqcore11:16:103
8305050,0irq/26-0000:00:0-21swapper/312:31:093
8305050,0irq/26-0000:00:0-21swapper/311:41:203
8305050,0irq/26-0000:00:0-21swapper/311:11:033
8305050,0irq/26-0000:00:0-21swapper/311:06:063
8305050,0irq/26-0000:00:0-21swapper/310:36:113
8305050,0irq/26-0000:00:0-21swapper/309:36:123
8305050,0irq/26-0000:00:0-21swapper/309:31:113
8305050,0irq/26-0000:00:0-21swapper/308:21:123
8305050,0irq/26-0000:00:0-21swapper/307:36:103
339950,0migration/3903-21kthreadcore12:21:113
339950,0migration/38235-21kthreadcore08:46:133
339950,0migration/36758-21latency_hist09:45:543
339950,0migration/332101-21munin-node07:16:043
339950,0migration/327017-21latency_hist11:25:553
339950,0migration/324628-21munin-node09:01:063
339950,0migration/324309-21kthreadcore12:16:133
339950,0migration/317952-21kthreadcore10:26:133
339950,0migration/313629-21kthreadcore10:06:123
339950,0migration/312542-21munin-node07:46:113
339950,0migration/312061-21kthreadcore08:51:123
339950,0migration/311105-21kthreadcore08:06:123
329950,0watchdog/330010-21cut09:56:133
329950,0watchdog/311165-21ssh11:32:173
29815050,0irq/29-eth0-tx-9133-21ntp_offset12:06:152
29815050,0irq/29-eth0-tx-6961-21munin-node12:26:122
29815050,0irq/29-eth0-tx-6722-21perl10:21:092
29815050,0irq/29-eth0-tx-5149-21kthreadcore10:36:122
29815050,0irq/29-eth0-tx-3524-21snmpd08:56:560
29815050,0irq/29-eth0-tx-26361-21ssh11:24:462
29815050,0irq/29-eth0-tx-26361-21ssh11:24:462
29815050,0irq/29-eth0-tx-25022-21munin-node09:56:062
29815050,0irq/29-eth0-tx-23376-21irqcore11:41:122
29815050,0irq/29-eth0-tx-22799-21cat11:05:542
29815050,0irq/29-eth0-tx-17358-21irqcore09:16:102
29815050,0irq/29-eth0-tx-13035-21munin-node11:36:242
29815050,0irq/29-eth0-tx-0-21swapper/211:35:122
29815050,0irq/29-eth0-tx-0-21swapper/211:26:112
29815050,0irq/29-eth0-tx-0-21swapper/210:47:042
29815050,0irq/29-eth0-tx-0-21swapper/210:41:122
29815050,0irq/29-eth0-tx-0-21swapper/210:26:122
29815050,0irq/29-eth0-tx-0-21swapper/210:03:082
29815050,0irq/29-eth0-tx-0-21swapper/209:54:442
29815050,0irq/29-eth0-tx-0-21swapper/209:33:012
29815050,0irq/29-eth0-tx-0-21swapper/209:21:162
29815050,0irq/29-eth0-tx-0-21swapper/008:53:250
29815050,0irq/29-eth0-tx-0-21swapper/007:39:500
29805050,0irq/28-eth0-rx-2142-21munin-node10:01:073
29805050,0irq/28-eth0-rx-0-21swapper/310:20:593
29805050,0irq/28-eth0-rx-0-21swapper/309:51:103
259950,0migration/29170-21cpuspeed_turbos11:16:062
259950,0migration/26219-21sh09:45:092
259950,0migration/24729-21kthreadcore11:46:132
259950,0migration/23595-21kthreadcore12:21:112
259950,0migration/232076-21kthreadcore08:41:132
259950,0migration/228094-21taskset10:50:552
259950,0migration/226006-21kthreadcore08:16:112
259950,0migration/224241-21sh11:57:182
259950,0migration/224047-21kthreadcore08:36:122
259950,0migration/217939-21sh10:06:452
259950,0migration/215703-21kthreadcore08:51:142
259950,0migration/214306-21aten_r1power_en07:26:032
259950,0migration/213068-21missed_timers09:11:132
243599955,0cyclictest0-21swapper/212:31:542
243599955,0cyclictest0-21swapper/209:40:322
243599955,0cyclictest0-21swapper/208:02:002
243599950,0cyclictest0-21swapper/212:18:112
243599950,0cyclictest0-21swapper/207:11:422
243589955,0cyclictest0-21swapper/111:03:181
243589955,0cyclictest0-21swapper/110:49:161
243589955,0cyclictest0-21swapper/110:43:541
243589955,0cyclictest0-21swapper/110:12:321
243589955,0cyclictest0-21swapper/109:28:041
243589950,0cyclictest0-21swapper/111:17:011
243589950,0cyclictest0-21swapper/109:56:171
243579955,0cyclictest0-21swapper/012:32:060
243579955,0cyclictest0-21swapper/012:28:290
243579955,0cyclictest0-21swapper/011:58:030
243579955,0cyclictest0-21swapper/009:48:560
243579955,0cyclictest0-21swapper/009:40:060
243579955,0cyclictest0-21swapper/008:04:050
243579953,0cyclictest8255-21kworker/0:010:13:130
243579950,0cyclictest0-21swapper/012:24:510
243579950,0cyclictest0-21swapper/012:15:050
243579950,0cyclictest0-21swapper/009:28:070
243579950,0cyclictest0-21swapper/009:16:460
19150,0ktimersoftd/10-21swapper/107:37:521
179950,0migration/15101-21iostat11:31:111
179950,0migration/15067-21munin-node11:46:151
179950,0migration/14564-21unixbench_multi12:21:191
179950,0migration/1417-21munin-node09:06:091
179950,0migration/132382-21kthreadcore08:21:121
179950,0migration/131835-21latency_hist07:15:541
179950,0migration/122166-21munin-node12:16:101
179950,0migration/121180-21kthreadcore11:21:131
179950,0migration/121180-21kthreadcore11:21:131
179950,0migration/120099-21kthreadcore11:56:111
179950,0migration/11751-21fschecks_time08:01:061
179950,0migration/115108-21kthreadcore09:31:141
149950,0migration/05909-21sh10:56:110
149950,0migration/05791-21kthreadcore12:06:120
149950,0migration/04517-21fschecks_count11:31:080
149950,0migration/031608-21df_abs11:46:080
149950,0migration/031504-21sh11:10:070
149950,0migration/031086-21kthreadcore12:01:130
149950,0migration/026545-21taskset10:47:480
149950,0migration/024629-21sh09:55:410
149950,0migration/019633-21sh10:10:430
149950,0migration/01659-21sh11:26:400
149950,0migration/014696-21kthreadcore11:36:160
149950,0migration/011128-21kthreadcore11:51:130
329940,0watchdog/30-21swapper/309:06:133
29815040,0irq/29-eth0-tx-6738-21diskmemload09:26:162
29815040,0irq/29-eth0-tx-26048-21ssh12:00:582
29815040,0irq/29-eth0-tx-19982-21ssh10:11:012
29815040,0irq/29-eth0-tx-1080-21md0_resync11:06:052
29815040,0irq/29-eth0-tx-0-21swapper/210:31:152
29815040,0irq/29-eth0-tx-0-21swapper/008:46:070
29815040,0irq/29-eth0-tx-0-21swapper/008:16:060
29815040,0irq/29-eth0-tx-0-21swapper/008:06:160
29815040,0irq/29-eth0-tx-0-21swapper/007:31:150
29815040,0irq/29-eth0-tx-0-21swapper/007:21:160
259940,0migration/29475-21kthreadcore10:56:142
259940,0migration/27889-21latency_hist11:50:542
259940,0migration/26816-21latency_hist09:10:542
259940,0migration/23944-21kthreadcore11:11:122
259940,0migration/222756-21runrttasks08:15:172
259940,0migration/222512-21fschecks_time07:31:092
259940,0migration/220426-21kthreadcore08:56:122
259940,0migration/211963-21fschecks_count07:46:082
243599944,0cyclictest0-21swapper/208:32:352
243599943,0cyclictest9464-21awk08:06:102
243599943,0cyclictest3524-21snmpd07:56:032
243599943,0cyclictest3524-21snmpd07:42:032
243589944,0cyclictest0-21swapper/109:46:031
243589944,0cyclictest0-21swapper/109:23:491
243589943,0cyclictest9578-21tr11:51:131
243589943,0cyclictest6050-21awk10:56:121
243589943,0cyclictest3524-21snmpd08:07:481
243589943,0cyclictest28590-21awk08:41:101
243589943,0cyclictest27284-21sadc08:40:531
243589943,0cyclictest25264-21awk10:31:091
243589943,0cyclictest24130-21awk11:06:091
243589943,0cyclictest14105-21awk11:36:141
243579944,0cyclictest0-21swapper/010:34:200
243579944,0cyclictest0-21swapper/010:21:050
243579944,0cyclictest0-21swapper/009:22:260
243579943,0cyclictest3571-21lldpd10:45:440
243579943,0cyclictest3524-21snmpd09:44:340
243579943,0cyclictest3250-21irqbalance10:17:000
243579943,0cyclictest3250-21irqbalance07:51:100
243579943,0cyclictest3250-21irqbalance07:46:300
243579943,0cyclictest3250-21irqbalance07:41:300
243579943,0cyclictest3250-21irqbalance07:26:200
243579943,0cyclictest3250-21irqbalance07:20:000
243579943,0cyclictest3250-21irqbalance07:11:300
243579943,0cyclictest28727-21cut08:41:110
243579943,0cyclictest26631-21ps09:56:120
243579943,0cyclictest21177-21awk08:36:100
243579943,0cyclictest16015-21awk10:26:100
243579943,0cyclictest10365-21kworker/0:109:33:490
249940,0watchdog/225419-21perl09:01:092
249940,0watchdog/20-21swapper/207:21:122
179940,0migration/1855-21kthreadcore10:16:121
179940,0migration/14095-21kthreadcore10:01:111
179940,0migration/129815-21latency_hist09:40:541
179940,0migration/128680-21kthreadcore12:01:121
179940,0migration/12531-21kthreadcore10:36:111
179940,0migration/12301-21kthreadcore11:11:111
179940,0migration/118208-21idleruntime08:56:101
179940,0migration/114688-21kthreadcore08:31:111
179940,0migration/114542-21kthreadcore07:46:121
179940,0migration/114193-21latency_hist07:25:541
179940,0migration/113947-21kthreadcore10:06:131
179940,0migration/110929-21kthreadcore12:26:131
169940,0watchdog/19301-21kthreadcore07:21:121
169940,0watchdog/18900-21grep12:06:131
169940,0watchdog/14575-21ssh09:41:211
169940,0watchdog/131973-21ssh12:36:261
169940,0watchdog/121250-21grep08:11:131
169940,0watchdog/119520-21kthreadcore12:31:141
169940,0watchdog/118555-21kthreadcore10:26:131
169940,0watchdog/113111-21cut08:51:131
169940,0watchdog/10-21swapper/110:51:091
169940,0watchdog/10-21swapper/109:13:211
169940,0watchdog/10-21swapper/107:40:561
169940,0watchdog/10-21swapper/107:31:121
149940,0migration/07503-21timerandwakeup10:36:200
149940,0migration/06225-21rt-features11:11:150
149940,0migration/04196-21kthreadcore10:01:110
149940,0migration/030156-21kthreadcore07:56:120
149940,0migration/028098-21sh10:50:560
149940,0migration/02641-21kthreadcore08:21:130
149940,0migration/025298-21aten2.4-expect12:36:080
149940,0migration/016628-21kthreadcore08:31:120
149940,0migration/010919-21kthreadcore11:16:100
243599932,0cyclictest30010-21awk07:36:092
243599930,0cyclictest3275-21perl08:46:082
243589932,0cyclictest6539-21fschecks_time10:21:091
243589932,0cyclictest20407-21sh12:15:021
243589932,0cyclictest19823-21awk07:51:081
243589932,0cyclictest16435-21cat09:16:061
243589932,0cyclictest16124-21cpuspeed_turbos09:51:051
243589930,0cyclictest25277-21perl07:11:101
243579933,0cyclictest24047-21kworker/0:211:20:590
243579933,0cyclictest24047-21kworker/0:211:20:590
243579933,0cyclictest13654-21kworker/0:012:15:560
243579933,0cyclictest13654-21kworker/0:011:41:040
243579933,0cyclictest10365-21kworker/0:109:10:590
243579933,0cyclictest10365-21kworker/0:108:25:550
243579933,0cyclictest10365-21kworker/0:108:10:590
243579932,0cyclictest14735-21tune2fs11:01:090
169930,0watchdog/131600-21kthreadcore07:56:131
169930,0watchdog/10-21swapper/111:41:131
169930,0watchdog/10-21swapper/111:25:571
169930,0watchdog/10-21swapper/108:26:131
249920,0watchdog/20-21swapper/207:50:572
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional