You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 02:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Sun Nov 23, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1210623725,0sleep00-21swapper/018:17:370
1215823623,0sleep10-21swapper/118:18:181
1194723627,0sleep20-21swapper/218:15:472
1224023126,0sleep30-21swapper/318:19:173
8315070,0irq/26-0000:00:0-21swapper/318:24:513
339970,0migration/34975-21fschecks_time23:15:033
329970,0watchdog/35790-21kthreadcore22:00:113
329970,0watchdog/331344-21sh21:19:233
329970,0watchdog/317365-21if_eth018:45:073
329970,0watchdog/30-21swapper/318:20:113
29925070,0irq/32-eth0-rx-787-21irqrtprio21:20:073
29925070,0irq/32-eth0-rx-4703-21sed20:45:113
29925070,0irq/32-eth0-rx-0-21swapper/323:36:183
339960,0migration/327612-21sh23:09:273
339960,0migration/326749-21munin-node21:35:073
339960,0migration/320054-21sh23:21:343
339960,0migration/319705-21idleruntime-cro19:09:503
339960,0migration/315952-21sh22:42:013
329960,0watchdog/33965-21sh21:00:393
329960,0watchdog/30-21swapper/322:50:223
329960,0watchdog/30-21swapper/322:46:303
329960,0watchdog/30-21swapper/322:33:263
329960,0watchdog/30-21swapper/322:10:073
329960,0watchdog/30-21swapper/321:11:273
329960,0watchdog/30-21swapper/320:58:113
329960,0watchdog/30-21swapper/320:35:113
329960,0watchdog/30-21swapper/320:25:393
329960,0watchdog/30-21swapper/320:10:073
329960,0watchdog/30-21swapper/320:07:273
329960,0watchdog/30-21swapper/319:50:153
329960,0watchdog/30-21swapper/319:30:113
329960,0watchdog/30-21swapper/319:20:113
329960,0watchdog/30-21swapper/319:09:513
329960,0watchdog/30-21swapper/318:55:513
329960,0watchdog/30-21swapper/318:34:513
329960,0watchdog/30-21swapper/318:30:033
29925060,0irq/32-eth0-rx-7511-21cut21:40:103
29925060,0irq/32-eth0-rx-5997-21ssh22:19:453
29925060,0irq/32-eth0-rx-24127-21perl23:45:053
29925060,0irq/32-eth0-rx-17766-21ssh23:01:013
29925060,0irq/32-eth0-rx-15323-21port_sshpriv20:30:123
29925060,0irq/32-eth0-rx-0-21swapper/323:11:553
29925060,0irq/32-eth0-rx-0-21swapper/322:54:543
29925060,0irq/32-eth0-rx-0-21swapper/322:35:103
29925060,0irq/32-eth0-rx-0-21swapper/322:27:383
29925060,0irq/32-eth0-rx-0-21swapper/322:07:103
29925060,0irq/32-eth0-rx-0-21swapper/321:55:343
29925060,0irq/32-eth0-rx-0-21swapper/321:46:123
29925060,0irq/32-eth0-rx-0-21swapper/321:30:093
29925060,0irq/32-eth0-rx-0-21swapper/321:26:523
29925060,0irq/32-eth0-rx-0-21swapper/321:04:553
29925060,0irq/32-eth0-rx-0-21swapper/320:20:073
123729965,0cyclictest16240-21cut22:25:072
123639966,0cyclictest0-21swapper/023:24:340
8315050,0irq/26-0000:00:0-21swapper/219:44:512
8315050,0irq/26-0000:00:0-21swapper/219:14:512
339950,0migration/37009-21fschecks_time22:20:043
339950,0migration/329004-21kthreadcore19:35:123
339950,0migration/324765-21kthreadcore23:25:083
339950,0migration/323483-21taskset18:50:053
339950,0migration/315756-21idleruntime23:40:063
329950,0watchdog/330425-21df_abs23:30:063
329950,0watchdog/325170-21needreboot21:50:113
329950,0watchdog/30-21swapper/320:49:553
329950,0watchdog/30-21swapper/320:00:273
329950,0watchdog/30-21swapper/319:56:273
329950,0watchdog/30-21swapper/319:44:513
329950,0watchdog/30-21swapper/319:40:553
329950,0watchdog/30-21swapper/319:25:193
329950,0watchdog/30-21swapper/319:15:553
329950,0watchdog/30-21swapper/318:59:553
329950,0watchdog/30-21swapper/318:39:553
29935050,0irq/33-eth0-tx-9247-21munin-node19:25:150
29935050,0irq/33-eth0-tx-629-21kthreadcore19:40:090
29935050,0irq/33-eth0-tx-32606-21ssh22:50:202
29935050,0irq/33-eth0-tx-3013-21interrupts21:40:082
29935050,0irq/33-eth0-tx-25492-21seq21:50:142
29935050,0irq/33-eth0-tx-23964-21aten2.4-expect20:20:020
29935050,0irq/33-eth0-tx-20830-21munin-node20:55:052
29935050,0irq/33-eth0-tx-19437-21munin-node23:05:062
29935050,0irq/33-eth0-tx-12944-21munin-node19:50:040
29935050,0irq/33-eth0-tx-0-21swapper/223:37:112
29935050,0irq/33-eth0-tx-0-21swapper/223:10:082
29935050,0irq/33-eth0-tx-0-21swapper/221:57:212
29935050,0irq/33-eth0-tx-0-21swapper/221:44:552
29935050,0irq/33-eth0-tx-0-21swapper/220:42:432
29935050,0irq/33-eth0-tx-0-21swapper/019:30:040
29935050,0irq/33-eth0-tx-0-21swapper/018:45:380
29925050,0irq/32-eth0-rx-0-21swapper/320:39:563
29925050,0irq/32-eth0-rx-0-21swapper/320:15:073
27150,0ktimersoftd/20-21swapper/222:20:182
259950,0migration/29185-21kthreadcore19:45:102
259950,0migration/26362-21kthreadcore18:35:102
259950,0migration/230998-21sh22:32:062
259950,0migration/230032-21kthreadcore20:00:092
259950,0migration/219151-21smart_sda19:50:142
259950,0migration/213709-21irqrtprio20:50:092
259950,0migration/210032-21gltestperf18:40:062
249950,0watchdog/27928-21missed_timers19:20:112
179950,0migration/18637-21aten_r1power_en22:40:011
179950,0migration/18081-21kthreadcore22:00:111
179950,0migration/13844-21sh23:14:231
179950,0migration/129496-21aten_r1power_cu21:00:021
179950,0migration/122652-21kthreadcore23:05:111
179950,0migration/120702-21kthreadcore20:35:101
179950,0migration/116473-21missed_timers21:45:111
169950,0watchdog/122955-21ssh22:28:421
149950,0migration/08013-21sh22:39:010
149950,0migration/07576-21kthreadcore22:55:100
149950,0migration/06034-21kthreadcore19:20:100
149950,0migration/05821-21kthreadcore20:05:110
149950,0migration/03882-21kthreadcore21:40:090
149950,0migration/03383-21rt-features21:00:130
149950,0migration/031069-21kthreadcore20:40:080
149950,0migration/018601-21kthreadcore22:25:090
149950,0migration/017894-21sh23:01:240
149950,0migration/015698-21timerandwakeup20:10:170
149950,0migration/015124-21kthreadcore22:05:080
149950,0migration/013314-21sh21:07:190
149950,0migration/010937-21munin-node21:45:050
123729955,0cyclictest2546-21nvme20:15:092
123729950,0cyclictest0-21swapper/223:33:182
123729950,0cyclictest0-21swapper/221:39:282
123649955,0cyclictest0-21swapper/120:32:081
123649950,0cyclictest0-21swapper/121:10:551
123649950,0cyclictest0-21swapper/120:53:531
123639955,0cyclictest0-21swapper/023:32:370
123639955,0cyclictest0-21swapper/022:50:310
123639955,0cyclictest0-21swapper/022:00:040
123639955,0cyclictest0-21swapper/021:19:420
123639954,0cyclictest29195-21sadc20:59:510
123639952,0cyclictest7897-21munin-run21:24:500
8315040,0irq/26-0000:00:24460-21perl19:35:082
8315040,0irq/26-0000:00:0-21swapper/223:44:502
8315040,0irq/26-0000:00:0-21swapper/220:24:512
8315040,0irq/26-0000:00:0-21swapper/219:54:512
29935040,0irq/33-eth0-tx-12689-21ssh22:04:542
29935040,0irq/33-eth0-tx-0-21swapper/223:25:152
29935040,0irq/33-eth0-tx-0-21swapper/222:59:572
29935040,0irq/33-eth0-tx-0-21swapper/222:44:592
29935040,0irq/33-eth0-tx-0-21swapper/222:39:542
29935040,0irq/33-eth0-tx-0-21swapper/222:11:282
29935040,0irq/33-eth0-tx-0-21swapper/222:00:022
29935040,0irq/33-eth0-tx-0-21swapper/221:25:062
29935040,0irq/33-eth0-tx-0-21swapper/221:09:532
29935040,0irq/33-eth0-tx-0-21swapper/221:06:132
29935040,0irq/33-eth0-tx-0-21swapper/221:00:032
29935040,0irq/33-eth0-tx-0-21swapper/220:44:562
29935040,0irq/33-eth0-tx-0-21swapper/220:30:042
29935040,0irq/33-eth0-tx-0-21swapper/019:10:050
29935040,0irq/33-eth0-tx-0-21swapper/018:55:080
259940,0migration/27684-21kthreadcore22:55:102
259940,0migration/230462-21latency_hist18:54:512
259940,0migration/22825-21kthreadcore22:15:082
259940,0migration/218931-21timerandwakeup18:20:182
259940,0migration/218791-21kthreadcore23:40:092
259940,0migration/218498-21kthreadcore21:30:092
259940,0migration/212219-21runrttasks19:01:432
259940,0migration/211396-21kthreadcore19:25:112
249940,0watchdog/224296-21python20:35:152
249940,0watchdog/224179-21sed20:20:032
249940,0watchdog/219845-21ssh23:20:582
249940,0watchdog/20-21swapper/218:45:072
19140,0ktimersoftd/10-21swapper/122:55:261
179940,0migration/19758-21kthreadcore19:00:091
179940,0migration/19708-21taskset20:45:181
179940,0migration/18955-21runrttasks19:24:211
179940,0migration/17313-21kthreadcore19:45:091
179940,0migration/13313-21irqrtprio21:40:091
179940,0migration/132229-21kthreadcore23:30:111
179940,0migration/131714-21kthreadcore21:55:091
179940,0migration/131232-21fschecks_count18:55:041
179940,0migration/127946-21hwlatdetect18:30:061
179940,0migration/124992-21kthreadcore21:15:071
179940,0migration/124518-21kthreadcore19:10:111
179940,0migration/120803-21kthreadcore21:50:081
179940,0migration/116965-21pmu-power23:00:111
179940,0migration/114630-21kthreadcore19:05:111
179940,0migration/113883-21kthreadcore21:25:091
179940,0migration/112759-21switchtime22:20:161
179940,0migration/111679-21kthreadcore18:40:101
179940,0migration/11055-21processes20:00:131
169940,0watchdog/132302-21ssh21:35:311
169940,0watchdog/1302-21kthreadcore19:15:111
169940,0watchdog/124178-21df20:20:031
169940,0watchdog/122669-21sensors19:30:151
169940,0watchdog/110471-21ntp_states23:15:101
169940,0watchdog/10-21swapper/123:40:061
169940,0watchdog/10-21swapper/122:30:061
169940,0watchdog/10-21swapper/119:40:031
169940,0watchdog/10-21swapper/118:50:071
169940,0watchdog/10-21swapper/118:20:111
149940,0migration/07152-21kthreadcore23:15:070
149940,0migration/05222-21kthreadcore20:45:120
149940,0migration/031611-21kthreadcore21:55:090
149940,0migration/029545-21taskset22:14:190
149940,0migration/028985-21kthreadcore18:50:120
149940,0migration/028954-21kthreadcore18:30:080
149940,0migration/027492-21aten_r1power_vo20:00:030
149940,0migration/027436-21cpuspeed_turbos19:15:050
149940,0migration/025641-21taskset20:37:480
149940,0migration/023155-21taskset21:30:130
149940,0migration/022713-21proc_pri20:15:130
149940,0migration/022160-21kthreadcore19:55:090
149940,0migration/021978-21runrttasks21:12:180
149940,0migration/014902-21kthreadcore19:05:110
149940,0migration/012857-21fschecks_time20:50:060
149940,0migration/011909-21kthreadcore20:30:090
149940,0migration/011354-21kthreadcore18:40:100
123729944,0cyclictest0-21swapper/221:20:042
123729944,0cyclictest0-21swapper/218:25:142
123729943,0cyclictest5392-21awk23:15:052
123729943,0cyclictest3599-21lldpd22:34:502
123729943,0cyclictest3245-21awk20:05:082
123729943,0cyclictest24315-21awk21:15:062
123729943,0cyclictest23871-21snmpd19:10:352
123729943,0cyclictest23871-21snmpd19:09:352
123729943,0cyclictest23871-21snmpd18:59:502
123729943,0cyclictest23871-21snmpd18:31:352
123729943,0cyclictest17489-21awk19:30:082
123649944,0cyclictest0-21swapper/122:04:501
123649944,0cyclictest0-21swapper/120:10:131
123649943,0cyclictest958-21awk22:35:081
123649943,0cyclictest6964-21awk21:05:081
123649943,0cyclictest406-21awk21:20:061
123649943,0cyclictest3599-21lldpd22:46:211
123649943,0cyclictest3599-21lldpd19:58:081
123649943,0cyclictest24716-21awk19:35:091
123649943,0cyclictest24300-21awk23:45:061
123649943,0cyclictest23871-21snmpd20:40:351
123649943,0cyclictest23871-21snmpd20:15:501
123649943,0cyclictest23871-21snmpd20:06:351
123649943,0cyclictest23013-21sed21:30:121
123649943,0cyclictest22030-21awk20:55:081
123649943,0cyclictest20859-21awk18:25:071
123649943,0cyclictest14061-21awk23:20:071
123649943,0cyclictest10281-21awk19:25:081
123639944,0cyclictest0-21swapper/023:26:180
123639944,0cyclictest0-21swapper/022:20:430
123639944,0cyclictest0-21swapper/020:28:470
123639943,0cyclictest6856-21awk19:45:080
123639943,0cyclictest3599-21lldpd22:49:210
123639943,0cyclictest3599-21lldpd21:27:100
123639943,0cyclictest2918-21awk18:35:080
123639943,0cyclictest29127-21awk23:10:090
123639943,0cyclictest26660-21awk21:35:070
123639943,0cyclictest23871-21snmpd23:36:220
123639943,0cyclictest23871-21snmpd22:41:220
123639943,0cyclictest20537-21awk23:05:090
123639943,0cyclictest15873-21awk23:40:070
123639943,0cyclictest13612-21awk18:20:080
29935030,0irq/33-eth0-tx-0-21swapper/019:35:100
249930,0watchdog/20-21swapper/220:10:072
179930,0migration/129005-21sh22:12:471
169930,0watchdog/10-21swapper/118:45:071
123649932,0cyclictest7285-21awk23:35:071
123649932,0cyclictest29741-21latency_hist22:14:501
123649932,0cyclictest2879-21awk18:35:081
123649932,0cyclictest25880-21munin-node22:50:071
123649932,0cyclictest23871-21snmpd20:26:351
123649932,0cyclictest22551-21awk23:25:061
123649932,0cyclictest14046-21awk19:50:071
123639933,0cyclictest8781-21kworker/0:221:50:090
123639932,0cyclictest5917-21sed19:00:040
123639932,0cyclictest29987-21ssh22:14:540
123639932,0cyclictest24806-21awk22:30:060
123639932,0cyclictest23216-21munin-node23:45:160
123639930,0cyclictest19796-21munin-node18:25:040
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional