You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 01:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Tue Nov 25, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1735523829,0sleep10-21swapper/118:18:291
1719423727,0sleep20-21swapper/218:16:382
1717523727,0sleep30-21swapper/318:16:233
1714323725,0sleep00-21swapper/018:16:000
175419988,0cyclictest23871-21snmpd20:25:022
175299988,0cyclictest23871-21snmpd22:25:240
329970,0watchdog/30-21swapper/323:30:033
329970,0watchdog/30-21swapper/322:20:313
329970,0watchdog/30-21swapper/321:50:273
329970,0watchdog/30-21swapper/320:35:043
29925070,0irq/32-eth0-rx-0-21swapper/323:15:023
29925070,0irq/32-eth0-rx-0-21swapper/321:55:023
8315060,0irq/26-0000:00:12626-21crond22:39:432
339960,0migration/330686-21kthreadcore23:45:013
339960,0migration/313488-21sh21:40:213
329960,0watchdog/330209-21cut22:10:033
329960,0watchdog/30-21swapper/323:27:353
329960,0watchdog/30-21swapper/323:20:313
329960,0watchdog/30-21swapper/323:10:513
329960,0watchdog/30-21swapper/323:01:353
329960,0watchdog/30-21swapper/322:51:353
329960,0watchdog/30-21swapper/322:37:033
329960,0watchdog/30-21swapper/322:28:353
329960,0watchdog/30-21swapper/322:14:593
329960,0watchdog/30-21swapper/321:25:113
329960,0watchdog/30-21swapper/321:15:393
329960,0watchdog/30-21swapper/321:13:513
329960,0watchdog/30-21swapper/320:54:483
329960,0watchdog/30-21swapper/320:44:043
329960,0watchdog/30-21swapper/319:30:043
329960,0watchdog/30-21swapper/318:45:003
29935060,0irq/33-eth0-tx-28624-21munin-node18:49:590
29935060,0irq/33-eth0-tx-14223-21munin-node18:40:020
29935060,0irq/33-eth0-tx-0-21swapper/019:44:550
29935060,0irq/33-eth0-tx-0-21swapper/019:39:540
29935060,0irq/33-eth0-tx-0-21swapper/019:35:030
29935060,0irq/33-eth0-tx-0-21swapper/018:48:420
29935060,0irq/33-eth0-tx-0-21swapper/018:34:570
29925060,0irq/32-eth0-rx-4439-21python21:35:073
29925060,0irq/32-eth0-rx-28511-21users21:30:083
29925060,0irq/32-eth0-rx-20407-21open_inodes20:30:053
29925060,0irq/32-eth0-rx-18314-21cut21:45:013
29925060,0irq/32-eth0-rx-10572-21ssh21:20:043
29925060,0irq/32-eth0-rx-0-21swapper/323:41:383
29925060,0irq/32-eth0-rx-0-21swapper/322:31:563
29925060,0irq/32-eth0-rx-0-21swapper/322:08:123
29925060,0irq/32-eth0-rx-0-21swapper/321:05:153
29925060,0irq/32-eth0-rx-0-21swapper/321:00:083
29925060,0irq/32-eth0-rx-0-21swapper/320:19:493
29925060,0irq/32-eth0-rx-0-21swapper/320:02:133
175419966,0cyclictest0-21swapper/222:44:432
175419960,0cyclictest0-21swapper/220:48:412
149960,0migration/030647-21df_abs22:49:560
149960,0migration/026400-21sh23:42:140
149960,0migration/026255-21if_eth1.219:10:010
149960,0migration/026094-21kthreadcore22:45:030
149960,0migration/02117-21sh23:46:310
149960,0migration/01863-21sendmail_mailqu22:30:050
8315050,0irq/26-0000:00:24674-21vmstat19:50:102
339950,0migration/329548-21kthreadcore19:55:033
339950,0migration/322142-21fschecks_count22:44:593
339950,0migration/315121-21kthreadcore22:40:023
339950,0migration/313929-21unixbench_singl20:05:103
329950,0watchdog/320438-21python18:40:083
329950,0watchdog/30-21swapper/323:34:593
329950,0watchdog/30-21swapper/321:59:433
329950,0watchdog/30-21swapper/320:49:483
329950,0watchdog/30-21swapper/320:44:443
329950,0watchdog/30-21swapper/320:15:123
329950,0watchdog/30-21swapper/320:11:203
329950,0watchdog/30-21swapper/319:49:443
329950,0watchdog/30-21swapper/319:45:043
329950,0watchdog/30-21swapper/319:39:523
329950,0watchdog/30-21swapper/319:34:523
329950,0watchdog/30-21swapper/319:25:323
329950,0watchdog/30-21swapper/319:20:043
329950,0watchdog/30-21swapper/319:16:243
329950,0watchdog/30-21swapper/319:05:003
329950,0watchdog/30-21swapper/319:01:083
329950,0watchdog/30-21swapper/318:55:203
329950,0watchdog/30-21swapper/318:55:203
329950,0watchdog/30-21swapper/318:53:083
329950,0watchdog/30-21swapper/318:35:003
329950,0watchdog/30-21swapper/318:29:563
329950,0watchdog/30-21swapper/318:29:003
329950,0watchdog/30-21swapper/318:20:563
29935050,0irq/33-eth0-tx-23593-21ntpq23:20:052
29935050,0irq/33-eth0-tx-18525-21kthreadcore23:00:012
29935050,0irq/33-eth0-tx-18403-21if_err_eth120:50:002
29935050,0irq/33-eth0-tx-16283-21ssh22:02:422
29935050,0irq/33-eth0-tx-15352-21munin-node21:45:092
29935050,0irq/33-eth0-tx-14376-21ssh20:29:552
29935050,0irq/33-eth0-tx-13135-21munin-node21:25:072
29935050,0irq/33-eth0-tx-10895-21munin-node21:05:062
29935050,0irq/33-eth0-tx-0-21swapper/221:15:212
29935050,0irq/33-eth0-tx-0-21swapper/221:00:562
29935050,0irq/33-eth0-tx-0-21swapper/020:15:130
29935050,0irq/33-eth0-tx-0-21swapper/019:17:040
29925050,0irq/32-eth0-rx-24973-21if_err_eth1.223:04:593
29925050,0irq/32-eth0-rx-0-21swapper/322:54:503
29925050,0irq/32-eth0-rx-0-21swapper/320:24:563
29925050,0irq/32-eth0-rx-0-21swapper/319:09:563
259950,0migration/25867-21irqcore21:20:012
259950,0migration/232463-21sh21:54:532
259950,0migration/231703-21taskset23:08:142
259950,0migration/22540-21sh22:30:522
259950,0migration/220226-21fschecks_time21:09:562
259950,0migration/215264-21munin-node23:15:072
259950,0migration/213302-21kthreadcore19:00:022
259950,0migration/212968-21kthreadcore19:45:012
259950,0migration/211178-21munin-node22:19:562
175419955,0cyclictest0-21swapper/221:43:422
175419955,0cyclictest0-21swapper/221:34:152
175419955,0cyclictest0-21swapper/220:23:182
175419950,0cyclictest0-21swapper/222:43:372
175419950,0cyclictest0-21swapper/220:15:152
175419950,0cyclictest0-21swapper/218:37:592
175309955,0cyclictest0-21swapper/122:24:341
175309955,0cyclictest0-21swapper/121:19:371
175309950,0cyclictest0-21swapper/123:34:531
175309950,0cyclictest0-21swapper/123:21:011
175309950,0cyclictest0-21swapper/121:02:351
175309950,0cyclictest0-21swapper/119:00:051
175299955,0cyclictest0-21swapper/023:15:000
179950,0migration/18698-21timerwakeupswit23:30:121
179950,0migration/132034-21sh21:54:181
179950,0migration/121716-21sh21:45:061
179950,0migration/120562-21kthreadcore22:05:021
179950,0migration/115979-21kthreadcore20:30:011
179950,0migration/113979-21latency_hist20:29:431
149950,0migration/09745-21kthreadcore20:05:010
149950,0migration/08987-21kthreadcore19:20:010
149950,0migration/07950-21kthreadcore18:55:010
149950,0migration/07950-21kthreadcore18:55:010
149950,0migration/06085-21sh21:38:500
149950,0migration/03079-21fschecks_time20:59:560
149950,0migration/030595-21irqrtprio20:20:010
149950,0migration/029065-21fschecks_time21:14:560
149950,0migration/028848-21kthreadcore23:05:020
149950,0migration/027125-21kthreadcore18:25:010
149950,0migration/024311-21kthreadcore19:30:030
149950,0migration/019514-21kthreadcore19:25:040
149950,0migration/018987-21kthreadcore21:45:010
149950,0migration/018574-21munin-node20:50:010
149950,0migration/017376-21timerandwakeup21:05:090
149950,0migration/016527-21sh22:03:250
149950,0migration/016183-21kthreadcore19:00:030
149950,0migration/015763-21fschecks_count22:59:570
149950,0migration/01396-21taskset22:13:420
149950,0migration/013554-21fschecks_count22:39:580
149950,0migration/013113-21munin-node21:40:070
149950,0migration/01149-21idleruntime23:09:590
149950,0migration/011234-21fschecks_count23:34:580
149950,0migration/011036-21cpuspeed_turbos22:19:560
29935040,0irq/33-eth0-tx-8960-21ssh23:30:312
29935040,0irq/33-eth0-tx-6315-21iceccd23:45:182
29935040,0irq/33-eth0-tx-31861-21ssh20:39:522
29935040,0irq/33-eth0-tx-12702-21ntp_states22:55:032
29935040,0irq/33-eth0-tx-0-21swapper/223:42:272
29935040,0irq/33-eth0-tx-0-21swapper/223:25:042
29935040,0irq/33-eth0-tx-0-21swapper/223:09:552
29935040,0irq/33-eth0-tx-0-21swapper/221:34:592
29935040,0irq/33-eth0-tx-0-21swapper/220:55:042
259940,0migration/25834-21kthreadcore18:55:002
259940,0migration/25834-21kthreadcore18:55:002
259940,0migration/229809-21diskmemload20:35:132
259940,0migration/229752-21kthreadcore19:10:032
259940,0migration/227986-21kthreadcore21:50:022
259940,0migration/22691-21kthreadcore22:50:012
259940,0migration/225901-21aten_r1power_en22:09:562
259940,0migration/222658-21kthreadcore22:05:022
259940,0migration/22134-21aten_r1power_fr22:14:542
259940,0migration/218688-21kthreadcore18:40:032
259940,0migration/216182-21kthreadcore23:35:022
259940,0migration/216018-21irqrtprio20:09:592
259940,0migration/21390-21kthreadcore19:35:022
175419943,0cyclictest8504-21awk20:05:002
175419943,0cyclictest3599-21lldpd19:57:122
175419943,0cyclictest3599-21lldpd19:05:122
175419943,0cyclictest23871-21snmpd18:21:502
175419943,0cyclictest22656-21awk19:30:012
175419943,0cyclictest20451-21awk22:25:012
175419943,0cyclictest1134-21awk19:59:592
175309944,0cyclictest28787-21diskmemload22:35:081
175309944,0cyclictest0-21swapper/121:56:401
175309943,0cyclictest5551-21awk21:19:591
175309943,0cyclictest3182-21awk22:14:581
175309943,0cyclictest3070-21sadc23:49:431
175309943,0cyclictest27130-21awk20:55:001
175309943,0cyclictest26187-21awk19:10:011
175309943,0cyclictest23871-21snmpd22:01:051
175309943,0cyclictest23871-21snmpd21:24:501
175309943,0cyclictest23871-21snmpd20:44:501
175309943,0cyclictest23871-21snmpd20:42:201
175309943,0cyclictest23871-21snmpd19:52:051
175309943,0cyclictest23871-21snmpd19:44:051
175309943,0cyclictest23871-21snmpd19:36:201
175309943,0cyclictest23871-21snmpd18:43:051
175309943,0cyclictest23871-21snmpd18:35:051
175309943,0cyclictest23871-21snmpd18:25:201
175309943,0cyclictest23871-21snmpd18:22:051
175309943,0cyclictest22504-21awk18:45:011
175309943,0cyclictest14054-21awk22:40:001
175309940,0cyclictest0-21swapper/120:15:551
175299943,0cyclictest801-21awk18:29:590
175299943,0cyclictest7990-21kworker/0:021:53:400
175299943,0cyclictest7990-21kworker/0:021:33:160
175299943,0cyclictest6489-21kworker/0:222:05:240
175299943,0cyclictest6489-21kworker/0:221:59:400
175299943,0cyclictest26388-21awk19:55:000
175299943,0cyclictest18961-21awk19:04:580
175299943,0cyclictest15677-21awk20:09:580
175299943,0cyclictest15380-21awk20:30:000
175299943,0cyclictest14339-21awk21:25:000
179940,0migration/18090-21kthreadcore18:55:011
179940,0migration/18090-21kthreadcore18:55:011
179940,0migration/17316-21latency_hist20:04:431
179940,0migration/17219-21aten_r1power_cu19:19:551
179940,0migration/16703-21runrttasks21:39:531
179940,0migration/1643-21kthreadcore18:50:031
179940,0migration/14361-21kthreadcore18:30:021
179940,0migration/12843-21kthreadcore21:35:021
179940,0migration/127887-21munin-node22:45:071
179940,0migration/126898-21kthreadcore23:05:011
179940,0migration/124948-21sh20:52:431
179940,0migration/123939-21kthreadcore21:30:001
179940,0migration/123659-21fschecks_count20:34:581
179940,0migration/123649-21kthreadcore23:40:051
179940,0migration/12242-21kthreadcore19:15:001
179940,0migration/119592-21kthreadcore23:00:021
179940,0migration/118680-21sh21:08:051
179940,0migration/111878-21kthreadcore23:15:021
169940,0watchdog/17311-21ssh23:11:031
169940,0watchdog/129069-21basename20:19:561
169940,0watchdog/126242-21ssh22:25:391
169940,0watchdog/125121-21kthreadcore19:30:041
169940,0watchdog/118513-21grep19:25:041
169940,0watchdog/11281-21sh20:00:001
169940,0watchdog/10-21swapper/122:29:591
169940,0watchdog/10-21swapper/121:09:591
149940,0migration/08479-21sensors_temp23:30:080
149940,0migration/023347-21cpuspeed_turbos20:34:570
149940,0migration/021614-21kthreadcore21:10:000
149940,0migration/018237-21aten_r1power_po19:49:560
8315030,0irq/26-0000:00:0-21swapper/218:29:442
249930,0watchdog/20-21swapper/219:25:042
175419933,0cyclictest3599-21lldpd18:26:112
175419932,0cyclictest6226-21cut19:40:012
175419932,0cyclictest2593-21cat18:50:102
175419932,0cyclictest22474-21awk18:45:012
175419931,0cyclictest17528-21cyclictest19:19:592
175309932,0cyclictest30984-21grep22:49:571
175309932,0cyclictest23871-21snmpd19:58:051
175309932,0cyclictest23871-21snmpd19:05:051
175309932,0cyclictest15631-21awk20:09:571
175299933,0cyclictest7990-21kworker/0:021:19:540
175299933,0cyclictest7990-21kworker/0:020:54:460
175299933,0cyclictest7990-21kworker/0:020:44:440
175299933,0cyclictest7990-21kworker/0:020:39:460
175299933,0cyclictest7990-21kworker/0:020:24:520
175299933,0cyclictest7990-21kworker/0:019:59:560
175299933,0cyclictest6489-21kworker/0:223:24:530
175299933,0cyclictest6489-21kworker/0:223:19:430
175299933,0cyclictest6489-21kworker/0:222:54:430
175299933,0cyclictest6489-21kworker/0:222:34:470
175299933,0cyclictest6489-21kworker/0:222:14:480
175299933,0cyclictest11962-21kworker/0:118:19:440
179930,0migration/111075-21aten_r1power_vo19:44:551
169930,0watchdog/10-21swapper/123:26:591
169930,0watchdog/10-21swapper/122:55:351
169930,0watchdog/10-21swapper/122:10:031
175419920,0cyclictest32079-21latency_hist19:14:442
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional