You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 20:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Apr 21, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1746023928,0sleep00-21swapper/006:56:450
1745723725,0sleep20-21swapper/206:56:432
304450350,0irq/25-eth10-21swapper/306:53:153
1736523424,0sleep10-21swapper/106:55:381
339960,0migration/37363-21sh09:30:223
339960,0migration/329903-21sh10:03:093
339960,0migration/329631-21sh11:23:413
339960,0migration/317127-21irqcore08:58:063
29715060,0irq/26-eth0-rx-32234-21cut11:48:043
29715060,0irq/26-eth0-rx-25509-21kthreadcore10:43:083
29715060,0irq/26-eth0-rx-15580-21iostat_ios12:18:053
176339966,0cyclictest0-21swapper/207:58:042
176339960,0cyclictest0-21swapper/212:03:042
149960,0migration/06200-21sh11:10:280
149960,0migration/023563-21sh09:40:540
149960,0migration/021519-21sh11:18:220
149960,0migration/016615-21munin-node10:58:050
8215050,0irq/24-0000:00:8227-21munin-run09:32:492
8215050,0irq/24-0000:00:27215-21latency_hist08:17:502
8215050,0irq/24-0000:00:25200-21idleruntime-cro07:27:502
35150,0ktimersoftd/30-21swapper/309:58:213
339950,0migration/39172-21kthreadcore11:13:063
339950,0migration/38873-21idleruntime10:13:043
339950,0migration/38212-21taskset08:27:463
339950,0migration/37717-21kernelversion07:38:063
339950,0migration/37046-21sh10:10:453
339950,0migration/35488-21missed_timers09:08:083
339950,0migration/332401-21munin-node11:08:033
339950,0migration/331095-21kthreadcore08:43:073
339950,0migration/33027-21kthreadcore12:08:073
339950,0migration/327911-21kthreadcore09:23:073
339950,0migration/327435-21taskset08:17:513
339950,0migration/326625-21kthreadcore08:38:073
339950,0migration/323899-21irqrtprio12:23:073
339950,0migration/321150-21latency10:38:083
339950,0migration/320790-21latency_hist08:12:503
339950,0migration/317076-21kthreadcore11:58:053
339950,0migration/316571-21aten_r1power_cu09:17:593
339950,0migration/315628-21taskset08:55:363
339950,0migration/313705-21fschecks_time07:43:033
339950,0migration/311127-21kthreadcore10:53:073
29725050,0irq/27-eth0-tx-31814-21munin-node10:48:112
29725050,0irq/27-eth0-tx-31157-21df12:08:022
29725050,0irq/27-eth0-tx-30488-21ssh11:26:112
29725050,0irq/27-eth0-tx-28424-21munin-node08:43:020
29725050,0irq/27-eth0-tx-26423-21kthreadcore09:03:072
29725050,0irq/27-eth0-tx-23651-21ssh10:01:492
29725050,0irq/27-eth0-tx-0-21swapper/209:53:062
29725050,0irq/27-eth0-tx-0-21swapper/007:40:470
29725050,0irq/27-eth0-tx-0-21swapper/007:28:160
29715050,0irq/26-eth0-rx-7854-21kthreadcore12:13:073
29715050,0irq/26-eth0-rx-5018-21unixbench-2d11:28:113
29715050,0irq/26-eth0-rx-30758-21ssh10:24:543
29715050,0irq/26-eth0-rx-27290-21cat09:03:073
29715050,0irq/26-eth0-rx-19404-21cut08:33:063
29715050,0irq/26-eth0-rx-142782chrt11:36:183
29715050,0irq/26-eth0-rx-0-21swapper/312:03:423
29715050,0irq/26-eth0-rx-0-21swapper/311:45:513
29715050,0irq/26-eth0-rx-0-21swapper/311:20:083
29715050,0irq/26-eth0-rx-0-21swapper/311:03:103
29715050,0irq/26-eth0-rx-0-21swapper/311:01:443
29715050,0irq/26-eth0-rx-0-21swapper/310:52:293
29715050,0irq/26-eth0-rx-0-21swapper/310:21:123
29715050,0irq/26-eth0-rx-0-21swapper/309:53:083
29715050,0irq/26-eth0-rx-0-21swapper/309:48:103
29715050,0irq/26-eth0-rx-0-21swapper/309:40:113
29715050,0irq/26-eth0-rx-0-21swapper/309:33:093
29715050,0irq/26-eth0-rx-0-21swapper/309:13:453
29715050,0irq/26-eth0-rx-0-21swapper/308:03:013
27150,0ktimersoftd/20-21swapper/210:14:122
259950,0migration/28309-21aten2.4-expect09:12:592
259950,0migration/217875-21munin-node06:58:012
259950,0migration/214249-21sh12:17:462
19150,0ktimersoftd/10-21swapper/110:14:121
176339954,0cyclictest3514-21snmpd07:14:182
176339950,0cyclictest0-21swapper/209:18:102
176319955,0cyclictest0-21swapper/110:37:131
176319954,0cyclictest3514-21snmpd11:49:041
176319950,0cyclictest0-21swapper/112:17:391
176319950,0cyclictest0-21swapper/111:45:391
176319950,0cyclictest0-21swapper/110:39:071
176319950,0cyclictest0-21swapper/109:58:071
176319950,0cyclictest0-21swapper/108:59:001
176309955,0cyclictest0-21swapper/010:48:120
176309950,0cyclictest0-21swapper/012:06:430
176309950,0cyclictest0-21swapper/010:28:300
149950,0migration/08964-21kthreadcore08:03:060
149950,0migration/08124-21taskset08:26:470
149950,0migration/07025-21latency_hist11:32:490
149950,0migration/05452-21missed_timers10:08:080
149950,0migration/04798-21kthreadcore07:33:080
149950,0migration/031972-21idleruntime07:08:040
149950,0migration/03055-21kthreadcore09:28:060
149950,0migration/029068-21timerandwakeup11:43:180
149950,0migration/027779-21kthreadcore10:43:090
149950,0migration/02691-21kthreadcore11:28:060
149950,0migration/026517-21aten_r1power_en07:53:000
149950,0migration/026065-21taskset07:50:580
149950,0migration/02336-21kthreadcore11:48:050
149950,0migration/019913-21kthreadcore09:58:060
149950,0migration/019783-21kthreadcore11:38:070
149950,0migration/019118-21munin-node07:23:020
149950,0migration/018360-21kthreadcore08:33:060
149950,0migration/018193-21kthreadcore08:58:070
149950,0migration/016140-21kthreadcore08:08:050
149950,0migration/015806-21taskset09:57:200
149950,0migration/011166-21kthreadcore08:53:060
149950,0migration/010469-21kthreadcore10:13:070
35140,0ktimersoftd/30-21swapper/308:49:093
339940,0migration/39508-21kthreadcore10:33:053
339940,0migration/325086-21taskset07:26:473
29725040,0irq/27-eth0-tx-7536-21ssh10:52:542
29725040,0irq/27-eth0-tx-32357-21munin-node09:08:022
29725040,0irq/27-eth0-tx-15290-21munin-node11:18:022
29725040,0irq/27-eth0-tx-14187-21ssh11:15:272
29725040,0irq/27-eth0-tx-0-21swapper/211:38:012
29725040,0irq/27-eth0-tx-0-21swapper/211:08:442
29725040,0irq/27-eth0-tx-0-21swapper/210:42:562
29725040,0irq/27-eth0-tx-0-21swapper/210:34:042
29725040,0irq/27-eth0-tx-0-21swapper/209:41:442
29725040,0irq/27-eth0-tx-0-21swapper/209:33:342
29725040,0irq/27-eth0-tx-0-21swapper/209:24:032
29725040,0irq/27-eth0-tx-0-21swapper/006:58:010
29715040,0irq/26-eth0-rx-5078-21cat07:33:093
29715040,0irq/26-eth0-rx-28995-21grep07:03:083
29715040,0irq/26-eth0-rx-0-21swapper/311:53:063
29715040,0irq/26-eth0-rx-0-21swapper/311:38:063
29715040,0irq/26-eth0-rx-0-21swapper/310:28:003
29715040,0irq/26-eth0-rx-0-21swapper/309:43:023
29715040,0irq/26-eth0-rx-0-21swapper/308:28:023
259940,0migration/29582-21aten_r1power_vo08:53:012
259940,0migration/28150-21fschecks_time08:03:032
259940,0migration/25218-21kthreadcore08:48:072
259940,0migration/229374-21kthreadcore07:53:062
259940,0migration/228541-21kthreadcore10:03:062
259940,0migration/227042-21kthreadcore10:23:092
259940,0migration/226029-21kthreadcore08:38:072
259940,0migration/225280-21kthreadcore11:03:062
259940,0migration/221492-21kthreadcore07:48:082
259940,0migration/221071-21missed_timers10:58:082
259940,0migration/219833-21munin-node08:08:102
259940,0migration/218277-21kthreadcore08:58:072
259940,0migration/217767-21kthreadcore07:43:072
259940,0migration/217100-21kthreadcore11:58:052
259940,0migration/216407-21fschecks_time10:38:032
259940,0migration/215950-21latency_hist10:17:492
249940,0watchdog/232131-21iostat11:48:032
249940,0watchdog/229420-21kthreadcore07:03:082
249940,0watchdog/229223-21ls08:43:042
249940,0watchdog/219935-21ssh12:18:072
249940,0watchdog/20-21swapper/211:53:032
249940,0watchdog/20-21swapper/208:23:042
249940,0watchdog/20-21swapper/207:28:082
176339943,0cyclictest9299-21awk08:28:052
176339943,0cyclictest8013-21awk11:33:042
176339943,0cyclictest7372-21awk07:38:052
176339943,0cyclictest448-21awk10:28:052
176339943,0cyclictest3556-21lldpd08:20:482
176339943,0cyclictest3514-21snmpd09:43:042
176339943,0cyclictest32516-21munin-node09:48:012
176339943,0cyclictest3241-21irqbalance07:33:082
176339943,0cyclictest32215-21awk11:28:032
176339943,0cyclictest23998-21awk11:43:092
176339943,0cyclictest23485-21awk12:23:052
176339943,0cyclictest16416-21ps08:33:052
176339941,0cyclictest0-21swapper/210:08:002
176319944,0cyclictest0-21swapper/109:36:501
176319944,0cyclictest0-21swapper/109:23:071
176319944,0cyclictest0-21swapper/109:13:491
176319944,0cyclictest0-21swapper/108:08:521
176319943,0cyclictest9203-21awk09:53:041
176319943,0cyclictest7831-21awk11:53:041
176319943,0cyclictest3556-21lldpd12:04:181
176319943,0cyclictest3556-21lldpd08:04:181
176319943,0cyclictest3556-21lldpd07:13:481
176319943,0cyclictest3514-21snmpd10:09:331
176319943,0cyclictest3514-21snmpd07:24:181
176319943,0cyclictest3241-21irqbalance10:28:381
176319943,0cyclictest3241-21irqbalance10:17:581
176319943,0cyclictest3241-21irqbalance09:29:481
176319943,0cyclictest3241-21irqbalance09:07:581
176319943,0cyclictest3241-21irqbalance09:06:081
176319943,0cyclictest3241-21irqbalance08:33:181
176319943,0cyclictest3241-21irqbalance08:29:181
176319943,0cyclictest3241-21irqbalance08:24:081
176319943,0cyclictest3241-21irqbalance07:04:091
176319943,0cyclictest3241-21irqbalance06:58:491
176319943,0cyclictest32071-21awk07:08:041
176319943,0cyclictest31767-21sadc10:27:491
176319943,0cyclictest24808-21awk10:43:061
176319943,0cyclictest24553-21awk11:03:041
176319943,0cyclictest23468-21awk12:23:051
176319943,0cyclictest16220-21awk11:18:041
176319940,0cyclictest0-21swapper/107:43:461
176309944,0cyclictest0-21swapper/012:21:420
176309944,0cyclictest0-21swapper/009:50:210
176309944,0cyclictest0-21swapper/009:13:080
176309943,0cyclictest3876-21irqrtprio08:48:060
176309943,0cyclictest3556-21lldpd08:30:180
176309943,0cyclictest3556-21lldpd07:03:480
176309943,0cyclictest3514-21snmpd07:47:180
176309943,0cyclictest3241-21irqbalance11:27:180
176309943,0cyclictest3241-21irqbalance10:32:580
176309943,0cyclictest3241-21irqbalance10:27:180
176309943,0cyclictest22600-21awk08:38:040
176309943,0cyclictest21656-21awk08:13:060
176309943,0cyclictest13065-21kworker/0:011:14:470
179940,0migration/18682-21unixbench_multi08:48:131
179940,0migration/132730-21fschecks_time07:33:041
179940,0migration/125753-21munin-node07:28:041
169940,0watchdog/132474-21if_err_eth1.211:08:031
169940,0watchdog/129858-21cat09:43:081
169940,0watchdog/121444-21fschecks_time08:13:041
169940,0watchdog/120026-21kthreadcore11:38:071
169940,0watchdog/112705-21cut10:53:071
169940,0watchdog/112096-21kthreadcore11:13:071
169940,0watchdog/11005-21if_err_eth1.309:48:041
169940,0watchdog/10-21swapper/112:08:071
169940,0watchdog/10-21swapper/111:28:031
169940,0watchdog/10-21swapper/111:25:191
169940,0watchdog/10-21swapper/108:18:081
169940,0watchdog/10-21swapper/107:18:081
149940,0migration/09321-21kthreadcore11:53:060
149940,0migration/04953-21runrttasks12:09:100
149940,0migration/029347-21sendmail_mailst11:03:100
149940,0migration/025492-21irqrtprio10:03:050
149940,0migration/020697-21switchtime11:58:110
149940,0migration/01713-21irqcore07:58:050
149940,0migration/016615-21kthreadcore07:18:070
149940,0migration/015693-21sh09:36:160
339930,0migration/318202-21runrttasks07:18:483
29715030,0irq/26-eth0-rx-0-21swapper/307:58:093
29715030,0irq/26-eth0-rx-0-21swapper/307:53:013
29715030,0irq/26-eth0-rx-0-21swapper/307:31:153
29715030,0irq/26-eth0-rx-0-21swapper/307:01:253
249930,0watchdog/20-21swapper/207:18:082
176339932,0cyclictest32095-21awk07:08:052
176319932,0cyclictest7939-21awk11:33:041
176319932,0cyclictest3514-21snmpd07:59:181
176319932,0cyclictest3241-21irqbalance08:37:581
176319932,0cyclictest19848-21stty07:48:031
176319932,0cyclictest17894-21kthreadcore12:18:061
176319932,0cyclictest17548-21awk09:18:031
176319932,0cyclictest17377-21awk09:38:041
176319932,0cyclictest10318-21awk08:53:041
176319932,0cyclictest10001-21kthreadcore07:38:071
176319930,0cyclictest0-21swapper/108:43:461
176309933,0cyclictest32405-21kworker/0:010:37:530
176309933,0cyclictest32405-21kworker/0:010:17:510
176309933,0cyclictest32405-21kworker/0:009:42:510
176309933,0cyclictest32405-21kworker/0:009:22:510
176309933,0cyclictest32405-21kworker/0:009:17:530
176309933,0cyclictest32405-21kworker/0:009:07:510
176309933,0cyclictest32405-21kworker/0:009:02:510
176309933,0cyclictest32405-21kworker/0:008:17:590
176309933,0cyclictest32405-21kworker/0:007:13:020
176309933,0cyclictest21590-21kworker/0:210:52:530
176309933,0cyclictest13065-21kworker/0:012:22:500
176309933,0cyclictest13065-21kworker/0:012:12:500
179930,0migration/130058-21munin-node10:03:101
169930,0watchdog/16559-21runrttasks10:57:511
169930,0watchdog/10-21swapper/111:58:031
169930,0watchdog/10-21swapper/110:48:031
169930,0watchdog/10-21swapper/107:52:561
29715020,0irq/26-eth0-rx-0-21swapper/308:08:063
29715020,0irq/26-eth0-rx-0-21swapper/307:48:143
29715020,0irq/26-eth0-rx-0-21swapper/307:12:583
29715020,0irq/26-eth0-rx-0-21swapper/307:07:503
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional