You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 19:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Wed Jun 03, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450560,0irq/25-eth10-21swapper/306:52:403
2202723627,0sleep20-21swapper/206:50:142
2218623423,0sleep00-21swapper/006:52:040
2199623127,0sleep10-21swapper/106:49:491
339960,0migration/35582-21gltestperf09:03:573
339960,0migration/33802-21sh10:01:153
339960,0migration/328618-21sh09:57:373
339960,0migration/327490-21sh08:55:343
339960,0migration/327462-21sh10:35:223
29715060,0irq/26-eth0-rx-20244-21ssh10:53:133
29715060,0irq/26-eth0-rx-18952-21python09:09:093
224159965,0cyclictest0-21swapper/108:19:031
8215050,0irq/24-0000:00:20459-21latency_hist10:53:452
8215050,0irq/24-0000:00:20459-21latency_hist10:53:452
8215050,0irq/24-0000:00:20044-21munin-run11:33:442
4150,0ktimersoftd/00-21swapper/010:10:000
339950,0migration/35015-21irqcore12:04:003
339950,0migration/34649-21irqcore07:04:023
339950,0migration/3410-21kthreadcore06:59:023
339950,0migration/33917-21kthreadcore07:49:033
339950,0migration/33850-21latency_hist11:43:443
339950,0migration/332161-21kthreadcore10:59:013
339950,0migration/331231-21kthreadcore11:59:013
339950,0migration/330262-21idleruntime09:39:003
339950,0migration/329990-21needreboot07:44:033
339950,0migration/329774-21taskset07:20:493
339950,0migration/326651-21kthreadcore09:34:013
339950,0migration/325515-21munin-node08:29:083
339950,0migration/321247-21latency_hist09:13:453
339950,0migration/320656-21fschecks_count11:53:593
339950,0migration/320512-21diskmemload10:13:033
339950,0migration/316565-21kthreadcore07:59:043
339950,0migration/316561-21taskset07:11:573
339950,0migration/312768-21cpuspeed_turbos11:08:583
339950,0migration/311881-21taskset07:56:523
339950,0migration/310728-21taskset11:45:293
29725050,0irq/27-eth0-tx-9274-21tr09:24:032
29725050,0irq/27-eth0-tx-3514-21snmpd07:00:020
29725050,0irq/27-eth0-tx-29300-21munin-node09:59:022
29725050,0irq/27-eth0-tx-29076-21sh10:38:562
29725050,0irq/27-eth0-tx-13000-21munin-node10:09:002
29725050,0irq/27-eth0-tx-11160-21ssh11:05:382
29725050,0irq/27-eth0-tx-10574-21ssh11:24:372
29725050,0irq/27-eth0-tx-0-21swapper/211:39:032
29725050,0irq/27-eth0-tx-0-21swapper/210:45:212
29725050,0irq/27-eth0-tx-0-21swapper/210:35:512
29725050,0irq/27-eth0-tx-0-21swapper/210:32:272
29725050,0irq/27-eth0-tx-0-21swapper/210:19:102
29725050,0irq/27-eth0-tx-0-21swapper/008:29:050
29715050,0irq/26-eth0-rx-785-21kthreadcore07:24:033
29715050,0irq/26-eth0-rx-29387-21cpuspeed_turbos10:19:013
29715050,0irq/26-eth0-rx-29063-21kthreadcore08:09:033
29715050,0irq/26-eth0-rx-29005-21iostat_ios12:19:003
29715050,0irq/26-eth0-rx-27029-21ssh10:54:393
29715050,0irq/26-eth0-rx-27029-21ssh10:54:393
29715050,0irq/26-eth0-rx-25959-21kthreadcore06:54:043
29715050,0irq/26-eth0-rx-23367-21grep08:04:043
29715050,0irq/26-eth0-rx-11878-21ssh12:08:473
29715050,0irq/26-eth0-rx-0-21swapper/312:14:373
29715050,0irq/26-eth0-rx-0-21swapper/311:38:133
29715050,0irq/26-eth0-rx-0-21swapper/311:29:013
29715050,0irq/26-eth0-rx-0-21swapper/311:24:013
29715050,0irq/26-eth0-rx-0-21swapper/311:14:573
29715050,0irq/26-eth0-rx-0-21swapper/310:39:013
29715050,0irq/26-eth0-rx-0-21swapper/310:15:523
29715050,0irq/26-eth0-rx-0-21swapper/310:05:093
29715050,0irq/26-eth0-rx-0-21swapper/309:27:303
29715050,0irq/26-eth0-rx-0-21swapper/308:59:033
259950,0migration/24103-21munin-node12:03:562
259950,0migration/22283-21timerwakeupswit10:59:072
259950,0migration/220586-21aten_r1power_en11:13:542
259950,0migration/218900-21rt-features09:09:082
259950,0migration/21105-21kernelversion08:14:032
259950,0migration/210898-21rt-features09:44:072
249950,0watchdog/219111-21ssh10:49:592
224219955,0cyclictest0-21swapper/211:58:472
224219950,0cyclictest0-21swapper/212:10:372
224219950,0cyclictest0-21swapper/211:43:582
224219950,0cyclictest0-21swapper/210:16:002
224219950,0cyclictest0-21swapper/210:03:512
224219950,0cyclictest0-21swapper/208:38:242
224219950,0cyclictest0-21swapper/208:00:152
224159954,0cyclictest3973-21crond11:23:451
224049954,0cyclictest4832-21awk12:03:590
224049950,0cyclictest0-21swapper/011:52:030
224049950,0cyclictest0-21swapper/010:26:070
19150,0ktimersoftd/10-21swapper/110:10:001
179950,0migration/15199-21fschecks_time10:43:581
179950,0migration/125194-21kthreadcore08:54:021
179950,0migration/117143-21timerwakeupswit07:34:091
169950,0watchdog/127150-21ssh12:17:111
149950,0migration/05880-21gltestperf09:43:590
149950,0migration/029852-21hddtemp_smartct08:59:000
149950,0migration/028588-21sh08:58:170
149950,0migration/028262-21taskset10:17:100
149950,0migration/028180-21kthreadcore07:44:020
149950,0migration/027698-21taskset10:56:360
149950,0migration/027698-21taskset10:56:360
149950,0migration/020145-21sh10:32:300
149950,0migration/013359-21kthreadcore07:34:030
149950,0migration/012061-21sh09:06:420
149950,0migration/011728-21runrttasks11:07:150
149950,0migration/011035-21taskset11:25:590
8215040,0irq/24-0000:00:0-21swapper/207:38:452
339940,0migration/38643-21kthreadcore08:19:033
339940,0migration/313617-21hddtemp_smartct10:28:583
339940,0migration/313422-21irqrtprio11:49:003
29725040,0irq/27-eth0-tx-4931-21aten_r1power_vo10:23:562
29725040,0irq/27-eth0-tx-3514-21snmpd07:30:080
29725040,0irq/27-eth0-tx-28309-21ssh12:18:562
29725040,0irq/27-eth0-tx-26500-21munin-node08:33:590
29725040,0irq/27-eth0-tx-0-21swapper/212:13:482
29725040,0irq/27-eth0-tx-0-21swapper/211:50:002
29725040,0irq/27-eth0-tx-0-21swapper/209:54:222
29725040,0irq/27-eth0-tx-0-21swapper/209:50:142
29725040,0irq/27-eth0-tx-0-21swapper/209:38:502
29725040,0irq/27-eth0-tx-0-21swapper/209:34:282
29725040,0irq/27-eth0-tx-0-21swapper/209:29:132
29725040,0irq/27-eth0-tx-0-21swapper/209:21:052
29725040,0irq/27-eth0-tx-0-21swapper/209:13:462
29725040,0irq/27-eth0-tx-0-21swapper/209:03:552
29715040,0irq/26-eth0-rx-0-21swapper/311:18:593
29715040,0irq/26-eth0-rx-0-21swapper/311:03:483
29715040,0irq/26-eth0-rx-0-21swapper/310:44:093
29715040,0irq/26-eth0-rx-0-21swapper/310:24:533
29715040,0irq/26-eth0-rx-0-21swapper/309:49:033
29715040,0irq/26-eth0-rx-0-21swapper/309:44:013
29715040,0irq/26-eth0-rx-0-21swapper/309:31:463
29715040,0irq/26-eth0-rx-0-21swapper/309:21:203
29715040,0irq/26-eth0-rx-0-21swapper/308:34:043
29715040,0irq/26-eth0-rx-0-21swapper/308:24:023
29715040,0irq/26-eth0-rx-0-21swapper/307:28:573
29715040,0irq/26-eth0-rx-0-21swapper/307:14:003
259940,0migration/231105-21kthreadcore11:19:002
259940,0migration/230383-21kernelversion08:59:032
259940,0migration/227433-21kthreadcore08:09:032
259940,0migration/221414-21kthreadcore08:29:032
259940,0migration/216365-21runrttasks07:09:372
259940,0migration/216011-21kthreadcore11:09:032
249940,0watchdog/223339-21cut11:54:032
249940,0watchdog/215933-21kthreadcore07:34:042
249940,0watchdog/20-21swapper/208:04:042
224219944,0cyclictest0-21swapper/208:54:392
224219943,0cyclictest7443-21awk08:19:012
224219943,0cyclictest3556-21lldpd08:25:502
224219943,0cyclictest3556-21lldpd07:19:192
224219943,0cyclictest3556-21lldpd07:01:192
224219943,0cyclictest3514-21snmpd08:40:332
224219943,0cyclictest3514-21snmpd07:04:182
224219943,0cyclictest3514-21snmpd06:56:182
224219943,0cyclictest15121-21awk08:49:012
224219940,0cyclictest20308-21munin-node11:33:572
224159943,0cyclictest6498-21awk07:54:011
224159943,0cyclictest5601-21awk07:29:011
224159943,0cyclictest4925-21awk11:43:591
224159943,0cyclictest3556-21lldpd09:08:501
224159943,0cyclictest3556-21lldpd08:25:201
224159943,0cyclictest3514-21snmpd10:27:481
224159943,0cyclictest3514-21snmpd09:14:181
224159943,0cyclictest3514-21snmpd07:22:481
224159943,0cyclictest3332-21sadc07:03:451
224159943,0cyclictest29049-21awk11:58:591
224159943,0cyclictest23500-21awk06:54:021
224159943,0cyclictest22262-21awk09:33:581
224159943,0cyclictest21213-21awk11:33:581
224159943,0cyclictest21167-21awk11:54:011
224159943,0cyclictest1592-21awk08:39:001
224049944,0cyclictest0-21swapper/011:35:290
224049943,0cyclictest5039-21awk11:44:000
224049943,0cyclictest3241-21irqbalance11:55:030
224049943,0cyclictest3241-21irqbalance11:14:430
224049943,0cyclictest3241-21irqbalance11:10:030
224049943,0cyclictest3241-21irqbalance10:49:130
224049943,0cyclictest3241-21irqbalance10:03:530
224049943,0cyclictest3241-21irqbalance09:19:340
224049943,0cyclictest3241-21irqbalance09:13:540
224049943,0cyclictest3241-21irqbalance08:51:340
224049943,0cyclictest3241-21irqbalance08:19:540
224049943,0cyclictest3241-21irqbalance08:14:040
224049943,0cyclictest3241-21irqbalance08:08:540
224049943,0cyclictest3241-21irqbalance08:06:040
224049943,0cyclictest32245-21awk07:49:010
224049943,0cyclictest29652-21awk10:58:590
224049943,0cyclictest24478-21awk07:19:000
224049943,0cyclictest22173-21awk09:54:000
224049943,0cyclictest21795-21awk10:33:590
224049943,0cyclictest20028-21sadc11:33:440
224049943,0cyclictest16731-21sadc07:13:450
224049943,0cyclictest14121-21awk08:24:000
224049943,0cyclictest14008-21awk09:09:010
224049943,0cyclictest13168-21awk07:59:010
179940,0migration/16432-21kthreadcore07:04:031
179940,0migration/15831-21kthreadcore12:04:001
179940,0migration/132259-21kthreadcore09:19:011
179940,0migration/131513-21kthreadcore10:59:011
179940,0migration/130309-21irqcore10:19:051
179940,0migration/127660-21irqcore08:34:031
179940,0migration/126811-21unixbench_singl10:54:091
179940,0migration/126811-21unixbench_singl10:54:091
179940,0migration/125673-21kthreadcore11:14:011
179940,0migration/124002-21kthreadcore09:54:021
179940,0migration/121001-21irqrtprio08:29:021
179940,0migration/120912-21aten_r1power_en10:33:551
179940,0migration/11961-21needreboot10:39:021
179940,0migration/118772-21taskset11:09:301
179940,0migration/118427-21fschecks_count07:38:591
179940,0migration/117049-21kthreadcore11:29:021
179940,0migration/116226-21kthreadcore08:49:021
179940,0migration/115640-21kthreadcore07:59:031
179940,0migration/115544-21kthreadcore12:09:031
179940,0migration/114985-21kthreadcore10:49:021
179940,0migration/111194-21runrttasks11:05:431
179940,0migration/110271-21latency_hist07:08:461
169940,0watchdog/15837-21if_err_eth1.210:03:591
169940,0watchdog/15530-21sensors_temp08:14:081
169940,0watchdog/122655-21python07:14:081
169940,0watchdog/122085-21kthreadcore08:04:041
169940,0watchdog/12091-21cut07:24:041
169940,0watchdog/120021-21ssh09:51:071
169940,0watchdog/119436-21ssh10:30:271
169940,0watchdog/118887-21ssh11:50:511
169940,0watchdog/10-21swapper/111:43:231
169940,0watchdog/10-21swapper/109:03:591
169940,0watchdog/10-21swapper/108:09:001
149940,0migration/06551-21kthreadcore10:44:020
149940,0migration/03276-21sh09:59:510
149940,0migration/031780-21kthreadcore11:59:010
149940,0migration/028105-21munin-node06:54:070
149940,0migration/025052-21kthreadcore09:34:010
149940,0migration/022196-21kthreadcore07:39:030
149940,0migration/019294-21unixbench_singl09:49:090
149940,0migration/018045-21kthreadcore09:29:010
149940,0migration/013472-21kthreadcore12:09:020
149940,0migration/012678-21sh09:27:380
35130,0ktimersoftd/30-21swapper/308:15:073
29725030,0irq/27-eth0-tx-0-21swapper/008:43:570
29715030,0irq/26-eth0-rx-0-21swapper/308:48:563
29715030,0irq/26-eth0-rx-0-21swapper/308:47:473
29715030,0irq/26-eth0-rx-0-21swapper/308:40:223
259930,0migration/26701-21kthreadcore07:29:022
249930,0watchdog/25564-21runrttasks07:53:482
249930,0watchdog/20-21swapper/207:26:002
224219933,0cyclictest3556-21lldpd07:52:502
224219932,0cyclictest7473-21munin-node08:43:582
224219932,0cyclictest30275-21sensors07:44:062
224219932,0cyclictest21907-21cut07:14:042
224159933,0cyclictest3556-21lldpd08:58:501
224159932,0cyclictest6848-21cat09:44:021
224159932,0cyclictest3556-21lldpd08:44:501
224159932,0cyclictest30380-21awk09:39:001
224159932,0cyclictest21950-21awk10:14:001
224159931,0cyclictest6078-21smartctl09:23:591
224049933,0cyclictest0-21swapper/010:38:530
224049932,0cyclictest6460-21sensors08:39:070
224049932,0cyclictest4507-21awk07:04:010
224049932,0cyclictest3514-21snmpd12:21:040
224049932,0cyclictest3514-21snmpd11:39:040
224049932,0cyclictest30429-21awk09:39:010
224049932,0cyclictest29359-21cpuspeed_turbos10:19:010
224049932,0cyclictest25720-21sensors12:14:060
224049932,0cyclictest1908-21sendmail_mailst11:19:050
224049932,0cyclictest0-21swapper/007:25:070
224049930,0cyclictest9276-21grep07:54:030
224049930,0cyclictest0-21swapper/007:13:590
169930,0watchdog/10-21swapper/112:18:551
169930,0watchdog/10-21swapper/111:24:031
169930,0watchdog/10-21swapper/109:28:591
169930,0watchdog/10-21swapper/107:43:561
29715020,0irq/26-eth0-rx-0-21swapper/307:38:583
29715020,0irq/26-eth0-rx-0-21swapper/307:33:573
224159920,0cyclictest0-21swapper/109:58:451
169920,0watchdog/10-21swapper/107:49:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional