You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 07:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun Sep 06, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450670,0irq/25-eth10-21swapper/318:42:173
1777723723,0sleep00-21swapper/018:42:220
1769123324,0sleep20-21swapper/218:41:242
1767023127,0sleep10-21swapper/118:41:071
180519976,0cyclictest0-21swapper/021:23:400
35160,0ktimersoftd/30-21swapper/322:50:193
339960,0migration/35015-21missed_timers21:35:043
339960,0migration/331669-21sh21:13:153
339960,0migration/330752-21sh21:32:013
339960,0migration/323092-21sh21:49:303
339960,0migration/316509-21taskset20:44:453
339960,0migration/314369-21sh21:20:353
339960,0migration/313578-21sh22:23:073
339960,0migration/310599-21sh00:02:063
27160,0ktimersoftd/20-21swapper/220:56:502
180539960,0cyclictest0-21swapper/222:35:032
180529960,0cyclictest0-21swapper/121:33:471
149960,0migration/08343-21hwlatdetect21:39:570
339950,0migration/35485-21aten_r1power_fr22:59:553
339950,0migration/34962-21kthreadcore20:35:013
339950,0migration/34011-21runrttasks22:15:033
339950,0migration/332155-21taskset19:18:163
339950,0migration/327883-21aten_r1power_en20:04:563
339950,0migration/325648-21kthreadcore22:30:013
339950,0migration/324429-21kthreadcore22:10:033
339950,0migration/321406-21latency21:25:053
339950,0migration/320906-21munin-node19:34:573
339950,0migration/320536-21if_err_eth1.200:10:013
339950,0migration/318735-21taskset19:07:123
339950,0migration/317203-21kthreadcore20:20:003
339950,0migration/312896-21kthreadcore21:00:023
339950,0migration/312782-21kthreadcore19:50:033
339950,0migration/312618-21cpuspeed_turbos23:44:563
339950,0migration/312472-21munin-node22:00:033
339950,0migration/311842-21timerandwakeup22:40:073
29725050,0irq/27-eth0-tx-5964-21munin-node22:40:002
29725050,0irq/27-eth0-tx-5810-21diskstats22:59:572
29725050,0irq/27-eth0-tx-3806-21ssh23:17:262
29725050,0irq/27-eth0-tx-3753-21ksmtuned00:10:402
29725050,0irq/27-eth0-tx-3114-21munin-node20:35:052
29725050,0irq/27-eth0-tx-20177-21munin-node19:35:012
29725050,0irq/27-eth0-tx-13521-21munin-node19:30:012
29725050,0irq/27-eth0-tx-0-21swapper/221:20:002
29725050,0irq/27-eth0-tx-0-21swapper/219:09:482
29715050,0irq/26-eth0-rx-3620-21pmu-power22:35:053
29715050,0irq/26-eth0-rx-29335-21sh23:14:543
29715050,0irq/26-eth0-rx-2885-21perf20:34:453
29715050,0irq/26-eth0-rx-28613-21diskstats23:54:583
29715050,0irq/26-eth0-rx-28610-21latency_hist23:34:433
29715050,0irq/26-eth0-rx-24525-21irqcore21:50:003
29715050,0irq/26-eth0-rx-21640-21if_err_eth023:29:583
29715050,0irq/26-eth0-rx-15310-21diskstats22:05:003
29715050,0irq/26-eth0-rx-13309-21smartctl21:40:053
29715050,0irq/26-eth0-rx-0-21swapper/323:25:103
29715050,0irq/26-eth0-rx-0-21swapper/323:21:363
29715050,0irq/26-eth0-rx-0-21swapper/323:13:363
29715050,0irq/26-eth0-rx-0-21swapper/323:07:363
29715050,0irq/26-eth0-rx-0-21swapper/322:56:353
29715050,0irq/26-eth0-rx-0-21swapper/322:47:073
29715050,0irq/26-eth0-rx-0-21swapper/322:25:063
29715050,0irq/26-eth0-rx-0-21swapper/321:17:363
29715050,0irq/26-eth0-rx-0-21swapper/321:06:293
29715050,0irq/26-eth0-rx-0-21swapper/320:52:553
29715050,0irq/26-eth0-rx-0-21swapper/319:39:553
29715050,0irq/26-eth0-rx-0-21swapper/300:09:193
259950,0migration/22707-21sh23:35:472
259950,0migration/225020-21kthreadcore22:10:032
259950,0migration/223865-21latency_hist21:29:452
259950,0migration/221768-21fschecks_count23:09:562
259950,0migration/218213-21kthreadcore23:05:002
259950,0migration/21500-21munin-node23:55:062
259950,0migration/211180-21missed_timers19:00:042
249950,0watchdog/25879-21cut19:45:032
19150,0ktimersoftd/10-21swapper/123:50:481
19150,0ktimersoftd/10-21swapper/120:58:001
180539955,0cyclictest0-21swapper/223:45:512
180539955,0cyclictest0-21swapper/223:40:012
180539955,0cyclictest0-21swapper/223:28:212
180539954,0cyclictest27914-21crond23:54:432
180539950,0cyclictest0-21swapper/221:06:462
180539950,0cyclictest0-21swapper/219:59:462
180529955,0cyclictest0-21swapper/123:34:241
180529955,0cyclictest0-21swapper/122:43:381
180529955,0cyclictest0-21swapper/122:11:001
180529950,0cyclictest0-21swapper/123:55:131
180529950,0cyclictest0-21swapper/123:47:101
180529950,0cyclictest0-21swapper/123:40:141
180529950,0cyclictest0-21swapper/123:02:141
180529950,0cyclictest0-21swapper/122:08:471
180529950,0cyclictest0-21swapper/100:13:491
180519955,0cyclictest9242-21kworker/0:121:19:280
180519955,0cyclictest9242-21kworker/0:121:06:160
180519955,0cyclictest6561-21kworker/0:123:57:030
180519955,0cyclictest6561-21kworker/0:123:19:130
180519955,0cyclictest6561-21kworker/0:123:06:090
180519955,0cyclictest6561-21kworker/0:122:50:070
180519955,0cyclictest29217-21kworker/0:022:18:410
180519955,0cyclictest29217-21kworker/0:021:50:070
180519955,0cyclictest21569-21kworker/0:218:50:020
180519955,0cyclictest0-21swapper/021:49:400
180519954,0cyclictest8738-21awk19:50:000
180519953,0cyclictest6561-21kworker/0:123:25:030
180519950,0cyclictest0-21swapper/023:09:460
180519950,0cyclictest0-21swapper/022:41:140
179950,0migration/16227-21latency_hist22:19:441
179950,0migration/11802-21kthreadcore22:35:021
179950,0migration/116328-21idleruntime20:19:581
179950,0migration/115595-21kthreadcore23:04:591
169950,0watchdog/122330-21python20:45:071
169950,0watchdog/113489-21memory21:20:021
149950,0migration/09704-21kthreadcore22:20:020
149950,0migration/08939-21sh20:14:560
149950,0migration/08796-21kthreadcore22:00:000
149950,0migration/08558-21taskset20:14:170
149950,0migration/07383-21kthreadcore23:40:010
149950,0migration/07144-21kthreadcore23:00:010
149950,0migration/06449-21latency19:45:030
149950,0migration/06402-21taskset19:22:300
149950,0migration/05132-21unixbench_multi18:55:090
149950,0migration/04418-21kthreadcore20:55:030
149950,0migration/032697-21timerandwakeup19:40:060
149950,0migration/030393-21kthreadcore23:35:010
149950,0migration/030369-21aten_r1power_en22:34:560
149950,0migration/028761-21kthreadcore20:50:020
149950,0migration/028675-21sh22:10:070
149950,0migration/028564-21kthreadcore21:30:030
149950,0migration/024937-21kthreadcore20:25:040
149950,0migration/024667-21kthreadcore23:30:010
149950,0migration/024651-21kthreadcore23:50:010
149950,0migration/024580-21aten_r1power_po21:09:560
149950,0migration/024493-21munin-node19:10:040
149950,0migration/024278-21taskset18:45:270
149950,0migration/023243-21sh20:47:090
149950,0migration/023121-21kthreadcore00:10:040
149950,0migration/021880-21kthreadcore19:35:000
149950,0migration/020235-21taskset22:45:410
149950,0migration/018645-21kthreadcore22:25:070
149950,0migration/016976-21kthreadcore22:05:040
149950,0migration/01609-21taskset20:06:570
149950,0migration/015785-21kthreadcore19:05:020
149950,0migration/010703-21taskset00:02:250
8215040,0irq/24-0000:00:0-21swapper/221:34:592
339940,0migration/33396-21kthreadcore19:45:023
339940,0migration/331764-21sh21:54:573
339940,0migration/331764-21sh21:54:573
339940,0migration/325465-21kthreadcore20:25:043
339940,0migration/32139-21kthreadcore20:55:023
339940,0migration/316076-21taskset20:42:253
339940,0migration/315621-21irqcore19:55:003
29725040,0irq/27-eth0-tx-6716-21kthreadcore00:00:022
29725040,0irq/27-eth0-tx-22180-21munin-node20:25:022
29725040,0irq/27-eth0-tx-11771-21munin-node00:05:012
29725040,0irq/27-eth0-tx-0-21swapper/223:30:472
29725040,0irq/27-eth0-tx-0-21swapper/223:20:082
29725040,0irq/27-eth0-tx-0-21swapper/222:19:532
29725040,0irq/27-eth0-tx-0-21swapper/221:10:002
29725040,0irq/27-eth0-tx-0-21swapper/220:49:582
29725040,0irq/27-eth0-tx-0-21swapper/219:24:572
29715040,0irq/26-eth0-rx-0-21swapper/323:49:543
29715040,0irq/26-eth0-rx-0-21swapper/323:40:573
259940,0migration/29251-21kthreadcore22:00:012
259940,0migration/23219-21kthreadcore22:15:022
259940,0migration/225028-21kthreadcore22:50:012
259940,0migration/224240-21hddtemp_smartct21:49:582
259940,0migration/222757-21munin-node22:29:572
259940,0migration/220471-21kthreadcore18:45:022
259940,0migration/216566-21fschecks_count21:25:002
259940,0migration/216046-21kthreadcore22:25:062
259940,0migration/21281-21irqrtprio21:15:022
259940,0migration/212438-21latency_hist19:04:452
259940,0migration/21053-21kthreadcore21:55:012
259940,0migration/21053-21kthreadcore21:55:012
249940,0watchdog/217360-21kthreadcore22:45:022
249940,0watchdog/20-21swapper/222:05:022
249940,0watchdog/20-21swapper/221:40:062
249940,0watchdog/20-21swapper/219:54:592
249940,0watchdog/20-21swapper/218:55:032
180539943,0cyclictest9326-21awk20:59:592
180539943,0cyclictest7885-21munin-node19:50:032
180539943,0cyclictest3556-21lldpd22:54:442
180539943,0cyclictest3556-21lldpd20:10:412
180539943,0cyclictest31310-21sadc18:54:452
180539943,0cyclictest29834-21awk20:29:592
180539943,0cyclictest26784-21awk19:15:002
180539943,0cyclictest17389-21awk20:44:592
180539943,0cyclictest16427-21awk21:45:012
180539943,0cyclictest15516-21munin-node20:20:062
180539943,0cyclictest1020-21awk19:20:012
180529943,0cyclictest417-21awk21:35:011
180529943,0cyclictest3514-21snmpd22:44:571
180529943,0cyclictest3514-21snmpd20:35:411
180529943,0cyclictest3514-21snmpd20:30:411
180529943,0cyclictest3514-21snmpd20:08:411
180529943,0cyclictest3514-21snmpd19:35:101
180529943,0cyclictest3241-21irqbalance20:00:031
180529943,0cyclictest30373-21sort22:54:581
180529943,0cyclictest2958-21awk20:10:001
180529943,0cyclictest26760-21awk19:15:001
180529943,0cyclictest15423-21awk19:54:591
180519944,0cyclictest9242-21kworker/0:121:34:580
180519944,0cyclictest9242-21kworker/0:121:25:140
180519944,0cyclictest6561-21kworker/0:122:54:570
180519944,0cyclictest6561-21kworker/0:122:29:510
180519944,0cyclictest6561-21kworker/0:100:04:490
180519944,0cyclictest29217-21kworker/0:021:54:590
180519944,0cyclictest29217-21kworker/0:021:54:590
180519944,0cyclictest21569-21kworker/0:220:34:580
180519944,0cyclictest21569-21kworker/0:220:34:280
180519944,0cyclictest21569-21kworker/0:220:19:460
180519944,0cyclictest21569-21kworker/0:219:24:520
180519944,0cyclictest21569-21kworker/0:219:15:000
180519944,0cyclictest0-21swapper/023:44:550
180519943,0cyclictest22129-21awk20:00:010
180519943,0cyclictest19941-21sadc19:34:450
179940,0migration/19685-21irqrtprio21:00:011
179940,0migration/18252-21kthreadcore19:25:021
179940,0migration/16502-21fschecks_time18:59:581
179940,0migration/15609-21kthreadcore00:00:011
179940,0migration/13158-21kthreadcore21:15:031
179940,0migration/131502-21aten_r1power_vo21:54:561
179940,0migration/131502-21aten_r1power_vo21:54:561
179940,0migration/131026-21sh20:51:281
179940,0migration/126003-21kthreadcore21:10:011
179940,0migration/121023-21kthreadcore21:05:061
179940,0migration/11942-21munin-node19:45:001
179940,0migration/117904-21kthreadcore21:25:041
179940,0migration/117396-21kthreadcore19:30:031
179940,0migration/114551-21munin-node20:15:071
179940,0migration/113665-21runrttasks22:01:541
179940,0migration/113315-21munin-node19:04:591
179940,0migration/112250-21kthreadcore20:40:031
179940,0migration/111931-21kthreadcore19:50:021
179940,0migration/110885-21unixbench-2d23:20:051
179940,0migration/11042-21kthreadcore23:15:001
169940,0watchdog/1971-21sh23:35:021
169940,0watchdog/16375-21ssh22:19:461
169940,0watchdog/1313032sleep118:54:431
169940,0watchdog/129417-21ssh22:32:541
169940,0watchdog/122505-21kthreadcore19:10:031
169940,0watchdog/121844-21ssh22:49:541
169940,0watchdog/118630-21python23:25:061
169940,0watchdog/114394-21ssh22:24:541
169940,0watchdog/113744-21ssh21:40:221
169940,0watchdog/11150-21grep18:55:031
169940,0watchdog/10-21swapper/120:25:031
149940,0migration/09424-21if_eth1.221:00:000
149940,0migration/07394-21kthreadcore19:00:020
149940,0migration/06699-21kthreadcore23:19:590
149940,0migration/017256-21kthreadcore19:55:010
149940,0migration/013556-21kthreadcore20:40:040
29715030,0irq/26-eth0-rx-0-21swapper/320:14:453
29715030,0irq/26-eth0-rx-0-21swapper/320:02:273
29715030,0irq/26-eth0-rx-0-21swapper/318:50:283
29715030,0irq/26-eth0-rx-0-21swapper/318:45:033
249930,0watchdog/20-21swapper/220:39:592
180539932,0cyclictest9774-21awk20:14:592
180539930,0cyclictest27831-21munin-node20:04:572
180539930,0cyclictest26919-21munin-node19:39:562
180529933,0cyclictest19244-21awk18:45:011
180529932,0cyclictest3514-21snmpd19:40:101
180529932,0cyclictest12621-21awk00:05:021
169930,0watchdog/10-21swapper/123:12:301
169930,0watchdog/10-21swapper/121:49:581
169930,0watchdog/10-21swapper/121:47:181
169930,0watchdog/10-21swapper/119:20:031
339920,0migration/332552-21iostat_ios18:55:023
29715020,0irq/26-eth0-rx-0-21swapper/320:14:563
29715020,0irq/26-eth0-rx-0-21swapper/319:30:223
29715020,0irq/26-eth0-rx-0-21swapper/319:25:073
29715020,0irq/26-eth0-rx-0-21swapper/319:19:493
29715020,0irq/26-eth0-rx-0-21swapper/319:09:473
29715020,0irq/26-eth0-rx-0-21swapper/318:59:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional