You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 02:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Wed May 13, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450420,0irq/25-eth10-21swapper/318:54:353
1048223526,0sleep20-21swapper/218:55:282
1038823127,0sleep10-21swapper/118:54:231
1018922915,0sleep00-21swapper/018:52:020
339960,0migration/39980-21kthreadcore00:16:063
339960,0migration/332464-21sh22:30:213
339960,0migration/330737-21sh23:48:033
339960,0migration/316722-21sh21:19:173
339960,0migration/314799-21sh23:59:033
339960,0migration/313885-21kthreadcore18:56:053
29715060,0irq/26-eth0-rx-22207-21sh23:21:113
29715060,0irq/26-eth0-rx-0-21swapper/321:36:053
149960,0migration/030403-21taskset23:26:330
149960,0migration/023493-21sh22:22:370
149960,0migration/023339-21sh22:01:200
149960,0migration/022739-21kthreadcore20:16:030
149960,0migration/018127-21kthreadcore23:41:040
105939960,0cyclictest0-21swapper/223:09:472
105849960,0cyclictest0-21swapper/022:06:060
339950,0migration/3866-21taskset20:23:283
339950,0migration/36035-21kthreadcore21:51:053
339950,0migration/35175-21kthreadcore22:51:043
339950,0migration/33651-21kthreadcore00:11:063
339950,0migration/327643-21kthreadcore19:06:063
339950,0migration/326398-21idleruntime21:26:013
339950,0migration/325719-21cpuspeed_turbos21:05:583
339950,0migration/324586-21taskset19:27:373
339950,0migration/324320-21runrttasks22:04:093
339950,0migration/323673-21taskset19:03:383
339950,0migration/322653-21munin-node21:01:073
339950,0migration/322071-21mdstat22:41:063
339950,0migration/320896-21fschecks_count20:15:593
339950,0migration/319179-21kthreadcore21:21:043
339950,0migration/317160-21cpuspeed_turbos22:21:003
339950,0migration/31405-21aten_r1power_fr20:26:013
29725050,0irq/27-eth0-tx-7910-21chrt22:34:312
29725050,0irq/27-eth0-tx-7621-21munin-node23:56:022
29725050,0irq/27-eth0-tx-4337-21cut23:31:042
29725050,0irq/27-eth0-tx-398-21if_err_eth1.323:51:022
29725050,0irq/27-eth0-tx-23899-21ssh23:45:552
29725050,0irq/27-eth0-tx-16957-21if_err_eth023:21:022
29725050,0irq/27-eth0-tx-0-21swapper/222:49:492
29725050,0irq/27-eth0-tx-0-21swapper/221:57:262
29725050,0irq/27-eth0-tx-0-21swapper/221:43:122
29725050,0irq/27-eth0-tx-0-21swapper/200:04:252
29725050,0irq/27-eth0-tx-0-21swapper/020:48:020
29725050,0irq/27-eth0-tx-0-21swapper/020:30:320
29725050,0irq/27-eth0-tx-0-21swapper/019:46:060
29725050,0irq/27-eth0-tx-0-21swapper/019:38:310
29725050,0irq/27-eth0-tx-0-21swapper/019:26:090
29715050,0irq/26-eth0-rx-6174-21ssh21:11:073
29715050,0irq/26-eth0-rx-580-21ssh21:50:363
29715050,0irq/26-eth0-rx-5204-21kthreadcore22:31:053
29715050,0irq/26-eth0-rx-3880-21kthreadcore20:51:033
29715050,0irq/26-eth0-rx-30441-21ssh23:26:393
29715050,0irq/26-eth0-rx-25363-21ssh23:06:033
29715050,0irq/26-eth0-rx-14072-21ssh23:36:193
29715050,0irq/26-eth0-rx-13760-21port_sshpriv23:16:073
29715050,0irq/26-eth0-rx-10619-21kthreadcore20:56:073
29715050,0irq/26-eth0-rx-10298-21cut22:56:063
29715050,0irq/26-eth0-rx-10199-21kthreadcore22:16:033
29715050,0irq/26-eth0-rx-0-21swapper/323:41:083
29715050,0irq/26-eth0-rx-0-21swapper/323:33:543
29715050,0irq/26-eth0-rx-0-21swapper/323:00:583
29715050,0irq/26-eth0-rx-0-21swapper/322:35:563
29715050,0irq/26-eth0-rx-0-21swapper/322:06:063
29715050,0irq/26-eth0-rx-0-21swapper/321:31:053
29715050,0irq/26-eth0-rx-0-21swapper/320:06:053
29715050,0irq/26-eth0-rx-0-21swapper/300:20:513
259950,0migration/2456-21sh21:09:392
259950,0migration/24556-21kthreadcore00:11:072
259950,0migration/226638-21kthreadcore23:26:042
259950,0migration/225235-21latency_hist21:05:482
259950,0migration/216985-21cpuspeed_turbos22:41:002
259950,0migration/216771-21unixbench_singl18:56:112
259950,0migration/215755-21sh22:18:172
259950,0migration/215452-21munin-node21:36:112
259950,0migration/215195-21sh20:56:192
259950,0migration/211639-21kthreadcore00:16:072
249950,0watchdog/28801-21diskmemload21:31:062
249950,0watchdog/22851-21cat21:11:062
249950,0watchdog/20-21swapper/223:18:332
179950,0migration/19316-21kthreadcore00:16:061
179950,0migration/131353-21munin-node21:06:081
179950,0migration/131273-21sh23:49:361
179950,0migration/118039-21latency19:46:051
169950,0watchdog/10-21swapper/121:31:061
149950,0migration/03519-21kthreadcore22:11:040
149950,0migration/031876-21timerandwakeup19:56:130
149950,0migration/026518-21kthreadcore22:46:060
149950,0migration/026392-21idleruntime21:26:010
149950,0migration/025718-21kernelversion23:06:040
149950,0migration/025629-21cpuspeed_turbos21:46:010
149950,0migration/023378-21sh23:24:340
149950,0migration/022758-21kthreadcore21:21:050
149950,0migration/019275-21kthreadcore00:01:050
149950,0migration/017000-21munin-node22:41:000
149950,0migration/016799-21df_abs23:01:000
149950,0migration/015722-21sh21:36:530
149950,0migration/015666-21kthreadcore20:11:060
149950,0migration/015657-21cpuspeed_turbos00:21:000
149950,0migration/014712-21taskset23:38:110
149950,0migration/014430-21taskset23:58:000
149950,0migration/011849-21kthreadcore22:16:030
105939955,0cyclictest0-21swapper/222:54:012
105939955,0cyclictest0-21swapper/200:10:382
105939954,0cyclictest3514-21snmpd22:58:332
105939950,0cyclictest0-21swapper/223:43:502
105939950,0cyclictest0-21swapper/223:36:352
105939950,0cyclictest0-21swapper/221:21:562
105939950,0cyclictest0-21swapper/219:12:402
105859955,0cyclictest23329-21sensors21:21:091
105859950,0cyclictest0-21swapper/123:59:561
105849955,0cyclictest0-21swapper/021:01:090
105849950,0cyclictest0-21swapper/022:51:490
105849950,0cyclictest0-21swapper/019:01:100
339940,0migration/38899-21kthreadcore19:41:043
339940,0migration/327007-21kthreadcore19:31:043
29725040,0irq/27-eth0-tx-29612-21cut21:26:042
29725040,0irq/27-eth0-tx-0-21swapper/223:10:572
29725040,0irq/27-eth0-tx-0-21swapper/223:01:302
29725040,0irq/27-eth0-tx-0-21swapper/222:35:552
29725040,0irq/27-eth0-tx-0-21swapper/222:05:582
29725040,0irq/27-eth0-tx-0-21swapper/221:46:002
29725040,0irq/27-eth0-tx-0-21swapper/200:20:532
29725040,0irq/27-eth0-tx-0-21swapper/020:20:570
29715040,0irq/26-eth0-rx-19517-21kthreadcore21:41:053
29715040,0irq/26-eth0-rx-0-21swapper/323:50:593
29715040,0irq/26-eth0-rx-0-21swapper/323:11:483
29715040,0irq/26-eth0-rx-0-21swapper/322:46:063
29715040,0irq/26-eth0-rx-0-21swapper/322:10:573
29715040,0irq/26-eth0-rx-0-21swapper/321:56:153
29715040,0irq/26-eth0-rx-0-21swapper/319:39:123
29715040,0irq/26-eth0-rx-0-21swapper/300:06:053
29715040,0irq/26-eth0-rx-0-21swapper/300:01:303
259940,0migration/26362-21kthreadcore20:51:042
259940,0migration/231547-21munin-node19:56:102
259940,0migration/229604-21kthreadcore22:26:062
259940,0migration/222054-21kthreadcore20:16:032
259940,0migration/219092-21kthreadcore19:01:062
259940,0migration/214289-21fschecks_count20:11:022
259940,0migration/21342-21aten_r1power_cu20:26:002
249940,0watchdog/27019-21if_err_eth1.319:41:022
249940,0watchdog/212765-21grep21:16:062
249940,0watchdog/21019-21basename21:50:582
249940,0watchdog/20-21swapper/220:21:022
249940,0watchdog/20-21swapper/219:36:062
249940,0watchdog/20-21swapper/219:26:022
179940,0migration/19308-21cpuspeed_turbos20:56:031
179940,0migration/1662-21irqcore23:51:031
179940,0migration/16061-21latency_hist19:40:481
179940,0migration/15303-21kthreadcore22:31:061
179940,0migration/14073-21kthreadcore22:11:041
179940,0migration/13817-21kthreadcore20:51:031
179940,0migration/13708-21missed_timers19:11:081
179940,0migration/12560-21irqrtprio20:26:051
179940,0migration/125113-21latency_hist21:45:481
179940,0migration/122080-21proc_pri23:01:091
179940,0migration/120843-21kthreadcore19:26:021
179940,0migration/119171-21kthreadcore00:21:051
179940,0migration/116189-21aten_r1power_fr23:20:591
179940,0migration/111621-21kthreadcore21:36:041
179940,0migration/110661-21kthreadcore20:31:051
169940,0watchdog/1969-21munin-node21:51:061
169940,0watchdog/15263-21munin-node23:31:051
169940,0watchdog/132382-21munin-node23:11:021
169940,0watchdog/130606-21ssh23:27:091
169940,0watchdog/125566-21fschecks_time22:26:021
169940,0watchdog/122508-21ssh23:42:331
169940,0watchdog/114993-21grep22:16:101
169940,0watchdog/110795-21chrt18:55:591
169940,0watchdog/10-21swapper/121:11:021
149940,0migration/03272-21kthreadcore21:51:040
149940,0migration/02738-21kthreadcore23:11:050
149940,0migration/025316-21cpu19:30:590
149940,0migration/013856-21kthreadcore21:16:060
105939943,0cyclictest8909-21awk20:31:022
105939943,0cyclictest3556-21lldpd20:02:022
105939943,0cyclictest3514-21snmpd22:21:332
105939943,0cyclictest3514-21snmpd20:06:342
105939943,0cyclictest3514-21snmpd19:32:042
105939943,0cyclictest3514-21snmpd19:24:342
105939943,0cyclictest3514-21snmpd19:17:342
105939943,0cyclictest3514-21snmpd19:06:042
105939943,0cyclictest1800-21irqcore22:11:032
105939943,0cyclictest17986-21awk22:01:022
105859944,0cyclictest0-21swapper/121:26:041
105859943,0cyclictest8858-21awk23:36:021
105859943,0cyclictest3556-21lldpd23:16:331
105859943,0cyclictest3556-21lldpd22:56:031
105859943,0cyclictest3556-21lldpd22:23:331
105859943,0cyclictest3556-21lldpd20:36:321
105859943,0cyclictest3556-21lldpd19:52:011
105859943,0cyclictest3556-21lldpd19:16:011
105859943,0cyclictest32660-21awk19:36:041
105859943,0cyclictest28924-21awk20:46:041
105859943,0cyclictest27905-21awk20:21:011
105859943,0cyclictest26211-21awk21:46:031
105859943,0cyclictest25574-21awk22:46:041
105859943,0cyclictest25079-21awk19:06:041
105859943,0cyclictest22232-21awk20:41:011
105859943,0cyclictest21399-21awk20:16:021
105859943,0cyclictest18356-21awk19:01:041
105859943,0cyclictest16560-21awk00:01:021
105859943,0cyclictest1342-21awk20:01:031
105859943,0cyclictest12777-21awk19:21:031
105849944,0cyclictest0-21swapper/021:42:460
105849944,0cyclictest0-21swapper/000:19:380
105849944,0cyclictest0-21swapper/000:15:230
105849943,0cyclictest9533-21awk22:36:020
105849943,0cyclictest6107-21munin-node19:41:000
105849943,0cyclictest5297-21kworker/0:222:57:310
105849943,0cyclictest505-21awk23:51:020
105849943,0cyclictest3514-21snmpd23:31:030
105849943,0cyclictest3514-21snmpd20:54:490
105849943,0cyclictest3241-21irqbalance21:57:260
105849943,0cyclictest3241-21irqbalance21:13:160
105849943,0cyclictest26348-21awk21:06:000
105849943,0cyclictest20305-21awk19:51:020
105849943,0cyclictest15637-21awk20:36:030
105849943,0cyclictest11727-21awk18:56:030
105849940,0cyclictest306-21munin-node20:01:040
339930,0migration/311170-21runrttasks19:16:213
29715030,0irq/26-eth0-rx-0-21swapper/320:36:093
29715030,0irq/26-eth0-rx-0-21swapper/320:31:063
29715030,0irq/26-eth0-rx-0-21swapper/320:11:003
29715030,0irq/26-eth0-rx-0-21swapper/320:00:593
29715030,0irq/26-eth0-rx-0-21swapper/319:58:453
29715030,0irq/26-eth0-rx-0-21swapper/319:54:153
29715030,0irq/26-eth0-rx-0-21swapper/319:48:003
29715030,0irq/26-eth0-rx-0-21swapper/319:13:223
249930,0watchdog/20-21swapper/220:46:022
249930,0watchdog/20-21swapper/219:45:582
169930,0watchdog/10-21swapper/122:50:581
169930,0watchdog/10-21swapper/122:01:021
169930,0watchdog/10-21swapper/121:57:061
169930,0watchdog/10-21swapper/120:11:061
169930,0watchdog/10-21swapper/100:11:091
169930,0watchdog/10-21swapper/100:05:491
105939932,0cyclictest24028-21kthreadcore19:51:052
105939932,0cyclictest22287-21awk20:41:022
105939930,0cyclictest16269-21grep20:36:052
105859933,0cyclictest3556-21lldpd19:57:311
105859932,0cyclictest9447-21hwlatdetect22:36:021
105859932,0cyclictest29884-21cut19:31:051
105859932,0cyclictest26046-21awk22:06:021
105859932,0cyclictest25282-21awk23:06:021
105859932,0cyclictest18335-21tr22:41:041
105859932,0cyclictest18104-21awk21:01:031
105859932,0cyclictest10491-21iostat_ios21:16:051
105849933,0cyclictest5297-21kworker/0:223:15:510
105849933,0cyclictest5297-21kworker/0:222:30:530
105849933,0cyclictest5297-21kworker/0:222:26:250
105849933,0cyclictest5297-21kworker/0:221:31:090
105849933,0cyclictest5297-21kworker/0:220:55:590
105849933,0cyclictest5297-21kworker/0:220:41:090
105849933,0cyclictest5297-21kworker/0:220:30:550
105849933,0cyclictest5297-21kworker/0:220:05:510
105849933,0cyclictest5297-21kworker/0:219:20:540
105849933,0cyclictest5297-21kworker/0:219:15:580
105849933,0cyclictest5297-21kworker/0:219:10:540
105849933,0cyclictest5297-21kworker/0:219:05:560
105849933,0cyclictest23782-21kworker/0:123:45:530
105849933,0cyclictest23782-21kworker/0:100:05:480
339920,0migration/322382-21runrttasks20:41:033
29715020,0irq/26-eth0-rx-0-21swapper/320:46:003
29715020,0irq/26-eth0-rx-0-21swapper/319:20:483
105859920,0cyclictest0-21swapper/120:05:481
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional