You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-22 - 03:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Wed Apr 22, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3059524637,0sleep20-21swapper/218:57:112
3037223623,0sleep00-21swapper/018:54:360
304450350,0irq/25-eth10-21swapper/318:53:163
3028923222,0sleep10-21swapper/118:53:371
339960,0migration/327412-21sh21:53:593
339960,0migration/326082-21sh23:13:173
339960,0migration/325197-21munin-node23:53:103
339960,0migration/320248-21sh22:12:063
339960,0migration/314154-21kernelversion19:33:033
339960,0migration/313746-21munin-node21:48:013
339960,0migration/311065-21sh23:06:043
307149960,0cyclictest0-21swapper/222:33:462
307149960,0cyclictest0-21swapper/221:28:012
307149960,0cyclictest0-21swapper/221:24:142
307149960,0cyclictest0-21swapper/221:20:272
29715060,0irq/26-eth0-rx-9980-21ntp_states22:23:073
29715060,0irq/26-eth0-rx-29225-21iostat_ios22:58:023
29715060,0irq/26-eth0-rx-18815-21sh21:28:073
29715060,0irq/26-eth0-rx-15385-21kthreadcore23:48:043
29715060,0irq/26-eth0-rx-0-21swapper/323:47:213
149960,0migration/028186-21sh22:57:560
149960,0migration/026260-21kthreadcore20:53:050
149960,0migration/019516-21sh19:58:090
149960,0migration/019339-21run-parts21:48:460
8215050,0irq/24-0000:00:13704-21cat19:57:472
339950,0migration/39511-21timerandwakeup23:23:103
339950,0migration/3448-21missed_timers23:58:053
339950,0migration/34104-21kthreadcore19:48:053
339950,0migration/332644-21latency_hist19:47:473
339950,0migration/331576-21kthreadcore21:58:033
339950,0migration/329329-21sh22:37:593
339950,0migration/327381-21sh22:14:423
339950,0migration/326135-21timerwakeupswit22:53:103
339950,0migration/323429-21kthreadcore19:38:043
339950,0migration/322477-21kthreadcore00:13:033
339950,0migration/321610-21cpuspeed_turbos21:12:593
339950,0migration/321324-21munin-node20:03:013
339950,0migration/321138-21taskset20:27:033
339950,0migration/317518-21timerandwakeup23:28:103
339950,0migration/315530-21kthreadcore19:08:043
339950,0migration/313960-21gltestperf21:08:013
307149955,0cyclictest0-21swapper/223:57:192
307149955,0cyclictest0-21swapper/223:30:222
307149955,0cyclictest0-21swapper/222:18:192
307149955,0cyclictest0-21swapper/221:05:382
307149955,0cyclictest0-21swapper/220:59:002
307149954,0cyclictest19579-21ssh23:12:332
307129955,0cyclictest0-21swapper/023:50:550
307129955,0cyclictest0-21swapper/023:14:300
307129950,0cyclictest0-21swapper/000:19:370
29725050,0irq/27-eth0-tx-6178-21munin-node19:28:080
29725050,0irq/27-eth0-tx-3367-21date23:22:462
29725050,0irq/27-eth0-tx-12278-21if_err_eth1.223:48:022
29725050,0irq/27-eth0-tx-0-21swapper/020:21:520
29725050,0irq/27-eth0-tx-0-21swapper/019:40:210
29725050,0irq/27-eth0-tx-0-21swapper/019:01:540
29715050,0irq/26-eth0-rx-6559-21runrttasks23:36:133
29715050,0irq/26-eth0-rx-4638-21kthreadcore00:23:053
29715050,0irq/26-eth0-rx-4299-21irqrtprio00:03:023
29715050,0irq/26-eth0-rx-30065-21hwlatdetect21:38:013
29715050,0irq/26-eth0-rx-27080-21ssh22:34:423
29715050,0irq/26-eth0-rx-18450-21unixbench_multi22:28:103
29715050,0irq/26-eth0-rx-0-21swapper/323:42:213
29715050,0irq/26-eth0-rx-0-21swapper/323:17:513
29715050,0irq/26-eth0-rx-0-21swapper/322:48:363
29715050,0irq/26-eth0-rx-0-21swapper/322:47:303
29715050,0irq/26-eth0-rx-0-21swapper/322:18:073
29715050,0irq/26-eth0-rx-0-21swapper/322:03:533
29715050,0irq/26-eth0-rx-0-21swapper/321:45:513
29715050,0irq/26-eth0-rx-0-21swapper/321:33:043
29715050,0irq/26-eth0-rx-0-21swapper/321:22:433
29715050,0irq/26-eth0-rx-0-21swapper/321:03:053
29715050,0irq/26-eth0-rx-0-21swapper/319:53:023
29715050,0irq/26-eth0-rx-0-21swapper/300:18:023
259950,0migration/27130-21kthreadcore22:43:052
259950,0migration/27071-21kthreadcore22:03:032
259950,0migration/232490-21kthreadcore18:58:032
259950,0migration/220893-21latency_hist21:52:462
259950,0migration/219112-21runrttasks22:50:252
249950,0watchdog/27203-21kthreadcore23:23:052
179950,0migration/15470-21kthreadcore00:23:051
179950,0migration/13882-21gltestperf00:02:591
179950,0migration/12723-21kthreadcore19:48:041
179950,0migration/124223-21kthreadcore21:13:041
179950,0migration/113620-21crond19:57:471
179950,0migration/111540-21taskset23:07:261
169950,0watchdog/127739-21aten2.4-expect23:37:561
149950,0migration/09899-21needreboot22:23:060
149950,0migration/0989-21fschecks_time19:48:010
149950,0migration/09146-21ntp_kernel_err23:23:060
149950,0migration/09054-21kthreadcore23:03:050
149950,0migration/07855-21taskset20:17:190
149950,0migration/07735-21kthreadcore21:23:050
149950,0migration/07256-21kthreadcore21:43:040
149950,0migration/07246-21aten_r1power_en19:52:590
149950,0migration/05009-21pmu-power19:23:080
149950,0migration/04954-21aten_r1power_po21:03:000
149950,0migration/031447-21latency19:43:050
149950,0migration/029512-21cpuspeed_turbos21:37:590
149950,0migration/02843-21aten_r1power_en00:22:590
149950,0migration/02803-21unixbench_singl20:58:120
149950,0migration/026741-21rt-features21:53:090
149950,0migration/023496-21kthreadcore23:33:040
149950,0migration/022877-21kthreadcore21:33:030
149950,0migration/022160-21taskset20:52:180
149950,0migration/021193-21kthreadcore00:13:030
149950,0migration/02069-21taskset20:37:080
149950,0migration/019945-21taskset21:09:470
149950,0migration/018879-21pmu-power21:28:070
149950,0migration/018336-21timerandwakeup22:28:090
149950,0migration/015424-21kthreadcore22:08:050
149950,0migration/015031-21kthreadcore23:28:040
149950,0migration/014718-21latency_hist20:22:470
149950,0migration/013038-21irqcore23:08:030
149950,0migration/012831-21latency_hist19:32:480
8215040,0irq/24-0000:00:26842-21latency_hist00:17:462
339940,0migration/39515-21kthreadcore20:18:043
339940,0migration/34281-21kthreadcore20:38:043
339940,0migration/331943-21aten2.4-expect19:22:593
339940,0migration/321841-21kthreadcore19:13:043
339940,0migration/320383-21kthreadcore20:48:043
339940,0migration/311475-21kthreadcore20:43:043
339940,0migration/310726-21kthreadcore19:28:053
307149943,0cyclictest3556-21lldpd23:02:522
307149943,0cyclictest3556-21lldpd20:12:222
307149943,0cyclictest3556-21lldpd19:37:512
307149943,0cyclictest3556-21lldpd19:27:512
307149943,0cyclictest3556-21lldpd19:21:212
307149943,0cyclictest29100-21awk20:33:022
307149943,0cyclictest10004-21awk20:43:022
307139944,0cyclictest0-21swapper/123:47:461
307139943,0cyclictest3556-21lldpd23:20:521
307139943,0cyclictest3556-21lldpd22:41:231
307139943,0cyclictest3556-21lldpd21:47:521
307139943,0cyclictest3556-21lldpd21:05:221
307139943,0cyclictest3556-21lldpd20:22:221
307139943,0cyclictest3556-21lldpd18:59:511
307139943,0cyclictest3514-21snmpd20:45:341
307139943,0cyclictest3241-21irqbalance22:44:251
307139943,0cyclictest29624-21awk22:18:011
307139943,0cyclictest27978-21awk00:18:001
307139943,0cyclictest22171-21awk21:33:011
307139943,0cyclictest21776-21awk22:13:031
307139943,0cyclictest21315-21awk22:33:011
307139943,0cyclictest14734-21awk19:58:021
307139940,0cyclictest0-21swapper/121:32:301
307129944,0cyclictest0-21swapper/020:37:470
307129943,0cyclictest5144-21tr22:43:040
307129943,0cyclictest4201-21awk23:42:590
307129943,0cyclictest3556-21lldpd23:59:220
307129943,0cyclictest3556-21lldpd22:22:230
307129943,0cyclictest3556-21lldpd21:12:510
307129943,0cyclictest3556-21lldpd19:15:210
307129943,0cyclictest3514-21snmpd20:46:040
307129943,0cyclictest29853-21awk21:58:010
307129943,0cyclictest28781-21awk23:18:000
307129943,0cyclictest28023-21awk20:08:020
307129943,0cyclictest22313-21awk20:28:010
29725040,0irq/27-eth0-tx-28229-21telnet22:57:572
29725040,0irq/27-eth0-tx-2699-21ssh22:39:442
29725040,0irq/27-eth0-tx-25623-21unixbench-2d23:33:092
29725040,0irq/27-eth0-tx-20715-21ssh22:12:492
29725040,0irq/27-eth0-tx-1831-21ssh23:39:292
29725040,0irq/27-eth0-tx-0-21swapper/223:42:532
29725040,0irq/27-eth0-tx-0-21swapper/223:12:532
29725040,0irq/27-eth0-tx-0-21swapper/222:52:532
29725040,0irq/27-eth0-tx-0-21swapper/221:48:192
29725040,0irq/27-eth0-tx-0-21swapper/221:33:232
29725040,0irq/27-eth0-tx-0-21swapper/221:12:592
29725040,0irq/27-eth0-tx-0-21swapper/221:07:592
29725040,0irq/27-eth0-tx-0-21swapper/200:25:182
29725040,0irq/27-eth0-tx-0-21swapper/200:08:032
29725040,0irq/27-eth0-tx-0-21swapper/200:03:562
29725040,0irq/27-eth0-tx-0-21swapper/019:03:000
29715040,0irq/26-eth0-rx-0-21swapper/323:07:593
29715040,0irq/26-eth0-rx-0-21swapper/321:23:003
29715040,0irq/26-eth0-rx-0-21swapper/320:58:513
29715040,0irq/26-eth0-rx-0-21swapper/319:18:093
29715040,0irq/26-eth0-rx-0-21swapper/319:02:593
29715040,0irq/26-eth0-rx-0-21swapper/300:08:223
259940,0migration/2863-21fschecks_count19:48:012
259940,0migration/25801-21kthreadcore20:38:052
259940,0migration/229948-21munin-node21:58:012
259940,0migration/227936-21fschecks_count23:58:002
259940,0migration/225342-21kthreadcore20:03:052
259940,0migration/222804-21kthreadcore00:13:032
259940,0migration/216475-21munin-node20:48:002
259940,0migration/216011-21kthreadcore22:28:042
259940,0migration/215385-21kthreadcore22:08:052
259940,0migration/214884-21aten_r1power_fr20:22:592
259940,0migration/214670-21idleruntime19:58:022
259940,0migration/213925-21kthreadcore19:08:042
259940,0migration/213924-21timerandwakeup20:18:092
259940,0migration/213729-21if_err_eth119:33:012
259940,0migration/210338-21timerandwakeup22:23:102
249940,0watchdog/25870-21perl19:03:022
249940,0watchdog/226951-21hddtemp_smartct19:43:012
249940,0watchdog/22652-21python21:38:092
249940,0watchdog/20-21swapper/221:43:012
179940,0migration/17416-21kthreadcore20:38:051
179940,0migration/14843-21aten_r1power_vo22:02:581
179940,0migration/14130-21idleruntime23:42:591
179940,0migration/1337-21kthreadcore20:33:041
179940,0migration/132309-21runrttasks19:44:271
179940,0migration/131888-21kthreadcore21:58:031
179940,0migration/130916-21kthreadcore22:58:031
179940,0migration/129921-21hwlatdetect20:58:031
179940,0migration/128833-21kthreadcore20:08:041
179940,0migration/12831-21switchtime21:18:091
179940,0migration/127678-21latency_hist23:17:461
179940,0migration/123502-21kthreadcore19:38:041
179940,0migration/120145-21fschecks_count23:32:591
179940,0migration/120112-21dump-pmu-power22:32:411
179940,0migration/119544-21munin-node23:52:591
179940,0migration/119199-21aten_r1power_fr00:12:571
179940,0migration/118348-21kthreadcore20:48:041
179940,0migration/116941-21switchtime00:08:091
179940,0migration/115768-21kthreadcore23:28:051
179940,0migration/113425-21kthreadcore23:08:031
179940,0migration/113178-21fschecks_count22:08:011
179940,0migration/112338-21cpu22:47:581
179940,0migration/110157-21kthreadcore19:28:051
169940,0watchdog/17202-21kthreadcore23:23:051
169940,0watchdog/16559-21runrttasks20:05:091
169940,0watchdog/128866-21diskmemload21:38:411
169940,0watchdog/122563-21kthreadcore22:53:051
169940,0watchdog/111615-21ssh21:44:451
169940,0watchdog/111285-21ssh22:25:171
169940,0watchdog/10-21swapper/121:53:051
169940,0watchdog/10-21swapper/121:23:011
169940,0watchdog/10-21swapper/120:23:011
169940,0watchdog/10-21swapper/120:13:051
169940,0watchdog/10-21swapper/119:08:021
169940,0watchdog/10-21swapper/119:03:061
149940,0migration/07234-21kthreadcore22:03:030
149940,0migration/025247-21kthreadcore20:03:050
149940,0migration/024284-21kthreadcore23:53:060
149940,0migration/02242-21sh22:38:260
149940,0migration/015502-21kthreadcore19:08:040
339930,0migration/328405-21irqrtprio20:08:033
339930,0migration/31140-21munin-node20:33:053
307149932,0cyclictest5927-21sadc19:27:472
307149932,0cyclictest3556-21lldpd19:15:512
307149932,0cyclictest22352-21awk20:28:022
307139933,0cyclictest3556-21lldpd20:54:521
307139933,0cyclictest3556-21lldpd19:19:511
307139932,0cyclictest323-21awk19:23:031
307139932,0cyclictest23509-21grep20:28:031
307129933,0cyclictest9941-21kworker/0:223:38:000
307129933,0cyclictest9941-21kworker/0:222:53:260
307129933,0cyclictest9941-21kworker/0:222:48:420
307129933,0cyclictest9941-21kworker/0:200:08:100
307129933,0cyclictest23291-21kworker/0:219:17:490
307129933,0cyclictest13739-21kworker/0:122:13:060
307129932,0cyclictest6153-21kthreadcore00:03:030
307129932,0cyclictest30374-21awk21:18:010
307129932,0cyclictest25948-21awk22:33:060
29715030,0irq/26-eth0-rx-0-21swapper/320:27:583
29715030,0irq/26-eth0-rx-0-21swapper/320:13:093
29715030,0irq/26-eth0-rx-0-21swapper/320:02:053
29715030,0irq/26-eth0-rx-0-21swapper/318:58:513
259930,0migration/21632-21cpuspeed_turbos20:13:002
179930,0migration/122817-21kthreadcore19:13:041
169930,0watchdog/132367-21kthreadcore23:58:041
169930,0watchdog/10-21swapper/121:08:051
169930,0watchdog/10-21swapper/119:33:091
307149920,0cyclictest0-21swapper/220:52:462
29715020,0irq/26-eth0-rx-0-21swapper/320:53:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional