You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 15:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Mar 31, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
72925546,0sleep30-21swapper/306:57:073
86425039,0sleep00-21swapper/006:58:120
2704823329,0sleep10-21swapper/106:54:531
58822622,0sleep20-21swapper/206:55:302
10979973,0cyclictest3556-21lldpd10:28:452
10979973,0cyclictest31864-21awk10:40:022
10979973,0cyclictest29692-21sadc08:09:492
10979973,0cyclictest24499-21awk09:55:042
10979973,0cyclictest15510-21awk07:10:022
10979972,0cyclictest21816-21awk12:15:032
339960,0migration/37030-21sh09:43:563
339960,0migration/35344-21sh10:20:523
339960,0migration/34935-21sh12:04:583
339960,0migration/34383-21sh12:04:003
339960,0migration/331220-21sh09:38:333
339960,0migration/329935-21sh09:55:493
339960,0migration/328462-21sh12:19:273
339960,0migration/326926-21sh11:55:083
339960,0migration/320787-21sh11:11:253
339960,0migration/320479-21sh11:52:573
339960,0migration/315327-21sh09:09:363
29715060,0irq/26-eth0-rx-394-21tr11:00:053
29715060,0irq/26-eth0-rx-22376-21munin-run10:54:483
29715060,0irq/26-eth0-rx-13256-21df12:10:003
19160,0ktimersoftd/115006-21perf10:09:481
149960,0migration/05628-21sh10:42:390
149960,0migration/031213-21fschecks_time11:00:010
10979965,0cyclictest0-21swapper/211:37:542
10979965,0cyclictest0-21swapper/211:37:542
10979962,0cyclictest32042-21cat11:00:042
10979962,0cyclictest25035-21awk08:30:042
10979962,0cyclictest23964-21awk10:35:042
10979962,0cyclictest22539-21irqcore11:55:052
10979962,0cyclictest2201-21awk07:00:032
10979962,0cyclictest15366-21telnet10:09:592
10979962,0cyclictest10804-21awk07:55:032
10979961,0cyclictest8853-21awk09:25:052
10979961,0cyclictest5691-21sh09:00:282
10979961,0cyclictest4956-21sh12:04:592
10979961,0cyclictest4752-21munin-node10:20:102
10979961,0cyclictest4001-21sh12:24:142
10979961,0cyclictest30025-21munin-node09:15:132
10979961,0cyclictest17982-21cut09:10:052
10979961,0cyclictest15521-21awk10:50:032
10979961,0cyclictest13320-21munin-node07:05:082
10979961,0cyclictest12837-21sh10:05:102
10979960,0cyclictest8801-21irqrtprio09:45:052
10979960,0cyclictest6685-21munin-node10:45:052
10979960,0cyclictest4363-21kthreadcore09:40:062
10979960,0cyclictest32649-21kthreadcore08:35:062
10979960,0cyclictest3200-17sedispatch07:24:492
10979960,0cyclictest27249-21kthreadcore10:55:062
10979960,0cyclictest26381-21cut08:55:052
10979960,0cyclictest26112-21cut09:35:052
10979960,0cyclictest22457-21cut07:15:052
10979960,0cyclictest22180-21sh10:33:192
10979960,0cyclictest2202-21cat10:00:052
10979960,0cyclictest19852-21ssh11:51:032
10979960,0cyclictest19536-21cut07:35:062
10979960,0cyclictest17205-21sed09:30:052
10979960,0cyclictest15301-21grep12:10:042
10979960,0cyclictest13950-21aten2.4-expect11:29:592
10979960,0cyclictest12855-21kthreadcore07:30:072
10979960,0cyclictest12461-21ssh11:47:452
10979960,0cyclictest12298-21ssh11:25:582
10979960,0cyclictest11497-21kthreadcore09:05:062
10979960,0cyclictest0-21swapper/212:00:032
10979960,0cyclictest0-21swapper/209:50:072
10979960,0cyclictest0-21swapper/209:19:592
10919966,0cyclictest31605-21diskmemload09:34:131
10919965,0cyclictest0-21swapper/111:25:121
339950,0migration/39033-21munin-node07:50:113
339950,0migration/38807-21munin-node07:05:043
339950,0migration/37412-21munin-node07:00:133
339950,0migration/36861-21munin-node11:25:053
339950,0migration/35411-21cpuspeed_turbos08:39:583
339950,0migration/35141-21timerandwakeup10:00:113
339950,0migration/332561-21gltestperf09:00:013
339950,0migration/328423-21munin-node10:55:113
339950,0migration/325097-21kthreadcore10:35:053
339950,0migration/322615-21diskmemload10:13:403
339950,0migration/321264-21sendmail_mailst07:35:093
339950,0migration/316912-21munin-node09:50:063
339950,0migration/315138-21aten_r1power_vo10:30:013
339950,0migration/311984-21taskset11:46:183
29725050,0irq/27-eth0-tx-23276-21munin-node08:05:090
29725050,0irq/27-eth0-tx-0-21swapper/007:41:120
29725050,0irq/27-eth0-tx-0-21swapper/007:11:510
29715050,0irq/26-eth0-rx-9114-21cut12:25:033
29715050,0irq/26-eth0-rx-24195-21munin-node08:30:003
29715050,0irq/26-eth0-rx-23933-21munin-node09:15:083
29715050,0irq/26-eth0-rx-23054-21perf08:04:493
29715050,0irq/26-eth0-rx-12798-21swap10:25:103
29715050,0irq/26-eth0-rx-0-21swapper/312:23:273
29715050,0irq/26-eth0-rx-0-21swapper/311:35:043
29715050,0irq/26-eth0-rx-0-21swapper/311:35:043
29715050,0irq/26-eth0-rx-0-21swapper/311:19:583
29715050,0irq/26-eth0-rx-0-21swapper/310:43:113
29715050,0irq/26-eth0-rx-0-21swapper/310:14:593
29715050,0irq/26-eth0-rx-0-21swapper/310:09:473
29715050,0irq/26-eth0-rx-0-21swapper/309:45:003
29715050,0irq/26-eth0-rx-0-21swapper/309:32:003
29715050,0irq/26-eth0-rx-0-21swapper/309:20:353
29715050,0irq/26-eth0-rx-0-21swapper/308:35:013
29715050,0irq/26-eth0-rx-0-21swapper/307:45:083
179950,0migration/131410-21aten_r1power_po10:20:001
179950,0migration/130052-21aten2.4-expect08:10:011
179950,0migration/129541-21taskset10:15:301
179950,0migration/124892-21gltestperf08:30:031
179950,0migration/113372-21missed_timers09:25:081
169950,0watchdog/112158-21cut07:05:071
149950,0migration/08540-21fschecks_time09:25:030
149950,0migration/07045-21sh09:04:030
149950,0migration/06684-21kthreadcore08:15:070
149950,0migration/05696-21timerwakeupswit09:20:100
149950,0migration/04655-21sh11:22:010
149950,0migration/031933-21fschecks_time10:20:020
149950,0migration/030894-21sh09:17:330
149950,0migration/028651-21processes10:35:100
149950,0migration/025350-21kthreadcore11:55:060
149950,0migration/024776-21kthreadcore11:35:050
149950,0migration/024776-21kthreadcore11:35:050
149950,0migration/022676-21sh09:12:180
149950,0migration/022281-21kthreadcore08:25:060
149950,0migration/021557-21kthreadcore08:50:030
149950,0migration/021353-21munin-node09:50:100
149950,0migration/020923-21taskset10:50:380
149950,0migration/017911-21kernelversion08:00:060
149950,0migration/015573-21cpuspeed_turbos10:10:000
149950,0migration/014683-21taskset10:08:530
149950,0migration/01340-21aten_r1power_en07:00:000
149950,0migration/013105-21sh10:46:050
149950,0migration/011676-21munin-node11:25:120
149950,0migration/010551-21kthreadcore11:05:050
10979951,0cyclictest8681-21kworker/2:207:49:492
10979951,0cyclictest29930-21hostname11:39:592
10979951,0cyclictest22274-21seq07:39:582
10979951,0cyclictest17530-21latency_hist08:24:492
10979951,0cyclictest0-21swapper/208:09:492
10979950,0cyclictest6559-21runrttasks07:24:492
10979950,0cyclictest6117-21latency_hist11:04:482
10979950,0cyclictest5164-21chrt08:39:562
10979950,0cyclictest4469-21cut12:24:572
10979950,0cyclictest4304-21munin-node08:15:022
10979950,0cyclictest3339-21dbus-daemon08:44:522
10979950,0cyclictest30467-21sort11:19:592
10979950,0cyclictest28866-21wc07:44:492
10979950,0cyclictest23243-21sh10:14:512
10979950,0cyclictest18537-21sh08:49:572
10979950,0cyclictest16574-21latency_hist07:59:492
10979950,0cyclictest14519-21grep11:10:002
10979950,0cyclictest11071-21egrep08:20:022
10979950,0cyclictest0-21swapper/211:14:592
10919955,0cyclictest31605-21diskmemload10:59:491
10919955,0cyclictest0-21swapper/109:42:151
10799950,0cyclictest0-21swapper/010:00:000
339940,0migration/33606-21kthreadcore07:25:073
339940,0migration/321997-21fschecks_time07:15:023
339940,0migration/321682-21kthreadcore08:25:053
339940,0migration/318698-21kthreadcore07:10:053
339940,0migration/318469-21latency_hist08:49:493
339940,0migration/317117-21taskset08:22:303
339940,0migration/316371-21taskset07:59:403
339940,0migration/31273-21kthreadcore08:10:063
29725040,0irq/27-eth0-tx-8958-21telnet07:30:000
29715040,0irq/26-eth0-rx-0-21swapper/311:40:003
29715040,0irq/26-eth0-rx-0-21swapper/311:29:563
29715040,0irq/26-eth0-rx-0-21swapper/311:16:163
29715040,0irq/26-eth0-rx-0-21swapper/311:06:393
29715040,0irq/26-eth0-rx-0-21swapper/309:25:003
29715040,0irq/26-eth0-rx-0-21swapper/309:09:503
29715040,0irq/26-eth0-rx-0-21swapper/308:05:073
179940,0migration/17897-21kthreadcore11:05:041
179940,0migration/13368-21kthreadcore10:40:051
179940,0migration/132170-21kthreadcore11:20:041
179940,0migration/128254-21kthreadcore09:15:061
179940,0migration/127035-21kthreadcore10:55:061
179940,0migration/125512-21kthreadcore09:35:051
179940,0migration/124287-21kthreadcore12:15:051
179940,0migration/122746-21if_eth011:35:021
179940,0migration/122746-21if_eth011:35:021
179940,0migration/121525-21kthreadcore08:50:031
179940,0migration/120508-21kthreadcore10:10:051
179940,0migration/116288-21taskset07:58:421
179940,0migration/112664-21kthreadcore08:20:061
179940,0migration/112235-21taskset11:47:041
169940,0watchdog/17818-21proc_pri07:25:111
169940,0watchdog/16559-21runrttasks08:40:111
169940,0watchdog/14518-21ssh12:04:221
169940,0watchdog/1385-21interrupts09:00:031
169940,0watchdog/13026-21ssh12:21:261
169940,0watchdog/121395-21ssh10:51:581
169940,0watchdog/11817-21kthreadcore08:35:071
169940,0watchdog/10-21swapper/112:24:581
169940,0watchdog/10-21swapper/110:30:061
169940,0watchdog/10-21swapper/108:55:031
169940,0watchdog/10-21swapper/107:44:511
149940,0migration/0816-21kthreadcore11:40:050
149940,0migration/06733-21kthreadcore07:50:060
149940,0migration/06233-21kthreadcore07:25:080
149940,0migration/05890-21gltestperf08:39:590
149940,0migration/04659-21kthreadcore09:40:060
149940,0migration/031710-21kthreadcore07:45:050
149940,0migration/030828-21if_eth008:10:040
149940,0migration/029357-21aten_r1power_cu12:00:000
149940,0migration/026739-21kthreadcore10:15:050
149940,0migration/01553-21kthreadcore12:20:040
149940,0migration/011962-21runrttasks12:07:440
149940,0migration/011565-21kthreadcore07:55:050
149940,0migration/011235-21kthreadcore09:05:060
149940,0migration/011008-21aten_r1power_po08:20:010
149940,0migration/010341-21kthreadcore11:45:050
10919944,0cyclictest31605-21diskmemload10:04:011
10919944,0cyclictest0-21swapper/111:14:381
10919944,0cyclictest0-21swapper/108:29:441
10919943,0cyclictest9885-21awk07:30:051
10919943,0cyclictest8553-21awk09:05:041
10919943,0cyclictest8540-21awk09:45:041
10919943,0cyclictest5893-21awk12:05:041
10919943,0cyclictest3556-21lldpd08:47:131
10919943,0cyclictest3514-21snmpd10:47:081
10919943,0cyclictest3514-21snmpd09:23:081
10919943,0cyclictest3514-21snmpd07:44:221
10919943,0cyclictest3514-21snmpd07:21:071
10919943,0cyclictest3514-21snmpd07:19:371
10919943,0cyclictest3241-21irqbalance09:14:471
10919943,0cyclictest24483-21awk09:55:041
10919943,0cyclictest24255-21awk08:05:051
10919943,0cyclictest22349-21awk11:55:041
10919943,0cyclictest17533-21awk08:00:041
10919943,0cyclictest13974-21grep12:10:031
10799944,0cyclictest5729-21kworker/0:112:17:010
10799943,0cyclictest5450-21awk12:25:010
10799943,0cyclictest3556-21lldpd10:29:450
10799943,0cyclictest3514-21snmpd11:51:380
10799943,0cyclictest3514-21snmpd11:30:080
10799943,0cyclictest3514-21snmpd08:32:220
10799943,0cyclictest26055-21awk08:55:040
10799943,0cyclictest23567-21awk10:55:030
10799943,0cyclictest18280-21sadc08:49:490
10799943,0cyclictest16879-21awk09:30:040
10799943,0cyclictest15834-21awk10:30:030
10799941,0cyclictest0-21swapper/008:35:010
339930,0migration/315241-21runrttasks07:33:073
29725030,0irq/27-eth0-tx-15684-21aten2.4-expect07:35:000
29715030,0irq/26-eth0-rx-0-21swapper/310:44:523
29715030,0irq/26-eth0-rx-0-21swapper/308:55:053
29715030,0irq/26-eth0-rx-0-21swapper/308:45:003
29715030,0irq/26-eth0-rx-0-21swapper/308:17:403
29715030,0irq/26-eth0-rx-0-21swapper/307:41:583
29715030,0irq/26-eth0-rx-0-21swapper/307:19:583
169930,0watchdog/10-21swapper/110:35:581
169930,0watchdog/10-21swapper/110:24:511
169930,0watchdog/10-21swapper/109:50:071
169930,0watchdog/10-21swapper/108:15:071
169930,0watchdog/10-21swapper/107:50:071
149930,0migration/021102-21sh11:12:200
10919932,0cyclictest3514-21snmpd07:39:371
10919932,0cyclictest3217-21sensors11:40:091
10919932,0cyclictest20344-21sensors07:10:091
10919932,0cyclictest19637-21sensors11:30:091
10919931,0cyclictest22618-21df_abs11:15:011
10799933,0cyclictest3041-21kworker/0:109:55:060
10799933,0cyclictest3041-21kworker/0:109:45:080
10799933,0cyclictest3041-21kworker/0:107:15:040
10799932,0cyclictest27975-21munin-node07:20:040
10799932,0cyclictest27601-21kthreadcore09:35:060
10799932,0cyclictest23154-21awk11:15:030
10799932,0cyclictest12644-21sh12:09:480
10799930,0cyclictest0-21swapper/007:05:000
169920,0watchdog/10-21swapper/106:59:511
10919921,0cyclictest13311-21cut11:49:481
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional