You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 06:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Tue Aug 04, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450890,0irq/25-eth10-21swapper/318:43:403
1736623632,0sleep20-21swapper/218:47:532
1700323624,0sleep10-21swapper/118:43:391
1703723322,0sleep00-21swapper/018:43:580
4170,0ktimersoftd/05740-21tr20:38:090
174409976,0cyclictest10203-21kthreadcore22:03:121
174399977,0cyclictest0-21swapper/020:49:510
35160,0ktimersoftd/30-21swapper/322:11:563
339960,0migration/322809-21munin-node22:33:083
29725060,0irq/27-eth0-tx-5259-21munin-node19:03:150
29725060,0irq/27-eth0-tx-20559-21munin-node20:03:130
29715060,0irq/26-eth0-rx-6533-21diskstats22:03:073
29715060,0irq/26-eth0-rx-29504-21python20:53:173
29715060,0irq/26-eth0-rx-29347-21cat23:38:093
174399960,0cyclictest0-21swapper/023:22:390
174399960,0cyclictest0-21swapper/021:31:020
149960,0migration/025732-21sh23:54:460
149960,0migration/011530-21sh23:06:130
8215050,0irq/24-0000:00:24929-21rsyslogd19:17:532
4150,0ktimersoftd/00-21swapper/020:13:090
339950,0migration/39444-21pmu-power23:23:153
339950,0migration/38543-21kthreadcore22:43:103
339950,0migration/38133-21latency_hist20:17:533
339950,0migration/37624-21munin-node19:53:073
339950,0migration/36786-21aten_r1power_en21:43:063
339950,0migration/35744-21kernelversion23:03:093
339950,0migration/35603-21taskset21:40:293
339950,0migration/35569-21cpuspeed_turbos19:03:053
339950,0migration/34465-21kthreadcore20:13:103
339950,0migration/33564-21diskmemload00:03:083
339950,0migration/33394-21munin-node22:18:143
339950,0migration/330527-21aten_r1power_cu21:58:023
339950,0migration/329139-21munin-node21:33:173
339950,0migration/328832-21aten_r1power_en22:58:033
339950,0migration/328351-21munin-node20:08:093
339950,0migration/328079-21munin-node23:58:093
339950,0migration/323763-21taskset18:49:263
339950,0migration/323571-21latency_hist21:12:533
339950,0migration/321149-21kthreadcore23:53:113
339950,0migration/320681-21kthreadcore19:13:103
339950,0migration/32044-21taskset20:36:583
339950,0migration/320012-21taskset19:59:193
339950,0migration/318753-21timerwakeupswit19:33:173
339950,0migration/312674-21runrttasks19:32:453
29725050,0irq/27-eth0-tx-4985-21munin-node22:43:142
29725050,0irq/27-eth0-tx-32106-21tr21:58:072
29725050,0irq/27-eth0-tx-31398-21cut23:58:112
29725050,0irq/27-eth0-tx-28615-21unixbench_multi21:53:162
29725050,0irq/27-eth0-tx-2558-21munin-node00:03:162
29725050,0irq/27-eth0-tx-12364-21munin-node23:08:042
29725050,0irq/27-eth0-tx-0-21swapper/223:27:232
29725050,0irq/27-eth0-tx-0-21swapper/222:15:512
29725050,0irq/27-eth0-tx-0-21swapper/221:08:242
29725050,0irq/27-eth0-tx-0-21swapper/221:04:552
29725050,0irq/27-eth0-tx-0-21swapper/019:17:410
29725050,0irq/27-eth0-tx-0-21swapper/018:53:180
29715050,0irq/26-eth0-rx-9026-21mailstats23:43:143
29715050,0irq/26-eth0-rx-3514-21snmpd21:48:393
29715050,0irq/26-eth0-rx-19118-21ssh23:32:033
29715050,0irq/26-eth0-rx-17861-21kthreadcore21:08:103
29715050,0irq/26-eth0-rx-12797-21if_err_eth1.319:08:073
29715050,0irq/26-eth0-rx-0-21swapper/323:50:053
29715050,0irq/26-eth0-rx-0-21swapper/323:37:443
29715050,0irq/26-eth0-rx-0-21swapper/322:38:133
29715050,0irq/26-eth0-rx-0-21swapper/322:32:473
29715050,0irq/26-eth0-rx-0-21swapper/322:13:093
29715050,0irq/26-eth0-rx-0-21swapper/321:57:103
29715050,0irq/26-eth0-rx-0-21swapper/321:31:353
29715050,0irq/26-eth0-rx-0-21swapper/320:52:293
29715050,0irq/26-eth0-rx-0-21swapper/319:23:193
259950,0migration/27776-21kthreadcore22:23:132
259950,0migration/26571-21kthreadcore23:43:102
259950,0migration/22974-21latency_hist23:42:522
259950,0migration/222070-21sh21:30:272
259950,0migration/221343-21hwlatdetect20:03:072
259950,0migration/217890-21sh23:28:232
259950,0migration/214800-21munin-node20:43:162
259950,0migration/21389-21kthreadcore21:38:102
259950,0migration/213073-21aten_r1power_fr22:48:042
249950,0watchdog/27080-21df_abs21:43:072
249950,0watchdog/230217-21sh21:36:072
249950,0watchdog/212350-21ksmtuned19:28:552
174449950,0cyclictest0-21swapper/220:58:522
174409950,0cyclictest0-21swapper/121:30:241
174399954,0cyclictest7099-21awk22:03:090
174399950,0cyclictest0-21swapper/022:49:510
174399950,0cyclictest0-21swapper/022:10:070
179950,0migration/19506-21sh00:05:311
179950,0migration/15124-21aten_r1power_vo22:43:041
179950,0migration/13920-21munin-node23:23:061
179950,0migration/13358-21run-parts22:38:531
179950,0migration/132460-21idleruntime21:18:081
179950,0migration/131057-21kthreadcore23:18:081
179950,0migration/130563-21aten_r1power_fr21:58:021
179950,0migration/127742-21kthreadcore19:18:121
179950,0migration/125825-21taskset19:40:401
179950,0migration/124493-21latency23:53:121
179950,0migration/12377-21timerwakeupswit22:58:151
179950,0migration/116701-21munin-node23:48:101
179950,0migration/112492-21kthreadcore20:43:101
169950,0watchdog/15881-21ssh21:41:191
149950,0migration/09847-21taskset00:06:340
149950,0migration/08666-21fschecks_count20:18:060
149950,0migration/07972-21kthreadcore22:43:090
149950,0migration/06990-21kthreadcore22:23:120
149950,0migration/04449-21irqrtprio23:43:090
149950,0migration/03924-21kthreadcore19:23:250
149950,0migration/030815-21kthreadcore23:38:100
149950,0migration/028762-21taskset22:15:040
149950,0migration/027281-21sh23:37:470
149950,0migration/02475-21unixbench_multi22:58:160
149950,0migration/023923-21kthreadcore19:38:120
149950,0migration/022998-21kthreadcore22:53:100
149950,0migration/022673-21kthreadcore23:13:100
149950,0migration/019620-21kthreadcore21:48:100
149950,0migration/018361-21taskset23:29:490
149950,0migration/017305-21kthreadcore22:28:120
149950,0migration/014662-21kthreadcore19:08:090
339940,0migration/37961-21cpuspeed_turbos21:03:063
339940,0migration/36578-21taskset19:48:193
339940,0migration/331827-21sh20:58:033
339940,0migration/311245-21kthreadcore21:23:133
339940,0migration/310879-21kthreadcore20:43:103
29725040,0irq/27-eth0-tx-21647-21aten2.4-expect20:28:060
29725040,0irq/27-eth0-tx-0-21swapper/223:52:552
29725040,0irq/27-eth0-tx-0-21swapper/223:48:092
29725040,0irq/27-eth0-tx-0-21swapper/223:33:052
29725040,0irq/27-eth0-tx-0-21swapper/223:18:002
29725040,0irq/27-eth0-tx-0-21swapper/223:03:462
29725040,0irq/27-eth0-tx-0-21swapper/222:32:552
29725040,0irq/27-eth0-tx-0-21swapper/222:17:542
29725040,0irq/27-eth0-tx-0-21swapper/222:09:482
29725040,0irq/27-eth0-tx-0-21swapper/222:03:162
29725040,0irq/27-eth0-tx-0-21swapper/221:23:052
29725040,0irq/27-eth0-tx-0-21swapper/221:12:552
29725040,0irq/27-eth0-tx-0-21swapper/200:12:562
29725040,0irq/27-eth0-tx-0-21swapper/020:43:160
29725040,0irq/27-eth0-tx-0-21swapper/019:48:050
29725040,0irq/27-eth0-tx-0-21swapper/019:32:530
29715040,0irq/26-eth0-rx-0-21swapper/323:18:273
29715040,0irq/26-eth0-rx-0-21swapper/323:13:103
29715040,0irq/26-eth0-rx-0-21swapper/323:08:063
29715040,0irq/26-eth0-rx-0-21swapper/322:53:163
29715040,0irq/26-eth0-rx-0-21swapper/322:48:403
29715040,0irq/26-eth0-rx-0-21swapper/322:24:473
29715040,0irq/26-eth0-rx-0-21swapper/321:19:293
29715040,0irq/26-eth0-rx-0-21swapper/300:13:133
29715040,0irq/26-eth0-rx-0-21swapper/300:08:113
259940,0migration/2312-21munin-node19:23:222
259940,0migration/228854-21aten2.4-expect22:58:032
259940,0migration/227394-21kthreadcore19:18:122
259940,0migration/224398-21aten_r1power_fr18:53:062
259940,0migration/220908-21aten2.4-expect22:53:042
259940,0migration/218627-21kthreadcore20:48:102
259940,0migration/214776-21irqrtprio22:28:102
259940,0migration/214302-21kthreadcore19:08:092
259940,0migration/213741-21idleruntime19:33:072
259940,0migration/211553-21kthreadcore20:18:102
259940,0migration/21046-21fschecks_time19:48:052
249940,0watchdog/27803-21perl19:53:072
249940,0watchdog/220783-21fschecks_count23:13:072
249940,0watchdog/20-21swapper/222:38:112
249940,0watchdog/20-21swapper/221:18:072
249940,0watchdog/20-21swapper/220:37:592
249940,0watchdog/20-21swapper/218:58:122
174449943,0cyclictest3556-21lldpd20:52:552
174449943,0cyclictest3556-21lldpd20:30:552
174449943,0cyclictest3556-21lldpd20:24:552
174449943,0cyclictest3556-21lldpd20:16:552
174449943,0cyclictest3556-21lldpd20:07:552
174449943,0cyclictest3556-21lldpd20:00:552
174449943,0cyclictest3556-21lldpd19:42:552
174449943,0cyclictest3556-21lldpd19:06:542
174449943,0cyclictest3241-21irqbalance20:33:112
174449943,0cyclictest18634-21awk18:48:092
174449943,0cyclictest15657-21awk21:48:082
174449943,0cyclictest11513-21awk00:08:082
174409944,0cyclictest0-21swapper/122:55:331
174409943,0cyclictest8241-21sed19:53:101
174409943,0cyclictest6113-21awk19:03:071
174409943,0cyclictest3556-21lldpd22:27:561
174409943,0cyclictest3556-21lldpd22:18:561
174409943,0cyclictest3556-21lldpd21:14:241
174409943,0cyclictest3556-21lldpd20:32:551
174409943,0cyclictest3556-21lldpd18:52:541
174409943,0cyclictest3556-21lldpd18:47:541
174409943,0cyclictest3241-21irqbalance23:16:011
174409943,0cyclictest3241-21irqbalance22:48:411
174409943,0cyclictest3241-21irqbalance20:22:211
174409943,0cyclictest27127-21awk19:43:071
174409943,0cyclictest14839-21awk19:58:101
174399944,0cyclictest0-21swapper/022:42:360
174399943,0cyclictest3556-21lldpd23:09:250
174399943,0cyclictest3556-21lldpd21:21:550
174399943,0cyclictest3556-21lldpd19:19:540
174399943,0cyclictest3514-21snmpd21:26:580
174399943,0cyclictest3241-21irqbalance23:48:510
174399943,0cyclictest28030-21awk20:08:070
174399943,0cyclictest24708-21awk20:53:100
174399943,0cyclictest17655-21munin-node18:48:110
179940,0migration/19203-21kthreadcore21:23:121
179940,0migration/15984-21kthreadcore23:43:101
179940,0migration/15479-21latency_hist22:22:531
179940,0migration/1393-21kthreadcore18:58:111
179940,0migration/131486-21kthreadcore23:38:101
179940,0migration/127517-21runrttasks20:08:051
179940,0migration/125900-21munin-node20:03:111
179940,0migration/125394-21runrttasks00:15:401
179940,0migration/123795-21kthreadcore23:33:101
179940,0migration/123701-21kthreadcore19:13:111
179940,0migration/123074-21irqcore22:13:071
179940,0migration/120564-21kthreadcore21:08:111
179940,0migration/116354-21kthreadcore23:08:111
179940,0migration/114590-21latency_hist21:47:531
179940,0migration/111192-21fschecks_time00:08:061
169940,0watchdog/15158-21gltestperf23:03:071
169940,0watchdog/132300-21munin-node19:23:201
169940,0watchdog/130941-21ssh20:56:191
169940,0watchdog/129842-21kthreadcore23:58:101
169940,0watchdog/127640-21sh22:33:231
169940,0watchdog/118775-21kthreadcore20:23:111
169940,0watchdog/116908-21grep22:08:111
169940,0watchdog/11475-21iostat_ios19:48:071
169940,0watchdog/10-21swapper/121:43:071
169940,0watchdog/10-21swapper/121:05:431
149940,0migration/06630-21latency_hist21:42:530
149940,0migration/04367-21diskmemload20:58:120
149940,0migration/04289-21fschecks_count23:23:070
149940,0migration/03589-21missed_timers21:58:090
149940,0migration/031145-21irqcore22:18:090
149940,0migration/029424-21kernelversion20:33:100
149940,0migration/028505-21latency21:33:110
149940,0migration/02645-21kthreadcore18:58:120
149940,0migration/024470-21kthreadcore22:33:080
149940,0migration/023687-21aten_r1power_cu21:13:040
149940,0migration/020356-21latency_hist20:02:530
149940,0migration/017465-21kthreadcore20:23:110
149940,0migration/013391-21sensors_fan21:03:150
29725030,0irq/27-eth0-tx-0-21swapper/019:43:120
29725030,0irq/27-eth0-tx-0-21swapper/019:28:040
29715030,0irq/26-eth0-rx-0-21swapper/320:37:573
29715030,0irq/26-eth0-rx-0-21swapper/320:28:063
29715030,0irq/26-eth0-rx-0-21swapper/320:24:123
29715030,0irq/26-eth0-rx-0-21swapper/320:05:073
29715030,0irq/26-eth0-rx-0-21swapper/319:47:193
29715030,0irq/26-eth0-rx-0-21swapper/319:39:093
29715030,0irq/26-eth0-rx-0-21swapper/319:18:063
249930,0watchdog/219558-40yum-updatesd-he19:37:592
174409933,0cyclictest3556-21lldpd20:15:251
174409932,0cyclictest9141-21cat19:28:101
174409932,0cyclictest18735-21grep20:48:101
174409932,0cyclictest13683-21gltestperf19:33:071
174409932,0cyclictest12602-21awk23:28:071
174409930,0cyclictest0-21swapper/120:37:591
174409930,0cyclictest0-21swapper/119:08:311
174399932,0cyclictest8097-21awk19:53:090
174399932,0cyclictest29087-21ssh21:54:260
174399932,0cyclictest27958-21awk23:58:080
174399932,0cyclictest11463-21awk00:08:080
174399930,0cyclictest0-21swapper/021:37:590
174399930,0cyclictest0-21swapper/000:13:070
169930,0watchdog/122038-21fschecks_time20:28:071
169930,0watchdog/10-21swapper/121:53:071
169930,0watchdog/10-21swapper/121:36:071
169930,0watchdog/10-21swapper/120:58:071
339920,0migration/324703-21diskstats18:53:083
29715020,0irq/26-eth0-rx-0-21swapper/318:58:063
174399920,0cyclictest0-21swapper/021:08:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional