You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 07:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sat May 23, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450550,0irq/25-eth10-21swapper/318:51:483
2908123424,0sleep10-21swapper/118:53:011
2893223425,0sleep20-21swapper/218:51:182
2872123423,0sleep00-21swapper/018:50:160
339970,0migration/329454-21aten_r1power_po00:20:003
29715070,0irq/26-eth0-rx-0-21swapper/323:35:343
293079974,0cyclictest6673-21munin-run21:24:510
35160,0ktimersoftd/30-21swapper/320:56:353
339960,0migration/35924-21sh22:03:563
339960,0migration/328835-21sh23:17:133
339960,0migration/320983-21sh22:31:183
339960,0migration/320165-21sh23:10:253
339960,0migration/320021-21sh00:11:503
339960,0migration/313534-21sh21:26:293
29725060,0irq/27-eth0-tx-25763-21munin-node20:05:090
29715060,0irq/26-eth0-rx-22588-21ssh22:14:523
293089960,0cyclictest22399-21diskmemload23:59:021
293079960,0cyclictest19850-21ssh00:11:210
149960,0migration/04879-21sh21:40:300
149960,0migration/023363-21taskset19:12:230
8215050,0irq/24-0000:00:14356-21perf22:09:512
4150,0ktimersoftd/00-21swapper/023:14:170
339950,0migration/38940-21kthreadcore22:45:073
339950,0migration/38907-21kthreadcore23:45:083
339950,0migration/3820-21latency_hist20:34:513
339950,0migration/36626-21sh21:04:543
339950,0migration/330753-21sh20:59:573
339950,0migration/328338-21pmu-power21:55:103
339950,0migration/327471-21kthreadcore22:55:073
339950,0migration/32510-21kthreadcore21:40:093
339950,0migration/318735-21kthreadcore19:10:073
339950,0migration/317363-21kthreadcore21:30:093
339950,0migration/317031-21kthreadcore21:10:063
339950,0migration/312650-21irqrtprio19:30:063
29725050,0irq/27-eth0-tx-6938-21munin-node21:25:072
29725050,0irq/27-eth0-tx-5458-21munin-node23:45:122
29725050,0irq/27-eth0-tx-4767-21ssh22:20:572
29725050,0irq/27-eth0-tx-23646-21if_eth1.322:15:062
29725050,0irq/27-eth0-tx-18116-21munin-node19:35:120
29725050,0irq/27-eth0-tx-14496-21munin-node22:30:052
29725050,0irq/27-eth0-tx-11463-21munin-node19:30:120
29725050,0irq/27-eth0-tx-0-21swapper/223:56:302
29725050,0irq/27-eth0-tx-0-21swapper/223:29:312
29725050,0irq/27-eth0-tx-0-21swapper/223:21:452
29725050,0irq/27-eth0-tx-0-21swapper/223:19:202
29725050,0irq/27-eth0-tx-0-21swapper/223:12:082
29725050,0irq/27-eth0-tx-0-21swapper/222:58:382
29725050,0irq/27-eth0-tx-0-21swapper/221:59:542
29725050,0irq/27-eth0-tx-0-21swapper/221:46:282
29725050,0irq/27-eth0-tx-0-21swapper/221:41:002
29725050,0irq/27-eth0-tx-0-21swapper/221:35:202
29725050,0irq/27-eth0-tx-0-21swapper/200:08:282
29725050,0irq/27-eth0-tx-0-21swapper/020:39:170
29715050,0irq/26-eth0-rx-8742-21kthreadcore22:05:083
29715050,0irq/26-eth0-rx-3200-17sedispatch22:11:193
29715050,0irq/26-eth0-rx-31334-21kthreadcore23:20:053
29715050,0irq/26-eth0-rx-29118-21yum21:35:163
29715050,0irq/26-eth0-rx-25757-21python20:25:133
29715050,0irq/26-eth0-rx-2462-21grep19:45:083
29715050,0irq/26-eth0-rx-18807-21chrt19:59:073
29715050,0irq/26-eth0-rx-18191-21kthreadcore22:50:083
29715050,0irq/26-eth0-rx-18191-21kthreadcore22:50:083
29715050,0irq/26-eth0-rx-18153-21cut19:35:023
29715050,0irq/26-eth0-rx-13728-21munin-node23:30:133
29715050,0irq/26-eth0-rx-12386-21ssh23:26:453
29715050,0irq/26-eth0-rx-10106-21cut20:15:083
29715050,0irq/26-eth0-rx-0-21swapper/323:55:073
29715050,0irq/26-eth0-rx-0-21swapper/323:50:093
29715050,0irq/26-eth0-rx-0-21swapper/323:41:153
29715050,0irq/26-eth0-rx-0-21swapper/322:35:073
29715050,0irq/26-eth0-rx-0-21swapper/322:27:303
29715050,0irq/26-eth0-rx-0-21swapper/321:50:323
29715050,0irq/26-eth0-rx-0-21swapper/321:45:033
29715050,0irq/26-eth0-rx-0-21swapper/321:20:563
29715050,0irq/26-eth0-rx-0-21swapper/321:16:293
29715050,0irq/26-eth0-rx-0-21swapper/320:05:353
29715050,0irq/26-eth0-rx-0-21swapper/319:24:513
29715050,0irq/26-eth0-rx-0-21swapper/319:15:133
29715050,0irq/26-eth0-rx-0-21swapper/300:17:373
29715050,0irq/26-eth0-rx-0-21swapper/300:00:073
293099955,0cyclictest0-21swapper/223:00:542
293099955,0cyclictest0-21swapper/221:04:432
293099950,0cyclictest0-21swapper/222:39:562
293089955,0cyclictest0-21swapper/121:41:421
293089950,0cyclictest0-21swapper/123:05:251
293079955,0cyclictest0-21swapper/022:44:000
293079955,0cyclictest0-21swapper/021:48:210
293079955,0cyclictest0-21swapper/021:00:260
293079954,0cyclictest0-21swapper/019:19:510
293079950,0cyclictest0-21swapper/021:07:410
259950,0migration/26786-21aten_r1power_po21:05:032
259950,0migration/24936-21aten_r1power_vo19:25:032
259950,0migration/230508-21munin-node23:40:022
259950,0migration/229999-21runrttasks20:58:362
259950,0migration/229066-21sh22:36:262
259950,0migration/228604-21munin-node21:15:112
259950,0migration/217003-21kthreadcore00:10:072
259950,0migration/214906-21munin-node21:10:012
249950,0watchdog/222399-21diskmemload00:01:332
249950,0watchdog/213348-21if_eth019:55:062
249950,0watchdog/213007-21awk22:26:212
149950,0migration/08181-21kthreadcore23:45:080
149950,0migration/0714-21kthreadcore19:20:060
149950,0migration/07047-21kthreadcore00:05:070
149950,0migration/06303-21kthreadcore19:25:080
149950,0migration/06108-21latency_hist22:44:510
149950,0migration/05250-21kthreadcore20:40:070
149950,0migration/04312-21sh23:42:190
149950,0migration/03844-21missed_timers22:20:090
149950,0migration/032527-21kthreadcore23:20:050
149950,0migration/028625-21sh23:37:210
149950,0migration/028044-21taskset00:16:540
149950,0migration/027097-21kthreadcore23:15:070
149950,0migration/025346-21kthreadcore21:35:070
149950,0migration/024852-21aten_r1power_fr19:40:020
149950,0migration/024517-21kthreadcore21:55:070
149950,0migration/023411-21kthreadcore20:25:080
149950,0migration/021990-21taskset21:12:510
149950,0migration/021439-21idleruntime-cro23:34:500
149950,0migration/018734-21kthreadcore21:30:100
149950,0migration/018346-21kthreadcore22:10:070
149950,0migration/016092-21kthreadcore19:55:080
149950,0migration/014879-21sh22:30:020
149950,0migration/013211-21sh23:08:240
149950,0migration/012831-21sh22:25:510
149950,0migration/012657-21unixbench-2d20:15:130
149950,0migration/012599-21kthreadcore19:05:080
149950,0migration/010276-21kthreadcore22:05:080
339940,0migration/32499-21latency_hist20:39:513
339940,0migration/322243-21taskset20:53:173
339940,0migration/313659-21cpuspeed_turbos20:20:013
29725040,0irq/27-eth0-tx-0-21swapper/223:34:592
29725040,0irq/27-eth0-tx-0-21swapper/222:50:172
29725040,0irq/27-eth0-tx-0-21swapper/222:50:172
29725040,0irq/27-eth0-tx-0-21swapper/222:09:582
29725040,0irq/27-eth0-tx-0-21swapper/221:55:032
29725040,0irq/27-eth0-tx-0-21swapper/200:21:262
29725040,0irq/27-eth0-tx-0-21swapper/200:15:072
29725040,0irq/27-eth0-tx-0-21swapper/020:54:520
29725040,0irq/27-eth0-tx-0-21swapper/019:00:020
29715040,0irq/26-eth0-rx-24768-21munin-node19:40:043
29715040,0irq/26-eth0-rx-0-21swapper/323:06:093
29715040,0irq/26-eth0-rx-0-21swapper/323:00:363
29715040,0irq/26-eth0-rx-0-21swapper/322:40:093
29715040,0irq/26-eth0-rx-0-21swapper/322:20:073
29715040,0irq/26-eth0-rx-0-21swapper/300:05:173
293099944,0cyclictest0-21swapper/223:50:342
293099944,0cyclictest0-21swapper/221:29:562
293099943,0cyclictest6859-21awk23:05:052
293099943,0cyclictest3514-21snmpd21:20:022
293099943,0cyclictest3514-21snmpd20:21:472
293099943,0cyclictest3514-21snmpd19:19:472
293099943,0cyclictest3241-21irqbalance23:30:492
293099943,0cyclictest31422-21awk19:20:042
293099943,0cyclictest25529-21sadc20:04:512
293099943,0cyclictest19095-21awk19:35:062
293099943,0cyclictest12331-21awk19:30:052
293099940,0cyclictest0-21swapper/220:42:032
293089944,0cyclictest0-21swapper/100:02:581
293089943,0cyclictest5755-21awk19:25:071
293089943,0cyclictest3556-21lldpd22:58:051
293089943,0cyclictest3556-21lldpd22:15:041
293089943,0cyclictest3556-21lldpd19:02:341
293089943,0cyclictest3556-21lldpd00:12:341
293089943,0cyclictest3514-21snmpd22:40:181
293089943,0cyclictest3514-21snmpd21:50:021
293089943,0cyclictest3514-21snmpd21:36:021
293089943,0cyclictest3514-21snmpd20:55:471
293089943,0cyclictest3514-21snmpd20:41:171
293089943,0cyclictest3514-21snmpd20:31:171
293089943,0cyclictest3514-21snmpd19:46:321
293089943,0cyclictest3241-21irqbalance22:45:091
293089943,0cyclictest3241-21irqbalance21:09:591
293089943,0cyclictest3241-21irqbalance19:41:401
293089943,0cyclictest3241-21irqbalance19:30:101
293089943,0cyclictest3241-21irqbalance19:10:201
293089943,0cyclictest3241-21irqbalance18:55:001
293089943,0cyclictest30741-21awk23:40:041
293089943,0cyclictest30299-21awk00:20:041
293089943,0cyclictest24682-21awk19:15:051
293089943,0cyclictest21442-21dump-pmu-power23:34:501
293089943,0cyclictest20042-21awk20:00:061
293089943,0cyclictest19023-21awk19:35:051
293089943,0cyclictest16877-21awk20:50:061
293089943,0cyclictest13342-21awk19:55:051
293089943,0cyclictest10148-21awk20:45:061
293079944,0cyclictest0-21swapper/022:02:230
293079944,0cyclictest0-21swapper/000:02:410
293079943,0cyclictest5772-21munin-node19:50:050
293079943,0cyclictest3241-21irqbalance23:50:490
293079943,0cyclictest3241-21irqbalance23:25:090
293079943,0cyclictest3241-21irqbalance23:01:290
293079943,0cyclictest3241-21irqbalance22:55:390
293079943,0cyclictest28384-21kworker/0:222:54:160
293079943,0cyclictest28384-21kworker/0:222:54:160
293079943,0cyclictest23502-21awk22:15:050
259940,0migration/29960-21kthreadcore22:45:072
259940,0migration/23246-21ksmtuned18:56:202
259940,0migration/221021-21runrttasks21:50:462
259940,0migration/21362-21kthreadcore19:45:082
249940,0watchdog/25396-21memory20:10:092
249940,0watchdog/20-21swapper/219:50:102
19140,0ktimersoftd/10-21swapper/122:09:401
179940,0migration/19518-21kthreadcore21:05:081
179940,0migration/17995-21kthreadcore00:05:071
179940,0migration/17798-21kthreadcore23:45:081
179940,0migration/16549-21taskset23:25:041
179940,0migration/15743-21latency_hist19:49:511
179940,0migration/126251-21kthreadcore22:35:071
179940,0migration/125860-21kthreadcore23:15:071
179940,0migration/122941-21cpuspeed_turbos21:55:021
179940,0migration/121432-21kthreadcore20:25:071
179940,0migration/114467-21runrttasks22:29:551
179940,0migration/110509-21aten_r1power_cu19:05:031
169940,0watchdog/132477-21grep23:20:051
169940,0watchdog/132184-21tr19:20:061
169940,0watchdog/131228-21cut21:20:011
169940,0watchdog/130865-21munin-node22:00:091
169940,0watchdog/125457-21cut23:35:091
169940,0watchdog/115300-21kthreadcore20:20:051
169940,0watchdog/114359-21ssh23:50:051
169940,0watchdog/112507-21switchtime21:45:131
169940,0watchdog/10-21swapper/121:28:571
149940,0migration/032235-21hwlatdetect19:45:050
149940,0migration/024475-21kthreadcore21:15:080
35130,0ktimersoftd/30-21swapper/320:12:353
29725030,0irq/27-eth0-tx-0-21swapper/018:55:080
29715030,0irq/26-eth0-rx-0-21swapper/320:45:543
29715030,0irq/26-eth0-rx-0-21swapper/319:22:093
29715030,0irq/26-eth0-rx-0-21swapper/318:59:573
29715030,0irq/26-eth0-rx-0-21swapper/318:55:023
293099933,0cyclictest3514-21snmpd20:15:472
293099933,0cyclictest0-21swapper/219:11:212
293099932,0cyclictest3514-21snmpd20:35:472
293099932,0cyclictest3514-21snmpd19:09:472
293099932,0cyclictest27647-21awk20:30:052
293099932,0cyclictest25621-21awk19:40:052
293099932,0cyclictest16950-21awk20:50:062
293099932,0cyclictest10187-21awk20:45:072
293089933,0cyclictest3556-21lldpd22:10:041
293089933,0cyclictest3556-21lldpd20:13:341
293089933,0cyclictest0-21swapper/122:24:541
293089932,0cyclictest7764-21awk20:15:061
293089932,0cyclictest26646-21awk20:05:041
293089932,0cyclictest23940-21awk21:15:061
293089932,0cyclictest1856-21awk20:35:061
293089932,0cyclictest15882-21awk21:30:071
293089930,0cyclictest30632-21df23:00:021
293079933,0cyclictest28384-21kworker/0:223:55:020
293079933,0cyclictest28384-21kworker/0:221:50:000
293079933,0cyclictest28384-21kworker/0:221:25:020
293079933,0cyclictest23213-21tr22:35:030
293079933,0cyclictest14266-21kworker/0:120:45:080
293079933,0cyclictest14266-21kworker/0:120:10:030
293079933,0cyclictest14266-21kworker/0:120:00:030
293079932,0cyclictest27608-21awk20:30:050
293079932,0cyclictest1988-21tr00:20:070
293079932,0cyclictest16791-21kthreadcore20:20:060
293079930,0cyclictest16365-21diskstats20:50:030
249930,0watchdog/226158-21munin-node20:05:012
169930,0watchdog/122399-21diskmemload23:10:011
169930,0watchdog/115079-21if_err_eth022:50:051
169930,0watchdog/115079-21if_err_eth022:50:051
169930,0watchdog/10-21swapper/122:20:051
169930,0watchdog/10-21swapper/120:59:571
169930,0watchdog/10-21swapper/100:14:561
339920,0migration/36808-21iostat_ios19:50:083
329920,0watchdog/30-21swapper/320:00:143
29715020,0irq/26-eth0-rx-0-21swapper/320:35:043
29715020,0irq/26-eth0-rx-0-21swapper/319:05:033
293099921,0cyclictest0-21swapper/220:24:512
249920,0watchdog/20-21swapper/219:00:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional