You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 06:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri May 22, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450720,0irq/25-eth10-21swapper/318:51:043
1726624544,0sleep00-21swapper/018:50:170
1726823525,0sleep20-21swapper/218:50:192
2274123324,0sleep10-21swapper/118:53:541
229049988,0cyclictest19873-21awk19:20:111
339970,0migration/312942-21sh23:50:183
339960,0migration/36468-21sh23:07:313
339960,0migration/36039-21sh23:27:173
339960,0migration/329558-21munin-node22:40:173
339960,0migration/324484-21sh21:39:593
339960,0migration/323988-21sh21:39:113
339960,0migration/322697-21sh21:55:563
229039960,0cyclictest0-21swapper/000:13:280
149960,0migration/030227-21sh23:01:080
149960,0migration/023476-21sh21:17:090
149960,0migration/017533-21irqcore21:35:130
149960,0migration/015387-21sh22:13:310
149960,0migration/014354-21timerandwakeup20:25:190
339950,0migration/39794-21cpuspeed_turbos20:50:093
339950,0migration/39608-21irqrtprio21:10:123
339950,0migration/39276-21kthreadcore19:10:143
339950,0migration/38308-21taskset21:50:033
339950,0migration/3576-21fschecks_count21:05:093
339950,0migration/35228-21munin-node19:30:153
339950,0migration/332535-21diskmemload21:45:033
339950,0migration/331802-21kthreadcore19:50:133
339950,0migration/329951-21kthreadcore20:40:123
339950,0migration/324935-21fschecks_count21:00:113
339950,0migration/324243-21aten_r1power_po22:20:083
339950,0migration/323004-21kthreadcore19:45:133
339950,0migration/322236-21munin-node20:35:093
339950,0migration/321934-21munin-node20:10:133
339950,0migration/321675-21kthreadcore19:20:133
339950,0migration/320095-21kthreadcore23:15:143
339950,0migration/316509-21fschecks_time22:35:093
339950,0migration/315198-21munin-node20:05:113
339950,0migration/313472-21latency22:10:153
29725050,0irq/27-eth0-tx-23609-21munin-node23:20:132
29725050,0irq/27-eth0-tx-16352-21munin-node21:55:182
29725050,0irq/27-eth0-tx-14884-21ssh23:54:562
29725050,0irq/27-eth0-tx-0-21swapper/223:40:382
29725050,0irq/27-eth0-tx-0-21swapper/222:49:022
29725050,0irq/27-eth0-tx-0-21swapper/222:44:262
29725050,0irq/27-eth0-tx-0-21swapper/221:46:442
29725050,0irq/27-eth0-tx-0-21swapper/221:41:072
29725050,0irq/27-eth0-tx-0-21swapper/221:30:112
29725050,0irq/27-eth0-tx-0-21swapper/020:46:180
29725050,0irq/27-eth0-tx-0-21swapper/020:33:170
29715050,0irq/26-eth0-rx-7939-21df_abs23:10:093
29715050,0irq/26-eth0-rx-7277-21munin-node23:30:123
29715050,0irq/26-eth0-rx-25169-21kthreadcore23:20:123
29715050,0irq/26-eth0-rx-25115-21kthreadcore23:40:133
29715050,0irq/26-eth0-rx-23807-21sshd21:18:033
29715050,0irq/26-eth0-rx-21293-21vmstat23:55:203
29715050,0irq/26-eth0-rx-0-21swapper/323:46:063
29715050,0irq/26-eth0-rx-0-21swapper/323:00:293
29715050,0irq/26-eth0-rx-0-21swapper/322:45:133
29715050,0irq/26-eth0-rx-0-21swapper/322:25:153
29715050,0irq/26-eth0-rx-0-21swapper/322:05:133
29715050,0irq/26-eth0-rx-0-21swapper/321:33:333
29715050,0irq/26-eth0-rx-0-21swapper/320:20:183
29715050,0irq/26-eth0-rx-0-21swapper/300:15:223
29715050,0irq/26-eth0-rx-0-21swapper/300:04:593
259950,0migration/2820-21munin-node19:30:122
259950,0migration/25324-21munin-node23:05:182
259950,0migration/24908-21switchtime23:45:202
259950,0migration/24192-21kthreadcore21:25:132
259950,0migration/229773-21sh00:22:172
259950,0migration/22707-21missed_timers19:00:152
259950,0migration/226145-21kthreadcore21:00:162
259950,0migration/223565-21latency_hist22:59:562
259950,0migration/220082-21munin-node19:45:082
259950,0migration/215173-21sh22:12:562
259950,0migration/214930-21sh21:51:422
259950,0migration/210636-21kthreadcore19:35:122
229059950,0cyclictest15959-21diskmemload22:36:092
229049955,0cyclictest0-21swapper/122:48:171
229049955,0cyclictest0-21swapper/100:15:561
229049955,0cyclictest0-21swapper/100:11:501
229049950,0cyclictest0-21swapper/123:55:451
229049950,0cyclictest0-21swapper/122:41:021
229049950,0cyclictest0-21swapper/122:37:241
229049950,0cyclictest0-21swapper/121:25:491
229049950,0cyclictest0-21swapper/119:35:091
229039955,0cyclictest0-21swapper/021:45:540
229039954,0cyclictest30987-21ps19:00:130
229039950,0cyclictest0-21swapper/022:31:190
179950,0migration/19301-21gltestperf20:25:091
179950,0migration/132470-21irqcore23:45:131
179950,0migration/124700-21irqcore23:20:121
179950,0migration/113506-21latency22:10:151
149950,0migration/08530-21kthreadcore23:50:120
149950,0migration/08074-21kthreadcore19:35:110
149950,0migration/06461-21unixbench_multi21:25:190
149950,0migration/05629-21missed_timers21:05:150
149950,0migration/04984-21taskset19:05:350
149950,0migration/03393-21kthreadcore00:05:140
149950,0migration/032336-21latency_hist22:04:570
149950,0migration/032289-21munin-node23:45:120
149950,0migration/032150-21taskset21:44:360
149950,0migration/031726-21latency_hist23:04:560
149950,0migration/030962-21kthreadcore20:40:130
149950,0migration/028912-21kthreadcore22:40:150
149950,0migration/028332-21kthreadcore18:55:150
149950,0migration/026687-21kthreadcore21:20:140
149950,0migration/025521-21kthreadcore00:00:120
149950,0migration/025339-21taskset19:24:440
149950,0migration/024032-21irqcore00:20:120
149950,0migration/023568-21kthreadcore20:35:120
149950,0migration/022287-21sh23:16:560
149950,0migration/017437-21kthreadcore22:35:120
149950,0migration/017001-21idleruntime22:15:110
149950,0migration/013863-21missed_timers21:30:140
149950,0migration/012708-21fschecks_count19:15:100
149950,0migration/011444-21kthreadcore20:00:140
339940,0migration/32762-21latency_hist20:44:573
339940,0migration/322373-21taskset22:15:383
339940,0migration/320288-21kthreadcore23:35:143
339940,0migration/31766-21kthreadcore19:00:143
339940,0migration/315275-21sh20:30:073
339940,0migration/310613-21kthreadcore19:35:123
29725040,0irq/27-eth0-tx-3514-21snmpd20:49:580
29725040,0irq/27-eth0-tx-31684-21rsyslogd20:14:040
29725040,0irq/27-eth0-tx-24140-21munin-node22:20:102
29725040,0irq/27-eth0-tx-23802-21munin-node23:00:152
29725040,0irq/27-eth0-tx-16937-21kthreadcore23:15:132
29725040,0irq/27-eth0-tx-15172-21munin-node23:55:072
29725040,0irq/27-eth0-tx-14947-21munin-node00:15:182
29725040,0irq/27-eth0-tx-13678-21ssh22:30:182
29725040,0irq/27-eth0-tx-0-21swapper/223:36:522
29725040,0irq/27-eth0-tx-0-21swapper/223:25:172
29725040,0irq/27-eth0-tx-0-21swapper/223:10:062
29725040,0irq/27-eth0-tx-0-21swapper/222:04:582
29725040,0irq/27-eth0-tx-0-21swapper/221:35:012
29725040,0irq/27-eth0-tx-0-21swapper/221:20:312
29725040,0irq/27-eth0-tx-0-21swapper/200:06:142
29725040,0irq/27-eth0-tx-0-21swapper/020:20:170
29725040,0irq/27-eth0-tx-0-21swapper/019:48:170
29725040,0irq/27-eth0-tx-0-21swapper/019:26:470
29715040,0irq/26-eth0-rx-23979-21if_err_eth1.218:55:123
29715040,0irq/26-eth0-rx-16994-21cut22:55:113
29715040,0irq/26-eth0-rx-0-21swapper/322:50:273
29715040,0irq/26-eth0-rx-0-21swapper/322:30:003
29715040,0irq/26-eth0-rx-0-21swapper/322:00:293
29715040,0irq/26-eth0-rx-0-21swapper/321:26:253
29715040,0irq/26-eth0-rx-0-21swapper/321:20:133
29715040,0irq/26-eth0-rx-0-21swapper/320:57:503
29715040,0irq/26-eth0-rx-0-21swapper/319:15:083
29715040,0irq/26-eth0-rx-0-21swapper/300:20:123
29715040,0irq/26-eth0-rx-0-21swapper/300:10:263
29715040,0irq/26-eth0-rx-0-21swapper/300:01:113
259940,0migration/232434-21latency_hist21:04:572
259940,0migration/228554-21kthreadcore19:25:122
259940,0migration/227873-21taskset20:35:192
259940,0migration/227662-21aten_r1power_po20:15:072
259940,0migration/22636-21kthreadcore19:55:122
259940,0migration/225238-21kthreadcore18:55:142
259940,0migration/218903-21munin-node19:40:192
259940,0migration/217901-21kthreadcore20:30:122
259940,0migration/217471-21munin-node22:15:132
259940,0migration/216698-21aten_r1power_en21:15:072
259940,0migration/214371-21aten_r1power_vo20:05:072
249940,0watchdog/28501-21ssh22:50:112
249940,0watchdog/232225-21cut22:25:072
249940,0watchdog/211389-21munin-node19:10:192
229059944,0cyclictest0-21swapper/200:00:112
229059943,0cyclictest9346-21awk21:10:112
229059943,0cyclictest8570-21awk20:00:122
229059943,0cyclictest7828-21awk00:10:112
229059943,0cyclictest3556-21lldpd20:21:182
229059943,0cyclictest3556-21lldpd19:16:472
229059943,0cyclictest3514-21snmpd23:30:272
229059943,0cyclictest25255-21awk22:00:112
229059943,0cyclictest17113-21awk20:55:122
229049944,0cyclictest0-21swapper/122:20:411
229049944,0cyclictest0-21swapper/122:15:391
229049943,0cyclictest8595-21awk20:00:121
229049943,0cyclictest822-21awk19:30:121
229049943,0cyclictest531-21awk22:25:101
229049943,0cyclictest3742-21awk20:45:121
229049943,0cyclictest3556-21lldpd23:30:501
229049943,0cyclictest3556-21lldpd19:10:171
229049943,0cyclictest3556-21lldpd00:09:201
229049943,0cyclictest3514-21snmpd21:20:111
229049943,0cyclictest3514-21snmpd20:18:251
229049943,0cyclictest30795-21awk19:00:121
229049943,0cyclictest27473-21awk19:50:101
229049943,0cyclictest26440-21awk19:25:101
229049943,0cyclictest25384-21awk22:00:121
229049943,0cyclictest23928-21awk00:20:111
229049943,0cyclictest17280-21awk21:55:111
229049943,0cyclictest16266-21awk23:35:121
229049943,0cyclictest15323-21sadc23:14:561
229039944,0cyclictest0-21swapper/022:56:230
229039944,0cyclictest0-21swapper/020:58:180
229039943,0cyclictest8198-21awk23:30:110
229039943,0cyclictest6470-21awk19:10:120
229039943,0cyclictest636-21awk22:25:110
229039943,0cyclictest32478-21awk23:25:110
229039943,0cyclictest25312-21awk21:00:130
229039943,0cyclictest25311-21awk22:00:110
229039943,0cyclictest24482-21awk23:20:100
229039943,0cyclictest21565-21kworker/0:200:15:030
229039943,0cyclictest20923-21kworker/0:119:55:080
229039943,0cyclictest16101-21awk23:55:110
229039941,0cyclictest0-21swapper/020:15:070
179940,0migration/132572-21aten_r1power_cu21:45:061
179940,0migration/131771-21latency_hist23:04:561
179940,0migration/130827-21runrttasks23:02:511
179940,0migration/129908-21taskset23:41:301
179940,0migration/129409-21kthreadcore21:00:171
179940,0migration/125469-21kthreadcore20:10:141
179940,0migration/125468-21kthreadcore18:55:141
179940,0migration/12532-21gltestperf20:20:091
179940,0migration/123764-21kthreadcore19:45:131
179940,0migration/122832-21taskset20:56:361
179940,0migration/122132-21cpuspeed_turbos20:35:081
179940,0migration/121530-21taskset20:34:061
179940,0migration/119438-21kthreadcore21:35:141
179940,0migration/117627-21kthreadcore19:40:151
179940,0migration/117228-21kthreadcore23:15:131
179940,0migration/115728-21kthreadcore20:05:121
179940,0migration/112133-21kthreadcore22:50:141
179940,0migration/11190-21irqcore21:05:121
169940,0watchdog/18543-21aten_r1power_fr21:30:071
169940,0watchdog/16961-21ssh23:29:501
169940,0watchdog/16369-21ssh22:05:431
169940,0watchdog/129447-21if_eth120:40:111
169940,0watchdog/126313-21tr21:40:151
169940,0watchdog/114431-21taskset21:50:191
169940,0watchdog/10-21swapper/121:16:231
169940,0watchdog/10-21swapper/120:50:111
169940,0watchdog/10-21swapper/119:55:151
169940,0watchdog/10-21swapper/100:00:101
149940,0migration/031656-21kthreadcore19:50:130
149940,0migration/026697-21kthreadcore22:20:140
149940,0migration/020853-21kthreadcore21:55:140
149940,0migration/012121-21kthreadcore23:10:150
149940,0migration/011745-21kthreadcore21:10:130
29725030,0irq/27-eth0-tx-0-21swapper/020:05:060
29715030,0irq/26-eth0-rx-0-21swapper/320:27:323
29715030,0irq/26-eth0-rx-0-21swapper/320:15:083
29715030,0irq/26-eth0-rx-0-21swapper/320:00:163
29715030,0irq/26-eth0-rx-0-21swapper/319:55:073
29715030,0irq/26-eth0-rx-0-21swapper/319:05:413
259930,0migration/29109-21runrttasks20:47:252
259930,0migration/224020-21kthreadcore20:10:142
259930,0migration/213186-21kthreadcore20:50:142
249930,0watchdog/20-21swapper/219:20:112
229059933,0cyclictest0-21swapper/219:08:222
229059932,0cyclictest9384-21awk20:25:102
229059932,0cyclictest3556-21lldpd19:50:472
229059932,0cyclictest31255-21grep20:40:132
229049933,0cyclictest3556-21lldpd19:05:471
229049932,0cyclictest8017-21awk23:50:101
229049932,0cyclictest3556-21lldpd22:30:211
229039933,0cyclictest3605-21kworker/0:022:45:040
229039933,0cyclictest20923-21kworker/0:119:30:060
229039932,0cyclictest9314-21awk21:50:110
229039932,0cyclictest8343-21fschecks_time22:50:100
229039932,0cyclictest3514-21snmpd23:39:570
229039932,0cyclictest14251-21awk19:40:130
229039930,0cyclictest15686-21diskstats23:35:090
169930,0watchdog/10-21swapper/121:11:231
169930,0watchdog/10-21swapper/119:16:071
35120,0ktimersoftd/329915-21kthreadcore19:25:133
339920,0migration/314297-21iostat_ios19:40:133
169920,0watchdog/10-21swapper/122:54:581
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional