You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-30 - 07:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Mon Mar 30, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
227924738,0sleep10-21swapper/118:59:361
304450410,0irq/25-eth10-21swapper/318:55:483
229423726,0sleep00-21swapper/018:59:470
210723329,0sleep20-21swapper/218:57:372
4160,0ktimersoftd/00-21swapper/023:09:590
35160,0ktimersoftd/30-21swapper/321:31:153
339960,0migration/37719-21sh22:03:263
339960,0migration/37645-21kthreadcore00:25:133
339960,0migration/36917-21sh22:43:073
339960,0migration/331075-21sh22:58:363
339960,0migration/324879-21sh21:14:523
339960,0migration/323258-21sh22:12:083
339960,0migration/313771-21sh00:09:093
29715060,0irq/26-eth0-rx-17190-21entropy21:50:093
29715060,0irq/26-eth0-rx-15202-21seq21:46:083
27160,0ktimersoftd/20-21swapper/200:08:012
23859960,0cyclictest0-21swapper/222:36:162
23839965,0cyclictest30046-21crond00:19:560
23839965,0cyclictest13171-21ssh23:46:050
23839960,0cyclictest0-21swapper/022:10:420
149960,0migration/08481-21sh21:23:500
149960,0migration/05532-21sh23:21:060
149960,0migration/026636-21kthreadcore22:35:150
149960,0migration/022142-21kthreadcore20:00:160
149960,0migration/022071-21timerandwakeup22:30:200
149960,0migration/015376-21sh21:46:390
149960,0migration/011468-21kthreadcore22:05:150
339950,0migration/39320-21latency_hist19:04:583
339950,0migration/38727-21kthreadcore20:40:133
339950,0migration/38403-21taskset21:04:073
339950,0migration/35290-21fschecks_time19:50:113
339950,0migration/3423-21diskmemload21:19:323
339950,0migration/34192-21runrttasks00:20:343
339950,0migration/330598-21munin-node19:20:113
339950,0migration/329655-21sh23:15:283
339950,0migration/326383-21kthreadcore19:40:133
339950,0migration/326014-21kthreadcore20:05:143
339950,0migration/325997-21munin-node21:55:123
339950,0migration/325321-21munin-node22:35:143
339950,0migration/325209-21aten_r1power_fr21:35:083
339950,0migration/325129-21kthreadcore19:15:153
339950,0migration/325076-21kthreadcore23:35:143
339950,0migration/323343-21taskset19:38:583
339950,0migration/322160-21kthreadcore20:25:143
339950,0migration/321476-21runrttasks23:31:003
339950,0migration/318545-21missed_timers20:45:143
339950,0migration/317970-21kthreadcore22:30:143
339950,0migration/317930-21latency_hist19:59:583
339950,0migration/316568-21taskset19:32:503
339950,0migration/315183-21munin-node23:50:083
339950,0migration/314010-21switchtime22:25:193
339950,0migration/313012-21gltestperf20:20:123
29725050,0irq/27-eth0-tx-6008-21kthreadcore21:20:152
29725050,0irq/27-eth0-tx-30630-21nfs_client21:35:162
29725050,0irq/27-eth0-tx-22360-21ntpq21:50:162
29725050,0irq/27-eth0-tx-19864-21munin-node20:50:210
29725050,0irq/27-eth0-tx-16508-21kthreadcore23:50:122
29725050,0irq/27-eth0-tx-16023-21munin-node19:10:160
29725050,0irq/27-eth0-tx-13652-21ssh23:47:302
29725050,0irq/27-eth0-tx-13234-21munin-node20:45:130
29725050,0irq/27-eth0-tx-0-21swapper/223:44:032
29725050,0irq/27-eth0-tx-0-21swapper/223:24:472
29725050,0irq/27-eth0-tx-0-21swapper/222:46:312
29725050,0irq/27-eth0-tx-0-21swapper/222:15:082
29725050,0irq/27-eth0-tx-0-21swapper/221:40:502
29725050,0irq/27-eth0-tx-0-21swapper/221:32:332
29725050,0irq/27-eth0-tx-0-21swapper/221:11:592
29725050,0irq/27-eth0-tx-0-21swapper/021:00:150
29725050,0irq/27-eth0-tx-0-21swapper/019:38:500
29725050,0irq/27-eth0-tx-0-21swapper/019:06:000
29715050,0irq/26-eth0-rx-6570-21seq23:24:073
29715050,0irq/26-eth0-rx-5078-21pmu-power23:00:163
29715050,0irq/26-eth0-rx-3898-21ps21:40:153
29715050,0irq/26-eth0-rx-2408-21kthreadcore22:20:143
29715050,0irq/26-eth0-rx-17076-21irqcore19:10:133
29715050,0irq/26-eth0-rx-11843-21kthreadcore21:25:123
29715050,0irq/26-eth0-rx-10149-21irqcore21:05:153
29715050,0irq/26-eth0-rx-0-21swapper/323:55:083
29715050,0irq/26-eth0-rx-0-21swapper/323:45:073
29715050,0irq/26-eth0-rx-0-21swapper/323:42:383
29715050,0irq/26-eth0-rx-0-21swapper/323:09:573
29715050,0irq/26-eth0-rx-0-21swapper/320:31:533
29715050,0irq/26-eth0-rx-0-21swapper/300:15:143
29715050,0irq/26-eth0-rx-0-21swapper/300:10:123
27150,0ktimersoftd/20-21swapper/221:01:022
259950,0migration/231275-21fschecks_time00:00:102
259950,0migration/230157-21sh22:55:542
259950,0migration/219647-21kthreadcore22:10:142
259950,0migration/215669-21kthreadcore00:10:122
259950,0migration/210189-21kthreadcore21:45:142
249950,0watchdog/230988-21tr19:20:122
249950,0watchdog/211180-21kthreadcore00:25:142
23859955,0cyclictest0-21swapper/223:03:232
23859954,0cyclictest10823-21grep22:25:142
23859950,0cyclictest0-21swapper/223:33:582
23859950,0cyclictest0-21swapper/222:02:502
23859950,0cyclictest0-21swapper/221:18:232
23859950,0cyclictest0-21swapper/220:09:572
23849954,0cyclictest27382-21awk20:55:121
23849950,0cyclictest0-21swapper/121:26:071
23839950,0cyclictest0-21swapper/023:26:040
23839950,0cyclictest0-21swapper/021:44:030
23839950,0cyclictest0-21swapper/000:26:170
179950,0migration/19381-21kthreadcore00:05:141
179950,0migration/17959-21kthreadcore19:50:141
179950,0migration/15126-21taskset00:02:141
179950,0migration/131682-21taskset20:31:561
179950,0migration/124589-21aten_r1power_fr22:15:081
179950,0migration/120498-21fschecks_time20:50:121
179950,0migration/118450-21kthreadcore19:35:141
179950,0migration/112030-21munin-node20:40:161
149950,0migration/06193-21taskset20:39:110
149950,0migration/05873-21taskset00:04:250
149950,0migration/05149-21taskset20:12:570
149950,0migration/04171-21runrttasks19:47:220
149950,0migration/03666-21kthreadcore23:00:130
149950,0migration/032357-21taskset21:18:070
149950,0migration/030653-21taskset20:05:530
149950,0migration/026806-21kthreadcore21:35:140
149950,0migration/025641-21idleruntime21:55:110
149950,0migration/025523-21taskset20:30:030
149950,0migration/024576-21kthreadcore19:15:150
149950,0migration/017975-21munin-node21:10:110
149950,0migration/017176-21kthreadcore22:50:110
149950,0migration/016874-21aten_r1power_fr21:50:080
149950,0migration/016827-21kthreadcore23:30:130
149950,0migration/016431-21taskset19:31:110
149950,0migration/014746-21taskset23:50:000
149950,0migration/013685-21kthreadcore19:55:130
149950,0migration/011555-21kthreadcore20:40:140
8215040,0irq/24-0000:00:0-21swapper/219:30:132
339940,0migration/34204-21taskset19:47:463
339940,0migration/326427-21kthreadcore22:15:133
339940,0migration/324527-21kthreadcore20:50:163
29725040,0irq/27-eth0-tx-494-21diskmemload23:05:242
29725040,0irq/27-eth0-tx-4327-21ssh00:20:582
29725040,0irq/27-eth0-tx-0-21swapper/223:55:092
29725040,0irq/27-eth0-tx-0-21swapper/223:35:212
29725040,0irq/27-eth0-tx-0-21swapper/223:14:592
29725040,0irq/27-eth0-tx-0-21swapper/222:40:122
29725040,0irq/27-eth0-tx-0-21swapper/222:30:022
29725040,0irq/27-eth0-tx-0-21swapper/222:20:052
29725040,0irq/27-eth0-tx-0-21swapper/222:05:102
29725040,0irq/27-eth0-tx-0-21swapper/221:55:042
29725040,0irq/27-eth0-tx-0-21swapper/221:05:262
29725040,0irq/27-eth0-tx-0-21swapper/200:15:132
29715040,0irq/26-eth0-rx-0-21swapper/323:25:083
29715040,0irq/26-eth0-rx-0-21swapper/323:05:163
29715040,0irq/26-eth0-rx-0-21swapper/322:50:403
29715040,0irq/26-eth0-rx-0-21swapper/322:46:373
29715040,0irq/26-eth0-rx-0-21swapper/322:06:373
29715040,0irq/26-eth0-rx-0-21swapper/321:19:573
29715040,0irq/26-eth0-rx-0-21swapper/319:25:083
29715040,0irq/26-eth0-rx-0-21swapper/319:25:083
29715040,0irq/26-eth0-rx-0-21swapper/300:00:013
259940,0migration/29134-21kthreadcore19:50:152
259940,0migration/28278-21kthreadcore19:25:152
259940,0migration/28278-21kthreadcore19:25:152
259940,0migration/28227-21kthreadcore20:40:132
259940,0migration/222091-21kthreadcore20:50:152
259940,0migration/218315-21kthreadcore23:10:142
259940,0migration/213153-21unixbench_multi23:25:202
249940,0watchdog/230397-21kthreadcore20:55:152
249940,0watchdog/20-21swapper/220:20:152
249940,0watchdog/20-21swapper/220:00:162
249940,0watchdog/20-21swapper/219:00:202
23859943,0cyclictest3556-21lldpd22:50:022
23859943,0cyclictest3514-21snmpd20:05:172
23859943,0cyclictest31200-21awk19:45:102
23859943,0cyclictest23574-21awk19:15:132
23859943,0cyclictest10350-21awk19:05:132
23859940,0cyclictest9172-21munin-node21:25:112
23849944,0cyclictest0-21swapper/123:02:271
23849944,0cyclictest0-21swapper/122:31:141
23849943,0cyclictest9978-21awk21:05:141
23849943,0cyclictest8697-21awk22:45:091
23849943,0cyclictest4611-21awk19:25:121
23849943,0cyclictest4611-21awk19:25:121
23849943,0cyclictest3514-21snmpd22:05:021
23849943,0cyclictest3241-21irqbalance23:51:051
23849943,0cyclictest3241-21irqbalance23:40:051
23849943,0cyclictest3241-21irqbalance23:20:251
23849943,0cyclictest3241-21irqbalance23:17:051
23849943,0cyclictest3241-21irqbalance23:10:251
23849943,0cyclictest3241-21irqbalance22:25:151
23849943,0cyclictest3241-21irqbalance22:22:151
23849943,0cyclictest3241-21irqbalance00:28:051
23849943,0cyclictest3241-21irqbalance00:20:051
23849943,0cyclictest32104-21awk20:10:111
23849943,0cyclictest25504-21awk20:05:131
23849943,0cyclictest24601-21awk19:40:111
23849943,0cyclictest23230-21awk00:15:101
23849943,0cyclictest18048-21awk21:10:121
23849943,0cyclictest1741-21awk21:00:121
23849943,0cyclictest15175-21awk00:10:091
23849943,0cyclictest1247-21awk22:00:111
23849943,0cyclictest11200-21awk19:30:131
23849942,0cyclictest19787-21grep20:00:151
23849940,0cyclictest0-21swapper/119:44:591
23839943,0cyclictest934-21awk22:20:120
23839943,0cyclictest3241-21irqbalance23:06:050
23839943,0cyclictest3241-21irqbalance22:55:050
23839943,0cyclictest3241-21irqbalance22:01:550
23839943,0cyclictest24426-21awk23:15:110
23839943,0cyclictest24006-21awk23:35:120
23839940,0cyclictest3647-21munin-node19:25:110
23839940,0cyclictest3647-21munin-node19:25:110
179940,0migration/18877-21kthreadcore20:15:131
179940,0migration/17585-21aten_r1power_po23:05:081
179940,0migration/16980-21runrttasks22:43:191
179940,0migration/15972-21kthreadcore19:00:151
179940,0migration/13313-21kthreadcore21:20:141
179940,0migration/127728-21kthreadcore21:15:181
179940,0migration/125557-21kthreadcore19:15:151
179940,0migration/12495-21kthreadcore21:40:141
179940,0migration/123485-21df_abs23:35:091
179940,0migration/121672-21munin-node19:10:181
179940,0migration/117612-21idleruntime21:50:111
179940,0migration/113980-21sh23:27:251
179940,0migration/111005-21kthreadcore19:05:141
169940,0watchdog/18801-21ssh21:45:071
169940,0watchdog/132737-21ssh21:39:471
169940,0watchdog/125466-21kthreadcore22:35:151
169940,0watchdog/122067-21ssh22:50:351
169940,0watchdog/1176472sleep119:59:001
169940,0watchdog/113248-21ssh23:46:191
169940,0watchdog/113219-40yum-updatesd-he20:44:591
169940,0watchdog/10-21swapper/123:55:111
169940,0watchdog/10-21swapper/122:10:191
169940,0watchdog/10-21swapper/121:59:231
169940,0watchdog/10-21swapper/120:20:191
149940,0migration/07784-21kthreadcore19:00:150
149940,0migration/031076-21runrttasks22:16:410
149940,0migration/023280-21fschecks_count23:55:100
149940,0migration/013837-21kthreadcore20:20:150
339930,0migration/3973-21irqrtprio20:35:123
29715030,0irq/26-eth0-rx-0-21swapper/320:55:093
29715030,0irq/26-eth0-rx-0-21swapper/320:15:083
29715030,0irq/26-eth0-rx-0-21swapper/320:10:193
29715030,0irq/26-eth0-rx-0-21swapper/319:00:093
259930,0migration/220770-21kthreadcore19:35:152
249930,0watchdog/20-21swapper/220:30:072
249930,0watchdog/20-21swapper/220:15:112
249930,0watchdog/20-21swapper/219:40:162
249930,0watchdog/20-21swapper/219:10:082
23859933,0cyclictest3556-21lldpd20:38:012
23859932,0cyclictest12157-21awk19:55:102
23859930,0cyclictest0-21swapper/220:45:002
23849932,0cyclictest30273-21awk19:20:101
23849932,0cyclictest19830-21awk20:25:121
23849932,0cyclictest16004-21awk23:30:111
23839933,0cyclictest3387-21kworker/0:223:41:060
23839933,0cyclictest3387-21kworker/0:222:40:060
23839933,0cyclictest3387-21kworker/0:221:30:080
23839933,0cyclictest3387-21kworker/0:221:25:060
23839933,0cyclictest3387-21kworker/0:221:05:060
23839933,0cyclictest3387-21kworker/0:220:25:110
23839933,0cyclictest3387-21kworker/0:200:20:000
23839933,0cyclictest3387-21kworker/0:200:05:100
23839932,0cyclictest9113-21awk22:25:120
23839932,0cyclictest3514-21snmpd00:10:020
23839932,0cyclictest30201-21awk19:20:100
23839932,0cyclictest26530-21munin-node20:55:120
23839932,0cyclictest23669-21munin-node19:40:090
23839932,0cyclictest13089-21kthreadcore22:45:120
23839930,0cyclictest7943-21cut19:50:140
23839930,0cyclictest0-21swapper/020:15:110
169930,0watchdog/10-21swapper/122:55:311
29715020,0irq/26-eth0-rx-0-21swapper/319:55:003
249920,0watchdog/20-21swapper/220:24:592
23849921,0cyclictest0-21swapper/121:30:011
169920,0watchdog/10-21swapper/120:35:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional