You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-18 - 01:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Wed Mar 18, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1730223625,0sleep00-21swapper/018:58:120
304450350,0irq/25-eth10-21swapper/318:58:373
1751923526,0sleep20-21swapper/219:00:462
1119823127,0sleep10-21swapper/118:56:091
35160,0ktimersoftd/30-21swapper/323:06:253
339960,0migration/38303-21munin-node20:51:263
339960,0migration/35111-21sh23:53:363
339960,0migration/329545-21sh23:28:023
339960,0migration/312977-21sh23:58:073
29715060,0irq/26-eth0-rx-431-21iostat_ios22:31:203
29715060,0irq/26-eth0-rx-23381-21port_sshpriv19:01:243
176209960,0cyclictest15778-21diskmemload22:18:210
149960,0migration/030147-21munin-node21:06:260
149960,0migration/023092-21sh21:04:120
339950,0migration/39885-21munin-node20:56:183
339950,0migration/39778-21kthreadcore21:56:233
339950,0migration/39752-21kthreadcore20:06:223
339950,0migration/3859-21if_err_eth1.221:51:213
339950,0migration/328471-21aten_r1power_po20:46:163
339950,0migration/324762-21munin-node21:06:153
339950,0migration/324179-21kthreadcore19:51:223
339950,0migration/323683-21kthreadcore20:16:233
339950,0migration/322818-21sh21:42:153
339950,0migration/322477-21sh21:02:353
339950,0migration/322374-21runrttasks23:04:223
339950,0migration/322061-21latency_hist00:06:043
339950,0migration/317150-21kthreadcore23:21:213
339950,0migration/316529-21kthreadcore19:21:213
339950,0migration/315749-21munin-node23:41:193
339950,0migration/31496-21kthreadcore22:51:213
29725050,0irq/27-eth0-tx-7851-21kthreadcore00:16:202
29725050,0irq/27-eth0-tx-32257-21munin-node22:51:182
29725050,0irq/27-eth0-tx-21933-21munin-node00:26:162
29725050,0irq/27-eth0-tx-19563-21kthreadcore22:41:192
29725050,0irq/27-eth0-tx-16072-21munin-node22:01:252
29725050,0irq/27-eth0-tx-16072-21munin-node22:01:252
29725050,0irq/27-eth0-tx-0-21swapper/223:02:492
29725050,0irq/27-eth0-tx-0-21swapper/221:57:392
29725050,0irq/27-eth0-tx-0-21swapper/200:00:352
29725050,0irq/27-eth0-tx-0-21swapper/020:46:170
29725050,0irq/27-eth0-tx-0-21swapper/020:13:290
29725050,0irq/27-eth0-tx-0-21swapper/019:01:150
29715050,0irq/26-eth0-rx-5869-21ssh23:34:583
29715050,0irq/26-eth0-rx-448-21kthreadcore00:11:213
29715050,0irq/26-eth0-rx-312162sleep319:11:153
29715050,0irq/26-eth0-rx-29636-21ssh22:06:233
29715050,0irq/26-eth0-rx-28596-21tr22:46:213
29715050,0irq/26-eth0-rx-19770-21smartctl00:21:283
29715050,0irq/26-eth0-rx-15127-21irqcore20:11:193
29715050,0irq/26-eth0-rx-13592-21python22:16:263
29715050,0irq/26-eth0-rx-0-21swapper/323:50:593
29715050,0irq/26-eth0-rx-0-21swapper/323:40:523
29715050,0irq/26-eth0-rx-0-21swapper/323:16:203
29715050,0irq/26-eth0-rx-0-21swapper/322:36:203
29715050,0irq/26-eth0-rx-0-21swapper/322:26:203
29715050,0irq/26-eth0-rx-0-21swapper/322:21:243
29715050,0irq/26-eth0-rx-0-21swapper/321:46:163
29715050,0irq/26-eth0-rx-0-21swapper/321:34:393
29715050,0irq/26-eth0-rx-0-21swapper/321:26:223
29715050,0irq/26-eth0-rx-0-21swapper/321:16:083
29715050,0irq/26-eth0-rx-0-21swapper/321:13:043
29715050,0irq/26-eth0-rx-0-21swapper/300:21:043
29715050,0irq/26-eth0-rx-0-21swapper/300:06:213
259950,0migration/229860-21sh22:47:052
259950,0migration/214530-21kthreadcore20:56:222
249950,0watchdog/227041-21tr21:06:192
19150,0ktimersoftd/115778-21diskmemload21:03:151
19150,0ktimersoftd/10-21swapper/123:28:561
176229955,0cyclictest0-21swapper/221:30:392
176229955,0cyclictest0-21swapper/221:17:152
176229950,0cyclictest0-21swapper/223:30:002
176229950,0cyclictest0-21swapper/223:09:402
176229950,0cyclictest0-21swapper/222:40:582
176229950,0cyclictest0-21swapper/222:11:392
176229950,0cyclictest0-21swapper/221:46:442
176229950,0cyclictest0-21swapper/200:01:232
176219950,0cyclictest0-21swapper/121:32:161
176219950,0cyclictest0-21swapper/100:06:361
176209955,0cyclictest0-21swapper/023:28:020
176209955,0cyclictest0-21swapper/022:32:420
176209955,0cyclictest0-21swapper/021:11:390
176209950,0cyclictest0-21swapper/022:06:220
179950,0migration/132358-21cpuspeed_turbos22:11:141
179950,0migration/131755-21sh21:29:021
179950,0migration/125426-21switchtime19:51:271
179950,0migration/123377-21aten_r1power_vo23:06:161
179950,0migration/11975-21kthreadcore20:01:211
179950,0migration/116907-21idleruntime22:01:201
179950,0migration/116907-21idleruntime22:01:201
179950,0migration/113997-21sh00:21:171
179950,0migration/110107-21kthreadcore23:56:211
169950,0watchdog/10-21swapper/120:41:231
149950,0migration/08756-21kthreadcore23:36:210
149950,0migration/08184-21kthreadcore19:41:210
149950,0migration/04010-21kthreadcore21:31:210
149950,0migration/032190-21sh21:30:170
149950,0migration/030293-21sh00:11:150
149950,0migration/028619-21kthreadcore22:46:210
149950,0migration/02844-21kthreadcore21:51:230
149950,0migration/025917-21kthreadcore20:16:230
149950,0migration/021070-21kthreadcore19:26:210
149950,0migration/020396-21kthreadcore21:41:210
149950,0migration/019920-21kthreadcore22:01:230
149950,0migration/019920-21kthreadcore22:01:230
149950,0migration/017957-21kthreadcore22:41:190
149950,0migration/01639-21kthreadcore23:11:220
149950,0migration/016045-21kthreadcore00:01:190
149950,0migration/013280-21kthreadcore21:56:240
149950,0migration/012498-21munin-node19:21:150
149950,0migration/010913-21kthreadcore23:16:200
8215040,0irq/24-0000:00:16146-21latency_hist21:41:052
339940,0migration/38928-21fschecks_count21:36:173
339940,0migration/38599-21cpuspeed_turbos20:31:143
339940,0migration/35612-21kthreadcore20:26:233
339940,0migration/326829-21fschecks_count19:56:183
339940,0migration/320682-21kthreadcore22:01:233
339940,0migration/320682-21kthreadcore22:01:233
339940,0migration/319243-21kthreadcore21:21:223
339940,0migration/316026-21if_eth1.320:36:193
339940,0migration/31313-21taskset20:25:023
29725040,0irq/27-eth0-tx-7082-21ssh23:16:152
29725040,0irq/27-eth0-tx-6574-21aten2.4-expect23:36:152
29725040,0irq/27-eth0-tx-27940-21ssh23:46:212
29725040,0irq/27-eth0-tx-26489-21telnet19:56:160
29725040,0irq/27-eth0-tx-24746-21ssh22:06:162
29725040,0irq/27-eth0-tx-16024-21munin-node21:01:262
29725040,0irq/27-eth0-tx-0-21swapper/223:51:422
29725040,0irq/27-eth0-tx-0-21swapper/223:21:162
29725040,0irq/27-eth0-tx-0-21swapper/222:57:032
29725040,0irq/27-eth0-tx-0-21swapper/222:16:552
29725040,0irq/27-eth0-tx-0-21swapper/221:21:102
29725040,0irq/27-eth0-tx-0-21swapper/221:12:262
29725040,0irq/27-eth0-tx-0-21swapper/200:21:182
29725040,0irq/27-eth0-tx-0-21swapper/200:06:282
29725040,0irq/27-eth0-tx-0-21swapper/020:51:210
29725040,0irq/27-eth0-tx-0-21swapper/020:06:190
29725040,0irq/27-eth0-tx-0-21swapper/019:31:160
29725040,0irq/27-eth0-tx-0-21swapper/019:16:210
29725040,0irq/27-eth0-tx-0-21swapper/019:11:230
29715040,0irq/26-eth0-rx-0-21swapper/323:13:093
29715040,0irq/26-eth0-rx-0-21swapper/322:56:203
29715040,0irq/26-eth0-rx-0-21swapper/322:42:083
29715040,0irq/26-eth0-rx-0-21swapper/322:11:243
29715040,0irq/26-eth0-rx-0-21swapper/320:01:173
29715040,0irq/26-eth0-rx-0-21swapper/300:26:323
259940,0migration/231770-21latency_hist22:31:052
259940,0migration/231083-21kthreadcore20:21:222
259940,0migration/229274-21kthreadcore19:56:212
259940,0migration/224792-21cpuspeed_turbos19:06:172
259940,0migration/219527-21fschecks_time19:26:182
259940,0migration/216036-21kthreadcore23:41:192
259940,0migration/214067-21proc_pri21:36:252
249940,0watchdog/26515-21kthreadcore20:51:232
249940,0watchdog/230898-21aten_r1power_en23:31:142
249940,0watchdog/218825-21iostat19:01:192
249940,0watchdog/20-21swapper/222:21:182
249940,0watchdog/20-21swapper/220:41:232
176229944,0cyclictest0-21swapper/220:48:532
176229944,0cyclictest0-21swapper/220:27:542
176229943,0cyclictest990-21awk21:51:212
176229943,0cyclictest9236-21awk20:31:172
176229943,0cyclictest611-21awk19:36:192
176229943,0cyclictest3556-21lldpd20:38:002
176229943,0cyclictest3556-21lldpd20:06:302
176229943,0cyclictest3556-21lldpd19:44:282
176229943,0cyclictest3556-21lldpd19:22:592
176229943,0cyclictest3514-21snmpd23:11:112
176229943,0cyclictest3514-21snmpd22:26:412
176229943,0cyclictest3514-21snmpd20:11:552
176229943,0cyclictest3514-21snmpd00:11:112
176229943,0cyclictest32062-21awk19:11:192
176229943,0cyclictest26308-21awk19:31:192
176229943,0cyclictest1186-21awk21:31:182
176219944,0cyclictest0-21swapper/122:36:071
176219943,0cyclictest832-21awk21:51:201
176219943,0cyclictest7344-21awk19:41:191
176219943,0cyclictest3556-21lldpd19:32:581
176219943,0cyclictest3514-21snmpd23:51:111
176219943,0cyclictest3514-21snmpd22:31:111
176219943,0cyclictest3514-21snmpd22:26:561
176219943,0cyclictest3514-21snmpd22:20:411
176219943,0cyclictest3514-21snmpd21:37:411
176219943,0cyclictest3514-21snmpd21:23:411
176219943,0cyclictest3514-21snmpd19:28:551
176219943,0cyclictest31293-21awk00:11:201
176219943,0cyclictest30488-21ssh21:07:151
176219943,0cyclictest29315-21awk20:46:201
176219943,0cyclictest25435-21awk19:06:201
176219943,0cyclictest25378-21awk21:46:181
176219943,0cyclictest22788-21awk00:26:181
176219943,0cyclictest21656-21awk20:16:211
176209943,0cyclictest9023-21awk21:16:170
176209943,0cyclictest8518-21awk22:36:180
176209943,0cyclictest3556-21lldpd19:09:290
176209943,0cyclictest3514-21snmpd23:53:110
176209943,0cyclictest3514-21snmpd21:46:410
176209943,0cyclictest3241-21irqbalance23:58:020
176209943,0cyclictest3241-21irqbalance21:21:130
176209943,0cyclictest29967-21kworker/0:022:13:180
176209943,0cyclictest23263-21awk00:06:190
176209943,0cyclictest22804-21awk00:26:180
176209943,0cyclictest21448-21sadc20:41:050
176209943,0cyclictest15424-21awk23:41:170
176209940,0cyclictest0-21swapper/020:56:180
179940,0migration/18392-21irqcore20:06:211
179940,0migration/17469-21kthreadcore19:16:211
179940,0migration/13589-21kthreadcore19:36:211
179940,0migration/131040-21sh22:08:491
179940,0migration/128781-21kthreadcore20:21:221
179940,0migration/12849-21kthreadcore19:11:211
179940,0migration/128488-21kthreadcore19:56:211
179940,0migration/126536-21kthreadcore23:46:201
179940,0migration/12147-21fschecks_count20:26:191
179940,0migration/120223-21runrttasks20:13:001
179940,0migration/119633-21latency_hist19:51:051
179940,0migration/116107-21fschecks_time22:41:151
179940,0migration/115468-21kthreadcore19:21:201
179940,0migration/113419-21latency21:56:241
179940,0migration/111052-21kthreadcore20:56:211
169940,0watchdog/19815-21grep20:31:191
169940,0watchdog/16517-21kthreadcore20:51:231
169940,0watchdog/16217-21ssh21:11:431
169940,0watchdog/117927-21aten_r1power_po19:01:151
169940,0watchdog/116389-21fschecks_time22:21:181
169940,0watchdog/114505-21ssh21:17:031
169940,0watchdog/10-21swapper/123:41:141
169940,0watchdog/10-21swapper/123:40:181
169940,0watchdog/10-21swapper/123:16:141
169940,0watchdog/10-21swapper/123:11:181
169940,0watchdog/10-21swapper/123:01:061
169940,0watchdog/10-21swapper/122:48:301
169940,0watchdog/10-21swapper/100:16:141
169940,0watchdog/10-21swapper/100:01:141
149940,0migration/028865-21diskmemload23:46:580
149940,0migration/028593-21sh23:06:230
149940,0migration/020289-21kthreadcore23:01:210
149940,0migration/01923-21kthreadcore23:31:210
149940,0migration/015976-21aten_r1power_vo22:21:170
149940,0migration/011628-21kthreadcore20:31:200
29715030,0irq/26-eth0-rx-0-21swapper/320:41:153
29715030,0irq/26-eth0-rx-0-21swapper/319:46:293
29715030,0irq/26-eth0-rx-0-21swapper/319:41:153
29715030,0irq/26-eth0-rx-0-21swapper/319:36:153
29715030,0irq/26-eth0-rx-0-21swapper/319:31:203
29715030,0irq/26-eth0-rx-0-21swapper/319:16:163
29715030,0irq/26-eth0-rx-0-21swapper/319:06:423
259930,0migration/25448-21runrttasks19:16:062
249930,0watchdog/20-21swapper/219:52:192
19130,0ktimersoftd/10-21swapper/120:41:011
176229933,0cyclictest3556-21lldpd19:46:282
176229932,0cyclictest3556-21lldpd20:04:302
176229930,0cyclictest22212-21grep20:16:222
176219932,0cyclictest8183-21awk22:56:191
176219932,0cyclictest17853-21grep23:21:211
176219931,0cyclictest0-21swapper/123:31:201
176209933,0cyclictest7050-21kworker/0:200:21:070
176209933,0cyclictest7050-21kworker/0:200:16:070
176209933,0cyclictest29967-21kworker/0:023:21:170
176209933,0cyclictest29967-21kworker/0:022:56:050
176209933,0cyclictest29967-21kworker/0:022:51:070
176209933,0cyclictest29967-21kworker/0:022:26:080
176209933,0cyclictest29967-21kworker/0:021:36:060
176209933,0cyclictest29967-21kworker/0:020:41:060
176209933,0cyclictest29967-21kworker/0:020:26:060
176209933,0cyclictest29967-21kworker/0:020:21:180
176209933,0cyclictest29967-21kworker/0:020:01:160
176209933,0cyclictest29967-21kworker/0:019:51:160
176209933,0cyclictest29967-21kworker/0:019:46:160
176209933,0cyclictest29967-21kworker/0:019:36:160
169930,0watchdog/10-21swapper/121:41:431
29715020,0irq/26-eth0-rx-0-21swapper/319:26:173
176219921,0cyclictest31406-21latency_hist22:51:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional