You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 02:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon May 11, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2464026361,0sleep20-21swapper/218:54:092
2460324029,0sleep00-21swapper/018:53:390
2442923725,0sleep30-21swapper/318:51:383
2477423228,0sleep10-21swapper/118:55:411
339960,0migration/331661-21fschecks_time22:21:123
339960,0migration/329762-21sh22:17:523
339960,0migration/323999-21df_abs21:16:133
339960,0migration/319970-21sh23:51:403
339960,0migration/315479-21sh21:11:013
29715060,0irq/26-eth0-rx-32469-21if_err_eth1.221:41:143
29715060,0irq/26-eth0-rx-26582-21kthreadcore21:36:173
29715060,0irq/26-eth0-rx-16512-21irqrtprio21:51:163
248719966,0cyclictest0-21swapper/123:06:121
8215050,0irq/24-0000:00:27723-21idleruntime-cro20:10:592
8215050,0irq/24-0000:00:22814-21latency_hist22:15:582
4150,0ktimersoftd/00-21swapper/023:50:240
339950,0migration/38401-21taskset20:17:323
339950,0migration/38298-21kthreadcore22:26:133
339950,0migration/38120-21munin-node22:06:163
339950,0migration/36533-21kthreadcore20:41:153
339950,0migration/36464-21munin-node19:06:113
339950,0migration/331974-21kthreadcore23:01:163
339950,0migration/331046-21kthreadcore19:46:173
339950,0migration/330832-21fschecks_count23:21:133
339950,0migration/330326-21fschecks_count00:01:123
339950,0migration/327366-21latency23:36:183
339950,0migration/326073-21aten_r1power_en19:21:113
339950,0migration/325617-21kthreadcore00:16:163
339950,0migration/325334-21kthreadcore23:56:163
339950,0migration/323906-21fschecks_time20:56:113
339950,0migration/321469-21timerwakeupswit20:26:223
339950,0migration/321436-21kthreadcore19:16:163
339950,0migration/32024-21hwlatdetect19:51:143
339950,0migration/319689-21kthreadcore22:51:183
339950,0migration/31638-21kthreadcore22:01:163
339950,0migration/315141-21taskset20:23:373
339950,0migration/312786-21diskmemload23:07:263
339950,0migration/312668-21latency_hist19:11:003
339950,0migration/312323-21taskset23:27:003
339950,0migration/312149-21taskset23:47:083
339950,0migration/311215-21kthreadcore19:31:163
29725050,0irq/27-eth0-tx-7023-21munin-node19:31:220
29725050,0irq/27-eth0-tx-30213-21ssh23:00:562
29725050,0irq/27-eth0-tx-23250-21munin-node20:56:212
29725050,0irq/27-eth0-tx-22778-21munin-node22:36:122
29725050,0irq/27-eth0-tx-22286-21munin-node20:31:150
29725050,0irq/27-eth0-tx-21389-21ssh22:53:332
29725050,0irq/27-eth0-tx-16369-21taskset21:11:132
29725050,0irq/27-eth0-tx-15644-21telnet20:26:100
29725050,0irq/27-eth0-tx-12653-21ssh00:09:202
29725050,0irq/27-eth0-tx-0-21swapper/223:47:172
29725050,0irq/27-eth0-tx-0-21swapper/222:46:192
29725050,0irq/27-eth0-tx-0-21swapper/222:32:002
29725050,0irq/27-eth0-tx-0-21swapper/222:15:272
29725050,0irq/27-eth0-tx-0-21swapper/222:06:232
29725050,0irq/27-eth0-tx-0-21swapper/200:15:142
29725050,0irq/27-eth0-tx-0-21swapper/019:26:110
29715050,0irq/26-eth0-rx-8430-21grep21:06:153
29715050,0irq/26-eth0-rx-701-21kthreadcore23:41:163
29715050,0irq/26-eth0-rx-27839-21nfs4_client23:16:183
29715050,0irq/26-eth0-rx-25540-21kthreadcore19:41:183
29715050,0irq/26-eth0-rx-22778-21cpuspeed_turbos22:56:113
29715050,0irq/26-eth0-rx-1890-21kthreadcore22:41:163
29715050,0irq/26-eth0-rx-15222-21if_eth123:11:113
29715050,0irq/26-eth0-rx-15120-21df22:31:113
29715050,0irq/26-eth0-rx-15115-21ssh21:30:263
29715050,0irq/26-eth0-rx-11722-21ssh00:06:383
29715050,0irq/26-eth0-rx-0-21swapper/323:31:063
29715050,0irq/26-eth0-rx-0-21swapper/321:05:563
29715050,0irq/26-eth0-rx-0-21swapper/320:11:173
29715050,0irq/26-eth0-rx-0-21swapper/300:11:043
259950,0migration/26644-21sh22:05:312
259950,0migration/26016-21aten_r1power_en23:26:112
259950,0migration/231494-21if_eth1.223:01:152
259950,0migration/230736-21sh21:18:512
259950,0migration/219850-21latency20:01:182
259950,0migration/215709-21sh21:31:102
259950,0migration/215709-21sh21:31:102
259950,0migration/212912-21munin-node21:26:172
259950,0migration/212842-21munin-node22:26:202
259950,0migration/21105-21kthreadcore22:41:152
248729955,0cyclictest0-21swapper/221:37:502
248719955,0cyclictest0-21swapper/123:46:061
248719955,0cyclictest0-21swapper/121:56:221
248719955,0cyclictest0-21swapper/121:47:041
248719955,0cyclictest0-21swapper/121:01:201
248719955,0cyclictest0-21swapper/100:15:441
248709955,0cyclictest0-21swapper/023:16:210
248709950,0cyclictest0-21swapper/021:40:580
248709950,0cyclictest0-21swapper/021:08:010
249950,0watchdog/25941-21ssh21:03:052
249950,0watchdog/214676-21diskstats23:11:092
179950,0migration/16315-21latency_hist22:45:581
179950,0migration/132611-21kthreadcore22:21:161
179950,0migration/128878-21taskset23:59:071
179950,0migration/127920-21kthreadcore20:56:151
179950,0migration/123733-21aten_r1power_fr21:16:121
179950,0migration/11063-21hwlatdetect19:26:131
169950,0watchdog/116560-21cut23:31:171
169950,0watchdog/10-21swapper/121:36:131
149950,0migration/09027-21kthreadcore23:06:160
149950,0migration/08518-21kthreadcore22:06:170
149950,0migration/029588-21latency_hist00:00:580
149950,0migration/024454-21kthreadcore22:16:160
149950,0migration/021579-21sh21:31:220
149950,0migration/021579-21sh21:31:220
149950,0migration/017475-21kthreadcore21:11:150
149950,0migration/016476-21kthreadcore00:11:160
149950,0migration/012996-21sh22:47:250
149950,0migration/011406-21kthreadcore20:21:160
8215040,0irq/24-0000:00:0-21swapper/221:45:592
339940,0migration/3389-21irqcore19:01:153
339940,0migration/326233-21irqcore18:56:153
339940,0migration/319480-21kthreadcore20:01:173
339940,0migration/311790-21kthreadcore21:46:173
29725040,0irq/27-eth0-tx-0-21swapper/223:56:062
29725040,0irq/27-eth0-tx-0-21swapper/223:41:222
29725040,0irq/27-eth0-tx-0-21swapper/223:36:122
29725040,0irq/27-eth0-tx-0-21swapper/223:31:052
29725040,0irq/27-eth0-tx-0-21swapper/223:17:572
29725040,0irq/27-eth0-tx-0-21swapper/222:21:022
29725040,0irq/27-eth0-tx-0-21swapper/221:56:032
29725040,0irq/27-eth0-tx-0-21swapper/221:41:232
29725040,0irq/27-eth0-tx-0-21swapper/221:07:372
29725040,0irq/27-eth0-tx-0-21swapper/200:22:222
29725040,0irq/27-eth0-tx-0-21swapper/200:16:152
29725040,0irq/27-eth0-tx-0-21swapper/020:51:220
29725040,0irq/27-eth0-tx-0-21swapper/020:41:090
29725040,0irq/27-eth0-tx-0-21swapper/019:01:200
29715040,0irq/26-eth0-rx-0-21swapper/322:46:053
29715040,0irq/26-eth0-rx-0-21swapper/322:36:463
29715040,0irq/26-eth0-rx-0-21swapper/322:11:563
29715040,0irq/26-eth0-rx-0-21swapper/321:57:193
29715040,0irq/26-eth0-rx-0-21swapper/321:31:283
29715040,0irq/26-eth0-rx-0-21swapper/321:31:283
29715040,0irq/26-eth0-rx-0-21swapper/321:22:153
29715040,0irq/26-eth0-rx-0-21swapper/320:31:113
29715040,0irq/26-eth0-rx-0-21swapper/300:21:043
259940,0migration/28750-21kthreadcore19:06:152
259940,0migration/23985-21kthreadcore19:26:162
259940,0migration/2344-21ksmtuned21:21:152
259940,0migration/231842-21timerandwakeup19:21:232
259940,0migration/230475-21fschecks_time00:01:122
259940,0migration/228770-21kthreadcore18:56:162
259940,0migration/22817-21fschecks_time20:16:122
259940,0migration/227875-21if_err_eth119:46:152
259940,0migration/217914-21kthreadcore23:51:162
259940,0migration/216873-21kernelversion20:26:152
259940,0migration/212689-21taskset23:07:102
259940,0migration/212671-21kthreadcore20:21:172
259940,0migration/211071-21irqrtprio20:46:152
248729943,0cyclictest4074-21awk20:41:132
248729943,0cyclictest3514-21snmpd20:54:332
248729943,0cyclictest3514-21snmpd19:03:332
248729943,0cyclictest31178-21awk23:21:152
248729943,0cyclictest28691-21sadc20:35:592
248729943,0cyclictest14657-21awk19:36:132
248719944,0cyclictest0-21swapper/123:15:211
248719943,0cyclictest7362-21awk22:46:141
248719943,0cyclictest6906-21awk19:06:131
248719943,0cyclictest4104-21awk20:41:131
248719943,0cyclictest3514-21snmpd23:39:331
248719943,0cyclictest3514-21snmpd21:53:181
248719943,0cyclictest3514-21snmpd21:41:331
248719943,0cyclictest3514-21snmpd19:46:031
248719943,0cyclictest3514-21snmpd19:36:031
248719943,0cyclictest3514-21snmpd19:17:331
248719943,0cyclictest3514-21snmpd19:01:031
248719943,0cyclictest3176-21awk20:16:141
248719943,0cyclictest22527-21awk00:16:141
248709944,0cyclictest0-21swapper/021:26:460
248709943,0cyclictest3514-21snmpd22:23:330
248709943,0cyclictest3514-21snmpd21:46:030
248709943,0cyclictest32755-21awk21:21:140
248709943,0cyclictest3241-21irqbalance23:11:170
248709943,0cyclictest3241-21irqbalance22:51:070
248709943,0cyclictest3241-21irqbalance21:02:280
248709943,0cyclictest3241-21irqbalance20:56:280
248709943,0cyclictest3241-21irqbalance20:36:080
248709943,0cyclictest3241-21irqbalance20:16:080
248709943,0cyclictest3241-21irqbalance20:01:580
248709943,0cyclictest3241-21irqbalance19:56:180
248709943,0cyclictest3241-21irqbalance00:08:470
248709943,0cyclictest31761-21awk22:41:140
248709943,0cyclictest30441-21awk00:21:130
248709943,0cyclictest28925-21awk20:11:150
248709943,0cyclictest27019-21awk19:21:150
248709943,0cyclictest22501-21sadc22:35:580
248709943,0cyclictest22239-21awk20:06:140
248709943,0cyclictest14832-21awk23:31:140
248709940,0cyclictest31509-21munin-run19:01:000
249940,0watchdog/221042-21cut19:41:142
249940,0watchdog/22025-21hwlatdetect19:51:142
249940,0watchdog/20-21swapper/221:51:172
179940,0migration/19660-21kthreadcore00:06:151
179940,0migration/18788-21kthreadcore19:31:151
179940,0migration/15989-21aten_r1power_cu23:26:111
179940,0migration/131786-21kthreadcore23:21:171
179940,0migration/130742-21cpu23:01:121
179940,0migration/130370-21sh00:21:121
179940,0migration/130037-21taskset22:18:391
179940,0migration/129560-21taskset22:38:111
179940,0migration/126975-21kthreadcore18:56:161
179940,0migration/125601-21kthreadcore20:06:161
179940,0migration/12550-21kthreadcore23:41:161
179940,0migration/115669-21aten_r1power_fr21:31:101
179940,0migration/115669-21aten_r1power_fr21:31:101
179940,0migration/115533-21fschecks_count22:11:121
179940,0migration/115284-21fschecks_time20:01:131
179940,0migration/111534-21kthreadcore19:56:161
169940,0watchdog/129020-21ssh22:57:291
169940,0watchdog/115173-21mdstat20:46:171
169940,0watchdog/114949-21grep22:51:131
169940,0watchdog/114815-21sh22:31:091
169940,0watchdog/113123-21diskstats19:11:141
169940,0watchdog/10-21swapper/123:51:051
169940,0watchdog/10-21swapper/122:08:051
169940,0watchdog/10-21swapper/119:41:141
149940,0migration/06689-21needreboot19:51:180
149940,0migration/06399-21munin-node23:26:130
149940,0migration/06272-21runrttasks22:04:300
149940,0migration/0491-21kthreadcore21:41:160
149940,0migration/04484-21sh23:01:450
149940,0migration/031027-21fschecks_time23:21:140
149940,0migration/030287-21kthreadcore19:46:170
149940,0migration/026292-21kthreadcore21:16:180
149940,0migration/019785-21runrttasks22:11:170
149940,0migration/012979-21kthreadcore20:46:160
29725030,0irq/27-eth0-tx-0-21swapper/019:16:110
29715030,0irq/26-eth0-rx-0-21swapper/320:46:023
29715030,0irq/26-eth0-rx-0-21swapper/320:37:403
29715030,0irq/26-eth0-rx-0-21swapper/320:10:003
29715030,0irq/26-eth0-rx-0-21swapper/320:00:463
29715030,0irq/26-eth0-rx-0-21swapper/319:39:583
29715030,0irq/26-eth0-rx-0-21swapper/319:26:103
248729933,0cyclictest0-21swapper/219:35:202
248729932,0cyclictest3514-21snmpd20:13:182
248729931,0cyclictest0-21swapper/220:36:112
248719932,0cyclictest3241-21irqbalance21:06:581
248719932,0cyclictest32395-21awk22:01:141
248719932,0cyclictest27715-21missed_timers23:16:171
248719932,0cyclictest26942-21ethtool19:21:151
248719932,0cyclictest23160-21awk20:31:151
248719932,0cyclictest1985-21open_inodes20:36:191
248719932,0cyclictest16468-21awk20:26:141
248719932,0cyclictest10973-21kthreadcore21:26:161
248709933,0cyclictest6233-21kworker/0:022:26:140
248709933,0cyclictest3556-21lldpd22:37:070
248709933,0cyclictest0-21swapper/023:37:570
248709932,0cyclictest9554-21grep19:06:150
248709932,0cyclictest30723-21awk00:01:140
248709932,0cyclictest27259-21ps00:16:190
248709932,0cyclictest24284-21awk21:56:130
248709932,0cyclictest21294-21awk19:41:160
248709932,0cyclictest19369-21sensors19:36:200
248709932,0cyclictest16298-21awk21:51:150
248709932,0cyclictest15934-21kthreadcore23:51:150
248709931,0cyclictest0-21swapper/019:15:170
249930,0watchdog/214034-21kthreadcore19:11:182
249930,0watchdog/20-21swapper/219:56:102
179930,0migration/117602-21irqcore20:51:171
169930,0watchdog/121048-21munin-node21:11:171
169930,0watchdog/112991-21ssh22:26:331
169930,0watchdog/10-21swapper/121:21:091
169930,0watchdog/10-21swapper/120:21:141
169930,0watchdog/10-21swapper/119:51:141
169930,0watchdog/10-21swapper/100:01:171
149930,0migration/028808-21sh22:56:530
29715020,0irq/26-eth0-rx-0-21swapper/320:51:123
248729920,0cyclictest0-21swapper/219:16:002
248719921,0cyclictest27894-21latency_hist20:11:001
248709920,0cyclictest0-21swapper/023:41:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional