You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 05:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Fri Apr 17, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1753721250,2sleep0201rcuc/022:47:140
1753721250,2sleep0201rcuc/022:47:140
1753721250,2sleep0201rcuc/022:47:140
3230721190,4chrt0-21swapper/323:27:133
1272221130,3sleep20-21swapper/221:07:192
1272221130,3sleep20-21swapper/221:07:192
372721050,3sleep10-21swapper/120:42:221
372721050,3sleep10-21swapper/120:42:221
372721050,3sleep10-21swapper/120:42:221
841321010,2sleep30-21swapper/323:47:233
841321010,2sleep30-21swapper/323:47:233
2124521010,3chrt0-21swapper/321:32:043
2124521010,3chrt0-21swapper/321:32:043
97312960,3chrt0-21swapper/223:52:202
97312960,3chrt0-21swapper/223:52:202
140492870,3sleep20-21swapper/200:02:302
140492870,3sleep20-21swapper/200:02:302
140182780,5chrt14021-21fschecks_count21:12:110
140182780,5chrt14021-21fschecks_count21:12:110
155632730,3sleep30-21swapper/300:07:213
155632730,3sleep30-21swapper/300:07:213
155632730,3sleep30-21swapper/300:07:213
26772720,2sleep20-21swapper/220:42:102
26772720,2sleep20-21swapper/220:42:102
26772720,2sleep20-21swapper/220:42:102
14462710,2chrt0-21swapper/320:37:173
14462710,2chrt0-21swapper/320:37:173
82252610,2sleep30-21swapper/320:57:073
309082610,3chrt0-21swapper/323:22:193
309082610,3chrt0-21swapper/323:22:193
140452610,3sleep30-21swapper/300:02:273
140452610,3sleep30-21swapper/300:02:273
27162600,1sleep0171ktimers/023:32:210
27162600,1sleep0171ktimers/023:32:210
82982590,2sleep20-21swapper/220:57:102
82982590,2sleep20-21swapper/220:57:102
82982590,2sleep20-21swapper/220:57:102
95762570,3chrt0-21swapper/319:32:173
274452570,2sleep30-21swapper/323:12:223
274452570,2sleep30-21swapper/323:12:223
175002570,2chrt0-21swapper/200:12:222
175002570,2chrt0-21swapper/200:12:222
60292560,2sleep20-21swapper/223:42:202
60292560,2sleep20-21swapper/223:42:202
177432560,1chrt0-21swapper/021:22:120
177432560,1chrt0-21swapper/021:22:120
177432560,1chrt0-21swapper/021:22:120
125322560,3sleep30-21swapper/321:07:163
125322560,3sleep30-21swapper/321:07:163
185182550,3sleep2351rcuc/219:57:122
185182550,3sleep2351rcuc/219:57:122
90682540,3sleep20-21swapper/219:32:122
239342520,3chrt0-21swapper/200:32:082
239342520,3chrt0-21swapper/200:32:082
191122490,1sleep00-21swapper/022:52:130
264682460,2sleep20-21swapper/220:17:182
264682460,2sleep20-21swapper/220:17:182
137272460,3sleep30-21swapper/319:42:223
137272460,3sleep30-21swapper/319:42:223
178799434,32cyclictest12654-21chrt00:02:052
178799434,32cyclictest12654-21chrt00:02:052
178799434,32cyclictest12654-21chrt00:02:052
143492430,3sleep20-21swapper/221:12:132
143492430,3sleep20-21swapper/221:12:132
114852420,3sleep30-21swapper/323:57:163
114852420,3sleep30-21swapper/323:57:163
114852420,3sleep30-21swapper/323:57:163
272002410,1sleep127322-21sendmail_mailtr23:12:211
272002410,1sleep127322-21sendmail_mailtr23:12:211
247672410,2sleep00-21swapper/023:07:120
1774993729,6cyclictest3890-21kernelversion23:37:150
1774993729,6cyclictest3890-21kernelversion23:37:150
1774993729,6cyclictest3890-21kernelversion23:37:150
1774993729,6cyclictest27616-21unixbench_multi23:12:230
1774993729,6cyclictest27616-21unixbench_multi23:12:230
303152360,3chrt0-21swapper/123:22:131
303152360,3chrt0-21swapper/123:22:131
303152360,3chrt0-21swapper/123:22:131
1774993630,4cyclictest12515-21latency_hist00:02:000
1774993630,4cyclictest12515-21latency_hist00:02:000
1774993630,4cyclictest12515-21latency_hist00:02:000
1774993629,5cyclictest1773-21cyclictest20:46:210
1774993629,5cyclictest1773-21cyclictest20:46:210
1774993629,5cyclictest1773-21cyclictest20:46:210
1774993628,6cyclictest19363-21threads00:17:220
1774993628,6cyclictest19363-21threads00:17:220
297072350,3sleep30-21swapper/320:27:123
297072350,3sleep30-21swapper/320:27:123
1774993528,5cyclictest29170-21head21:52:140
135342350,3sleep30-21swapper/322:37:123
135342350,3sleep30-21swapper/322:37:123
135342350,3sleep30-21swapper/322:37:123
1782993415,3cyclictest0-21swapper/122:28:281
1782993415,3cyclictest0-21swapper/122:28:281
1774993428,4cyclictest1773-21cyclictest19:57:350
1774993428,4cyclictest1773-21cyclictest19:57:350
1774993427,6cyclictest8358-21sshd19:28:220
1774993427,6cyclictest8358-21sshd19:28:220
196772330,2sleep20-21swapper/222:52:192
1782993328,3cyclictest16260-21cron00:11:591
1782993328,3cyclictest16260-21cron00:11:591
1782993328,3cyclictest16260-21cron00:11:591
1782993327,4cyclictest729-21dbus-daemon22:21:591
1782993327,4cyclictest729-21dbus-daemon22:21:591
1782993327,4cyclictest729-21dbus-daemon22:21:591
1782993326,5cyclictest14955-21sendmail22:42:001
1782993326,5cyclictest14955-21sendmail22:42:001
1782993326,5cyclictest14955-21sendmail22:42:001
1774993328,4cyclictest7510-21gltestperf23:47:130
1774993328,4cyclictest7510-21gltestperf23:47:130
270852320,3sleep20-21swapper/200:37:232
270852320,3sleep20-21swapper/200:37:232
270852320,3sleep20-21swapper/200:37:232
263832320,7sleep026386-21ntp_kernel_err20:17:160
263832320,7sleep026386-21ntp_kernel_err20:17:160
1782993226,4cyclictest19994-21cron00:22:001
1782993226,4cyclictest19994-21cron00:22:001
1782993225,5cyclictest729-21dbus-daemon21:02:001
1782993225,5cyclictest729-21dbus-daemon21:02:001
1782993225,5cyclictest729-21dbus-daemon21:02:001
1782993225,5cyclictest474-21sshd23:27:211
1782993225,5cyclictest474-21sshd23:27:211
1782993225,5cyclictest12511-21latency_hist00:02:111
1782993225,5cyclictest12511-21latency_hist00:02:111
1774993227,3cyclictest7457-21latency_hist22:21:590
1774993227,3cyclictest7457-21latency_hist22:21:590
1774993227,3cyclictest7457-21latency_hist22:21:590
1774993226,4cyclictest572-21cups-browsed22:12:210
1774993226,4cyclictest572-21cups-browsed22:12:210
1782993126,3cyclictest9698-21kworker/u9:0-nfsiod00:02:541
1782993126,3cyclictest9698-21kworker/u9:0-nfsiod00:02:541
1782993125,4cyclictest1773-21cyclictest21:47:221
1782993125,4cyclictest1773-21cyclictest21:47:221
1782993122,7cyclictest8660-21sort19:32:001
1782993122,7cyclictest8660-21sort19:32:001
1782993122,7cyclictest19082-21fschecks_count22:52:131
1774993127,3cyclictest17515-21latency_hist21:22:040
1774993127,3cyclictest17515-21latency_hist21:22:040
1774993125,3cyclictest13653-21latency_hist21:11:590
1774993125,3cyclictest13653-21latency_hist21:11:590
1774993124,5cyclictest12598-21latency_hist19:42:080
1774993124,5cyclictest12598-21latency_hist19:42:080
1774993124,5cyclictest11449-21cpu22:32:110
1774993124,5cyclictest11449-21cpu22:32:110
1782993026,2cyclictest0-21swapper/100:27:141
1782993026,2cyclictest0-21swapper/100:27:141
1782993025,3cyclictest9218-21gltestperf19:32:131
1782993023,5cyclictest8786-21latency_hist23:52:001
1782993023,5cyclictest8786-21latency_hist23:52:001
1782993023,5cyclictest15491-21threads19:47:221
1782993023,5cyclictest15491-21threads19:47:221
1782993022,6cyclictest11076-21fschecks_time23:57:131
1774993025,3cyclictest10551-21latency_hist19:37:000
1774993023,5cyclictest32360-21gltestperf23:27:140
1774993023,5cyclictest32360-21gltestperf23:27:140
290322290,2sleep20-21swapper/223:17:192
290322290,2sleep20-21swapper/223:17:192
1782992924,4cyclictest572-21cups-browsed22:02:211
1782992924,4cyclictest572-21cups-browsed22:02:211
1782992924,3cyclictest8038-21gltestperf22:22:131
1782992924,3cyclictest8038-21gltestperf22:22:131
1782992924,3cyclictest1773-21cyclictest22:57:251
1782992924,3cyclictest1773-21cyclictest22:57:251
1782992924,3cyclictest1773-21cyclictest22:57:251
1782992922,6cyclictest1773-21cyclictest21:57:221
1782992922,6cyclictest1773-21cyclictest21:57:221
1782992922,5cyclictest22083-21fschecks_count20:07:111
1782992922,5cyclictest22083-21fschecks_count20:07:111
1774992923,5cyclictest5779-21kernelversion23:42:160
1774992923,5cyclictest5779-21kernelversion23:42:160
1774992923,4cyclictest1346-21avahi-daemon22:37:210
1774992923,4cyclictest1346-21avahi-daemon22:37:210
1774992923,4cyclictest1346-21avahi-daemon22:37:210
178299285,3cyclictest700-21192.168.115.40-manager00:12:561
178299285,3cyclictest700-21192.168.115.40-manager00:12:561
1782992824,3cyclictest4279-21gltestperf22:12:131
1782992824,3cyclictest4279-21gltestperf22:12:131
1782992823,4cyclictest572-21cups-browsed23:32:191
1782992823,4cyclictest572-21cups-browsed23:32:191
1782992823,4cyclictest21062-21switchtime20:02:211
1782992823,4cyclictest21062-21switchtime20:02:211
1782992823,3cyclictest547-21rs:main1
1782992823,3cyclictest547-21rs:main1
1782992822,4cyclictest6645-21fschecks_time20:52:131
1782992822,4cyclictest545-21in:imuxsock21:17:211
1782992822,4cyclictest545-21in:imuxsock21:17:211
1782992821,6cyclictest24310-21gltestperf00:32:141
1782992821,5cyclictest26222-21idleruntime-cro23:12:011
1782992821,5cyclictest18880-21needreboot19:57:171
1782992821,5cyclictest18880-21needreboot19:57:171
1782992821,5cyclictest1773-21cyclictest20:37:231
1782992821,5cyclictest1773-21cyclictest20:37:231
1782992821,5cyclictest16828-21kernelversion19:52:141
1782992821,5cyclictest16828-21kernelversion19:52:141
1774992823,3cyclictest13280-21latency00:02:140
1774992823,3cyclictest13280-21latency00:02:140
1774992822,4cyclictest0-21swapper/023:52:250
1774992822,4cyclictest0-21swapper/023:52:250
1774992821,5cyclictest23176-21latency_hist21:37:070
1774992821,5cyclictest23176-21latency_hist21:37:070
1774992821,5cyclictest23011-21sshd00:27:220
1774992821,5cyclictest23011-21sshd00:27:220
1774992820,6cyclictest6631-21cut20:52:130
1774992820,6cyclictest26724-21grep00:37:210
1774992820,6cyclictest26724-21grep00:37:210
1774992820,6cyclictest26724-21grep00:37:210
1782992723,3cyclictest15131-21/usr/sbin/munin22:42:191
1782992723,3cyclictest15131-21/usr/sbin/munin22:42:191
1782992722,4cyclictest12098-21needreboot22:32:181
1782992722,4cyclictest12098-21needreboot22:32:181
1782992721,5cyclictest5389-21rt-features20:47:181
1782992721,4cyclictest24898-21sendmail21:41:591
1782992721,4cyclictest24898-21sendmail21:41:591
1782992721,4cyclictest24898-21sendmail21:41:591
1782992721,4cyclictest19998-21kernelversion21:27:121
1782992720,5cyclictest6052-21open_files23:42:211
1782992720,5cyclictest6052-21open_files23:42:211
1782992719,6cyclictest10753-21wc19:37:101
1782992719,6cyclictest10753-21wc19:37:101
1774992721,4cyclictest0-21swapper/022:22:130
1774992721,4cyclictest0-21swapper/022:22:130
1774992721,4cyclictest0-21swapper/000:34:310
1774992720,5cyclictest31224-21latency_hist20:32:060
1774992720,5cyclictest31224-21latency_hist20:32:060
1774992719,6cyclictest0-21swapper/020:07:200
1774992719,6cyclictest0-21swapper/020:07:200
8852260,2sleep20-21swapper/220:37:122
8852260,2sleep20-21swapper/220:37:122
1782992621,2cyclictest700-21192.168.115.40-manager21:43:221
1782992621,2cyclictest700-21192.168.115.40-manager21:43:221
1782992621,2cyclictest700-21192.168.115.40-manager21:43:221
1782992620,5cyclictest21140-21latency_hist21:32:071
1782992620,5cyclictest21140-21latency_hist21:32:071
1782992620,4cyclictest28227-21needreboot20:22:161
1782992620,4cyclictest28227-21needreboot20:22:161
1782992618,6cyclictest3499-21ls23:37:111
1782992618,6cyclictest3499-21ls23:37:111
1782992618,6cyclictest3499-21ls23:37:111
1782992618,6cyclictest26884-21sort00:37:221
1782992618,6cyclictest26884-21sort00:37:221
1782992618,6cyclictest26884-21sort00:37:221
1782992618,6cyclictest13541-21cut19:42:211
1782992618,6cyclictest13541-21cut19:42:211
1774992622,2cyclictest0-21swapper/023:22:230
1774992622,2cyclictest0-21swapper/023:22:230
1774992622,2cyclictest0-21swapper/023:22:230
1774992621,4cyclictest3916-21latency_hist22:12:070
1774992621,4cyclictest3916-21latency_hist22:12:070
1774992621,3cyclictest0-21swapper/019:52:120
1774992621,3cyclictest0-21swapper/019:52:120
1774992620,5cyclictest28780-21kernelversion23:17:150
1774992620,5cyclictest28780-21kernelversion23:17:150
1774992620,4cyclictest0-21swapper/021:57:220
1774992620,4cyclictest0-21swapper/021:57:220
1774992620,4cyclictest0-21swapper/021:47:210
1774992620,4cyclictest0-21swapper/021:47:210
1774992619,5cyclictest15118-21kernelversion00:07:150
1774992619,5cyclictest15118-21kernelversion00:07:150
1774992619,5cyclictest15118-21kernelversion00:07:150
1774992619,5cyclictest11152-21threads21:02:190
1774992619,5cyclictest11152-21threads21:02:190
1774992619,5cyclictest11152-21threads21:02:190
1788992517,5cyclictest230562sleep323:02:143
1788992517,5cyclictest230562sleep323:02:143
178299257,3cyclictest264702ps20:17:181
178299257,3cyclictest264702ps20:17:181
1782992521,3cyclictest29929-21latency_hist23:22:001
1782992521,3cyclictest29929-21latency_hist23:22:001
1782992520,3cyclictest9698-21kworker/u9:0-rpciod21:22:151
1782992520,3cyclictest9698-21kworker/u9:0-rpciod21:22:151
1774992521,3cyclictest1773-21cyclictest19:22:210
1774992521,3cyclictest1773-21cyclictest19:22:210
1774992521,2cyclictest0-21swapper/020:57:150
1774992521,2cyclictest0-21swapper/020:57:150
1774992521,2cyclictest0-21swapper/020:57:150
1774992521,2cyclictest0-21swapper/020:47:200
1774992520,3cyclictest0-21swapper/022:27:220
1774992520,3cyclictest0-21swapper/022:27:220
1774992520,3cyclictest0-21swapper/021:27:200
1774992520,3cyclictest0-21swapper/021:27:200
1774992519,5cyclictest28298-21ntp_offset20:22:180
1774992519,5cyclictest28298-21ntp_offset20:22:180
1774992519,5cyclictest22234-21sshd22:59:180
1774992519,5cyclictest22234-21sshd22:59:180
1774992519,5cyclictest22234-21sshd22:59:180
1774992519,4cyclictest0-21swapper/023:02:120
1774992519,4cyclictest0-21swapper/023:02:120
1782992420,2cyclictest0-21swapper/121:12:211
1782992420,2cyclictest0-21swapper/121:12:211
1782992416,6cyclictest11000-21gunzip21:02:171
1782992416,6cyclictest11000-21gunzip21:02:171
1782992416,6cyclictest11000-21gunzip21:02:171
1782992412,10cyclictest142-21systemd-journal21:36:211
1782992412,10cyclictest142-21systemd-journal21:36:211
1774992420,3cyclictest15825-21needreboot22:42:160
1774992420,3cyclictest15825-21needreboot22:42:160
1774992420,3cyclictest1514-21ntp_offset20:37:190
1774992420,3cyclictest1514-21ntp_offset20:37:190
1774992419,4cyclictest24012-21fschecks_time20:12:100
1774992419,4cyclictest24012-21fschecks_time20:12:100
1774992419,3cyclictest0-21swapper/020:32:160
1774992417,5cyclictest26841-21latency_hist21:47:040
1774992417,5cyclictest26841-21latency_hist21:47:040
1774992417,5cyclictest26841-21latency_hist21:47:040
1774992417,5cyclictest20916-21needreboot00:22:170
1774992417,5cyclictest20916-21needreboot00:22:170
1774992417,5cyclictest20916-21needreboot00:22:170
95522230,3sleep30-21swapper/323:52:173
95522230,3sleep30-21swapper/323:52:173
1782992319,2cyclictest0-21swapper/122:47:211
1782992319,2cyclictest0-21swapper/122:47:211
1782992319,2cyclictest0-21swapper/122:47:211
1782992317,3cyclictest0-21swapper/119:27:101
1782992316,5cyclictest0-21swapper/123:02:131
1782992316,5cyclictest0-21swapper/123:02:131
1782992315,6cyclictest23904-21ls20:12:081
1782992315,6cyclictest23904-21ls20:12:081
1782992314,7cyclictest20773-21ls00:22:151
1782992314,7cyclictest20773-21ls00:22:151
1782992314,7cyclictest20773-21ls00:22:151
177499238,4cyclictest36350irq/48-eth020:02:500
177499238,4cyclictest36350irq/48-eth020:02:500
1774992318,3cyclictest0-21swapper/019:42:210
1774992318,3cyclictest0-21swapper/019:42:210
1774992317,4cyclictest1346-21avahi-daemon19:47:210
1774992317,4cyclictest1346-21avahi-daemon19:47:210
1774992317,4cyclictest0-21swapper/022:07:110
1774992317,4cyclictest0-21swapper/022:07:110
1774992314,7cyclictest24963-21taskset21:42:020
1774992314,7cyclictest24963-21taskset21:42:020
1774992314,7cyclictest24963-21taskset21:42:020
1782992217,3cyclictest0-21swapper/123:52:251
1782992217,3cyclictest0-21swapper/123:52:251
1782992217,3cyclictest0-21swapper/120:32:161
164052220,2sleep20-21swapper/221:17:162
164052220,2sleep20-21swapper/221:17:162
212952210,3chrt0-21swapper/221:32:072
212952210,3chrt0-21swapper/221:32:072
1782992115,4cyclictest0-21swapper/120:28:571
1782992115,4cyclictest0-21swapper/120:28:571
1782992114,5cyclictest29847-21threads21:52:221
227142200,2sleep30-21swapper/320:07:183
227142200,2sleep30-21swapper/320:07:183
1788992015,3cyclictest33202chrt23:37:063
1788992015,3cyclictest33202chrt23:37:063
1782992013,5cyclictest5425-21gltestperf19:22:121
1782992011,7cyclictest2575-21kernelversion22:07:151
1782992011,7cyclictest2575-21kernelversion22:07:151
1774992013,5cyclictest0-21swapper/000:12:210
1774992013,5cyclictest0-21swapper/000:12:210
178799191,13cyclictest19546-21taskset21:27:092
178799191,13cyclictest19546-21taskset21:27:092
178799191,13cyclictest19546-21taskset21:27:092
178899181,12cyclictest10292-21chrt21:02:093
178899181,12cyclictest10292-21chrt21:02:093
178899181,11cyclictest27956-21chrt20:22:143
178899181,11cyclictest27956-21chrt20:22:143
178799182,12cyclictest5535-21chrt22:16:212
178799182,12cyclictest5535-21chrt22:16:212
178799182,12cyclictest0-21swapper/223:27:162
178799182,12cyclictest0-21swapper/223:27:162
178899172,11cyclictest0-21swapper/322:22:143
178899172,11cyclictest0-21swapper/322:22:143
1788991711,4cyclictest174142chrt21:21:593
1788991711,4cyclictest174142chrt21:21:593
178899171,12cyclictest22553-21chrt00:27:153
178899171,12cyclictest22553-21chrt00:27:153
178899171,11cyclictest7875-21chrt19:27:183
178899171,11cyclictest7875-21chrt19:27:183
178899171,11cyclictest25096-21chrt21:42:093
178899171,11cyclictest25096-21chrt21:42:093
178799171,12cyclictest26702-21chrt23:12:132
178799171,12cyclictest26702-21chrt23:12:132
178799171,11cyclictest5253-21chrt19:22:112
178799171,11cyclictest5253-21chrt19:22:112
178799171,11cyclictest1793-21chrt23:32:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional