You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-03 - 02:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Fri Jul 03, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2628521170,2sleep30-21swapper/320:38:333
2628521170,2sleep30-21swapper/320:38:333
140442790,3sleep30-21swapper/321:33:283
140442790,3sleep30-21swapper/321:33:283
140442790,3sleep30-21swapper/321:33:283
132342750,3sleep30-21swapper/300:23:333
132342750,3sleep30-21swapper/300:23:333
132342750,3sleep30-21swapper/300:23:333
60272600,2chrt4496-40mandb00:04:051
60272600,2chrt4496-40mandb00:04:051
43482600,2chrt4349-21/usr/sbin/munin21:08:250
43482600,2chrt4349-21/usr/sbin/munin21:08:250
43482600,2chrt4349-21/usr/sbin/munin21:08:250
270162590,2sleep027086-21grep23:33:340
270162590,2sleep027086-21grep23:33:340
196642560,3sleep30-21swapper/300:43:193
196642560,3sleep30-21swapper/300:43:193
196642560,3sleep30-21swapper/300:43:193
45192550,4sleep20-21swapper/200:03:142
45192550,4sleep20-21swapper/200:03:142
45192550,4sleep20-21swapper/200:03:142
93812540,2chrt0-21swapper/222:48:242
93812540,2chrt0-21swapper/222:48:242
93812540,2chrt0-21swapper/222:48:242
138062520,2sleep10-21swapper/122:58:321
138062520,2sleep10-21swapper/122:58:321
138062520,2sleep10-21swapper/122:58:321
25899995040,5cyclictest0-21swapper/000:03:180
25899995040,5cyclictest0-21swapper/000:03:180
25899995040,5cyclictest0-21swapper/000:03:180
111452470,3chrt0-21swapper/119:58:311
111452470,3chrt0-21swapper/119:58:311
80802450,3sleep30-21swapper/322:43:313
80802450,3sleep30-21swapper/322:43:313
80802450,3sleep30-21swapper/322:43:313
90622430,4sleep1271rcuc/100:13:261
90622430,4sleep1271rcuc/100:13:261
90622430,4sleep1271rcuc/100:13:261
234102420,3sleep20-21swapper/221:58:282
234102420,3sleep20-21swapper/221:58:282
234102420,3sleep20-21swapper/221:58:282
191602420,3sleep20-21swapper/221:48:232
25905994132,7cyclictest25801-21netstat20:38:281
25905994132,7cyclictest25801-21netstat20:38:281
2589999413,29cyclictest5973-21users00:03:350
2589999413,29cyclictest5973-21users00:03:350
6462400,3sleep00-21swapper/020:58:240
6462400,3sleep00-21swapper/020:58:240
6462400,3sleep00-21swapper/020:58:240
244492400,3sleep20-21swapper/220:33:332
244492400,3sleep20-21swapper/220:33:332
244492400,3sleep20-21swapper/220:33:332
211982400,2sleep30-21swapper/323:18:323
211982400,2sleep30-21swapper/323:18:323
211982400,2sleep30-21swapper/323:18:323
25899993930,7cyclictest20631-21latency_hist21:53:090
25899993930,7cyclictest20631-21latency_hist21:53:090
25899993930,7cyclictest20631-21latency_hist21:53:090
322602370,2sleep032259-21open_files23:48:320
322602370,2sleep032259-21open_files23:48:320
322602370,2sleep032259-21open_files23:48:320
25899993728,6cyclictest575-21thd19:28:320
25899993728,6cyclictest575-21thd19:28:320
234792370,2sleep00-21swapper/020:33:230
234792370,2sleep00-21swapper/020:33:230
234792370,2sleep00-21swapper/020:33:230
67772360,2chrt0-21swapper/319:48:243
67772360,2chrt0-21swapper/319:48:243
319182360,3sleep20-21swapper/219:28:252
319182360,3sleep20-21swapper/219:28:252
25905993625,4cyclictest60-21kcompactd022:22:051
25905993625,4cyclictest60-21kcompactd022:22:051
25899993630,4cyclictest32416-21gltestperf22:23:250
25899993630,4cyclictest32416-21gltestperf22:23:250
25899993630,4cyclictest32416-21gltestperf22:23:250
146002360,2sleep10-21swapper/120:08:281
146002360,2sleep10-21swapper/120:08:281
146002360,2sleep10-21swapper/120:08:281
25899993526,6cyclictest12143-21fschecks_count20:03:220
25899993526,6cyclictest12143-21fschecks_count20:03:220
25905993427,5cyclictest10683-21idleruntime-cro22:53:101
25905993427,5cyclictest10683-21idleruntime-cro22:53:101
25905993427,5cyclictest10683-21idleruntime-cro22:53:101
25905993425,2cyclictest181rcu_preempt19:53:011
25905993425,2cyclictest181rcu_preempt19:53:011
25899993424,8cyclictest8929-21kernelversion00:13:250
25899993424,8cyclictest8929-21kernelversion00:13:250
25899993424,8cyclictest8929-21kernelversion00:13:250
156662340,3sleep20-21swapper/221:38:262
156662340,3sleep20-21swapper/221:38:262
156662340,3sleep20-21swapper/221:38:262
25905993328,3cyclictest14678-21systemd-journal00:03:131
25905993328,3cyclictest14678-21systemd-journal00:03:131
25905993328,3cyclictest14678-21systemd-journal00:03:131
25905993327,4cyclictest281ktimers/120:19:311
25905993327,4cyclictest281ktimers/120:19:311
25905993327,4cyclictest281ktimers/120:19:311
2590599332,6cyclictest281ktimers/120:05:311
2590599332,6cyclictest281ktimers/120:05:311
25899993327,4cyclictest17490-21cut21:43:230
25899993327,4cyclictest17490-21cut21:43:230
2590599326,3cyclictest181rcu_preempt21:41:041
2590599326,3cyclictest181rcu_preempt21:41:041
2590599326,3cyclictest181rcu_preempt21:41:041
25905993225,5cyclictest7666-21kernelversion22:43:251
25905993225,5cyclictest7666-21kernelversion22:43:251
25905993225,5cyclictest7666-21kernelversion22:43:251
25905993225,5cyclictest10312-21fschecks_count19:58:221
25899993225,5cyclictest8475-21cut19:53:230
25899993225,5cyclictest8475-21cut19:53:230
25899993225,5cyclictest8475-21cut19:53:230
25899993225,5cyclictest19715-21cpu00:43:210
25899993225,5cyclictest19715-21cpu00:43:210
321602310,3sleep30-21swapper/319:28:313
321602310,3sleep30-21swapper/319:28:313
2590599313,5cyclictest281ktimers/100:19:081
2590599313,5cyclictest281ktimers/100:19:081
2590599313,5cyclictest281ktimers/100:19:081
25905993126,4cyclictest572-21cups-browsed19:43:301
25905993126,4cyclictest572-21cups-browsed19:43:301
25905993126,4cyclictest572-21cups-browsed19:43:301
25905993125,5cyclictest23203-21kernelversion21:58:251
25905993125,5cyclictest23203-21kernelversion21:58:251
25905993125,5cyclictest23203-21kernelversion21:58:251
25905993124,4cyclictest0-21swapper/122:58:221
25905993124,4cyclictest0-21swapper/122:58:221
25905993122,3cyclictest181rcu_preempt23:10:401
25905993122,3cyclictest181rcu_preempt23:10:401
25905993122,3cyclictest181rcu_preempt23:10:401
25905993118,4cyclictest181rcu_preempt23:15:581
25905993118,4cyclictest181rcu_preempt23:15:581
25905993118,4cyclictest181rcu_preempt23:15:581
25899993123,6cyclictest30295-21needreboot23:43:270
25899993123,6cyclictest30295-21needreboot23:43:270
25905993024,4cyclictest545-21in:imuxsock19:43:091
25905993024,4cyclictest545-21in:imuxsock19:43:091
25905993024,4cyclictest545-21in:imuxsock19:43:091
25905993023,5cyclictest9993-21gltestperf21:23:251
25905993023,5cyclictest9993-21gltestperf21:23:251
25905993023,5cyclictest9993-21gltestperf21:23:251
25905993023,5cyclictest20545-21gltestperf23:18:241
25905993023,5cyclictest20545-21gltestperf23:18:241
25905993023,5cyclictest20545-21gltestperf23:18:241
25905993022,6cyclictest30348-21sshd20:50:511
25905993022,6cyclictest30348-21sshd20:50:511
25905993022,6cyclictest30348-21sshd20:50:511
25905993019,2cyclictest181rcu_preempt20:54:031
25905993019,2cyclictest181rcu_preempt20:54:031
25899993024,3cyclictest0-21swapper/022:33:100
25899993024,3cyclictest0-21swapper/022:33:100
25899993023,5cyclictest14594-21kernelversion00:28:290
25899993023,5cyclictest14594-21kernelversion00:28:290
229662300,3chrt0-21swapper/323:23:313
229662300,3chrt0-21swapper/323:23:313
229662300,3chrt0-21swapper/323:23:313
41872290,3sleep30-21swapper/321:08:243
41872290,3sleep30-21swapper/321:08:243
41872290,3sleep30-21swapper/321:08:243
2590599296,19cyclictest281ktimers/122:07:531
2590599296,19cyclictest281ktimers/122:07:531
2590599296,19cyclictest281ktimers/122:07:531
25905992923,4cyclictest23944-21latency_hist23:28:191
25905992923,4cyclictest23944-21latency_hist23:28:191
25905992923,4cyclictest23944-21latency_hist23:28:191
25905992922,5cyclictest18315-21kernelversion00:38:251
25905992922,5cyclictest18315-21kernelversion00:38:251
25905992922,5cyclictest18315-21kernelversion00:38:251
25905992922,4cyclictest181rcu_preempt23:45:381
25905992922,4cyclictest181rcu_preempt23:45:381
25905992921,2cyclictest181rcu_preempt23:53:271
25905992921,2cyclictest181rcu_preempt23:53:271
25905992920,7cyclictest31629-21fschecks_count23:48:251
25905992920,7cyclictest31629-21fschecks_count23:48:251
25905992920,7cyclictest31629-21fschecks_count23:48:251
25905992911,3cyclictest0-21swapper/119:53:231
25905992911,3cyclictest0-21swapper/119:53:231
25899992922,5cyclictest0-21swapper/019:43:090
25899992922,5cyclictest0-21swapper/019:43:090
25899992922,5cyclictest0-21swapper/019:43:090
158782290,3sleep30-21swapper/300:33:183
158782290,3sleep30-21swapper/300:33:183
25905992823,3cyclictest0-21swapper/123:43:101
25905992823,3cyclictest0-21swapper/123:43:101
25905992822,5cyclictest14389-21threads21:33:341
25905992822,5cyclictest14389-21threads21:33:341
25905992822,5cyclictest14389-21threads21:33:341
25905992822,5cyclictest12530-21gltestperf00:23:241
25905992822,5cyclictest12530-21gltestperf00:23:241
25905992822,5cyclictest12530-21gltestperf00:23:241
25905992822,4cyclictest11227-21gltestperf22:53:241
25905992814,4cyclictest181rcu_preempt22:26:091
25905992814,4cyclictest181rcu_preempt22:26:091
25905992814,4cyclictest181rcu_preempt22:26:091
25905992814,4cyclictest181rcu_preempt22:10:501
25905992814,4cyclictest181rcu_preempt22:10:501
25905992814,4cyclictest181rcu_preempt22:10:501
25905992812,3cyclictest25882-21cyclictest20:44:211
25905992812,3cyclictest25882-21cyclictest20:44:211
25899992821,5cyclictest5403-21fschecks_count22:38:210
25899992821,5cyclictest5403-21fschecks_count22:38:210
25899992821,5cyclictest16281-21sshd23:07:510
25899992821,5cyclictest16281-21sshd23:07:510
25899992821,5cyclictest11803-21gltestperf21:28:230
25899992821,4cyclictest0-21swapper/020:43:210
25899992815,11cyclictest0-21swapper/020:48:220
101832280,2sleep30-21swapper/322:48:333
101832280,2sleep30-21swapper/322:48:333
101832280,2sleep30-21swapper/322:48:333
318432270,3sleep20-21swapper/223:48:262
318432270,3sleep20-21swapper/223:48:262
318432270,3sleep20-21swapper/223:48:262
25905992723,3cyclictest12349-21sendmail_mailqu21:28:311
25905992723,3cyclictest12349-21sendmail_mailqu21:28:311
25905992722,3cyclictest90-1pr/ttyS019:29:301
25905992722,3cyclictest90-1pr/ttyS019:29:301
25905992721,5cyclictest20246-21ntp_offset20:23:301
25905992721,5cyclictest20246-21ntp_offset20:23:301
25905992721,5cyclictest20246-21ntp_offset20:23:301
25905992721,4cyclictest0-21swapper/122:18:101
25905992721,4cyclictest0-21swapper/122:18:101
25905992720,2cyclictest181rcu_preempt22:33:101
25905992720,2cyclictest181rcu_preempt22:33:101
25905992719,6cyclictest5974-21taskset22:38:271
25905992719,6cyclictest5974-21taskset22:38:271
25905992719,6cyclictest25882-21cyclictest20:18:211
25905992719,6cyclictest25882-21cyclictest20:18:211
25905992719,6cyclictest22220-21awk20:28:281
25905992719,6cyclictest22220-21awk20:28:281
25905992719,6cyclictest22220-21awk20:28:281
25905992719,5cyclictest281ktimers/123:37:071
25905992719,5cyclictest281ktimers/123:37:071
25905992716,2cyclictest181rcu_preempt22:36:211
25899992721,5cyclictest17394-21latency_hist20:18:210
25899992721,5cyclictest17394-21latency_hist20:18:210
25899992721,3cyclictest0-21swapper/023:28:350
25899992721,3cyclictest0-21swapper/023:28:350
25899992720,5cyclictest7813-21sshd19:50:420
25899992720,5cyclictest7813-21sshd19:50:420
25899992720,5cyclictest6507-21needreboot21:13:260
25899992720,5cyclictest6507-21needreboot21:13:260
25899992720,5cyclictest6507-21needreboot21:13:260
25899992720,5cyclictest15503-21sshd20:12:510
25899992720,5cyclictest15503-21sshd20:12:510
25899992720,5cyclictest15503-21sshd20:12:510
25899992719,6cyclictest25882-21cyclictest21:23:350
25899992719,6cyclictest25882-21cyclictest21:23:350
25899992718,7cyclictest31300-21sort22:18:330
25899992718,7cyclictest31300-21sort22:18:330
25899992718,7cyclictest11132-21threads19:58:310
25899992718,7cyclictest11132-21threads19:58:310
25899992713,4cyclictest90-1pr/ttyS000:21:220
25899992713,4cyclictest90-1pr/ttyS000:21:220
25899992713,4cyclictest90-1pr/ttyS000:21:220
65112260,3sleep30-21swapper/321:13:263
65112260,3sleep30-21swapper/321:13:263
65112260,3sleep30-21swapper/321:13:263
2590599265,3cyclictest281ktimers/123:32:071
2590599265,3cyclictest281ktimers/123:32:071
2590599263,10cyclictest29-21ksoftirqd/121:44:411
2590599263,10cyclictest29-21ksoftirqd/121:44:411
2590599262,4cyclictest281ktimers/120:13:321
25905992619,6cyclictest14369-21fschecks_time00:28:271
25905992619,6cyclictest14369-21fschecks_time00:28:271
25905992618,6cyclictest21173-21gltestperf21:53:231
25905992618,6cyclictest21173-21gltestperf21:53:231
25905992618,6cyclictest21173-21gltestperf21:53:231
25905992613,7cyclictest90-1pr/ttyS021:08:511
25905992613,7cyclictest90-1pr/ttyS021:08:511
25905992613,7cyclictest90-1pr/ttyS021:08:511
25905992613,3cyclictest181rcu_preempt23:03:261
25905992613,3cyclictest181rcu_preempt23:03:261
25899992621,4cyclictest14678-21systemd-journal20:25:480
25899992621,4cyclictest14678-21systemd-journal20:25:480
25899992621,4cyclictest14678-21systemd-journal20:25:480
25899992621,3cyclictest749-21runrttasks00:38:100
25899992621,3cyclictest749-21runrttasks00:38:100
25899992621,3cyclictest0-21swapper/023:38:240
25899992621,3cyclictest0-21swapper/023:38:240
25899992620,4cyclictest0-21swapper/020:23:100
25899992620,4cyclictest0-21swapper/020:23:100
25899992620,4cyclictest0-21swapper/020:23:100
25899992619,5cyclictest28875-21latency_hist20:48:210
25899992619,5cyclictest28456-21fschecks_count22:13:210
25899992619,5cyclictest28456-21fschecks_count22:13:210
25899992619,5cyclictest28456-21fschecks_count22:13:210
25899992619,5cyclictest12861-21sshd21:29:500
25899992619,5cyclictest12861-21sshd21:29:500
25899992619,4cyclictest0-21swapper/022:58:250
25899992619,4cyclictest0-21swapper/022:58:250
25899992619,4cyclictest0-21swapper/022:58:250
25899992618,7cyclictest26654-21fschecks_time22:08:220
25899992618,7cyclictest26654-21fschecks_time22:08:220
25899992618,7cyclictest26654-21fschecks_time22:08:220
25899992615,9cyclictest8721-21threads21:18:340
25899992615,9cyclictest8721-21threads21:18:340
245312260,2chrt24528-21cat22:03:200
245312260,2chrt24528-21cat22:03:200
245312260,2chrt24528-21cat22:03:200
2590599252,19cyclictest281ktimers/121:49:041
2590599252,19cyclictest281ktimers/121:49:041
2590599252,19cyclictest281ktimers/121:49:041
25905992521,2cyclictest0-21swapper/100:33:311
25905992521,2cyclictest0-21swapper/100:33:311
25905992520,3cyclictest23592-21gltestperf20:33:241
25905992520,3cyclictest23592-21gltestperf20:33:241
25905992520,3cyclictest23592-21gltestperf20:33:241
25905992519,4cyclictest271rcuc/121:18:281
25905992519,4cyclictest271rcuc/121:18:281
25905992519,4cyclictest25882-21cyclictest21:05:131
25905992519,4cyclictest25882-21cyclictest21:05:131
25905992519,4cyclictest25882-21cyclictest21:05:131
25899992520,3cyclictest0-21swapper/023:13:100
25899992520,3cyclictest0-21swapper/023:13:100
25899992520,3cyclictest0-21swapper/023:13:100
25899992519,5cyclictest22619-21unixbench_singl20:28:310
25899992519,5cyclictest22619-21unixbench_singl20:28:310
25899992519,5cyclictest22619-21unixbench_singl20:28:310
25899992519,4cyclictest0-21swapper/020:53:320
25899992519,4cyclictest0-21swapper/020:53:320
25899992518,5cyclictest4809-21gltestperf19:43:230
25899992518,5cyclictest4809-21gltestperf19:43:230
25899992518,5cyclictest4809-21gltestperf19:43:230
25899992517,6cyclictest3379-21/usr/sbin/munin22:33:280
25899992517,6cyclictest2568-21latency_hist23:58:110
25899992517,6cyclictest2568-21latency_hist23:58:110
25899992513,4cyclictest90-1pr/ttyS021:33:500
25899992513,4cyclictest90-1pr/ttyS021:33:500
25899992513,4cyclictest90-1pr/ttyS021:33:500
25917992418,3cyclictest248612sleep322:03:233
25917992418,3cyclictest248612sleep322:03:233
25905992416,2cyclictest181rcu_preempt21:13:301
25905992416,2cyclictest181rcu_preempt21:13:301
25905992416,2cyclictest181rcu_preempt21:13:301
25905992415,2cyclictest181rcu_preempt00:13:211
25905992415,2cyclictest181rcu_preempt00:13:211
2590599241,12cyclictest29-21ksoftirqd/119:38:141
2590599241,12cyclictest29-21ksoftirqd/119:38:141
25899992419,3cyclictest0-21swapper/022:18:230
25899992419,3cyclictest0-21swapper/022:18:230
25899992418,4cyclictest25882-21cyclictest20:38:240
25899992416,6cyclictest30739-21latency_hist20:53:200
25899992416,6cyclictest30739-21latency_hist20:53:200
2589999241,3cyclictest0-21swapper/022:49:060
2589999241,3cyclictest0-21swapper/022:49:060
2589999241,3cyclictest0-21swapper/022:49:060
25899992412,10cyclictest23787-21cron23:28:100
25899992412,10cyclictest23787-21cron23:28:100
25899992412,10cyclictest23787-21cron23:28:100
25905992313,4cyclictest90-1pr/ttyS020:58:341
25905992313,4cyclictest90-1pr/ttyS020:58:341
25905992313,4cyclictest90-1pr/ttyS020:58:341
25899992319,2cyclictest0-21swapper/022:48:090
25899992319,2cyclictest0-21swapper/022:48:090
25899992319,2cyclictest0-21swapper/022:48:090
25899992319,2cyclictest0-21swapper/020:48:260
25899992318,3cyclictest90-1pr/ttyS021:38:340
25899992318,3cyclictest90-1pr/ttyS021:38:340
25899992318,3cyclictest90-1pr/ttyS021:38:340
25899992318,3cyclictest0-21swapper/020:13:320
25899992317,4cyclictest0-21swapper/019:33:230
25899992317,4cyclictest0-21swapper/019:33:230
25899992317,4cyclictest0-21swapper/000:23:330
25899992317,4cyclictest0-21swapper/000:23:330
25899992317,4cyclictest0-21swapper/000:23:330
25899992317,3cyclictest0-21swapper/023:13:280
25899992317,3cyclictest0-21swapper/023:13:280
25899992317,3cyclictest0-21swapper/023:13:280
2589999231,19cyclictest22588-21latency_hist21:58:160
2589999231,19cyclictest22588-21latency_hist21:58:160
2589999231,19cyclictest22588-21latency_hist21:58:160
2589999231,12cyclictest20221-21/usr/sbin/munin23:18:300
2589999231,12cyclictest20221-21/usr/sbin/munin23:18:300
2589999231,12cyclictest20221-21/usr/sbin/munin23:18:300
25899992215,4cyclictest0-21swapper/021:03:340
25899992215,4cyclictest0-21swapper/021:03:340
25899992215,4cyclictest0-21swapper/021:03:340
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional