You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 22:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Tue Jul 21, 2026 12:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1566921030,3chrt0-21swapper/007:45:460
1566921030,3chrt0-21swapper/007:45:460
1566921030,3chrt0-21swapper/007:45:460
141532750,2chrt0-21swapper/007:43:330
141532750,2chrt0-21swapper/007:43:330
249802720,4chrt0-21swapper/311:03:433
249802720,4chrt0-21swapper/311:03:433
101912690,2sleep0171ktimers/010:23:440
101912690,2sleep0171ktimers/010:23:440
101912690,2sleep0171ktimers/010:23:440
249902630,2sleep024994-21sed12:28:500
249902630,2sleep024994-21sed12:28:500
169632620,2sleep016966-21latency_hist10:43:380
169632620,2sleep016966-21latency_hist10:43:380
327472600,3sleep30-21swapper/309:58:433
327472600,3sleep30-21swapper/309:58:433
92562560,3sleep30-21swapper/311:48:413
92562560,3sleep30-21swapper/311:48:413
226302530,2sleep30-21swapper/312:23:433
226302530,2sleep30-21swapper/312:23:433
132552490,3sleep30-21swapper/311:58:463
132552490,3sleep30-21swapper/311:58:463
132552490,3sleep30-21swapper/311:58:463
247192460,2sleep124723-21tail11:03:411
247192460,2sleep124723-21tail11:03:411
96052450,3sleep00-21swapper/011:48:460
96052450,3sleep00-21swapper/011:48:460
82432440,3chrt0-21swapper/011:43:520
82432440,3chrt0-21swapper/011:43:520
82432440,3chrt0-21swapper/011:43:520
64302430,4sleep00-21swapper/010:13:470
64302430,4sleep00-21swapper/010:13:470
64302430,4sleep00-21swapper/010:13:470
271332430,1sleep027126-21idleruntime-cro08:18:260
271332430,1sleep027126-21idleruntime-cro08:18:260
4192420,3sleep30-21swapper/311:23:493
4192420,3sleep30-21swapper/311:23:493
134392420,3chrt0-21swapper/310:33:403
207442410,1sleep020747-21tail12:18:460
207442410,1sleep020747-21tail12:18:460
40602390,2sleep10-21swapper/111:33:481
40602390,2sleep10-21swapper/111:33:481
3148993911,25cyclictest9353-21ntp_states07:28:481
3148993911,25cyclictest9353-21ntp_states07:28:481
276752390,3chrt0-21swapper/012:38:390
314899387,6cyclictest20557-21ntp_kernel_pll_07:58:461
314899382,9cyclictest5664-21/usr/sbin/munin10:13:481
314899382,9cyclictest5664-21/usr/sbin/munin10:13:481
314899382,9cyclictest5664-21/usr/sbin/munin10:13:481
314899381,9cyclictest6804-21wc08:48:441
314899381,9cyclictest6804-21wc08:48:441
314899381,9cyclictest6804-21wc08:48:441
314899381,11cyclictest500-21/usr/sbin/munin08:33:411
314899381,11cyclictest500-21/usr/sbin/munin08:33:411
314899381,11cyclictest13109-21latency11:58:441
314899381,11cyclictest13109-21latency11:58:441
314899381,11cyclictest13109-21latency11:58:441
314899381,10cyclictest311-21gltestperf09:58:431
314899381,10cyclictest311-21gltestperf09:58:431
314899381,10cyclictest29174-21awk09:48:421
314899381,10cyclictest29174-21awk09:48:421
314899381,10cyclictest29174-21awk09:48:421
3148993716,19cyclictest0-21swapper/112:03:521
3148993716,19cyclictest0-21swapper/112:03:521
3148993716,19cyclictest0-21swapper/112:03:521
314899371,12cyclictest14977-21ntp_offset07:43:461
314899371,12cyclictest14977-21ntp_offset07:43:461
314899371,12cyclictest14977-21ntp_offset07:43:461
314899371,11cyclictest19103-21/usr/sbin/munin10:48:391
314899371,11cyclictest19103-21/usr/sbin/munin10:48:391
314899371,11cyclictest19103-21/usr/sbin/munin10:48:391
314899371,11cyclictest15348-21grep10:38:401
314899371,11cyclictest15348-21grep10:38:401
3148993629,5cyclictest18239-21ntp_kernel_pll_09:18:481
3148993629,5cyclictest18239-21ntp_kernel_pll_09:18:481
3148993629,5cyclictest16757-21needreboot07:48:451
3148993629,5cyclictest16757-21needreboot07:48:451
3148993628,6cyclictest22987-21threads12:23:481
3148993628,6cyclictest22987-21threads12:23:481
314899361,9cyclictest2325-21ls11:28:521
314899361,9cyclictest2325-21ls11:28:521
3148993614,20cyclictest0-21swapper/108:43:461
3148993614,20cyclictest0-21swapper/108:43:461
3148993614,20cyclictest0-21swapper/108:43:461
314899361,13cyclictest18712-21ntp_states07:53:441
314899361,13cyclictest18712-21ntp_states07:53:441
314899361,12cyclictest9588-21uniq10:23:381
314899361,12cyclictest9588-21uniq10:23:381
314899361,12cyclictest9588-21uniq10:23:381
314899361,11cyclictest5279-21fschecks_count11:38:401
314899361,11cyclictest5279-21fschecks_count11:38:401
314899361,11cyclictest4587-21ps10:08:471
314899361,11cyclictest4587-21ps10:08:471
314899361,11cyclictest15144-21cat10:38:381
314899361,11cyclictest13995-21latency_hist07:43:361
314899361,11cyclictest13995-21latency_hist07:43:361
314899361,10cyclictest6717-21/usr/sbin/munin07:23:491
314899361,10cyclictest31776-21needreboot08:28:461
314899361,10cyclictest31776-21needreboot08:28:461
314899361,10cyclictest29330-21grep11:13:501
314899361,10cyclictest29330-21grep11:13:501
314899361,10cyclictest22082-21head08:03:421
314899361,10cyclictest22082-21head08:03:421
314899361,10cyclictest13594-21/usr/sbin/munin09:08:371
314899361,10cyclictest13594-21/usr/sbin/munin09:08:371
314899361,10cyclictest13594-21/usr/sbin/munin09:08:371
314899360,9cyclictest2673-21ntp_states10:03:471
314899360,9cyclictest2673-21ntp_states10:03:471
3148993529,4cyclictest1346-21avahi-daemon09:03:271
3148993529,4cyclictest1346-21avahi-daemon09:03:271
314899351,9cyclictest26747-21timerandwakeup12:33:511
314899351,9cyclictest26747-21timerandwakeup12:33:511
314899351,9cyclictest26747-21timerandwakeup12:33:511
314899351,9cyclictest17126-21open_inodes12:08:471
314899351,9cyclictest17126-21open_inodes12:08:471
314899351,8cyclictest31853-21sed09:53:511
314899351,8cyclictest31853-21sed09:53:511
314899351,8cyclictest21595-21gltestperf09:28:411
314899351,8cyclictest21595-21gltestperf09:28:411
314899351,8cyclictest21595-21gltestperf09:28:411
314899351,8cyclictest19961-21ls09:23:431
314899351,8cyclictest19961-21ls09:23:431
314899351,8cyclictest19961-21ls09:23:431
314899351,12cyclictest20616-21sshd12:18:441
314899351,12cyclictest20616-21sshd12:18:441
314899351,12cyclictest14598-21cut09:08:471
314899351,12cyclictest14598-21cut09:08:471
314899351,12cyclictest14598-21cut09:08:471
314899351,11cyclictest8555-21timerandwakeup10:18:501
314899351,11cyclictest8555-21timerandwakeup10:18:501
314899351,11cyclictest7622-21missed_timers11:43:451
314899351,11cyclictest7622-21missed_timers11:43:451
314899351,11cyclictest7622-21missed_timers11:43:451
314899351,11cyclictest749-21runrttasks09:14:181
314899351,11cyclictest749-21runrttasks09:14:181
314899351,11cyclictest749-21runrttasks09:14:181
314899351,11cyclictest3266-21grep08:38:491
314899351,11cyclictest23504-21idleruntime09:33:411
314899351,11cyclictest23504-21idleruntime09:33:411
314899351,11cyclictest23504-21idleruntime09:33:411
314899351,11cyclictest23444-21awk10:58:471
314899351,11cyclictest23444-21awk10:58:471
314899351,11cyclictest21232-21latency10:53:421
314899351,11cyclictest21232-21latency10:53:421
314899351,10cyclictest24336-21grep12:28:421
314899351,10cyclictest24336-21grep12:28:421
314899351,10cyclictest18717-21kernelversion12:13:441
314899351,10cyclictest18717-21kernelversion12:13:441
3148993510,23cyclictest9059-21sort08:53:501
3148993510,23cyclictest9059-21sort08:53:501
3143993528,5cyclictest32601-21netstat11:23:450
3143993528,5cyclictest32601-21netstat11:23:450
314899341,9cyclictest9334-21/usr/sbin/munin11:48:421
314899341,9cyclictest9334-21/usr/sbin/munin11:48:421
314899341,9cyclictest27051-21open_inodes11:08:481
314899341,9cyclictest27051-21open_inodes11:08:481
314899341,9cyclictest27051-21open_inodes11:08:481
314899341,9cyclictest18747-21/usr/sbin/munin10:48:441
314899341,9cyclictest18747-21/usr/sbin/munin10:48:441
314899341,9cyclictest18747-21/usr/sbin/munin10:48:441
314899341,7cyclictest29364-21forks08:23:411
314899341,7cyclictest29364-21forks08:23:411
314899341,7cyclictest29364-21forks08:23:411
3148993413,19cyclictest0-21swapper/108:18:411
3148993413,19cyclictest0-21swapper/108:18:411
3148993413,19cyclictest0-21swapper/108:18:411
314899341,11cyclictest30466-21if_wlan011:18:431
314899341,11cyclictest28282-21ntp_kernel_pll_12:38:451
314899341,11cyclictest28282-21ntp_kernel_pll_12:38:451
314899341,11cyclictest26625-21cron09:43:271
314899341,11cyclictest26625-21cron09:43:271
314899341,11cyclictest26625-21cron09:43:271
314899341,11cyclictest16323-21sed10:38:501
314899341,11cyclictest16323-21sed10:38:501
314899341,11cyclictest12519-21grep10:28:521
314899341,11cyclictest12519-21grep10:28:521
314899341,11cyclictest12519-21grep10:28:521
314899341,10cyclictest23575-21/usr/sbin/munin08:08:431
314899341,10cyclictest23575-21/usr/sbin/munin08:08:431
3143993422,10cyclictest0-21swapper/008:23:440
3143993422,10cyclictest0-21swapper/008:23:440
3143993422,10cyclictest0-21swapper/008:23:440
314899331,7cyclictest12671-21/usr/sbin/munin11:58:401
314899331,11cyclictest25396-21/usr/sbin/munin08:13:451
314899331,11cyclictest25396-21/usr/sbin/munin08:13:451
314899330,11cyclictest27724-21awk09:43:471
314899330,11cyclictest27724-21awk09:43:471
314899330,11cyclictest27724-21awk09:43:471
3143993327,4cyclictest3131-21cyclictest08:03:270
3143993326,5cyclictest16375-21ssh09:13:460
3143993326,5cyclictest16375-21ssh09:13:460
3143993326,5cyclictest16375-21ssh09:13:460
30112330,2sleep00-21swapper/010:03:500
30112330,2sleep00-21swapper/010:03:500
315599321,3cyclictest0-21swapper/211:22:412
314899321,10cyclictest32474-21latency11:23:431
314899321,10cyclictest32474-21latency11:23:431
314899321,10cyclictest11400-21needreboot11:53:451
314899321,10cyclictest11400-21needreboot11:53:451
314899320,8cyclictest11349-21mailstats07:33:481
314899320,8cyclictest11349-21mailstats07:33:481
180342320,2sleep20-21swapper/207:53:362
180342320,2sleep20-21swapper/207:53:362
315599311,3cyclictest0-21swapper/208:01:332
3143993126,4cyclictest4573-21gltestperf08:43:410
3143993126,4cyclictest4573-21gltestperf08:43:410
3143993126,4cyclictest4573-21gltestperf08:43:410
3143993125,4cyclictest5374-21gltestperf11:38:410
3143993125,4cyclictest5374-21gltestperf11:38:410
3143993125,3cyclictest4845-21idleruntime-cro11:38:270
3143993125,3cyclictest4845-21idleruntime-cro11:38:270
3143993123,6cyclictest11022-21grep11:53:410
3143993123,6cyclictest11022-21grep11:53:410
3143993123,6cyclictest11022-21grep11:53:410
315599301,3cyclictest0-21swapper/208:51:342
315599301,3cyclictest0-21swapper/208:51:342
315599301,3cyclictest0-21swapper/208:51:342
3143993024,4cyclictest25879-21fschecks_count12:33:420
3143993024,4cyclictest25879-21fschecks_count12:33:420
3143993024,4cyclictest20456-21sh10:53:270
3143993024,4cyclictest20456-21sh10:53:270
3143993024,4cyclictest20456-21sh10:53:270
3143993022,6cyclictest25095-21needreboot11:03:450
3143993022,6cyclictest25095-21needreboot11:03:450
3143993022,6cyclictest11660-21unixbench_multi07:33:510
3143993022,6cyclictest11660-21unixbench_multi07:33:510
160502300,2sleep016158-21latency09:13:430
160502300,2sleep016158-21latency09:13:430
160502300,2sleep016158-21latency09:13:430
315599291,3cyclictest0-21swapper/210:46:242
315599291,3cyclictest0-21swapper/210:46:242
315599291,3cyclictest0-21swapper/210:46:242
315599291,3cyclictest0-21swapper/210:19:072
315599291,3cyclictest0-21swapper/210:19:072
3143992924,3cyclictest31656-21wc11:23:270
269502290,3sleep30-21swapper/311:08:463
269502290,3sleep30-21swapper/311:08:463
269502290,3sleep30-21swapper/311:08:463
315599281,3cyclictest0-21swapper/209:25:452
315599281,3cyclictest0-21swapper/209:25:452
315599281,3cyclictest0-21swapper/209:25:452
315599281,3cyclictest0-21swapper/208:26:182
315599281,3cyclictest0-21swapper/208:26:182
315599281,3cyclictest0-21swapper/208:26:182
315599281,3cyclictest0-21swapper/207:24:482
3143992824,3cyclictest21170-21kernelversion10:53:420
3143992824,3cyclictest21170-21kernelversion10:53:420
3143992824,2cyclictest0-21swapper/008:48:500
3143992824,2cyclictest0-21swapper/008:48:500
3143992824,2cyclictest0-21swapper/008:48:500
3143992822,5cyclictest19460-21cpu09:23:380
3143992822,5cyclictest19460-21cpu09:23:380
3143992822,5cyclictest19460-21cpu09:23:380
3143992821,5cyclictest16103-21/usr/sbin/munin07:48:440
3143992821,5cyclictest16103-21/usr/sbin/munin07:48:440
3143992821,5cyclictest15442-21latency_hist10:38:400
3143992820,6cyclictest18383-21cut12:13:410
3143992820,6cyclictest18383-21cut12:13:410
3143992820,6cyclictest14295-21basename12:03:270
3143992820,6cyclictest14295-21basename12:03:270
3143992820,6cyclictest14295-21basename12:03:270
3143992815,11cyclictest0-21swapper/008:03:410
3143992815,11cyclictest0-21swapper/008:03:410
315599271,3cyclictest0-21swapper/211:55:012
315599271,3cyclictest0-21swapper/211:55:012
315599271,3cyclictest0-21swapper/211:55:012
315599271,3cyclictest0-21swapper/211:04:172
315599271,3cyclictest0-21swapper/211:04:172
315599271,3cyclictest0-21swapper/209:38:052
315599271,3cyclictest0-21swapper/209:38:052
315599271,3cyclictest0-21swapper/209:38:052
315599271,3cyclictest0-21swapper/209:16:162
315599271,3cyclictest0-21swapper/209:16:162
315599271,3cyclictest0-21swapper/209:16:162
315599271,3cyclictest0-21swapper/208:39:042
315599271,3cyclictest0-21swapper/208:33:342
315599271,3cyclictest0-21swapper/208:33:342
315599271,2cyclictest0-21swapper/210:09:362
315599271,2cyclictest0-21swapper/210:09:362
3143992722,4cyclictest14780-21gltestperf12:03:420
3143992722,4cyclictest14780-21gltestperf12:03:420
3143992722,4cyclictest14780-21gltestperf12:03:420
3143992722,3cyclictest24179-21latency_hist11:03:260
3143992722,3cyclictest24179-21latency_hist11:03:260
3143992722,3cyclictest0-21swapper/012:43:380
3143992722,3cyclictest0-21swapper/012:43:380
3143992720,5cyclictest28642-21kernelversion11:13:430
3143992720,5cyclictest28642-21kernelversion11:13:430
3143992720,5cyclictest11549-21cut10:28:410
3143992720,5cyclictest11549-21cut10:28:410
3143992720,5cyclictest11549-21cut10:28:410
3143992720,4cyclictest0-21swapper/012:13:340
3143992720,4cyclictest0-21swapper/012:13:340
3143992719,7cyclictest10219-21gltestperf08:58:410
3143992719,7cyclictest10219-21gltestperf08:58:410
315599261,3cyclictest0-21swapper/211:59:132
315599261,3cyclictest0-21swapper/211:59:132
315599261,3cyclictest0-21swapper/211:59:132
315599261,3cyclictest0-21swapper/211:49:082
315599261,3cyclictest0-21swapper/211:49:082
315599261,3cyclictest0-21swapper/211:37:572
315599261,3cyclictest0-21swapper/211:37:572
315599261,3cyclictest0-21swapper/211:28:092
315599261,3cyclictest0-21swapper/211:28:092
315599261,3cyclictest0-21swapper/210:35:222
315599261,3cyclictest0-21swapper/210:35:222
315599261,3cyclictest0-21swapper/210:35:222
3143992621,3cyclictest0-21swapper/008:23:260
3143992621,3cyclictest0-21swapper/008:23:260
3143992621,3cyclictest0-21swapper/008:23:260
3143992619,5cyclictest8814-21cut07:28:420
3143992619,5cyclictest8814-21cut07:28:420
3143992619,5cyclictest24286-21ntp_kernel_err08:08:470
3143992619,5cyclictest24286-21ntp_kernel_err08:08:470
3143992618,6cyclictest26928-21ls09:43:380
3143992618,6cyclictest26928-21ls09:43:380
3143992618,6cyclictest26928-21ls09:43:380
141102260,4sleep30-21swapper/310:33:483
141102260,4sleep30-21swapper/310:33:483
141102260,4sleep30-21swapper/310:33:483
315599251,3cyclictest0-21swapper/210:49:492
315599251,3cyclictest0-21swapper/210:49:492
315599251,3cyclictest0-21swapper/210:49:492
315599251,3cyclictest0-21swapper/208:44:412
315599251,3cyclictest0-21swapper/208:44:412
315599251,3cyclictest0-21swapper/208:44:412
315599251,3cyclictest0-21swapper/207:49:042
315599251,3cyclictest0-21swapper/207:49:042
315599251,3cyclictest0-21swapper/207:44:272
315599251,3cyclictest0-21swapper/207:44:272
315599251,3cyclictest0-21swapper/207:44:272
315599251,3cyclictest0-21swapper/207:31:152
315599251,3cyclictest0-21swapper/207:31:152
3143992521,2cyclictest0-21swapper/009:48:470
3143992521,2cyclictest0-21swapper/009:48:470
3143992521,2cyclictest0-21swapper/009:48:470
3143992520,4cyclictest30958-21cut09:53:420
3143992520,4cyclictest30958-21cut09:53:420
3143992520,3cyclictest0-21swapper/010:33:490
3143992520,3cyclictest0-21swapper/010:33:490
3143992519,5cyclictest18249-21ntp_offset09:18:480
3143992519,5cyclictest18249-21ntp_offset09:18:480
3143992519,3cyclictest0-21swapper/009:28:490
3143992519,3cyclictest0-21swapper/009:28:490
3143992519,3cyclictest0-21swapper/009:28:490
3143992519,3cyclictest0-21swapper/008:33:270
3143992519,3cyclictest0-21swapper/008:33:270
3143992518,5cyclictest28104-21latency_hist11:13:380
3143992518,5cyclictest28104-21latency_hist11:13:380
3143992518,5cyclictest28104-21latency_hist11:13:380
3143992511,12cyclictest14678-21systemd-journal09:08:270
3143992511,12cyclictest14678-21systemd-journal09:08:270
3143992511,12cyclictest14678-21systemd-journal09:08:270
315599249,12cyclictest296642chrt11:17:012
315599249,12cyclictest296642chrt11:17:012
315599247,14cyclictest286532chrt08:19:132
315599247,14cyclictest286532chrt08:19:132
315599247,14cyclictest286532chrt08:19:132
315599245,14cyclictest138282chrt07:41:222
315599245,14cyclictest138282chrt07:41:222
315599244,16cyclictest0-21swapper/212:37:532
315599244,16cyclictest0-21swapper/212:37:532
315599244,16cyclictest0-21swapper/212:37:532
315599242,5cyclictest0-21swapper/211:28:402
315599242,5cyclictest0-21swapper/211:28:402
315599241,3cyclictest0-21swapper/212:25:262
315599241,3cyclictest0-21swapper/212:25:262
315599241,3cyclictest0-21swapper/212:15:572
315599241,3cyclictest0-21swapper/212:15:572
315599241,3cyclictest0-21swapper/212:11:552
315599241,3cyclictest0-21swapper/212:11:552
315599241,3cyclictest0-21swapper/210:42:172
315599241,3cyclictest0-21swapper/210:42:172
315599241,3cyclictest0-21swapper/208:37:342
315599241,3cyclictest0-21swapper/208:37:342
315599241,3cyclictest0-21swapper/208:05:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional