You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 23:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Thu Jul 23, 2026 12:44:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
107392710,3sleep30-21swapper/309:03:513
107392710,3sleep30-21swapper/309:03:513
147222690,3chrt0-21swapper/309:13:523
147222690,3chrt0-21swapper/309:13:523
108002620,2sleep010799-21sh10:33:280
108002620,2sleep010799-21sh10:33:280
140792580,3sleep00-21swapper/009:13:450
140792580,3sleep00-21swapper/009:13:450
147282520,2sleep014739-21latency_hist10:43:390
147282520,2sleep014739-21latency_hist10:43:390
305112510,3sleep30-21swapper/311:23:493
305112510,3sleep30-21swapper/311:23:493
305112510,3sleep30-21swapper/311:23:493
305112510,3sleep30-21swapper/311:23:493
258882510,2chrt0-21swapper/009:43:520
258882510,2chrt0-21swapper/009:43:520
258882510,2chrt0-21swapper/009:43:520
9662480,2sleep30-21swapper/308:38:463
9662480,2sleep30-21swapper/308:38:463
160552440,3sleep30-21swapper/310:43:533
160552440,3sleep30-21swapper/310:43:533
158852420,3chrt0-21swapper/210:43:522
158852420,3chrt0-21swapper/210:43:522
1171994118,21cyclictest0-21swapper/009:53:440
1171994118,21cyclictest0-21swapper/009:53:440
1171994118,21cyclictest0-21swapper/009:53:440
1171994115,3cyclictest0-21swapper/011:43:460
1171994115,3cyclictest0-21swapper/011:43:460
1171994115,3cyclictest0-21swapper/011:43:460
1171994020,17cyclictest0-21swapper/009:08:420
1171994020,17cyclictest0-21swapper/009:08:420
1171994016,2cyclictest0-21swapper/009:40:500
1171994016,2cyclictest0-21swapper/009:40:500
1171994016,2cyclictest0-21swapper/009:40:500
117199401,15cyclictest13038-21grep10:38:410
122672390,3sleep30-21swapper/310:33:523
122672390,3sleep30-21swapper/310:33:523
1171993917,2cyclictest0-21swapper/009:28:500
1171993917,2cyclictest0-21swapper/009:28:500
1171993917,2cyclictest0-21swapper/009:28:500
1171993816,2cyclictest0-21swapper/010:53:280
1171993816,2cyclictest0-21swapper/010:53:280
1171993816,2cyclictest0-21swapper/010:53:280
1171993812,4cyclictest0-21swapper/010:13:420
1171993812,4cyclictest0-21swapper/010:13:420
1171993812,4cyclictest0-21swapper/010:13:420
162082370,3sleep30-21swapper/309:18:503
162082370,3sleep30-21swapper/309:18:503
66922360,2sleep06720-21awk08:53:480
66922360,2sleep06720-21awk08:53:480
262012360,2sleep025356-21/usr/sbin/munin12:38:480
262012360,2sleep025356-21/usr/sbin/munin12:38:480
117199363,8cyclictest59572latency10:18:450
1171993616,18cyclictest0-21swapper/010:43:450
1171993616,18cyclictest0-21swapper/010:43:450
1171993614,2cyclictest0-21swapper/008:58:520
1171993614,2cyclictest0-21swapper/008:58:520
117199361,11cyclictest31727-21fschecks_count11:28:420
117199361,11cyclictest31727-21fschecks_count11:28:420
117199361,11cyclictest31727-21fschecks_count11:28:420
117199361,11cyclictest31727-21fschecks_count11:28:420
1172993529,4cyclictest12083-21cron12:03:291
1172993529,4cyclictest12083-21cron12:03:291
1172993528,5cyclictest29345-21gltestperf08:28:421
1172993528,5cyclictest21790-21latency_hist12:28:401
1172993528,5cyclictest21790-21latency_hist12:28:401
1172993528,5cyclictest21790-21latency_hist12:28:401
1171993515,2cyclictest0-21swapper/012:23:420
1171993514,2cyclictest0-21swapper/008:18:420
1171993514,2cyclictest0-21swapper/008:18:420
117199351,10cyclictest10974-21/usr/sbin/munin10:33:460
266172340,2sleep026671-21netstat11:13:460
266172340,2sleep026671-21netstat11:13:460
266172340,2sleep026671-21netstat11:13:460
209262340,2sleep121051-21df09:33:411
209262340,2sleep121051-21df09:33:411
209262340,2sleep121051-21df09:33:411
1172993426,6cyclictest12656-21cron10:38:281
1172993426,6cyclictest12656-21cron10:38:281
188032330,2sleep0201rcuc/007:58:510
188032330,2sleep0201rcuc/007:58:510
1172993324,7cyclictest5893-21kernelversion10:18:451
1171993324,7cyclictest20859-21latency_hist10:58:430
117299327,5cyclictest281ktimers/111:40:061
117299327,5cyclictest281ktimers/111:40:061
1172993225,5cyclictest550-21cron08:13:301
1172993225,5cyclictest550-21cron08:13:301
1171993227,3cyclictest0-21swapper/011:03:290
1171993227,3cyclictest0-21swapper/011:03:290
1171993227,3cyclictest0-21swapper/011:03:290
1171993222,8cyclictest28042-21cut11:18:410
1171993222,8cyclictest28042-21cut11:18:410
1171993222,8cyclictest28042-21cut11:18:410
1171993219,11cyclictest0-21swapper/007:28:410
1171993219,11cyclictest0-21swapper/007:28:410
229312310,3sleep20-21swapper/211:03:462
229312310,3sleep20-21swapper/211:03:462
229312310,3sleep20-21swapper/211:03:462
229312310,3sleep20-21swapper/211:03:462
162222310,3sleep00-21swapper/012:13:430
162222310,3sleep00-21swapper/012:13:430
1172993124,5cyclictest17916-21needreboot09:23:461
1172993124,5cyclictest17916-21needreboot09:23:461
1172993123,5cyclictest550-21cron07:43:391
1172993123,5cyclictest550-21cron07:43:391
1172993122,7cyclictest19470-21switchtime10:53:511
1172993122,7cyclictest19470-21switchtime10:53:511
1172993122,7cyclictest19470-21switchtime10:53:511
1172993116,3cyclictest90-1pr/ttyS012:21:381
1172993116,3cyclictest90-1pr/ttyS012:21:381
1172993116,3cyclictest90-1pr/ttyS012:21:381
1171993124,6cyclictest25641-21fschecks_time12:38:400
1171993124,6cyclictest25641-21fschecks_time12:38:400
1171993124,6cyclictest25641-21fschecks_time12:38:400
1172993025,3cyclictest14547-21latency_hist10:43:281
1172993025,3cyclictest14547-21latency_hist10:43:281
1171993023,5cyclictest1110-21sshd10:04:530
1171993023,5cyclictest1110-21sshd10:04:530
1171993022,6cyclictest21974-21grep12:28:410
117299298,5cyclictest271rcuc/108:15:421
117299298,5cyclictest271rcuc/108:15:421
117299298,5cyclictest271rcuc/108:15:421
1172992923,4cyclictest9567-21latency_hist09:03:311
1172992923,4cyclictest9567-21latency_hist09:03:311
1172992923,4cyclictest281ktimers/111:29:361
1172992923,4cyclictest281ktimers/111:29:361
1172992923,4cyclictest281ktimers/111:29:361
1172992922,5cyclictest1-21systemd08:03:291
1172992922,5cyclictest1-21systemd08:03:291
1172992921,6cyclictest21978-21threads09:33:511
1172992921,6cyclictest21978-21threads09:33:511
1172992921,6cyclictest21978-21threads09:33:511
1171992922,5cyclictest550-21cron11:53:420
1171992922,5cyclictest550-21cron11:53:420
1171992922,5cyclictest550-21cron11:53:420
1171992922,5cyclictest550-21cron08:53:290
1171992922,5cyclictest550-21cron08:53:290
1171992922,5cyclictest550-21cron08:53:290
1171992916,10cyclictest29516-21idleruntime-cro11:23:280
1171992916,10cyclictest29516-21idleruntime-cro11:23:280
1171992916,10cyclictest29516-21idleruntime-cro11:23:280
1171992916,10cyclictest29516-21idleruntime-cro11:23:280
235302280,3sleep20-21swapper/208:13:412
235302280,3sleep20-21swapper/208:13:412
235302280,3sleep20-21swapper/208:13:412
233442280,4sleep3431rcuc/311:03:513
233442280,4sleep3431rcuc/311:03:513
233442280,4sleep3431rcuc/311:03:513
233442280,4sleep3431rcuc/311:03:513
117299285,13cyclictest28529-21taskset08:25:081
117299285,13cyclictest28529-21taskset08:25:081
1172992818,2cyclictest181rcu_preempt11:14:061
1172992818,2cyclictest181rcu_preempt11:14:061
1172992818,2cyclictest181rcu_preempt11:14:061
1172992815,2cyclictest181rcu_preempt11:56:571
1172992815,2cyclictest181rcu_preempt11:56:571
1172992815,2cyclictest181rcu_preempt11:56:571
1171992822,5cyclictest22701-21threads12:28:500
1171992822,5cyclictest22701-21threads12:28:500
1171992822,5cyclictest22701-21threads12:28:500
1171992821,5cyclictest1170-21cyclictest09:21:010
1171992821,5cyclictest1170-21cyclictest09:21:010
1171992819,7cyclictest31529-21/usr/sbin/munin11:28:470
1171992819,7cyclictest31529-21/usr/sbin/munin11:28:470
1171992819,7cyclictest31529-21/usr/sbin/munin11:28:470
215812270,3sleep20-21swapper/210:58:512
215812270,3sleep20-21swapper/210:58:512
215812270,3sleep20-21swapper/210:58:512
117299277,6cyclictest29-21ksoftirqd/108:48:491
117299277,6cyclictest29-21ksoftirqd/108:48:491
117299277,6cyclictest29-21ksoftirqd/108:48:491
117299276,4cyclictest281ktimers/110:59:351
117299276,4cyclictest281ktimers/110:59:351
117299276,4cyclictest281ktimers/110:59:351
117299275,9cyclictest749-21runrttasks08:38:311
117299275,9cyclictest749-21runrttasks08:38:311
117299275,4cyclictest281ktimers/111:34:361
117299275,4cyclictest281ktimers/111:34:361
1172992721,4cyclictest271rcuc/111:23:461
1172992721,4cyclictest271rcuc/111:23:461
1172992721,4cyclictest24868-21ntp_offset11:08:481
1172992721,4cyclictest24868-21ntp_offset11:08:481
1172992721,4cyclictest24868-21ntp_offset11:08:481
1172992720,5cyclictest31102-21processes09:58:471
1172992720,5cyclictest31102-21processes09:58:471
1172992720,5cyclictest27242-21head09:48:481
1172992720,5cyclictest27242-21head09:48:481
1172992720,5cyclictest27242-21head09:48:481
1172992720,5cyclictest10741-21threads09:03:511
1172992720,5cyclictest10741-21threads09:03:511
1172992719,6cyclictest9060-21missed_timers07:33:441
1172992719,6cyclictest9060-21missed_timers07:33:441
1171992722,3cyclictest14570-21cut12:08:440
1171992721,5cyclictest7406-21fschecks_count10:23:400
1171992721,5cyclictest7406-21fschecks_count10:23:400
1171992721,5cyclictest3414-21kernelversion11:38:450
1171992721,5cyclictest3414-21kernelversion11:38:450
1171992721,5cyclictest32063-21unixbench_multi08:33:510
1171992721,5cyclictest32063-21unixbench_multi08:33:510
1171992719,6cyclictest17772-21latency_hist07:58:360
117299268,3cyclictest281ktimers/108:41:231
117299268,3cyclictest281ktimers/108:41:231
117299268,3cyclictest271rcuc/107:40:551
117299268,3cyclictest271rcuc/107:40:551
117299266,4cyclictest281ktimers/112:38:421
117299266,4cyclictest281ktimers/112:38:421
1172992620,4cyclictest281ktimers/111:18:521
1172992620,4cyclictest281ktimers/111:18:521
1172992620,4cyclictest23852-21tail12:33:411
1172992620,4cyclictest10831-21latency_hist10:33:301
1172992620,4cyclictest10831-21latency_hist10:33:301
1172992618,6cyclictest16754-21needreboot12:13:481
1172992618,6cyclictest16754-21needreboot12:13:481
1171992621,4cyclictest21488-21latency_hist08:08:380
1171992621,4cyclictest21488-21latency_hist08:08:380
1171992620,4cyclictest1-21systemd10:03:290
1171992620,4cyclictest1-21systemd10:03:290
1171992620,3cyclictest0-21swapper/009:33:450
1171992620,3cyclictest0-21swapper/009:33:450
1171992620,3cyclictest0-21swapper/009:33:450
1171992619,5cyclictest15655-21sshd07:52:540
1171992619,5cyclictest15655-21sshd07:52:540
1171992619,5cyclictest1324-21threads08:38:520
1171992619,5cyclictest1324-21threads08:38:520
117299258,5cyclictest281ktimers/107:49:301
117299258,5cyclictest281ktimers/107:49:301
117299256,5cyclictest281ktimers/109:19:321
117299256,5cyclictest281ktimers/109:19:321
117299256,3cyclictest281ktimers/112:33:391
117299256,3cyclictest281ktimers/112:33:391
117299255,6cyclictest29-21ksoftirqd/107:53:521
117299255,6cyclictest29-21ksoftirqd/107:53:521
117299255,5cyclictest29-21ksoftirqd/109:53:431
117299255,5cyclictest29-21ksoftirqd/109:53:431
117299255,5cyclictest29-21ksoftirqd/109:53:431
117299255,4cyclictest281ktimers/111:43:441
117299255,4cyclictest281ktimers/111:43:441
117299255,4cyclictest281ktimers/111:43:441
1172992519,5cyclictest281ktimers/111:49:361
1172992519,5cyclictest281ktimers/111:49:361
1172992518,5cyclictest281ktimers/112:35:381
1172992518,5cyclictest281ktimers/112:35:381
1172992518,5cyclictest281ktimers/112:35:381
1172992518,5cyclictest14222-21ntp_offset09:13:471
1172992518,5cyclictest14222-21ntp_offset09:13:471
1172992518,4cyclictest0-21swapper/109:08:491
1172992518,4cyclictest0-21swapper/109:08:491
1172992513,7cyclictest90-1pr/ttyS010:50:381
1172992513,7cyclictest90-1pr/ttyS010:50:381
1172992513,7cyclictest90-1pr/ttyS010:50:381
1172992513,6cyclictest90-1pr/ttyS011:06:201
1172992513,6cyclictest90-1pr/ttyS011:06:201
1172992513,6cyclictest90-1pr/ttyS011:06:201
1172992513,6cyclictest90-1pr/ttyS011:06:201
1172992513,6cyclictest90-1pr/ttyS010:11:461
1172992513,6cyclictest90-1pr/ttyS010:11:461
1172992513,6cyclictest90-1pr/ttyS010:11:461
117299250,9cyclictest6148-21grep08:53:411
117299250,9cyclictest6148-21grep08:53:411
1171992521,3cyclictest27738-21unixbench-2d09:48:530
1171992521,3cyclictest27738-21unixbench-2d09:48:530
1171992521,3cyclictest27738-21unixbench-2d09:48:530
1171992520,3cyclictest0-21swapper/011:48:520
1171992520,3cyclictest0-21swapper/011:48:520
1171992520,3cyclictest0-21swapper/008:43:510
1171992520,3cyclictest0-21swapper/008:43:510
1171992520,3cyclictest0-21swapper/008:43:510
1171992520,3cyclictest0-21swapper/008:23:490
1171992520,3cyclictest0-21swapper/008:23:490
1171992520,3cyclictest0-21swapper/008:08:450
1171992520,3cyclictest0-21swapper/008:08:450
1171992518,5cyclictest2700-21idleruntime-cro11:38:290
1171992518,5cyclictest2700-21idleruntime-cro11:38:290
1171992518,5cyclictest17890-21/usr/sbin/munin12:18:470
1171992518,5cyclictest17890-21/usr/sbin/munin12:18:470
117299246,7cyclictest29-21ksoftirqd/108:43:441
117299246,7cyclictest29-21ksoftirqd/108:43:441
117299246,7cyclictest29-21ksoftirqd/108:43:441
117299245,4cyclictest281ktimers/108:04:201
117299245,4cyclictest281ktimers/108:04:201
117299245,3cyclictest281ktimers/111:23:521
117299245,3cyclictest281ktimers/111:23:521
117299245,3cyclictest281ktimers/111:23:521
117299245,3cyclictest281ktimers/111:23:521
117299244,6cyclictest29-21ksoftirqd/109:43:441
117299244,6cyclictest29-21ksoftirqd/109:43:441
117299244,6cyclictest29-21ksoftirqd/109:43:441
1172992418,4cyclictest0-21swapper/108:18:521
1172992418,4cyclictest0-21swapper/108:18:521
1172992418,2cyclictest181rcu_preempt07:24:291
1172992417,4cyclictest0-21swapper/110:13:441
1172992417,4cyclictest0-21swapper/110:13:441
1172992417,4cyclictest0-21swapper/110:13:441
1171992420,2cyclictest0-21swapper/012:23:480
1171992420,2cyclictest0-21swapper/012:23:480
1171992419,3cyclictest0-21swapper/011:08:420
1171992419,3cyclictest0-21swapper/011:08:420
1171992419,3cyclictest0-21swapper/011:08:420
1171992419,3cyclictest0-21swapper/011:08:420
1171992419,3cyclictest0-21swapper/009:03:500
1171992419,3cyclictest0-21swapper/009:03:500
1171992419,3cyclictest0-21swapper/008:13:470
1171992419,3cyclictest0-21swapper/008:13:470
1171992419,3cyclictest0-21swapper/008:13:470
1171992418,5cyclictest20207-21latency_hist10:58:310
1171992418,5cyclictest20207-21latency_hist10:58:310
1171992418,4cyclictest1170-21cyclictest12:13:460
1171992418,4cyclictest1170-21cyclictest12:13:460
1171992418,4cyclictest0-21swapper/010:08:500
1171992418,4cyclictest0-21swapper/010:08:500
1171992418,4cyclictest0-21swapper/010:08:500
1171992415,7cyclictest17591-21awk07:58:290
1172992319,2cyclictest0-21swapper/107:28:441
1172992319,2cyclictest0-21swapper/107:28:441
117299231,7cyclictest13722-21taskset12:05:001
117299231,7cyclictest13722-21taskset12:05:001
117299231,7cyclictest13722-21taskset12:05:001
1172992316,2cyclictest181rcu_preempt09:43:291
1172992316,2cyclictest181rcu_preempt09:43:291
1172992316,2cyclictest181rcu_preempt09:43:291
1172992315,2cyclictest181rcu_preempt10:03:441
1172992315,2cyclictest181rcu_preempt10:03:441
117299230,8cyclictest15383-21memory10:43:451
117299230,8cyclictest15383-21memory10:43:451
1171992319,2cyclictest0-21swapper/007:23:450
1171992317,3cyclictest0-21swapper/007:43:440
1171992317,3cyclictest0-21swapper/007:43:440
1171992316,5cyclictest0-21swapper/007:33:500
1171992316,5cyclictest0-21swapper/007:33:500
1171992310,11cyclictest17239-21/usr/sbin/munin09:23:390
1171992310,11cyclictest17239-21/usr/sbin/munin09:23:390
122472220,2sleep30-21swapper/309:08:453
122472220,2sleep30-21swapper/309:08:453
117299221,8cyclictest7818-21/usr/sbin/munin10:23:431
117299221,8cyclictest7818-21/usr/sbin/munin10:23:431
1172992216,4cyclictest281ktimers/112:08:541
1172992216,4cyclictest281ktimers/112:08:541
1172992216,4cyclictest281ktimers/112:08:541
1171992217,3cyclictest0-21swapper/011:58:500
1171992217,3cyclictest0-21swapper/011:58:500
1171992215,4cyclictest0-21swapper/007:38:400
1171992215,4cyclictest0-21swapper/007:38:400
1171992117,2cyclictest0-21swapper/008:28:460
1171992116,3cyclictest0-21swapper/012:05:370
1171992116,3cyclictest0-21swapper/012:05:370
1171992116,3cyclictest0-21swapper/012:05:370
117599191,11cyclictest25395-21chrt12:38:383
117599191,11cyclictest25395-21chrt12:38:383
117599191,11cyclictest25395-21chrt12:38:383
1173991913,3cyclictest253802sleep209:43:462
1173991913,3cyclictest253802sleep209:43:462
1173991913,3cyclictest253802sleep209:43:462
117599183,9cyclictest0-21swapper/307:23:543
117599183,8cyclictest0-21swapper/309:49:243
117599183,8cyclictest0-21swapper/309:49:243
117599183,8cyclictest0-21swapper/309:49:243
117599171,12cyclictest4174-21chrt08:48:393
117599171,11cyclictest22895-21chrt09:38:403
117599171,11cyclictest22895-21chrt09:38:403
117599171,11cyclictest22895-21chrt09:38:403
117399173,10cyclictest0-21swapper/211:29:322
117399173,10cyclictest0-21swapper/211:29:322
117399173,10cyclictest0-21swapper/211:29:322
117399171,11cyclictest3266-21chrt11:38:432
117399171,11cyclictest3266-21chrt11:38:432
117599163,9cyclictest0-21swapper/311:43:543
117599163,9cyclictest0-21swapper/311:43:543
117599163,9cyclictest0-21swapper/311:43:543
117599162,9cyclictest0-21swapper/311:34:533
117599162,9cyclictest0-21swapper/311:34:533
117599161,11cyclictest6957-21chrt08:53:503
117599161,11cyclictest6957-21chrt08:53:503
117599161,11cyclictest31397-21chrt08:33:443
117599161,11cyclictest31397-21chrt08:33:443
117599161,11cyclictest10401-21chrt07:38:403
117599161,11cyclictest10401-21chrt07:38:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional