You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 21:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Tue Jun 02, 2026 12:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
292382930,2sleep0171ktimers/011:12:470
292382930,2sleep0171ktimers/011:12:470
130042820,2sleep013001-21latency_hist09:02:500
191892690,1chrt0-21swapper/010:43:040
191892690,1chrt0-21swapper/010:43:040
191892690,1chrt0-21swapper/010:43:040
280772660,2sleep028215-21grep09:42:550
280772660,2sleep028215-21grep09:42:550
280772660,2sleep028215-21grep09:42:550
280772660,2sleep028215-21grep09:42:550
190572620,3sleep20-21swapper/207:52:452
190572620,3sleep20-21swapper/207:52:452
190572620,3sleep20-21swapper/207:52:452
43532610,3sleep00-21swapper/008:38:020
43532610,3sleep00-21swapper/008:38:020
15842560,3chrt0-21swapper/208:32:452
15842560,3chrt0-21swapper/208:32:452
286942530,3sleep128696-21needreboot09:42:591
286942530,3sleep128696-21needreboot09:42:591
286942530,3sleep128696-21needreboot09:42:591
436499510,49cyclictest700-21192.168.115.40-manager08:08:250
436499510,49cyclictest700-21192.168.115.40-manager08:08:250
75702490,3sleep20-21swapper/210:12:592
436499490,3cyclictest700-21192.168.115.40-manager10:31:040
436499490,3cyclictest700-21192.168.115.40-manager10:31:040
436499484,29cyclictest20435-21cat10:47:580
436499481,3cyclictest0-21swapper/012:08:220
436499481,3cyclictest0-21swapper/012:08:220
436499480,2cyclictest700-21192.168.115.40-manager10:37:340
436499480,2cyclictest700-21192.168.115.40-manager10:37:340
436499470,2cyclictest700-21192.168.115.40-manager10:00:120
436499470,2cyclictest700-21192.168.115.40-manager10:00:120
436499470,2cyclictest700-21192.168.115.40-manager10:00:120
436499470,2cyclictest700-21192.168.115.40-manager08:13:440
436499470,2cyclictest700-21192.168.115.40-manager08:13:440
189372470,3sleep30-21swapper/309:17:573
189372470,3sleep30-21swapper/309:17:573
436499461,9cyclictest17274-21taskset12:06:360
436499461,9cyclictest17274-21taskset12:06:360
436499461,9cyclictest17274-21taskset12:06:360
436499461,31cyclictest30860-21chrt11:15:530
436499461,31cyclictest30860-21chrt11:15:530
436499461,31cyclictest30860-21chrt11:15:530
436499460,5cyclictest700-21192.168.115.40-manager09:20:320
436499460,5cyclictest700-21192.168.115.40-manager09:20:320
436499460,4cyclictest700-21192.168.115.40-manager12:25:530
436499460,4cyclictest700-21192.168.115.40-manager12:25:530
436499460,3cyclictest700-21192.168.115.40-manager07:50:570
436499460,3cyclictest700-21192.168.115.40-manager07:50:570
436499460,3cyclictest700-21192.168.115.40-manager07:50:570
436499451,8cyclictest8381-21taskset10:15:420
436499451,8cyclictest749-21runrttasks12:17:110
436499451,8cyclictest749-21runrttasks12:17:110
436499451,2cyclictest700-21192.168.115.40-manager07:40:120
436499451,2cyclictest700-21192.168.115.40-manager07:40:120
436499450,9cyclictest22562-21taskset08:00:470
436499450,9cyclictest22562-21taskset08:00:470
436499450,2cyclictest700-21192.168.115.40-manager11:55:540
436499450,2cyclictest700-21192.168.115.40-manager11:55:540
176072450,2sleep017695-21sendmail_mailqu09:13:040
176072450,2sleep017695-21sendmail_mailqu09:13:040
436499441,8cyclictest749-21runrttasks08:53:210
436499441,8cyclictest749-21runrttasks08:53:210
436499441,7cyclictest10223-21taskset10:20:010
436499441,2cyclictest0-21swapper/011:21:050
436499441,2cyclictest0-21swapper/011:21:050
436499441,11cyclictest8806-21taskset08:49:590
436499441,11cyclictest8806-21taskset08:49:590
436499440,4cyclictest700-21192.168.115.40-manager10:10:140
436499440,4cyclictest700-21192.168.115.40-manager10:10:140
436499440,3cyclictest700-21192.168.115.40-manager11:42:230
436499440,3cyclictest700-21192.168.115.40-manager10:26:340
436499440,3cyclictest700-21192.168.115.40-manager10:26:340
436499440,3cyclictest700-21192.168.115.40-manager10:26:340
436499440,3cyclictest700-21192.168.115.40-manager08:23:440
436499440,3cyclictest700-21192.168.115.40-manager08:23:440
436499440,2cyclictest700-21192.168.115.40-manager09:54:510
436499440,2cyclictest700-21192.168.115.40-manager09:54:510
436499440,2cyclictest700-21192.168.115.40-manager09:54:510
436499431,8cyclictest4471-21seq10:03:360
436499431,8cyclictest4471-21seq10:03:360
436499431,8cyclictest17636-21taskset10:38:540
436499431,8cyclictest17636-21taskset10:38:540
436499431,7cyclictest25763-21chrt09:35:300
436499431,7cyclictest25763-21chrt09:35:300
436499431,7cyclictest25763-21chrt09:35:300
436499431,6cyclictest749-21runrttasks09:09:050
436499431,6cyclictest749-21runrttasks09:09:050
436499431,6cyclictest749-21runrttasks09:09:050
436499431,6cyclictest13263-21seq07:36:310
436499431,6cyclictest13263-21seq07:36:310
436499431,4cyclictest30157-21taskset12:39:180
436499431,4cyclictest30157-21taskset12:39:180
436499431,4cyclictest30157-21taskset12:39:180
436499431,1cyclictest0-21swapper/008:22:450
436499431,1cyclictest0-21swapper/008:22:450
436499430,8cyclictest16810-21taskset07:43:350
436499430,8cyclictest16810-21taskset07:43:350
436499430,7cyclictest26359-21chrt12:28:210
436499430,7cyclictest26359-21chrt12:28:210
436499430,7cyclictest26359-21chrt12:28:210
436499430,6cyclictest5773-21taskset11:33:330
436499430,6cyclictest5773-21taskset11:33:330
436499430,3cyclictest700-21192.168.115.40-manager11:58:540
436499430,3cyclictest700-21192.168.115.40-manager11:58:540
436499430,2cyclictest700-21192.168.115.40-manager07:55:500
436499430,2cyclictest700-21192.168.115.40-manager07:55:500
436499421,7cyclictest6863-21taskset08:43:560
436499421,7cyclictest6863-21taskset08:43:560
436499421,7cyclictest28440-21taskset12:36:130
436499421,7cyclictest28440-21taskset12:36:130
436499421,7cyclictest28440-21taskset12:36:130
436499421,5cyclictest2233-21taskset11:26:250
436499421,5cyclictest2233-21taskset11:26:250
436499421,1cyclictest0-21swapper/009:48:490
436499421,1cyclictest0-21swapper/009:48:490
436499421,1cyclictest0-21swapper/009:48:490
436499420,7cyclictest14487-21taskset09:05:150
436499420,7cyclictest14487-21taskset09:05:150
436499420,4cyclictest700-21192.168.115.40-manager12:19:310
436499420,3cyclictest700-21192.168.115.40-manager08:28:150
436499420,3cyclictest700-21192.168.115.40-manager08:28:150
436499420,2cyclictest700-21192.168.115.40-manager11:51:020
436499420,2cyclictest700-21192.168.115.40-manager11:51:020
436499420,2cyclictest700-21192.168.115.40-manager11:31:530
436499420,2cyclictest700-21192.168.115.40-manager11:31:530
436499420,2cyclictest700-21192.168.115.40-manager09:23:460
436499420,2cyclictest700-21192.168.115.40-manager09:23:460
436499420,2cyclictest700-21192.168.115.40-manager08:05:540
436499420,2cyclictest700-21192.168.115.40-manager08:05:540
436499420,2cyclictest700-21192.168.115.40-manager08:05:540
436499411,7cyclictest24037-21chrt09:32:170
436499411,7cyclictest24037-21chrt09:32:170
436499411,7cyclictest24037-21chrt09:32:170
436499411,1cyclictest0-21swapper/009:01:440
436499410,5cyclictest23528-21seq10:57:390
436499410,3cyclictest700-21192.168.115.40-manager11:06:350
436499410,3cyclictest700-21192.168.115.40-manager11:06:350
436499410,3cyclictest700-21192.168.115.40-manager11:06:350
436499410,3cyclictest700-21192.168.115.40-manager07:27:120
436499410,3cyclictest700-21192.168.115.40-manager07:27:120
143282410,2sleep114360-21ntp_states07:38:021
143282410,2sleep114360-21ntp_states07:38:021
436499400,6cyclictest29482-21taskset09:45:110
436499400,6cyclictest29482-21taskset09:45:110
436499400,6cyclictest29482-21taskset09:45:110
436499400,3cyclictest700-21192.168.115.40-manager11:43:490
436499400,3cyclictest700-21192.168.115.40-manager11:43:490
188102400,3chrt0-21swapper/209:17:562
188102400,3chrt0-21swapper/209:17:562
47752390,3sleep30-21swapper/311:32:563
47752390,3sleep30-21swapper/311:32:563
436499390,4cyclictest700-21192.168.115.40-manager08:33:480
436499390,4cyclictest700-21192.168.115.40-manager08:33:480
436499390,2cyclictest700-21192.168.115.40-manager10:58:470
436499390,2cyclictest700-21192.168.115.40-manager10:58:470
4372993715,6cyclictest4352-21cyclictest07:33:401
4372993715,6cyclictest4352-21cyclictest07:33:401
436499370,4cyclictest700-21192.168.115.40-manager07:30:120
436499370,4cyclictest700-21192.168.115.40-manager07:30:120
164282370,4sleep20-21swapper/207:43:042
164282370,4sleep20-21swapper/207:43:042
4372993527,6cyclictest1519-21cat08:32:421
4372993527,6cyclictest1519-21cat08:32:421
4372993324,7cyclictest7110-21fschecks_time10:12:551
4372993323,3cyclictest181rcu_preempt08:50:121
4372993323,3cyclictest181rcu_preempt08:50:121
437299326,13cyclictest281ktimers/110:52:491
4372993225,5cyclictest19315-21latency_hist12:12:461
4372993225,5cyclictest19315-21latency_hist12:12:461
4372993218,3cyclictest90-1pr/ttyS010:03:001
4372993218,3cyclictest90-1pr/ttyS010:03:001
137682320,3sleep30-21swapper/309:03:013
137682320,3sleep30-21swapper/309:03:013
437299313,11cyclictest271rcuc/107:52:171
437299313,11cyclictest271rcuc/107:52:171
437299313,11cyclictest271rcuc/107:52:171
4372993126,3cyclictest15481-21latency_hist12:02:421
4372993126,3cyclictest15481-21latency_hist12:02:421
4372993124,5cyclictest5165-21fschecks_count10:07:551
4372993124,5cyclictest5165-21fschecks_count10:07:551
4372993025,3cyclictest4352-21cyclictest12:03:031
4372993025,3cyclictest4352-21cyclictest12:03:031
4372993025,3cyclictest4352-21cyclictest12:03:031
4372993021,7cyclictest550-21cron11:37:421
4372993021,7cyclictest550-21cron11:37:421
4372993019,3cyclictest181rcu_preempt08:12:491
4372993019,3cyclictest181rcu_preempt08:12:491
437299298,9cyclictest29-21ksoftirqd/110:42:311
437299298,9cyclictest29-21ksoftirqd/110:42:311
437299295,11cyclictest271rcuc/108:13:281
437299295,11cyclictest271rcuc/108:13:281
437299294,23cyclictest271rcuc/110:55:081
437299294,10cyclictest271rcuc/108:42:381
437299294,10cyclictest271rcuc/108:42:381
4372992921,6cyclictest28989-21fschecks_time12:37:561
4372992921,6cyclictest28989-21fschecks_time12:37:561
4372992921,6cyclictest28989-21fschecks_time12:37:561
4372992921,6cyclictest13851-21processes09:03:031
4372992921,6cyclictest13851-21processes09:03:031
437299292,13cyclictest271rcuc/109:30:071
437299292,13cyclictest271rcuc/109:30:071
437299292,13cyclictest271rcuc/109:30:071
4372992921,2cyclictest181rcu_preempt11:43:041
4372992921,2cyclictest181rcu_preempt11:43:041
4372992915,3cyclictest181rcu_preempt11:48:001
4372992915,3cyclictest181rcu_preempt11:48:001
4372992915,3cyclictest181rcu_preempt11:37:561
4372992915,2cyclictest181rcu_preempt10:01:101
4372992913,2cyclictest181rcu_preempt11:13:011
4372992913,2cyclictest181rcu_preempt11:13:011
4372992913,2cyclictest181rcu_preempt11:13:011
4372992911,16cyclictest271rcuc/112:34:501
4372992911,16cyclictest271rcuc/112:34:501
4372992911,16cyclictest271rcuc/112:34:501
135582290,4sleep113557-21xargs10:28:061
135582290,4sleep113557-21xargs10:28:061
437299286,9cyclictest29-21ksoftirqd/111:23:141
437299286,9cyclictest29-21ksoftirqd/111:23:141
4372992819,2cyclictest181rcu_preempt12:18:021
4372992816,2cyclictest181rcu_preempt08:17:531
4372992816,2cyclictest181rcu_preempt08:17:531
4372992815,2cyclictest181rcu_preempt12:27:511
4372992815,2cyclictest181rcu_preempt12:27:511
4372992814,3cyclictest181rcu_preempt07:57:581
4372992814,3cyclictest181rcu_preempt07:57:581
4372992813,3cyclictest181rcu_preempt11:12:561
4372992813,3cyclictest181rcu_preempt11:12:561
4372992812,4cyclictest281ktimers/111:07:561
4372992812,4cyclictest281ktimers/111:07:561
4372992812,4cyclictest281ktimers/111:07:561
4372992810,4cyclictest0-21swapper/109:08:051
4372992810,4cyclictest0-21swapper/109:08:051
4372992810,4cyclictest0-21swapper/109:08:051
437299278,3cyclictest271rcuc/107:53:021
437299278,3cyclictest271rcuc/107:53:021
437299273,12cyclictest271rcuc/107:28:411
437299273,12cyclictest271rcuc/107:28:411
4372992723,3cyclictest90-1pr/ttyS009:37:421
4372992723,3cyclictest90-1pr/ttyS009:37:421
4372992723,3cyclictest90-1pr/ttyS009:37:421
4372992721,5cyclictest3235-21needreboot11:28:001
4372992721,5cyclictest3235-21needreboot11:28:001
4372992721,4cyclictest0-21swapper/111:02:421
4372992721,4cyclictest0-21swapper/111:02:421
4372992720,5cyclictest27171-21threads09:38:051
4372992714,2cyclictest181rcu_preempt08:43:051
4372992714,2cyclictest181rcu_preempt08:43:051
4372992713,2cyclictest181rcu_preempt07:27:531
4372992713,2cyclictest181rcu_preempt07:27:531
437299271,13cyclictest271rcuc/109:16:241
437299271,13cyclictest271rcuc/109:16:241
437299268,4cyclictest0-21swapper/109:18:031
437299268,4cyclictest0-21swapper/109:18:031
437299266,9cyclictest29-21ksoftirqd/111:18:331
437299266,9cyclictest29-21ksoftirqd/111:18:331
437299265,4cyclictest281ktimers/108:27:521
437299265,4cyclictest281ktimers/108:27:521
437299264,4cyclictest281ktimers/110:43:051
437299264,4cyclictest281ktimers/110:43:051
437299264,4cyclictest281ktimers/110:43:051
437299262,21cyclictest271rcuc/110:36:241
437299262,21cyclictest271rcuc/110:36:241
4372992621,3cyclictest0-21swapper/112:32:421
4372992621,3cyclictest0-21swapper/112:32:421
4372992621,3cyclictest0-21swapper/112:32:421
4372992621,3cyclictest0-21swapper/109:57:541
4372992621,3cyclictest0-21swapper/109:57:541
4372992621,3cyclictest0-21swapper/109:57:541
4372992619,5cyclictest30209-21gltestperf09:47:561
4372992619,5cyclictest30209-21gltestperf09:47:561
4372992619,5cyclictest30209-21gltestperf09:47:561
4372992616,2cyclictest181rcu_preempt08:07:131
4372992616,2cyclictest181rcu_preempt08:07:131
4372992616,2cyclictest181rcu_preempt08:07:131
4372992611,2cyclictest181rcu_preempt10:18:021
4372992611,2cyclictest181rcu_preempt09:23:041
4372992611,2cyclictest181rcu_preempt09:23:041
305642260,4sleep20-21swapper/208:22:522
305642260,4sleep20-21swapper/208:22:522
256282260,3sleep20-21swapper/211:02:522
437299258,8cyclictest20191-21ntp_states12:13:011
437299258,8cyclictest20191-21ntp_states12:13:011
437299255,4cyclictest271rcuc/111:53:441
437299255,4cyclictest271rcuc/111:53:441
4372992520,3cyclictest281ktimers/108:52:581
4372992520,3cyclictest281ktimers/108:52:581
4372992519,4cyclictest281ktimers/110:22:571
4372992519,4cyclictest281ktimers/110:22:571
4372992519,4cyclictest281ktimers/110:22:571
4372992511,2cyclictest181rcu_preempt10:08:021
4372992511,2cyclictest181rcu_preempt10:08:021
4372992510,2cyclictest181rcu_preempt08:58:061
4372992510,2cyclictest181rcu_preempt08:58:061
4372992510,13cyclictest281ktimers/108:35:461
4372992510,13cyclictest281ktimers/108:35:461
437299248,3cyclictest0-21swapper/107:42:571
437299248,3cyclictest0-21swapper/107:42:571
4372992410,4cyclictest271rcuc/109:52:181
4372992410,4cyclictest271rcuc/109:52:181
4372992410,4cyclictest271rcuc/109:52:181
4380992215,4cyclictest165822sleep212:03:022
4380992215,4cyclictest165822sleep212:03:022
4380992215,4cyclictest165822sleep212:03:022
4388992014,4cyclictest117422chrt11:52:423
4388992014,4cyclictest117422chrt11:52:423
438899201,15cyclictest26903-21chrt12:32:543
438899201,15cyclictest26903-21chrt12:32:543
4380992015,3cyclictest204642chrt07:53:052
4380992015,3cyclictest204642chrt07:53:052
438899191,12cyclictest30598-21chrt12:42:513
438899191,12cyclictest30598-21chrt12:42:513
438899191,12cyclictest30598-21chrt12:42:513
438899181,12cyclictest26787-21chrt08:12:523
438899181,12cyclictest26787-21chrt08:12:523
438099181,13cyclictest14103-21taskset07:37:582
438099181,13cyclictest14103-21taskset07:37:582
438099181,12cyclictest0-21swapper/212:33:442
438099181,12cyclictest0-21swapper/212:33:442
438099181,12cyclictest0-21swapper/212:33:442
438099181,11cyclictest28992-21chrt09:43:042
438099181,11cyclictest28992-21chrt09:43:042
438099181,11cyclictest28992-21chrt09:43:042
438099173,10cyclictest0-21swapper/208:40:132
438099173,10cyclictest0-21swapper/208:40:132
438099171,12cyclictest16509-21chrt09:12:212
438099171,12cyclictest16509-21chrt09:12:212
438099171,12cyclictest16509-21chrt09:12:212
438099171,10cyclictest0-21swapper/211:02:532
438099171,10cyclictest0-21swapper/211:02:532
438099171,10cyclictest0-21swapper/211:02:532
438899163,9cyclictest0-21swapper/311:13:073
438899163,9cyclictest0-21swapper/311:13:073
438899163,9cyclictest0-21swapper/311:13:073
438899163,9cyclictest0-21swapper/308:25:133
438899163,9cyclictest0-21swapper/308:25:133
438899161,11cyclictest9974-21chrt11:47:473
438899161,11cyclictest9974-21chrt11:47:473
438899161,11cyclictest30124-21chrt09:47:563
438899161,11cyclictest30124-21chrt09:47:563
438899161,11cyclictest30124-21chrt09:47:563
4388991610,4cyclictest165102sleep307:43:053
4388991610,4cyclictest165102sleep307:43:053
438099162,9cyclictest0-21swapper/208:12:492
438099162,9cyclictest0-21swapper/208:12:492
438099161,12cyclictest27211-21chrt08:12:552
438099161,12cyclictest27211-21chrt08:12:552
438099161,12cyclictest0-21swapper/210:17:542
438099161,11cyclictest7778-21taskset08:47:572
438099161,11cyclictest7778-21taskset08:47:572
438099161,11cyclictest29134-21chrt08:17:582
438099161,11cyclictest29134-21chrt08:17:582
438099161,11cyclictest1859-21chrt09:58:002
438099161,11cyclictest1859-21chrt09:58:002
438099161,11cyclictest1859-21chrt09:58:002
438099161,11cyclictest11053-21taskset10:22:562
438099161,11cyclictest11053-21taskset10:22:562
438099161,11cyclictest11053-21taskset10:22:562
438099161,11cyclictest0-21swapper/209:07:522
438099161,11cyclictest0-21swapper/209:07:522
438099161,10cyclictest703-21chrt11:22:542
438099161,10cyclictest703-21chrt11:22:542
438899153,9cyclictest0-21swapper/312:33:263
438899153,9cyclictest0-21swapper/312:33:263
438899153,9cyclictest0-21swapper/312:33:263
438899153,8cyclictest0-21swapper/312:03:133
438899153,8cyclictest0-21swapper/312:03:133
438899153,8cyclictest0-21swapper/312:03:133
438899151,12cyclictest20001-21chrt07:53:023
438899151,12cyclictest20001-21chrt07:53:023
438899151,11cyclictest6374-21chrt08:43:033
438899151,11cyclictest6374-21chrt08:43:033
438899151,11cyclictest27492-21chrt08:13:003
438899151,11cyclictest27492-21chrt08:13:003
438899151,11cyclictest23691-21chrt12:22:573
438899151,11cyclictest23691-21chrt12:22:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional