You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 08:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Thu Jul 16, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2291221230,3sleep30-21swapper/300:03:343
2291221230,3sleep30-21swapper/300:03:343
2291221230,3sleep30-21swapper/300:03:343
1260321180,1sleep00-21swapper/020:43:280
1260321180,1sleep00-21swapper/020:43:280
661421160,2sleep3431rcuc/323:18:403
661421160,2sleep3431rcuc/323:18:403
1017721090,3sleep30-21swapper/323:28:373
1017721090,3sleep30-21swapper/323:28:373
2515521080,2sleep30-21swapper/321:13:463
2515521080,2sleep30-21swapper/321:13:463
31202980,2sleep30-21swapper/321:43:373
31202980,2sleep30-21swapper/321:43:373
31202980,2sleep30-21swapper/321:43:373
147022890,8sleep014706-21needreboot22:13:400
147022890,8sleep014706-21needreboot22:13:400
171602730,3sleep30-21swapper/319:28:353
171602730,3sleep30-21swapper/319:28:353
294652680,2sleep029466-21uname22:53:360
294652680,2sleep029466-21uname22:53:360
318612610,2sleep031863-21if_eth021:33:370
318612610,2sleep031863-21if_eth021:33:370
318612610,2sleep031863-21if_eth021:33:370
238782600,3sleep30-21swapper/319:43:453
246392580,2chrt0-21swapper/300:08:333
246392580,2chrt0-21swapper/300:08:333
246392580,2chrt0-21swapper/300:08:333
11644995739,13cyclictest23548-21grep00:03:411
11644995739,13cyclictest23548-21grep00:03:411
11644995739,13cyclictest23548-21grep00:03:411
97522540,3chrt0-21swapper/121:59:551
97522540,3chrt0-21swapper/121:59:551
203942540,2chrt0-21swapper/321:03:333
203942540,2chrt0-21swapper/321:03:333
203942540,2chrt0-21swapper/321:03:333
210952530,4sleep021097-21fschecks_time23:58:330
210952530,4sleep021097-21fschecks_time23:58:330
210952530,4sleep021097-21fschecks_time23:58:330
109202520,3chrt0-21swapper/322:03:393
109202520,3chrt0-21swapper/322:03:393
189682510,3sleep30-21swapper/323:53:293
189682510,3sleep30-21swapper/323:53:293
171402500,2sleep00-21swapper/020:53:420
171402500,2sleep00-21swapper/020:53:420
86692480,2sleep30-21swapper/323:23:423
86692480,2sleep30-21swapper/323:23:423
86692480,2sleep30-21swapper/323:23:423
231532470,2chrt23152-21ls22:38:221
231532470,2chrt23152-21ls22:38:221
231532470,2chrt23152-21ls22:38:221
11643994523,14cyclictest263522awk19:53:330
11643994523,14cyclictest263522awk19:53:330
11643994518,16cyclictest16075-21taskset20:51:510
11643994518,16cyclictest16075-21taskset20:51:510
11643994418,15cyclictest20941-21taskset22:28:590
11643994418,15cyclictest20941-21taskset22:28:590
11643994418,15cyclictest20941-21taskset22:28:590
11643994418,10cyclictest31503-21taskset20:05:370
11643994418,10cyclictest31503-21taskset20:05:370
11643994418,10cyclictest31503-21taskset20:05:370
145172430,2sleep114619-21memory22:13:381
145172430,2sleep114619-21memory22:13:381
11643994317,15cyclictest749-21runrttasks21:09:570
11643994317,15cyclictest749-21runrttasks21:09:570
11643994317,15cyclictest749-21runrttasks21:09:570
175442420,4sleep20-21swapper/219:28:372
175442420,4sleep20-21swapper/219:28:372
11643993832,4cyclictest700-21192.168.115.40-manager22:24:160
11643993832,4cyclictest700-21192.168.115.40-manager22:24:160
11643993832,4cyclictest700-21192.168.115.40-manager22:24:160
11643993731,4cyclictest700-21192.168.115.40-manager21:42:220
11643993731,4cyclictest700-21192.168.115.40-manager21:42:220
11643993731,4cyclictest700-21192.168.115.40-manager21:42:220
11643993717,10cyclictest2119-21acpi23:08:320
11643993717,10cyclictest2119-21acpi23:08:320
11643993717,10cyclictest2119-21acpi23:08:320
11643993715,11cyclictest18563-21runrttasks19:32:200
11643993715,11cyclictest18563-21runrttasks19:32:200
11643993631,3cyclictest700-21192.168.115.40-manager20:21:490
11643993631,3cyclictest700-21192.168.115.40-manager20:21:490
11643993631,3cyclictest700-21192.168.115.40-manager20:21:490
11643993629,5cyclictest29837-21latency_hist20:03:210
11643993629,5cyclictest29837-21latency_hist20:03:210
11643993628,6cyclictest0-21swapper/000:03:280
11643993628,6cyclictest0-21swapper/000:03:280
11643993628,6cyclictest0-21swapper/000:03:280
11643993624,9cyclictest10926-21bingrep22:03:390
11643993624,9cyclictest10926-21bingrep22:03:390
11643993623,10cyclictest25023-21timerwakeupswit21:13:450
11643993623,10cyclictest25023-21timerwakeupswit21:13:450
11643993617,9cyclictest21627-21seq21:06:070
11643993617,9cyclictest21627-21seq21:06:070
11643993617,7cyclictest197392sleep021:01:040
11643993617,7cyclictest197392sleep021:01:040
11643993617,7cyclictest197392sleep021:01:040
11643993616,10cyclictest749-21runrttasks21:25:040
11643993616,10cyclictest749-21runrttasks21:25:040
11643993616,10cyclictest749-21runrttasks21:25:040
11643993615,8cyclictest261202chrt00:10:590
11643993615,8cyclictest261202chrt00:10:590
11643993615,8cyclictest261202chrt00:10:590
11643993612,13cyclictest6142-21seq21:51:460
11643993612,13cyclictest6142-21seq21:51:460
11643993612,12cyclictest2983-21taskset00:35:180
11643993612,12cyclictest2983-21taskset00:35:180
11643993612,12cyclictest2983-21taskset00:35:180
11643993611,14cyclictest749-21runrttasks22:21:430
11643993611,14cyclictest749-21runrttasks22:21:430
308642350,3sleep20-21swapper/220:03:422
308642350,3sleep20-21swapper/220:03:422
308642350,3sleep20-21swapper/220:03:422
199452350,2chrt0-21swapper/322:28:353
199452350,2chrt0-21swapper/322:28:353
199452350,2chrt0-21swapper/322:28:353
11643993529,4cyclictest700-21192.168.115.40-manager21:21:060
11643993529,4cyclictest700-21192.168.115.40-manager21:21:060
11643993527,6cyclictest0-21swapper/022:38:230
11643993527,6cyclictest0-21swapper/022:38:230
11643993527,6cyclictest0-21swapper/022:38:230
11643993522,10cyclictest9421-21cat21:58:440
11643993522,10cyclictest9421-21cat21:58:440
11643993522,10cyclictest32366-21tune2fs00:28:350
11643993522,10cyclictest32366-21tune2fs00:28:350
11643993517,7cyclictest27914-21runrttasks00:15:050
11643993517,7cyclictest27914-21runrttasks00:15:050
11643993516,8cyclictest749-21runrttasks20:16:500
11643993516,8cyclictest749-21runrttasks20:16:500
11643993516,7cyclictest11199-21chrt23:31:390
11643993516,7cyclictest11199-21chrt23:31:390
11643993514,11cyclictest24964-21taskset22:42:540
11643993514,11cyclictest24964-21taskset22:42:540
11643993512,9cyclictest6359-21if_wlan023:18:360
11643993512,9cyclictest6359-21if_wlan023:18:360
11643993512,8cyclictest16943-21taskset23:48:020
11643993512,8cyclictest16943-21taskset23:48:020
11643993512,7cyclictest23357-21ntp_states19:43:400
11643993512,7cyclictest0-21swapper/020:47:370
11643993512,7cyclictest0-21swapper/020:47:370
282672340,2sleep30-21swapper/319:58:323
282672340,2sleep30-21swapper/319:58:323
11643993426,6cyclictest0-21swapper/020:08:340
11643993426,6cyclictest0-21swapper/020:08:340
11643993418,13cyclictest28764-21idleruntime-cro22:53:220
11643993418,13cyclictest28764-21idleruntime-cro22:53:220
11643993416,6cyclictest25954-21seq19:51:220
11643993416,6cyclictest25954-21seq19:51:220
11643993414,4cyclictest12226-21strings23:33:390
11643993414,4cyclictest12226-21strings23:33:390
11643993412,7cyclictest297802chrt00:20:110
11643993412,7cyclictest297802chrt00:20:110
11643993412,12cyclictest9140-21taskset23:24:060
11643993412,12cyclictest9140-21taskset23:24:060
11643993412,12cyclictest9140-21taskset23:24:060
11643993411,8cyclictest318782chrt00:27:510
11643993411,8cyclictest18571-21runrttasks23:50:000
11643993411,8cyclictest18571-21runrttasks23:50:000
11643993411,7cyclictest4677-21bingrep23:13:380
11643993411,7cyclictest4677-21bingrep23:13:380
11643993410,8cyclictest0-21swapper/021:28:530
11643993410,8cyclictest0-21swapper/021:28:530
11643993410,8cyclictest0-21swapper/021:28:530
11643993410,8cyclictest0-21swapper/020:31:090
11643993410,8cyclictest0-21swapper/020:31:090
11643993410,8cyclictest0-21swapper/000:04:180
11643993410,8cyclictest0-21swapper/000:04:180
11643993410,8cyclictest0-21swapper/000:04:180
63492330,3chrt0-21swapper/320:23:453
63492330,3chrt0-21swapper/320:23:453
219942330,2sleep20-21swapper/223:58:432
219942330,2sleep20-21swapper/223:58:432
219942330,2sleep20-21swapper/223:58:432
11644993324,7cyclictest13975-21latency_hist22:13:321
11644993324,7cyclictest13975-21latency_hist22:13:321
11644993324,7cyclictest13975-21latency_hist22:13:321
1164399339,11cyclictest749-21runrttasks00:43:270
1164399339,11cyclictest749-21runrttasks00:43:270
11643993328,4cyclictest572-21cups-browsed00:28:430
11643993328,4cyclictest572-21cups-browsed00:28:430
11643993321,9cyclictest22589-21/usr/sbin/munin19:43:320
11643993321,7cyclictest14178-21/usr/sbin/munin23:38:410
11643993321,7cyclictest14178-21/usr/sbin/munin23:38:410
11643993321,7cyclictest14178-21/usr/sbin/munin23:38:410
11643993318,11cyclictest32454-21seq23:03:090
11643993318,11cyclictest32454-21seq23:03:090
11643993318,11cyclictest26649-21chrt22:45:160
11643993318,11cyclictest26649-21chrt22:45:160
11643993315,7cyclictest6658-21taskset20:26:340
11643993315,7cyclictest6658-21taskset20:26:340
11643993311,10cyclictest8032-21sshd21:57:060
11643993311,10cyclictest8032-21sshd21:57:060
86872320,3sleep30-21swapper/320:33:233
86872320,3sleep30-21swapper/320:33:233
60452320,2sleep10-21swapper/120:23:431
60452320,2sleep10-21swapper/120:23:431
278642320,3sleep30-21swapper/322:48:413
278642320,3sleep30-21swapper/322:48:413
1164499322,26cyclictest22868-21df00:03:331
1164499322,26cyclictest22868-21df00:03:331
1164499322,26cyclictest22868-21df00:03:331
1164399328,11cyclictest8888-21/usr/sbin/munin20:33:350
1164399328,11cyclictest8888-21/usr/sbin/munin20:33:350
1164399327,11cyclictest20164-21runrttasks19:34:000
1164399327,11cyclictest20164-21runrttasks19:34:000
11643993221,8cyclictest0-21swapper/019:38:350
11643993218,11cyclictest2768-21sed23:08:380
11643993218,11cyclictest2768-21sed23:08:380
11643993218,11cyclictest2768-21sed23:08:380
11643993217,11cyclictest2948-21fschecks_time21:43:360
11643993217,11cyclictest2948-21fschecks_time21:43:360
11643993217,11cyclictest2948-21fschecks_time21:43:360
11644993124,5cyclictest29841-21sendmail20:03:211
11644993124,5cyclictest29841-21sendmail20:03:211
11644993122,7cyclictest9588-21needreboot20:33:391
11644993122,7cyclictest9588-21needreboot20:33:391
11644993120,3cyclictest700-21192.168.115.40-manager23:54:151
11644993120,3cyclictest700-21192.168.115.40-manager23:54:151
11644993120,3cyclictest700-21192.168.115.40-manager23:54:151
11643993116,11cyclictest13605-21seq22:11:110
11643993116,11cyclictest13605-21seq22:11:110
11643993116,11cyclictest13605-21seq22:11:110
1164499307,20cyclictest281ktimers/120:07:041
1164499307,20cyclictest281ktimers/120:07:041
1164499307,20cyclictest281ktimers/120:07:041
1164499303,8cyclictest700-21192.168.115.40-manager19:30:061
1164499303,8cyclictest700-21192.168.115.40-manager19:30:061
11644993023,6cyclictest14162-21ntp_kernel_err23:38:401
11644993023,6cyclictest14162-21ntp_kernel_err23:38:401
11644993023,6cyclictest14162-21ntp_kernel_err23:38:401
91602290,4sleep30-21swapper/320:33:343
91602290,4sleep30-21swapper/320:33:343
11644992924,3cyclictest0-21swapper/100:08:351
11644992924,3cyclictest0-21swapper/100:08:351
11644992924,3cyclictest0-21swapper/100:08:351
11644992923,5cyclictest29348-21latency_hist21:28:221
11644992923,5cyclictest29348-21latency_hist21:28:221
11644992923,5cyclictest29348-21latency_hist21:28:221
11644992923,5cyclictest16593-21fschecks_count20:53:351
11644992923,5cyclictest16593-21fschecks_count20:53:351
11644992921,6cyclictest9889-21fschecks_time23:28:351
11644992921,6cyclictest9889-21fschecks_time23:28:351
11644992920,2cyclictest181rcu_preempt22:23:061
11644992920,2cyclictest181rcu_preempt22:23:061
11644992919,7cyclictest29708-21ntp_offset22:53:401
11644992919,7cyclictest29708-21ntp_offset22:53:401
1164699281,6cyclictest3065-21chrt00:36:223
1164699281,6cyclictest3065-21chrt00:36:223
1164699281,6cyclictest3065-21chrt00:36:223
1164499287,9cyclictest281ktimers/122:59:171
1164499287,9cyclictest281ktimers/122:59:171
1164499287,10cyclictest281ktimers/119:49:161
1164499287,10cyclictest281ktimers/119:49:161
1164499286,10cyclictest281ktimers/122:44:161
1164499286,10cyclictest281ktimers/122:44:161
1164499286,10cyclictest281ktimers/120:39:161
1164499286,10cyclictest281ktimers/120:39:161
1164499285,9cyclictest29-21ksoftirqd/119:33:461
1164499285,9cyclictest29-21ksoftirqd/119:33:461
11644992823,3cyclictest5090-21latency_hist00:43:221
11644992823,3cyclictest5090-21latency_hist00:43:221
11644992822,4cyclictest10652-21idleruntime22:03:361
11644992822,4cyclictest10652-21idleruntime22:03:361
11644992819,7cyclictest18957-21needreboot20:58:391
11644992819,7cyclictest18957-21needreboot20:58:391
11644992819,7cyclictest18957-21needreboot20:58:391
11644992818,7cyclictest28753-21sh22:53:241
11644992818,7cyclictest28753-21sh22:53:241
11644992817,2cyclictest181rcu_preempt20:21:291
11644992817,2cyclictest181rcu_preempt20:21:291
11644992817,2cyclictest181rcu_preempt20:21:291
1164699271,6cyclictest9401-21chrt21:58:443
1164699271,6cyclictest9401-21chrt21:58:443
1164699271,6cyclictest30516-21chrt22:57:093
1164699271,6cyclictest30516-21chrt22:57:093
1164499277,3cyclictest281ktimers/121:19:161
1164499277,3cyclictest281ktimers/121:19:161
1164499276,5cyclictest281ktimers/100:14:171
1164499276,5cyclictest281ktimers/100:14:171
1164499276,10cyclictest281ktimers/123:09:171
1164499276,10cyclictest281ktimers/123:09:171
1164499276,10cyclictest281ktimers/123:09:171
1164499273,11cyclictest29-21ksoftirqd/100:18:381
1164499273,11cyclictest29-21ksoftirqd/100:18:381
11644992721,5cyclictest20348-21netstat22:28:401
11644992721,5cyclictest20348-21netstat22:28:401
11644992721,5cyclictest20348-21netstat22:28:401
11644992720,5cyclictest550-21cron20:18:211
11644992720,5cyclictest550-21cron20:18:211
11644992719,6cyclictest10368-21needreboot23:28:391
11644992719,6cyclictest10368-21needreboot23:28:391
11644992718,4cyclictest181rcu_preempt23:23:541
11644992714,7cyclictest90-1pr/ttyS023:53:331
11644992711,10cyclictest0-21swapper/123:08:301
11644992711,10cyclictest0-21swapper/123:08:301
11644992711,10cyclictest0-21swapper/123:08:301
1164699269,14cyclictest0-21swapper/321:38:223
1164699269,14cyclictest0-21swapper/321:38:223
1164699269,14cyclictest0-21swapper/321:38:223
1164699261,6cyclictest905-21chrt20:10:183
1164699261,6cyclictest905-21chrt20:10:183
1164699261,6cyclictest8187-21chrt21:58:243
1164699261,6cyclictest8187-21chrt21:58:243
1164699261,6cyclictest6101-21chrt21:51:123
1164699261,6cyclictest6101-21chrt21:51:123
1164699261,6cyclictest31604-21chrt20:06:573
1164699261,6cyclictest31604-21chrt20:06:573
1164699261,6cyclictest31604-21chrt20:06:573
1164699261,6cyclictest30851-21chrt21:28:573
1164699261,6cyclictest30851-21chrt21:28:573
1164699261,6cyclictest30851-21chrt21:28:573
1164699261,6cyclictest2838-21chrt20:16:053
1164699261,6cyclictest2838-21chrt20:16:053
1164699261,5cyclictest29748-21chrt00:19:463
1164699261,5cyclictest29748-21chrt00:19:463
1164699261,10cyclictest23686-21taskset21:13:223
1164699261,10cyclictest23686-21taskset21:13:223
1164699261,10cyclictest23686-21taskset21:13:223
1164699260,6cyclictest20597-21chrt23:57:073
1164699260,6cyclictest20597-21chrt23:57:073
1164699260,6cyclictest20597-21chrt23:57:073
1164699260,6cyclictest15935-21chrt20:50:023
1164699260,6cyclictest15935-21chrt20:50:023
1164499268,3cyclictest281ktimers/123:18:431
1164499268,3cyclictest281ktimers/123:18:431
1164499266,9cyclictest281ktimers/123:49:551
1164499265,10cyclictest281ktimers/121:34:161
1164499265,10cyclictest281ktimers/121:34:161
1164499265,10cyclictest281ktimers/121:34:161
1164499265,10cyclictest281ktimers/119:39:161
1164499265,10cyclictest281ktimers/119:39:161
1164499262,11cyclictest281ktimers/121:10:061
1164499262,11cyclictest281ktimers/121:10:061
1164499262,11cyclictest281ktimers/121:10:061
11644992620,5cyclictest32265-21fschecks_count00:28:341
11644992620,5cyclictest32265-21fschecks_count00:28:341
11644992620,4cyclictest749-21runrttasks21:08:231
11644992620,4cyclictest749-21runrttasks21:08:231
11644992619,5cyclictest6730-21fschecks_time21:53:351
11644992619,5cyclictest6730-21fschecks_time21:53:351
11644992618,6cyclictest30287-21ntp_kernel_pll_21:28:401
11644992618,6cyclictest30287-21ntp_kernel_pll_21:28:401
11644992618,6cyclictest30287-21ntp_kernel_pll_21:28:401
11644992618,5cyclictest19583-21/usr/sbin/munin22:28:351
11644992618,5cyclictest19583-21/usr/sbin/munin22:28:351
11644992618,5cyclictest19583-21/usr/sbin/munin22:28:351
1164699251,9cyclictest18148-21chrt20:58:263
1164699251,9cyclictest18148-21chrt20:58:263
1164699251,8cyclictest14262-21chrt20:47:293
1164699251,8cyclictest14262-21chrt20:47:293
1164699251,6cyclictest5602-21chrt23:17:193
1164699251,6cyclictest5602-21chrt23:17:193
1164699251,6cyclictest32183-21chrt22:59:383
1164699251,6cyclictest32183-21chrt22:59:383
1164699251,6cyclictest28985-21chrt21:23:553
1164699251,6cyclictest28985-21chrt21:23:553
1164699251,6cyclictest28985-21chrt21:23:553
1164699251,6cyclictest22023-21chrt19:38:593
1164699251,6cyclictest22023-21chrt19:38:593
1164699251,6cyclictest21734-21chrt21:07:313
1164699251,6cyclictest21734-21chrt21:07:313
1164699251,6cyclictest12969-21chrt23:35:243
1164699251,6cyclictest12969-21chrt23:35:243
1164699251,5cyclictest12323-21chrt20:42:063
1164699251,5cyclictest12323-21chrt20:42:063
1164699251,5cyclictest1127-21chrt00:30:323
1164699251,5cyclictest1127-21chrt00:30:323
1164699251,5cyclictest0-21swapper/323:08:303
1164699251,5cyclictest0-21swapper/323:08:303
1164699251,5cyclictest0-21swapper/323:08:303
1164699251,4cyclictest0-21swapper/323:13:343
1164699251,4cyclictest0-21swapper/323:13:343
1164699251,4cyclictest0-21swapper/323:13:343
1164699251,4cyclictest0-21swapper/300:43:353
1164699250,6cyclictest4616-21chrt20:19:593
1164699250,6cyclictest4616-21chrt20:19:593
1164699250,6cyclictest4616-21chrt20:19:593
1164699250,6cyclictest31668-21chrt00:25:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional