You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 19:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Wed Jul 15, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3179221380,3sleep30-21swapper/309:58:443
3179221380,3sleep30-21swapper/309:58:443
1894621020,3sleep01580-21cyclictest07:58:410
1894621020,3sleep01580-21cyclictest07:58:410
1894621020,3sleep01580-21cyclictest07:58:410
169972900,1sleep00-21swapper/012:13:360
169972900,1sleep00-21swapper/012:13:360
169972900,1sleep00-21swapper/012:13:360
69952890,2chrt0-21swapper/308:53:403
69952890,2chrt0-21swapper/308:53:403
89412730,3chrt0-21swapper/310:23:443
89412730,3chrt0-21swapper/310:23:443
89412730,3chrt0-21swapper/310:23:443
52902640,2sleep20-21swapper/207:23:342
52902640,2sleep20-21swapper/207:23:342
267972590,3sleep00-21swapper/012:38:440
267972590,3sleep00-21swapper/012:38:440
267972590,3sleep00-21swapper/012:38:440
54062560,3chrt0-21swapper/011:43:330
54062560,3chrt0-21swapper/011:43:330
54062560,3chrt0-21swapper/011:43:330
32062500,2sleep00-21swapper/011:38:220
32062500,2sleep00-21swapper/011:38:220
213292500,2sleep20-21swapper/209:33:262
213292500,2sleep20-21swapper/209:33:262
233092490,1sleep2351rcuc/209:38:332
8872460,2sleep20-21swapper/211:28:442
8872460,2sleep20-21swapper/211:28:442
8872460,2sleep20-21swapper/211:28:442
208482440,3sleep30-21swapper/309:28:453
208482440,3sleep30-21swapper/309:28:453
109732420,2sleep00-21swapper/009:03:430
109732420,2sleep00-21swapper/009:03:430
253592410,2sleep125558-21grep09:43:351
253592410,2sleep125558-21grep09:43:351
1596994134,4cyclictest201rcuc/007:29:310
1596994134,4cyclictest201rcuc/007:29:310
74742400,2sleep17575-21memory07:28:371
74742400,2sleep17575-21memory07:28:371
1596994032,5cyclictest201rcuc/012:04:150
1596994032,5cyclictest201rcuc/012:04:150
1596994032,5cyclictest201rcuc/012:04:150
1596993821,4cyclictest90-1pr/ttyS008:33:430
1596993821,4cyclictest90-1pr/ttyS008:33:430
1596993821,4cyclictest90-1pr/ttyS008:33:430
1596993811,7cyclictest201rcuc/007:44:460
1596993811,7cyclictest201rcuc/007:44:460
1596993811,7cyclictest201rcuc/007:44:460
181532370,5sleep20-21swapper/210:48:442
181532370,5sleep20-21swapper/210:48:442
181532370,5sleep20-21swapper/210:48:442
1596993729,6cyclictest201rcuc/009:38:000
1596993729,6cyclictest201rcuc/009:38:000
1596993713,20cyclictest201rcuc/008:42:580
1596993713,20cyclictest201rcuc/008:42:580
69452360,1sleep06985-21/usr/sbin/munin07:28:320
69452360,1sleep06985-21/usr/sbin/munin07:28:320
92712350,3sleep30-21swapper/307:33:363
92712350,3sleep30-21swapper/307:33:363
55912350,3sleep20-21swapper/211:43:352
55912350,3sleep20-21swapper/211:43:352
55912350,3sleep20-21swapper/211:43:352
52572350,3chrt0-21swapper/208:48:402
52572350,3chrt0-21swapper/208:48:402
1596993528,4cyclictest201rcuc/008:09:330
1596993528,4cyclictest201rcuc/008:09:330
1596993520,4cyclictest181rcu_preempt11:58:260
1596993520,4cyclictest181rcu_preempt11:58:260
124482350,3sleep20-21swapper/209:08:382
124482350,3sleep20-21swapper/209:08:382
124482350,3sleep20-21swapper/209:08:382
124482350,3sleep20-21swapper/209:08:382
1596993428,4cyclictest201rcuc/011:28:170
1596993428,4cyclictest201rcuc/011:28:170
1596993427,5cyclictest20140-21grep09:28:370
1596993427,5cyclictest20140-21grep09:28:370
264722330,2chrt0-21swapper/208:18:412
264722330,2chrt0-21swapper/208:18:412
1602993328,4cyclictest1580-21cyclictest11:13:351
1602993328,4cyclictest1580-21cyclictest11:13:351
1602993328,4cyclictest1580-21cyclictest11:13:351
1602993325,6cyclictest7974-21processes11:48:431
1602993325,6cyclictest7974-21processes11:48:431
1602993325,6cyclictest21272-21latency_hist09:33:231
1602993325,6cyclictest21272-21latency_hist09:33:231
1596993329,3cyclictest12312-21processes10:33:410
1596993329,3cyclictest12312-21processes10:33:410
1596993326,5cyclictest8854-21latency_hist11:53:230
1596993326,5cyclictest8854-21latency_hist11:53:230
1596993325,6cyclictest27735-21fschecks_count08:23:340
1596993325,6cyclictest27735-21fschecks_count08:23:340
1596993324,2cyclictest181rcu_preempt11:13:350
1596993324,2cyclictest181rcu_preempt11:13:350
1596993324,2cyclictest181rcu_preempt11:13:350
1596993225,5cyclictest10878-21cut07:38:350
1596993225,5cyclictest10878-21cut07:38:350
1596993224,6cyclictest1580-21cyclictest10:13:340
1602993124,5cyclictest29003-21needreboot11:18:391
1602993124,5cyclictest29003-21needreboot11:18:391
1602993124,5cyclictest29003-21needreboot11:18:391
1602993123,6cyclictest5310-21fschecks_count07:23:341
1602993123,6cyclictest5310-21fschecks_count07:23:341
1602993122,7cyclictest8049-21latency_hist08:58:211
1602993122,7cyclictest8049-21latency_hist08:58:211
1596993127,3cyclictest24111-21fschecks_count12:33:330
1596993127,3cyclictest24111-21fschecks_count12:33:330
1596993127,3cyclictest24111-21fschecks_count12:33:330
1596993125,4cyclictest15057-21sendmail10:43:210
1596993125,4cyclictest15057-21sendmail10:43:210
1596993124,5cyclictest17572-21unixbench_multi07:53:450
1596993124,5cyclictest17572-21unixbench_multi07:53:450
1596993124,5cyclictest17572-21unixbench_multi07:53:450
1596993124,5cyclictest171ktimers/010:23:310
1596993124,5cyclictest171ktimers/010:23:310
1596993123,6cyclictest749-21runrttasks10:00:150
1596993123,6cyclictest749-21runrttasks10:00:150
163752300,3sleep2351rcuc/207:53:332
163752300,3sleep2351rcuc/207:53:332
163752300,3sleep2351rcuc/207:53:332
159699306,5cyclictest171ktimers/009:24:290
159699306,5cyclictest171ktimers/009:24:290
1596993024,4cyclictest90-1pr/ttyS009:22:290
1596993024,4cyclictest90-1pr/ttyS009:22:290
1596993024,4cyclictest90-1pr/ttyS009:22:290
1596993024,4cyclictest90-1pr/ttyS009:22:290
1596993023,5cyclictest7657-21unixbench_singl08:53:480
1596993023,5cyclictest7657-21unixbench_singl08:53:480
1596993020,8cyclictest17841-21netstat10:48:400
1596993020,8cyclictest17841-21netstat10:48:400
1596993020,8cyclictest17841-21netstat10:48:400
1596993016,5cyclictest90-1pr/ttyS011:10:460
1596993016,5cyclictest90-1pr/ttyS011:10:460
1596993016,5cyclictest90-1pr/ttyS011:10:460
1596993016,5cyclictest181rcu_preempt09:48:220
1596993016,5cyclictest181rcu_preempt09:48:220
1596993016,4cyclictest90-1pr/ttyS012:14:020
1596993016,4cyclictest90-1pr/ttyS012:14:020
1596993016,4cyclictest90-1pr/ttyS012:14:020
1596993016,4cyclictest181rcu_preempt09:17:080
1596993016,4cyclictest181rcu_preempt09:17:080
1596993016,4cyclictest181rcu_preempt09:17:080
271372290,1sleep2351rcuc/211:13:402
271372290,1sleep2351rcuc/211:13:402
271372290,1sleep2351rcuc/211:13:402
161672290,3sleep20-21swapper/209:18:352
161672290,3sleep20-21swapper/209:18:352
161672290,3sleep20-21swapper/209:18:352
161672290,3sleep20-21swapper/209:18:352
1602992920,7cyclictest12744-21netstat09:08:421
1602992920,7cyclictest12744-21netstat09:08:421
1602992920,7cyclictest12744-21netstat09:08:421
1602992920,7cyclictest12744-21netstat09:08:421
159699297,4cyclictest201rcuc/011:04:360
159699297,4cyclictest201rcuc/011:04:360
1596992923,4cyclictest31194-21latency_hist08:33:260
1596992923,4cyclictest31194-21latency_hist08:33:260
1596992923,4cyclictest29982-21latency_hist11:23:330
1596992923,4cyclictest29982-21latency_hist11:23:330
1596992923,4cyclictest29982-21latency_hist11:23:330
1596992922,5cyclictest8426-21cut08:58:340
1596992922,5cyclictest8426-21cut08:58:340
1596992922,5cyclictest25296-21sshd08:16:000
1596992918,9cyclictest21761-21cron08:08:210
1596992918,9cyclictest21761-21cron08:08:210
1596992918,9cyclictest21761-21cron08:08:210
1596992918,9cyclictest21761-21cron08:08:210
1596992916,11cyclictest25669-21latency_hist12:38:210
1596992916,11cyclictest25669-21latency_hist12:38:210
1596992916,11cyclictest25669-21latency_hist12:38:210
1596992916,11cyclictest25669-21latency_hist12:38:210
1596992913,8cyclictest1580-21cyclictest10:43:420
1596992913,8cyclictest1580-21cyclictest10:43:420
1596992913,8cyclictest1580-21cyclictest10:43:420
266842280,1sleep026844-21/usr/sbin/munin08:18:440
266842280,1sleep026844-21/usr/sbin/munin08:18:440
1602992822,5cyclictest26375-21kernelversion12:38:391
1602992822,5cyclictest26375-21kernelversion12:38:391
1602992822,4cyclictest1648-21ntpd11:08:431
1602992822,4cyclictest1648-21ntpd11:08:431
1602992822,4cyclictest1648-21ntpd11:08:431
1602992821,5cyclictest15364-21needreboot12:08:391
1602992821,5cyclictest15364-21needreboot12:08:391
1602992821,5cyclictest15364-21needreboot12:08:391
1602992821,5cyclictest15235-21tr07:48:401
1602992821,5cyclictest15235-21tr07:48:401
1602992821,5cyclictest15235-21tr07:48:401
1602992820,6cyclictest31037-21fschecks_time09:58:341
1602992820,6cyclictest29714-21fschecks_time08:28:341
1602992820,6cyclictest29714-21fschecks_time08:28:341
159699289,17cyclictest171ktimers/008:48:310
159699289,17cyclictest171ktimers/008:48:310
159699285,4cyclictest201rcuc/009:42:000
159699285,4cyclictest201rcuc/009:42:000
1596992823,3cyclictest0-21swapper/010:08:440
1596992822,4cyclictest22283-21fschecks_count12:28:350
1596992822,4cyclictest22283-21fschecks_count12:28:350
1596992822,4cyclictest22283-21fschecks_count12:28:350
1596992822,4cyclictest0-21swapper/010:03:450
1596992822,4cyclictest0-21swapper/010:03:450
215712270,3sleep30-21swapper/310:58:423
215712270,3sleep30-21swapper/310:58:423
215712270,3sleep30-21swapper/310:58:423
1602992722,3cyclictest0-21swapper/109:48:451
1602992722,3cyclictest0-21swapper/109:48:451
1602992722,3cyclictest0-21swapper/109:48:451
1602992722,2cyclictest0-21swapper/107:48:231
1602992722,2cyclictest0-21swapper/107:48:231
1602992722,2cyclictest0-21swapper/107:48:231
1602992721,5cyclictest31076-21rt-features11:23:431
1602992721,5cyclictest31076-21rt-features11:23:431
160299271,24cyclictest79402sleep110:23:341
160299271,24cyclictest79402sleep110:23:341
159699275,20cyclictest171ktimers/010:27:570
159699275,20cyclictest171ktimers/010:27:570
159699275,20cyclictest171ktimers/010:27:570
1596992723,3cyclictest27366-21gltestperf09:48:350
1596992723,3cyclictest27366-21gltestperf09:48:350
1596992723,3cyclictest27366-21gltestperf09:48:350
1596992722,3cyclictest90-1pr/ttyS010:17:010
1596992722,3cyclictest90-1pr/ttyS010:17:010
1596992722,3cyclictest90-1pr/ttyS010:17:010
1596992722,3cyclictest0-21swapper/008:48:210
1596992721,3cyclictest0-21swapper/010:28:460
1596992721,3cyclictest0-21swapper/010:28:460
1596992721,3cyclictest0-21swapper/010:28:460
1596992720,5cyclictest4843-21sshd11:41:020
1596992720,5cyclictest4843-21sshd11:41:020
1596992720,4cyclictest0-21swapper/009:58:340
5592260,1chrt0-21swapper/011:28:420
5592260,1chrt0-21swapper/011:28:420
5592260,1chrt0-21swapper/011:28:420
1602992621,3cyclictest0-21swapper/107:38:431
1602992621,3cyclictest0-21swapper/107:38:431
1602992621,3cyclictest0-21swapper/107:38:431
1602992620,4cyclictest1346-21avahi-daemon11:41:031
1602992620,4cyclictest1346-21avahi-daemon11:41:031
1602992619,5cyclictest10174-21sshd07:34:001
1602992619,5cyclictest10174-21sshd07:34:001
1602992612,11cyclictest729-21dbus-daemon11:03:421
1602992612,11cyclictest729-21dbus-daemon11:03:421
159699266,4cyclictest0-21swapper/009:08:360
159699266,4cyclictest0-21swapper/009:08:360
159699266,4cyclictest0-21swapper/009:08:360
159699266,4cyclictest0-21swapper/009:08:360
1596992620,3cyclictest0-21swapper/007:48:440
1596992620,3cyclictest0-21swapper/007:48:440
1596992620,3cyclictest0-21swapper/007:48:440
1596992611,12cyclictest0-21swapper/009:53:350
1602992521,2cyclictest0-21swapper/109:23:441
1602992521,2cyclictest0-21swapper/109:23:441
1602992520,3cyclictest0-21swapper/110:28:341
1602992520,3cyclictest0-21swapper/110:28:341
1602992518,5cyclictest28230-21kernelversion12:43:381
1602992518,5cyclictest28230-21kernelversion12:43:381
1602992518,5cyclictest28230-21kernelversion12:43:381
1602992518,5cyclictest14114-21ntp_kernel_pll_10:38:411
1602992518,5cyclictest14114-21ntp_kernel_pll_10:38:411
1602992518,4cyclictest0-21swapper/112:03:451
1602992518,4cyclictest0-21swapper/112:03:451
1602992518,4cyclictest0-21swapper/112:03:451
1602992517,6cyclictest1-21systemd08:47:001
1602992516,7cyclictest25059-21latency_hist09:43:251
1602992516,7cyclictest25059-21latency_hist09:43:251
1596992520,4cyclictest90-1pr/ttyS012:18:470
1596992520,4cyclictest90-1pr/ttyS012:18:470
1596992520,4cyclictest90-1pr/ttyS012:18:470
1596992519,3cyclictest0-21swapper/011:58:350
1596992519,3cyclictest0-21swapper/011:58:350
1596992519,3cyclictest0-21swapper/011:58:350
1596992518,5cyclictest1580-21cyclictest10:55:100
1596992518,5cyclictest1580-21cyclictest10:55:100
1596992514,8cyclictest14678-21systemd-journal07:42:000
1596992514,8cyclictest14678-21systemd-journal07:42:000
1596992514,8cyclictest14678-21systemd-journal07:42:000
1596992511,4cyclictest201rcuc/010:58:410
1596992511,4cyclictest201rcuc/010:58:410
1596992511,4cyclictest201rcuc/010:58:410
227052240,3sleep30-21swapper/308:08:413
227052240,3sleep30-21swapper/308:08:413
171852240,3sleep3431rcuc/310:48:333
171852240,3sleep3431rcuc/310:48:333
1602992418,4cyclictest9042-21ntp_states08:58:411
1602992418,4cyclictest9042-21ntp_states08:58:411
1602992417,5cyclictest10837-21latency_hist11:58:291
1602992417,5cyclictest10837-21latency_hist11:58:291
1602992416,6cyclictest4674-21fschecks_count08:48:351
1602992416,6cyclictest4674-21fschecks_count08:48:351
1602992416,6cyclictest17600-21latency_hist09:23:271
1602992416,6cyclictest17600-21latency_hist09:23:271
1602992416,6cyclictest17600-21latency_hist09:23:271
1602992416,6cyclictest17600-21latency_hist09:23:271
1609992314,7cyclictest22114-21chrt12:28:332
1609992314,7cyclictest22114-21chrt12:28:332
1609992314,7cyclictest22114-21chrt12:28:332
1602992318,3cyclictest0-21swapper/112:03:221
1602992318,3cyclictest0-21swapper/112:03:221
1602992318,3cyclictest0-21swapper/112:03:221
1602992318,3cyclictest0-21swapper/108:08:401
1602992318,3cyclictest0-21swapper/108:08:401
1602992318,2cyclictest0-21swapper/112:23:371
1602992318,2cyclictest0-21swapper/112:23:371
1602992318,2cyclictest0-21swapper/112:23:371
1602992318,2cyclictest0-21swapper/110:58:361
1602992318,2cyclictest0-21swapper/110:58:361
1602992318,2cyclictest0-21swapper/110:58:361
1602992317,3cyclictest0-21swapper/110:53:471
1602992317,3cyclictest0-21swapper/110:53:471
1602992316,5cyclictest31267-21kernelversion09:58:361
1602992316,5cyclictest31267-21kernelversion09:58:361
1602992314,7cyclictest21934-21kernelversion09:33:351
1602992314,7cyclictest21934-21kernelversion09:33:351
160299231,2cyclictest700-21192.168.115.40-manager12:38:211
160299231,2cyclictest700-21192.168.115.40-manager12:38:211
1602992312,9cyclictest14678-21systemd-journal11:33:211
1602992312,9cyclictest14678-21systemd-journal11:33:211
1602992312,9cyclictest14678-21systemd-journal11:33:211
220092220,4sleep30-21swapper/309:33:353
220092220,4sleep30-21swapper/309:33:353
160999221,17cyclictest219082sleep209:33:342
160299228,3cyclictest181rcu_preempt09:06:401
160299228,3cyclictest181rcu_preempt09:06:401
1602992218,2cyclictest0-21swapper/107:53:451
1602992218,2cyclictest0-21swapper/107:53:451
1602992218,2cyclictest0-21swapper/107:53:451
1602992217,3cyclictest90-1pr/ttyS011:33:401
1602992217,3cyclictest90-1pr/ttyS011:33:401
1602992217,3cyclictest0-21swapper/110:33:461
1602992217,3cyclictest0-21swapper/110:33:461
1602992217,3cyclictest0-21swapper/110:33:321
1602992217,3cyclictest0-21swapper/110:33:321
1602992216,4cyclictest18382-21grep10:48:451
1602992216,4cyclictest18382-21grep10:48:451
1602992216,4cyclictest18382-21grep10:48:451
1602992216,4cyclictest0-21swapper/110:13:211
1602992216,4cyclictest0-21swapper/110:13:211
1602992216,3cyclictest0-21swapper/112:28:461
1602992216,3cyclictest0-21swapper/112:28:461
1602992216,3cyclictest0-21swapper/112:28:461
1602992216,3cyclictest0-21swapper/112:23:431
1602992216,3cyclictest0-21swapper/112:23:431
1602992216,3cyclictest0-21swapper/112:23:431
1602992216,3cyclictest0-21swapper/107:58:391
1602992216,3cyclictest0-21swapper/107:58:391
1602992216,3cyclictest0-21swapper/107:58:391
1602992215,5cyclictest2641-21latency_hist08:43:321
1602992214,6cyclictest30009-21timerwakeupswit09:53:451
1602992211,9cyclictest14678-21systemd-journal09:17:281
1602992211,9cyclictest14678-21systemd-journal09:17:281
1602992211,9cyclictest14678-21systemd-journal09:17:281
66112210,3sleep30-21swapper/310:18:423
66112210,3sleep30-21swapper/310:18:423
1602992117,2cyclictest0-21swapper/110:33:311
1602992116,3cyclictest90-1pr/ttyS008:08:361
1602992116,3cyclictest90-1pr/ttyS008:08:361
1602992116,3cyclictest90-1pr/ttyS008:08:361
1602992116,3cyclictest0-21swapper/112:18:211
1602992116,3cyclictest0-21swapper/112:18:211
1602992116,3cyclictest0-21swapper/112:18:211
1602992116,3cyclictest0-21swapper/108:23:351
1602992116,3cyclictest0-21swapper/108:23:351
1602992115,5cyclictest1580-21cyclictest08:38:391
160299211,3cyclictest700-21192.168.115.40-manager08:14:251
46532200,3sleep30-21swapper/311:38:463
46532200,3sleep30-21swapper/311:38:463
1617992011,6cyclictest431rcuc/307:43:263
1617992011,6cyclictest431rcuc/307:43:263
1617992011,6cyclictest431rcuc/307:43:263
160999203,9cyclictest0-21swapper/207:34:002
160999203,9cyclictest0-21swapper/207:34:002
1602992016,2cyclictest0-21swapper/108:18:351
1602992016,2cyclictest0-21swapper/108:18:351
1602992016,2cyclictest0-21swapper/108:03:461
1602992015,3cyclictest0-21swapper/110:13:351
1602992015,3cyclictest0-21swapper/110:13:351
1602992015,3cyclictest0-21swapper/110:13:351
1602992015,3cyclictest0-21swapper/110:03:451
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional