You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 04:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sun Jul 12, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1324721050,3sleep30-21swapper/321:48:323
1324721050,3sleep30-21swapper/321:48:323
1324721050,3sleep30-21swapper/321:48:323
308872900,3sleep30-21swapper/300:03:193
308872900,3sleep30-21swapper/300:03:193
32562840,3sleep30-21swapper/322:48:333
32562840,3sleep30-21swapper/322:48:333
32562840,3sleep30-21swapper/322:48:333
146472770,3sleep10-21swapper/123:18:341
283362740,9sleep1281ktimers/121:03:231
283362740,9sleep1281ktimers/121:03:231
283362740,9sleep1281ktimers/121:03:231
151242730,4chrt0-21swapper/320:27:593
151242730,4chrt0-21swapper/320:27:593
151242730,4chrt0-21swapper/320:27:593
81642720,3chrt0-21swapper/300:28:313
81642720,3chrt0-21swapper/300:28:313
81642720,3chrt0-21swapper/300:28:313
292512700,2chrt29254-21cpuspeed23:58:301
95212690,2chrt9523-21latency_hist20:13:181
95212690,2chrt9523-21latency_hist20:13:181
170232660,3sleep30-21swapper/320:33:183
170232660,3sleep30-21swapper/320:33:183
170232660,3sleep30-21swapper/320:33:183
53302650,1chrt0-21swapper/100:18:401
53302650,1chrt0-21swapper/100:18:401
253102630,1sleep10-21swapper/123:48:211
253102630,1sleep10-21swapper/123:48:211
19895995425,27cyclictest24493-21latency_hist20:53:180
19895995425,27cyclictest24493-21latency_hist20:53:180
152472530,3sleep10-21swapper/121:53:321
152472530,3sleep10-21swapper/121:53:321
152472530,3sleep10-21swapper/121:53:321
248252500,2chrt0-21swapper/322:18:353
248252500,2chrt0-21swapper/322:18:353
19895995041,7cyclictest0-21swapper/000:03:230
19895995041,7cyclictest0-21swapper/000:03:230
153202500,2sleep20-21swapper/221:53:332
153202500,2sleep20-21swapper/221:53:332
153202500,2sleep20-21swapper/221:53:332
303602490,3sleep30-21swapper/321:08:323
303602490,3sleep30-21swapper/321:08:323
262362480,3sleep30-21swapper/319:28:423
262362480,3sleep30-21swapper/319:28:423
1989599480,6cyclictest5628-21chrt20:01:550
1989599480,6cyclictest5628-21chrt20:01:550
262552470,2sleep20-21swapper/223:48:392
262552470,2sleep20-21swapper/223:48:392
1989599461,8cyclictest26951-21taskset23:51:260
1989599461,8cyclictest26951-21taskset23:51:260
1989599460,8cyclictest28958-21seq23:58:170
1989599460,8cyclictest28958-21seq23:58:170
1989599460,8cyclictest28525-21seq19:34:460
1989599460,8cyclictest28525-21seq19:34:460
1989599460,8cyclictest28525-21seq19:34:460
1989599451,7cyclictest1688-21taskset19:49:290
1989599451,7cyclictest1688-21taskset19:49:290
1989599441,6cyclictest23260-21seq23:42:090
1989599441,6cyclictest23260-21seq23:42:090
1989599441,6cyclictest23260-21seq23:42:090
1989599421,6cyclictest5571-21taskset00:19:000
1989599421,6cyclictest5571-21taskset00:19:000
19895994211,27cyclictest700-21192.168.115.40-manager20:24:130
19895994211,27cyclictest700-21192.168.115.40-manager20:24:130
19895994211,27cyclictest700-21192.168.115.40-manager20:24:130
19895994121,18cyclictest14678-21systemd-journal22:25:590
19895994121,18cyclictest14678-21systemd-journal22:25:590
19895994121,18cyclictest14678-21systemd-journal22:25:590
1989599402,6cyclictest749-21runrttasks00:11:200
1989599402,6cyclictest749-21runrttasks00:11:200
1989599402,6cyclictest749-21runrttasks00:11:200
10972400,10sleep1281ktimers/119:48:381
10972400,10sleep1281ktimers/119:48:381
19895993931,6cyclictest18233-21fschecks_time23:28:310
19895993930,6cyclictest21343-21mailstats22:08:400
19895993930,6cyclictest21343-21mailstats22:08:400
19895993930,6cyclictest21343-21mailstats22:08:400
1989599391,9cyclictest15570-21ntp_states21:53:370
1989599391,9cyclictest15570-21ntp_states21:53:370
1989599391,9cyclictest15570-21ntp_states21:53:370
29432380,2chrt2945-21latency00:13:341
29432380,2chrt2945-21latency00:13:341
19903993829,4cyclictest0-21swapper/100:03:421
19903993829,4cyclictest0-21swapper/100:03:421
19903993829,4cyclictest0-21swapper/100:03:421
19895993829,7cyclictest31019-21/usr/sbin/munin00:03:380
19895993829,7cyclictest31019-21/usr/sbin/munin00:03:380
19895993829,7cyclictest31019-21/usr/sbin/munin00:03:380
1989599381,9cyclictest32707-21kernelversion21:13:340
1989599381,9cyclictest32707-21kernelversion21:13:340
1989599381,9cyclictest19198-21latency_hist20:38:300
1989599381,8cyclictest9514-21gltestperf21:38:310
1989599381,8cyclictest9514-21gltestperf21:38:310
1989599381,7cyclictest7971-21/usr/sbin/munin00:28:350
1989599381,7cyclictest7971-21/usr/sbin/munin00:28:350
1989599381,7cyclictest18008-21cpuspeed23:28:290
1989599381,7cyclictest18008-21cpuspeed23:28:290
1989599380,35cyclictest4195-21/usr/sbin/munin21:23:350
1989599380,35cyclictest4195-21/usr/sbin/munin21:23:350
270622370,3sleep2351rcuc/222:23:402
270622370,3sleep2351rcuc/222:23:402
270622370,3sleep2351rcuc/222:23:402
1989599374,5cyclictest11958-21cut20:18:310
1989599374,5cyclictest11958-21cut20:18:310
1989599373,5cyclictest0-21swapper/020:46:430
1989599373,5cyclictest0-21swapper/020:46:430
1989599371,9cyclictest25555-21fschecks_time19:28:330
1989599371,9cyclictest25555-21fschecks_time19:28:330
1989599371,9cyclictest1838-21ntp_states22:43:380
1989599371,9cyclictest1838-21ntp_states22:43:380
1989599371,9cyclictest17122-21grep20:33:240
1989599371,9cyclictest17122-21grep20:33:240
1989599371,9cyclictest17122-21grep20:33:240
1989599371,9cyclictest10993-21timerwakeupswit00:33:420
1989599371,9cyclictest10993-21timerwakeupswit00:33:420
19895993719,16cyclictest0-21swapper/019:43:350
19895993719,16cyclictest0-21swapper/019:43:350
1989599371,8cyclictest16498-21if_eth023:23:320
1989599371,8cyclictest16498-21if_eth023:23:320
1989599371,8cyclictest16498-21if_eth023:23:320
1989599371,5cyclictest17430-21ntp_states21:58:400
1989599371,5cyclictest17430-21ntp_states21:58:400
1989599371,10cyclictest4914-21df22:53:300
1989599371,10cyclictest4914-21df22:53:300
289882360,2sleep10-21swapper/119:38:291
289882360,2sleep10-21swapper/119:38:291
289882360,2sleep10-21swapper/119:38:291
1989599362,8cyclictest11775-21/usr/sbin/munin00:38:410
1989599362,8cyclictest11775-21/usr/sbin/munin00:38:410
1989599361,9cyclictest29314-21tr19:38:320
1989599361,9cyclictest29314-21tr19:38:320
1989599361,9cyclictest29314-21tr19:38:320
1989599361,9cyclictest24151-21/usr/sbin/munin19:23:370
1989599361,8cyclictest27843-21users20:58:440
1989599361,8cyclictest27843-21users20:58:440
1989599361,8cyclictest27843-21users20:58:440
1989599361,8cyclictest21635-21latency_hist23:38:270
1989599361,8cyclictest21635-21latency_hist23:38:270
1989599361,8cyclictest21635-21latency_hist23:38:270
1989599361,8cyclictest10080-21grep20:13:290
1989599361,8cyclictest10080-21grep20:13:290
1989599361,7cyclictest25617-21sshd22:21:000
1989599361,7cyclictest25617-21sshd22:21:000
1989599361,7cyclictest10289-21awk23:08:190
1989599361,7cyclictest10289-21awk23:08:190
1989599361,6cyclictest14256-21sshd21:48:590
1989599361,6cyclictest14256-21sshd21:48:590
1989599361,6cyclictest14256-21sshd21:48:590
1989599361,5cyclictest0-21swapper/022:06:460
1989599361,5cyclictest0-21swapper/022:06:460
1989599361,4cyclictest700-21192.168.115.40-manager23:09:180
1989599361,4cyclictest700-21192.168.115.40-manager23:09:180
1989599361,4cyclictest700-21192.168.115.40-manager23:09:180
1989599361,4cyclictest3852-21latency_hist19:58:270
1989599361,4cyclictest3852-21latency_hist19:58:270
1989599361,4cyclictest3852-21latency_hist19:58:270
1989599351,9cyclictest7180-21sort21:33:230
1989599351,9cyclictest7180-21sort21:33:230
1989599351,9cyclictest6465-21gltestperf00:23:320
1989599351,9cyclictest6465-21gltestperf00:23:320
1989599351,9cyclictest6465-21gltestperf00:23:320
1989599351,9cyclictest5144-21gltestperf22:53:320
1989599351,9cyclictest5144-21gltestperf22:53:320
1989599351,9cyclictest5144-21gltestperf22:53:320
1989599351,8cyclictest749-21runrttasks20:33:310
1989599351,8cyclictest749-21runrttasks20:33:310
1989599351,8cyclictest32559-21mailstats22:38:390
1989599351,8cyclictest32559-21mailstats22:38:390
1989599351,7cyclictest9591-21sed20:13:200
1989599351,7cyclictest9591-21sed20:13:200
1989599351,7cyclictest7894-21sed22:58:430
1989599351,7cyclictest7894-21sed22:58:430
1989599351,7cyclictest7894-21sed22:58:430
1989599351,7cyclictest7722-21if_eth021:33:330
1989599351,7cyclictest7722-21if_eth021:33:330
1989599351,7cyclictest3157-21/usr/sbin/munin22:48:330
1989599351,7cyclictest24691-21/usr/sbin/munin20:53:340
1989599351,7cyclictest24691-21/usr/sbin/munin20:53:340
1989599351,7cyclictest23356-21latency_hist23:43:330
1989599351,7cyclictest23356-21latency_hist23:43:330
1989599351,7cyclictest23045-21ntp_states22:13:360
1989599351,7cyclictest23045-21ntp_states22:13:360
1989599351,7cyclictest13648-21taskset23:14:060
1989599351,7cyclictest13648-21taskset23:14:060
1989599351,7cyclictest13648-21taskset23:14:060
1989599351,5cyclictest28396-21irqstats22:28:340
1989599351,5cyclictest28396-21irqstats22:28:340
1989599351,4cyclictest11556-21irqstats21:43:330
1989599351,4cyclictest11556-21irqstats21:43:330
1989599351,10cyclictest3310-21grep00:13:390
1989599351,10cyclictest3310-21grep00:13:390
1989599350,8cyclictest19411-21chrt23:30:390
1989599350,8cyclictest19411-21chrt23:30:390
1989599350,32cyclictest29119-21bingrep21:03:350
1989599350,32cyclictest29119-21bingrep21:03:350
47022340,5sleep14706-21unixbench-2d21:23:411
47022340,5sleep14706-21unixbench-2d21:23:411
1989599341,9cyclictest31545-21unixbench_singl21:08:440
1989599341,9cyclictest31545-21unixbench_singl21:08:440
1989599341,9cyclictest20162-21sort20:38:410
1989599341,9cyclictest20162-21sort20:38:410
1989599341,9cyclictest20162-21sort20:38:410
1989599341,8cyclictest6751-21open_files20:03:380
1989599341,8cyclictest6751-21open_files20:03:380
1989599341,8cyclictest6751-21open_files20:03:380
1989599341,6cyclictest0-21swapper/022:38:240
1989599341,6cyclictest0-21swapper/022:38:240
1989599341,10cyclictest1649-21/usr/sbin/munin21:18:330
1989599341,10cyclictest1649-21/usr/sbin/munin21:18:330
257282330,3sleep30-21swapper/320:53:403
257282330,3sleep30-21swapper/320:53:403
19903993326,5cyclictest30053-21gltestperf22:33:311
19903993226,4cyclictest25560-21cron22:20:191
19903993226,4cyclictest25560-21cron22:20:191
19903993124,5cyclictest11516-21latency_hist20:18:231
19903993124,5cyclictest11516-21latency_hist20:18:231
197392310,3sleep20-21swapper/222:03:422
197392310,3sleep20-21swapper/222:03:422
19903993024,5cyclictest15241-21threads23:18:431
19903993024,5cyclictest15241-21threads23:18:431
19903993024,5cyclictest15241-21threads23:18:431
19903993024,4cyclictest32033-21sh21:13:211
19903993024,4cyclictest32033-21sh21:13:211
19903993024,3cyclictest0-21swapper/123:38:321
19903993024,3cyclictest0-21swapper/123:38:321
19903993024,3cyclictest0-21swapper/123:38:321
19903993023,6cyclictest9674-21kernelversion21:38:321
19903993023,6cyclictest9674-21kernelversion21:38:321
19903992924,3cyclictest4268-21fschecks_count19:58:311
19903992924,3cyclictest4268-21fschecks_count19:58:311
19903992923,5cyclictest1346-21avahi-daemon20:23:391
19903992923,5cyclictest1346-21avahi-daemon20:23:391
19903992923,5cyclictest1346-21avahi-daemon20:23:391
19903992923,4cyclictest19885-21cyclictest20:54:271
19903992923,4cyclictest19885-21cyclictest20:54:271
19903992923,4cyclictest12378-21missed_timers00:38:381
19903992923,4cyclictest12378-21missed_timers00:38:381
19903992922,6cyclictest22417-21fschecks_count22:13:291
19903992922,6cyclictest22417-21fschecks_count22:13:291
19903992922,6cyclictest22417-21fschecks_count22:13:291
19903992922,5cyclictest32447-21cut21:13:321
19903992922,5cyclictest32447-21cut21:13:321
19903992922,5cyclictest19220-21needreboot22:03:371
19903992922,5cyclictest19220-21needreboot22:03:371
19903992921,5cyclictest19885-21cyclictest20:38:271
19903992921,5cyclictest19885-21cyclictest20:38:271
19903992921,5cyclictest19885-21cyclictest20:38:271
19903992824,2cyclictest0-21swapper/122:53:321
19903992824,2cyclictest0-21swapper/122:53:321
19903992824,2cyclictest0-21swapper/122:53:321
19903992822,5cyclictest13637-21sshd23:13:591
19903992822,5cyclictest13637-21sshd23:13:591
19903992822,4cyclictest729-21dbus-daemon00:03:181
19903992822,4cyclictest729-21dbus-daemon00:03:181
19903992821,5cyclictest8162-21wc21:33:391
19903992821,5cyclictest8162-21wc21:33:391
19903992820,6cyclictest22923-21sort22:13:331
19903992820,6cyclictest22923-21sort22:13:331
19903992723,2cyclictest0-21swapper/121:23:181
19903992723,2cyclictest0-21swapper/121:23:181
19903992722,4cyclictest547-21rs:main1
19903992722,4cyclictest547-21rs:main1
19903992721,5cyclictest18371-21latency_hist22:03:191
19903992721,5cyclictest18371-21latency_hist22:03:191
19903992721,5cyclictest11562-21kernelversion21:43:331
19903992721,5cyclictest11562-21kernelversion21:43:331
19903992721,4cyclictest10972-21kernelversion23:08:351
19903992721,4cyclictest10972-21kernelversion23:08:351
19903992721,4cyclictest10972-21kernelversion23:08:351
19903992721,3cyclictest0-21swapper/119:58:301
19903992720,5cyclictest19885-21cyclictest20:29:201
19903992720,5cyclictest19885-21cyclictest20:29:201
19903992720,5cyclictest19885-21cyclictest20:29:201
19903992720,4cyclictest0-21swapper/122:23:401
19903992720,4cyclictest0-21swapper/122:23:401
19903992720,4cyclictest0-21swapper/122:23:401
105822270,4sleep30-21swapper/320:13:363
105822270,4sleep30-21swapper/320:13:363
60902260,3sleep30-21swapper/300:23:253
60902260,3sleep30-21swapper/300:23:253
44222260,2sleep30-21swapper/300:18:303
44222260,2sleep30-21swapper/300:18:303
19903992622,3cyclictest2573-21gltestperf19:53:321
19903992622,3cyclictest2573-21gltestperf19:53:321
19903992622,3cyclictest20200-21head23:33:321
19903992622,2cyclictest0-21swapper/122:48:181
19903992622,2cyclictest0-21swapper/122:48:181
308412250,3chrt0-21swapper/122:33:401
308412250,3chrt0-21swapper/122:33:401
1992199254,12cyclictest0-21swapper/300:03:423
1992199254,12cyclictest0-21swapper/300:03:423
1992199254,12cyclictest0-21swapper/300:03:423
19903992521,2cyclictest0-21swapper/121:08:181
19903992521,2cyclictest0-21swapper/121:08:181
19903992521,2cyclictest0-21swapper/100:08:321
19903992521,2cyclictest0-21swapper/100:08:321
19903992521,2cyclictest0-21swapper/100:08:321
19903992519,4cyclictest6164-21/usr/sbin/munin00:23:391
19903992519,4cyclictest6164-21/usr/sbin/munin00:23:391
19903992519,4cyclictest6164-21/usr/sbin/munin00:23:391
19903992419,4cyclictest6478-21kernelversion20:03:341
19903992419,4cyclictest6478-21kernelversion20:03:341
19903992419,4cyclictest6478-21kernelversion20:03:341
19903992419,4cyclictest23105-21fschecks_time20:48:311
19903992419,4cyclictest23105-21fschecks_time20:48:311
19903992419,3cyclictest0-21swapper/100:36:511
19903992419,3cyclictest0-21swapper/100:36:511
19903992418,4cyclictest749-21runrttasks21:28:331
19903992418,4cyclictest749-21runrttasks21:28:331
19903992418,3cyclictest0-21swapper/122:58:421
19903992418,3cyclictest0-21swapper/122:58:421
19903992418,3cyclictest0-21swapper/122:58:421
19903992417,4cyclictest0-21swapper/119:43:411
19903992417,4cyclictest0-21swapper/119:43:411
19903992416,7cyclictest21196-21fschecks_time20:43:311
19903992416,7cyclictest21196-21fschecks_time20:43:311
19903992416,6cyclictest28193-21gltestperf22:28:331
19903992319,2cyclictest0-21swapper/120:38:391
19903992319,2cyclictest0-21swapper/120:38:391
19903992319,2cyclictest0-21swapper/120:38:391
19903992318,3cyclictest0-21swapper/123:28:271
19903992318,3cyclictest0-21swapper/123:28:271
19903992318,3cyclictest0-21swapper/123:28:271
19903992318,3cyclictest0-21swapper/123:08:181
19903992318,3cyclictest0-21swapper/123:08:181
19903992317,5cyclictest4865-21cpuspeed22:53:301
19903992317,5cyclictest4865-21cpuspeed22:53:301
19903992317,4cyclictest0-21swapper/123:33:401
19903992317,4cyclictest0-21swapper/123:33:401
19903992317,4cyclictest0-21swapper/123:33:401
19903992317,3cyclictest0-21swapper/122:43:311
19903992316,5cyclictest25960-21kernelversion23:48:341
19903992316,5cyclictest25960-21kernelversion23:48:341
19903992218,3cyclictest13260-21latency_hist20:23:181
19903992218,3cyclictest13260-21latency_hist20:23:181
19903992217,3cyclictest0-21swapper/123:58:181
19903992217,2cyclictest0-21swapper/119:38:331
19903992217,2cyclictest0-21swapper/119:38:331
19903992217,2cyclictest0-21swapper/119:38:331
19903992216,4cyclictest0-21swapper/122:38:411
19903992115,4cyclictest0-21swapper/123:28:411
19903992115,3cyclictest0-21swapper/122:48:411
19903992112,7cyclictest13461-21ls21:48:331
19903992112,7cyclictest13461-21ls21:48:331
19903992112,7cyclictest13461-21ls21:48:331
171812210,3sleep30-21swapper/321:58:373
171812210,3sleep30-21swapper/321:58:373
19921992012,5cyclictest12282chrt22:43:313
19911992014,4cyclictest200582sleep223:33:312
19903992015,3cyclictest0-21swapper/119:28:181
1992199191,14cyclictest29682-21chrt22:33:263
1992199191,14cyclictest29682-21chrt22:33:263
19903991914,4cyclictest8551-21latency00:28:341
19903991914,4cyclictest8551-21latency00:28:341
1992199184,8cyclictest0-21swapper/319:48:383
1992199184,8cyclictest0-21swapper/319:48:383
19921991813,3cyclictest129662sleep300:38:443
19921991813,3cyclictest129662sleep300:38:443
1992199181,13cyclictest29086-21chrt23:58:243
1992199181,13cyclictest29086-21chrt23:58:243
1992199181,12cyclictest25079-21chrt19:28:183
1992199181,12cyclictest18003-21chrt23:28:293
1992199181,12cyclictest18003-21chrt23:28:293
1992199181,12cyclictest18003-21chrt23:28:293
1991199182,11cyclictest0-21swapper/222:28:352
1991199182,11cyclictest0-21swapper/222:28:352
1991199181,11cyclictest24736-21taskset23:43:402
1991199181,11cyclictest24736-21taskset23:43:402
248882170,3sleep20-21swapper/220:53:312
248882170,3sleep20-21swapper/220:53:312
19921991711,3cyclictest289372sleep321:03:333
19921991711,3cyclictest289372sleep321:03:333
1992199171,10cyclictest23248-21taskset23:42:003
1992199171,10cyclictest23248-21taskset23:42:003
1992199171,10cyclictest23248-21taskset23:42:003
19921991710,5cyclictest211852sleep322:08:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional