You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-09 - 22:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Mon Mar 09, 2026 12:44:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2719021400,3chrt0-21swapper/110:26:451
2719021400,3chrt0-21swapper/110:26:451
1468621340,2sleep20-21swapper/211:21:382
1468621340,2sleep20-21swapper/211:21:382
2169221310,3sleep30-21swapper/308:46:413
2471021200,2sleep10-21swapper/110:21:391
2471021200,2sleep10-21swapper/110:21:391
2471021200,2sleep10-21swapper/110:21:391
104162850,3chrt0-21swapper/212:36:392
104162850,3chrt0-21swapper/212:36:392
247262790,3sleep30-21swapper/307:27:143
247262790,3sleep30-21swapper/307:27:143
60592730,3sleep30-21swapper/310:56:463
60592730,3sleep30-21swapper/310:56:463
256292720,8sleep125633-21fschecks_time11:51:361
154472720,3sleep30-21swapper/309:56:403
154472720,3sleep30-21swapper/309:56:403
325342710,3sleep00-21swapper/010:41:440
283992700,3sleep30-21swapper/309:06:313
293082680,2chrt29307-21fschecks_count12:01:371
293082680,2chrt29307-21fschecks_count12:01:371
197652660,1chrt0-21swapper/111:36:321
197652660,1chrt0-21swapper/111:36:321
304232620,5sleep30-21swapper/310:36:403
304232620,5sleep30-21swapper/310:36:403
4612610,2sleep029702-21kworker/0:2+events09:16:420
4612610,2sleep029702-21kworker/0:2+events09:16:420
245012600,3sleep20-21swapper/208:56:252
135532590,2chrt13551-21needreboot09:51:390
135532590,2chrt13551-21needreboot09:51:390
175182580,3chrt0-21swapper/308:36:363
175182580,3chrt0-21swapper/308:36:363
175182580,3chrt0-21swapper/308:36:363
122332560,2sleep10-21swapper/111:16:301
122332560,2sleep10-21swapper/111:16:301
323112540,3sleep20-21swapper/209:16:352
323112540,3sleep20-21swapper/209:16:352
253942530,4sleep30-21swapper/311:51:343
97152510,3sleep30-21swapper/308:16:353
97152510,3sleep30-21swapper/308:16:353
262102510,3sleep20-21swapper/207:31:432
262102510,3sleep20-21swapper/207:31:432
91092490,1sleep10-21swapper/109:41:311
91092490,1sleep10-21swapper/109:41:311
52272470,3sleep00-21swapper/012:21:470
52272470,3sleep00-21swapper/012:21:470
79732440,2sleep20-21swapper/212:31:332
288692440,1chrt0-21swapper/109:06:371
288692440,1chrt0-21swapper/109:06:371
274262410,2sleep20-21swapper/211:56:352
274262410,2sleep20-21swapper/211:56:352
205082400,3sleep30-21swapper/310:11:373
205082400,3sleep30-21swapper/310:11:373
114222380,3chrt0-21swapper/208:21:242
114222380,3chrt0-21swapper/208:21:242
316862360,2sleep20-21swapper/207:46:432
288282360,2sleep10-21swapper/110:31:441
288282360,2sleep10-21swapper/110:31:441
258592360,3sleep30-21swapper/308:56:463
258592360,3sleep30-21swapper/308:56:463
258592360,3sleep30-21swapper/308:56:463
17922993629,5cyclictest25168-21latency_hist11:51:230
17922993629,5cyclictest25168-21latency_hist11:51:230
17924993530,3cyclictest28814-21sendmail07:41:231
17924993530,3cyclictest28814-21sendmail07:41:231
17924993529,4cyclictest1346-21avahi-daemon11:41:461
17924993529,4cyclictest1346-21avahi-daemon11:41:461
17924993528,5cyclictest20793-21threads11:36:451
17924993528,5cyclictest20793-21threads11:36:451
17924993528,5cyclictest20793-21threads11:36:451
279392340,3chrt0-21swapper/307:36:433
279392340,3chrt0-21swapper/307:36:433
17924993325,5cyclictest6661-21latency_hist11:01:321
17924993325,5cyclictest6661-21latency_hist11:01:321
17924993226,4cyclictest15310-21latency_hist08:31:301
17924993225,6cyclictest12793-21kernelversion11:16:391
17924993225,6cyclictest12793-21kernelversion11:16:391
11512320,2sleep2351rcuc/212:11:392
17924993127,3cyclictest847-21gltestperf12:11:361
17924993125,5cyclictest10214-21kernelversion08:16:391
17924993125,5cyclictest10214-21kernelversion08:16:391
17924993125,4cyclictest28900-21wc12:01:231
17924993125,4cyclictest28900-21wc12:01:231
17924993125,4cyclictest0-21swapper/109:01:231
17924993125,4cyclictest0-21swapper/109:01:231
17924993125,4cyclictest0-21swapper/109:01:231
17924993124,5cyclictest2717-21threads09:21:451
17924993124,5cyclictest2717-21threads09:21:451
17924993124,5cyclictest142-21systemd-journal08:46:311
17922993124,5cyclictest550-21cron12:31:230
17922993124,5cyclictest550-21cron12:31:230
178062310,3sleep20-21swapper/210:01:462
178062310,3sleep20-21swapper/210:01:462
17924993026,2cyclictest0-21swapper/112:21:391
17924993026,2cyclictest0-21swapper/112:21:391
17924993022,6cyclictest0-21swapper/109:51:361
17924993022,6cyclictest0-21swapper/109:51:361
17922993024,5cyclictest10852-21sshd08:16:470
17922993024,5cyclictest10852-21sshd08:16:470
17922993022,6cyclictest7719-21threads11:01:450
17922993020,7cyclictest16818-21fschecks_count10:01:360
17922993020,7cyclictest16818-21fschecks_count10:01:360
17924992923,4cyclictest11701-21latency_hist12:41:341
17924992923,4cyclictest11701-21latency_hist12:41:341
17924992922,5cyclictest3155-21proc_pri12:16:431
17924992922,5cyclictest3155-21proc_pri12:16:431
17924992922,5cyclictest2744-21latency_hist10:51:231
17924992922,5cyclictest2744-21latency_hist10:51:231
17924992922,5cyclictest11315-21cut09:46:351
17924992922,5cyclictest11315-21cut09:46:351
1792299299,14cyclictest13624-21kworker/0:0+events_power_efficient07:41:420
17922992916,4cyclictest90-1pr/ttyS009:07:530
17922992916,4cyclictest90-1pr/ttyS009:07:530
17924992822,4cyclictest20298-21latency_hist10:11:301
17924992821,5cyclictest23820-21threads08:51:451
17924992821,5cyclictest10742-21sshd09:45:311
17924992821,5cyclictest10742-21sshd09:45:311
17924992820,6cyclictest7855-21/usr/sbin/munin08:11:421
17924992820,6cyclictest7855-21/usr/sbin/munin08:11:421
17922992824,3cyclictest22010-21gltestperf11:41:390
17922992824,3cyclictest22010-21gltestperf11:41:390
17922992822,4cyclictest17920-21cyclictest09:44:080
17922992822,4cyclictest17920-21cyclictest09:44:080
17922992821,5cyclictest17126-21timerwakeupswit11:26:460
17922992821,5cyclictest17126-21timerwakeupswit11:26:460
17922992821,5cyclictest11618-21latency_hist12:41:300
17922992821,5cyclictest11618-21latency_hist12:41:300
17922992820,6cyclictest23584-21fschecks_count07:26:350
17922992820,6cyclictest23584-21fschecks_count07:26:350
17922992820,6cyclictest21623-21needreboot08:46:400
17922992819,6cyclictest7766-21latency_hist08:11:290
17922992819,6cyclictest7766-21latency_hist08:11:290
51452270,2sleep20-21swapper/208:01:462
51452270,2sleep20-21swapper/208:01:462
17924992723,3cyclictest18322-21latency_hist10:06:231
17924992723,3cyclictest18322-21latency_hist10:06:231
17924992723,3cyclictest15848-21latency08:31:361
17924992723,3cyclictest15848-21latency08:31:361
17924992723,2cyclictest0-21swapper/110:36:441
17924992723,2cyclictest0-21swapper/110:36:441
17924992722,4cyclictest13259-21latency_hist08:26:231
17924992722,4cyclictest13259-21latency_hist08:26:231
17924992721,5cyclictest32637-21latency_hist07:51:261
17924992719,6cyclictest3101-21grep07:56:421
17924992719,6cyclictest3101-21grep07:56:421
17924992719,6cyclictest3101-21grep07:56:421
17924992717,8cyclictest700-21192.168.115.40-manager09:27:241
17924992717,8cyclictest700-21192.168.115.40-manager09:27:241
17922992722,3cyclictest26877-21gltestperf09:01:360
17922992722,3cyclictest26877-21gltestperf09:01:360
17922992721,5cyclictest20341-21munin-node10:14:250
17922992721,5cyclictest20341-21munin-node10:14:250
17922992721,4cyclictest17920-21cyclictest11:41:230
17922992721,4cyclictest17920-21cyclictest11:41:230
17922992721,4cyclictest17920-21cyclictest11:41:230
17922992720,5cyclictest6694-21latency_hist11:01:330
17922992720,5cyclictest17920-21cyclictest10:51:430
17922992720,5cyclictest17920-21cyclictest10:51:430
17922992719,6cyclictest5682-21fschecks_time09:31:350
17922992719,6cyclictest5682-21fschecks_time09:31:350
17922992719,6cyclictest24447-21cron08:56:230
17922992718,7cyclictest10649-21fschecks_time11:11:380
17922992718,7cyclictest10649-21fschecks_time11:11:380
17922992714,2cyclictest181rcu_preempt08:41:420
17922992714,2cyclictest181rcu_preempt08:41:420
17922992710,3cyclictest25222-21latency08:56:380
17922992710,3cyclictest25222-21latency08:56:380
17922992710,3cyclictest25222-21latency08:56:380
17924992622,3cyclictest14417-21switchtime08:26:441
17924992622,2cyclictest0-21swapper/111:31:471
17924992622,2cyclictest0-21swapper/111:26:431
17924992622,2cyclictest0-21swapper/111:26:431
17924992621,3cyclictest0-21swapper/112:06:361
17924992620,4cyclictest22893-21needreboot10:16:411
17924992620,4cyclictest22893-21needreboot10:16:411
17924992620,4cyclictest22893-21needreboot10:16:411
17924992619,5cyclictest23769-21cut11:46:361
17924992619,5cyclictest23769-21cut11:46:361
17922992621,4cyclictest17920-21cyclictest10:32:280
17922992621,4cyclictest17920-21cyclictest10:32:280
17922992621,4cyclictest142-21systemd-journal11:32:310
17922992621,4cyclictest142-21systemd-journal11:32:310
17922992621,4cyclictest142-21systemd-journal11:32:310
17922992621,3cyclictest0-21swapper/012:11:440
17922992620,5cyclictest17920-21cyclictest11:21:440
17922992620,5cyclictest17920-21cyclictest11:21:440
17922992620,4cyclictest0-21swapper/009:26:230
17922992620,4cyclictest0-21swapper/009:26:230
17922992619,5cyclictest8856-21sshd09:40:310
17922992619,5cyclictest8856-21sshd09:40:310
17922992619,5cyclictest12604-21sshd09:50:310
17922992619,5cyclictest12604-21sshd09:50:310
17922992619,4cyclictest0-21swapper/012:21:230
17922992619,4cyclictest0-21swapper/012:21:230
17922992613,8cyclictest181rcu_preempt11:11:240
17922992613,8cyclictest181rcu_preempt11:11:240
17924992521,3cyclictest17920-21cyclictest09:01:401
17924992521,3cyclictest17920-21cyclictest09:01:401
17924992521,2cyclictest0-21swapper/112:31:381
17924992521,2cyclictest0-21swapper/112:31:381
17924992520,3cyclictest5355-21unixbench_singl08:01:471
17924992520,3cyclictest5355-21unixbench_singl08:01:471
17924992520,3cyclictest0-21swapper/110:11:361
17924992520,3cyclictest0-21swapper/110:11:361
17924992520,3cyclictest0-21swapper/108:51:231
17924992520,3cyclictest0-21swapper/108:41:441
17924992519,5cyclictest5923-21kernelversion09:31:381
17924992519,5cyclictest5923-21kernelversion09:31:381
17924992519,5cyclictest23644-21cut07:26:351
17924992519,5cyclictest23644-21cut07:26:351
17922992520,4cyclictest23645-21cron10:18:230
17922992520,4cyclictest23645-21cron10:18:230
17922992520,4cyclictest23645-21cron10:18:230
17922992519,5cyclictest601-21snmpd12:01:370
17922992519,5cyclictest601-21snmpd12:01:370
17922992519,5cyclictest27150-21latency_hist11:56:290
17922992519,4cyclictest6687-21latency_hist11:01:330
17922992519,4cyclictest31064-21fschecks_count07:46:350
17922992519,4cyclictest0-21swapper/009:26:360
17922992519,4cyclictest0-21swapper/009:26:360
17922992518,6cyclictest17329-21fschecks_count08:36:350
17922992518,6cyclictest17329-21fschecks_count08:36:350
17922992518,6cyclictest17329-21fschecks_count08:36:350
17922992518,6cyclictest13623-21fschecks_count08:26:350
17922992518,6cyclictest13623-21fschecks_count08:26:350
17922992518,5cyclictest28901-21sendmail12:01:230
17922992518,5cyclictest28901-21sendmail12:01:230
17922992516,7cyclictest24305-21fschecks_count10:21:360
17922992516,7cyclictest24305-21fschecks_count10:21:360
17922992516,7cyclictest24305-21fschecks_count10:21:360
17922992515,8cyclictest25950-21needreboot07:31:390
17922992515,8cyclictest25950-21needreboot07:31:390
17922992514,10cyclictest17920-21cyclictest08:11:510
17922992514,10cyclictest17920-21cyclictest08:11:510
17922992512,3cyclictest90-1pr/ttyS008:31:430
17922992512,3cyclictest90-1pr/ttyS008:31:430
17924992420,2cyclictest0-21swapper/111:26:231
17924992420,2cyclictest0-21swapper/111:26:231
17924992420,2cyclictest0-21swapper/110:46:231
17924992420,2cyclictest0-21swapper/109:21:231
17924992420,2cyclictest0-21swapper/109:21:231
17924992419,3cyclictest0-21swapper/108:41:241
17924992419,3cyclictest0-21swapper/108:41:241
17924992419,3cyclictest0-21swapper/108:41:241
17924992418,4cyclictest0-21swapper/108:06:341
17924992418,4cyclictest0-21swapper/108:06:341
17922992419,3cyclictest2242-21/usr/sbin/munin07:56:420
17922992419,3cyclictest2242-21/usr/sbin/munin07:56:420
17922992419,3cyclictest2242-21/usr/sbin/munin07:56:420
17922992419,3cyclictest18683-21fschecks_count10:06:360
17922992419,3cyclictest0-21swapper/012:31:350
17922992419,3cyclictest0-21swapper/012:31:350
17922992418,5cyclictest31175-21rt-features09:11:450
17922992418,5cyclictest31175-21rt-features09:11:450
17922992418,4cyclictest14036-21latency_hist11:21:260
17922992418,4cyclictest14036-21latency_hist11:21:260
17922992417,5cyclictest32491-21sshd12:09:310
17922992416,6cyclictest500-21fschecks_time07:51:350
17924992319,2cyclictest0-21swapper/112:26:411
17924992319,2cyclictest0-21swapper/112:26:411
17924992319,2cyclictest0-21swapper/110:51:461
17924992319,2cyclictest0-21swapper/110:51:461
17924992319,2cyclictest0-21swapper/107:41:461
17924992318,3cyclictest0-21swapper/111:06:441
17924992318,3cyclictest0-21swapper/111:06:441
17922992311,2cyclictest181rcu_preempt09:56:340
17922992311,2cyclictest181rcu_preempt09:56:340
17924992218,2cyclictest0-21swapper/109:56:341
17924992218,2cyclictest0-21swapper/109:56:341
17924992217,3cyclictest663-21sed07:51:361
17924992217,3cyclictest0-21swapper/109:12:531
17924992217,3cyclictest0-21swapper/109:12:531
17924992217,3cyclictest0-21swapper/107:31:361
17924992217,3cyclictest0-21swapper/107:31:361
17924992213,7cyclictest142-21systemd-journal11:01:561
17922992217,4cyclictest17920-21cyclictest10:30:120
17922992217,4cyclictest17920-21cyclictest10:30:120
17922992217,3cyclictest0-21swapper/010:46:360
17922992217,3cyclictest0-21swapper/010:46:360
17922992215,5cyclictest12716-21wc08:21:440
17922992215,5cyclictest12716-21wc08:21:440
1792299221,14cyclictest27415-21kworker/0:1+events_power_efficient10:36:340
1792299221,14cyclictest27415-21kworker/0:1+events_power_efficient10:36:340
17922992115,4cyclictest4344-21fschecks_time08:01:370
17922992115,4cyclictest4344-21fschecks_time08:01:370
34732200,3sleep30-21swapper/307:56:453
34732200,3sleep30-21swapper/307:56:453
34732200,3sleep30-21swapper/307:56:453
211302201,2sleep20-21swapper/210:11:442
211302201,2sleep20-21swapper/210:11:442
17922992016,3cyclictest17920-21cyclictest07:36:340
17922992016,3cyclictest17920-21cyclictest07:36:340
68682190,2sleep20-21swapper/211:01:362
17925991812,4cyclictest289372sleep209:06:382
17925991812,4cyclictest289372sleep209:06:382
17926991711,4cyclictest63632sleep309:31:453
17926991711,4cyclictest63632sleep309:31:453
17926991710,5cyclictest199372sleep308:41:443
17926991710,5cyclictest199372sleep308:41:443
156772170,3sleep30-21swapper/308:31:343
156772170,3sleep30-21swapper/308:31:343
18952161,2sleep20-21swapper/207:54:522
17926991611,3cyclictest117452chrt11:12:143
17926991611,3cyclictest117452chrt11:12:143
1792699161,12cyclictest2428-21chrt10:47:263
1792699161,12cyclictest2428-21chrt10:47:263
1792699161,12cyclictest13444-21taskset09:51:383
1792699161,12cyclictest13444-21taskset09:51:383
1792699161,11cyclictest4318-21chrt09:26:413
1792699161,11cyclictest4318-21chrt09:26:413
1792699161,10cyclictest16757-21chrt10:01:353
1792699161,10cyclictest16757-21chrt10:01:353
1792599161,13cyclictest271402sleep207:36:342
1792599161,13cyclictest271402sleep207:36:342
1792599161,12cyclictest21653-21taskset08:46:402
1792599161,11cyclictest31593-21chrt10:41:342
1792599161,11cyclictest31593-21chrt10:41:342
1792599161,11cyclictest28997-21chrt10:31:452
1792599161,11cyclictest28997-21chrt10:31:452
1792699159,4cyclictest46042chrt08:01:393
1792699159,4cyclictest46042chrt08:01:393
17926991511,3cyclictest0-21swapper/309:14:233
17926991511,3cyclictest0-21swapper/309:14:233
1792699151,11cyclictest3715-21chrt10:51:443
1792699151,11cyclictest3715-21chrt10:51:443
1792699151,11cyclictest32194-21chrt10:41:393
1792699151,11cyclictest2298-21chrt09:21:393
1792699151,11cyclictest2298-21chrt09:21:393
1792699151,10cyclictest4407-21taskset12:21:373
1792699151,10cyclictest4407-21taskset12:21:373
1792699151,10cyclictest28549-21chrt09:06:353
1792699151,10cyclictest28549-21chrt09:06:353
1792599159,4cyclictest239302sleep210:21:252
1792599159,4cyclictest239302sleep210:21:252
1792599159,4cyclictest239302sleep210:21:252
1792599158,4cyclictest37092chrt07:59:132
1792599158,4cyclictest37092chrt07:59:132
1792599158,4cyclictest37092chrt07:59:132
1792599152,6cyclictest0-21swapper/210:27:552
1792599152,6cyclictest0-21swapper/210:27:552
1792599151,9cyclictest370-21chrt12:11:312
1792599151,9cyclictest25790-21chrt11:51:372
1792599151,11cyclictest6660-21taskset11:01:322
1792599151,11cyclictest6660-21taskset11:01:322
1792599151,11cyclictest2206-21chrt09:21:382
1792599151,11cyclictest2206-21chrt09:21:382
1792599151,11cyclictest20799-21chrt11:36:452
1792599151,11cyclictest20799-21chrt11:36:452
1792599151,11cyclictest20799-21chrt11:36:452
1792599151,11cyclictest17339-21chrt08:36:352
1792599151,11cyclictest17339-21chrt08:36:352
1792599151,11cyclictest17339-21chrt08:36:352
1792599151,11cyclictest13046-21chrt11:16:432
1792599151,11cyclictest13046-21chrt11:16:432
1792599151,10cyclictest6222-21chrt12:26:342
1792699149,3cyclictest255532chrt10:23:343
1792699149,3cyclictest255532chrt10:23:343
1792699149,3cyclictest255532chrt10:23:343
1792699149,3cyclictest18282chrt07:53:593
1792699149,2cyclictest0-21swapper/309:47:083
1792699149,2cyclictest0-21swapper/309:47:083
1792699142,9cyclictest17620-21chrt11:30:313
1792699142,9cyclictest17620-21chrt11:30:313
1792699141,11cyclictest29123-21chrt10:31:463
1792699141,11cyclictest29123-21chrt10:31:463
1792699141,10cyclictest8852-21chrt11:06:353
1792699141,10cyclictest8852-21chrt11:06:353
1792699141,10cyclictest8443-21chrt08:11:403
1792699141,10cyclictest8443-21chrt08:11:403
1792699141,10cyclictest8083-21chrt12:31:343
1792699141,10cyclictest8083-21chrt12:31:343
1792699141,10cyclictest7034-21chrt11:01:373
1792699141,10cyclictest6891-21chrt08:06:423
1792699141,10cyclictest6891-21chrt08:06:423
1792699141,10cyclictest31130-21chrt07:46:363
1792699141,10cyclictest29885-21taskset12:01:433
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional