You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 13:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun Apr 19, 2026 12:43:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3890919996549,8cyclictest0-21swapper/109:15:451
3890920995738,16cyclictest421-21dbus-daemon11:40:012
3890920995346,3cyclictest0-21swapper/211:00:402
3890918995338,10cyclictest0-21swapper/011:25:010
3890920995141,6cyclictest420-21avahi-daemon08:40:312
3890920995126,22cyclictest3076692-21cups-browsed08:10:462
3890920995126,22cyclictest3076692-21cups-browsed08:10:462
3890920995023,23cyclictest3076692-21cups-browsed09:40:302
3890918994836,9cyclictest421-21dbus-daemon12:00:000
3890918994834,10cyclictest0-21swapper/009:30:460
3890918994834,10cyclictest0-21swapper/009:30:460
3890920994740,4cyclictest4093090-21munin-plugin-st12:35:012
3890920994738,5cyclictest4036396-21systemd-run11:25:012
3890920994737,7cyclictest0-21swapper/209:10:002
3890919994739,5cyclictest4003969-21systemd-run10:45:011
3890920994623,20cyclictest0-21swapper/210:05:462
3890919994638,5cyclictest3904407-21systemd-run07:55:011
3890918994637,6cyclictest3997117-21ssh10:35:430
3890920994538,3cyclictest0-21swapper/209:55:442
389092099452,7cyclictest3902392-21ntpq07:45:442
389091999454,35cyclictest4053812-21ssh11:45:391
3890919994533,8cyclictest3076692-21cups-browsed11:04:531
3890918994529,13cyclictest3076692-21cups-browsed11:35:300
3890918994529,11cyclictest0-21swapper/009:10:000
389092099442,6cyclictest3896335-21ntpq07:25:392
3890920994416,24cyclictest3911936-21systemd-run08:20:012
3890920994416,24cyclictest3911936-21systemd-run08:20:012
3890919994436,5cyclictest3951378-21munin-plugin-st09:40:001
389092099436,13cyclictest4074921-21sshd12:11:112
389092099433,6cyclictest3891781-21ntpq07:10:382
389092099432,6cyclictest3906833-21ntpq08:00:402
3890920994323,17cyclictest4072860-21latency_hist12:10:022
389092099431,15cyclictest4040431-21awk11:30:012
3890919994333,6cyclictest1054-21mta-sts-daemon12:00:391
3890919994333,5cyclictest0-21swapper/112:20:011
3890919994332,7cyclictest0-21swapper/110:10:401
3890918994333,5cyclictest0-21swapper/010:55:440
389092099428,30cyclictest3931246-21ssh09:15:082
3890920994233,6cyclictest4024198-21systemd-run11:10:022
3890920994233,6cyclictest4024198-21systemd-run11:10:022
3890920994232,6cyclictest4062486-21unixbench_multi11:55:512
3890920994224,14cyclictest0-21swapper/210:50:292
389092099422,15cyclictest4001082-21ntpq10:40:382
3890920994216,23cyclictest4044466-21latency_hist11:35:022
3890920994215,22cyclictest3988245-21/usr/sbin/munin10:25:522
3890920994215,22cyclictest3988245-21/usr/sbin/munin10:25:522
389092099421,15cyclictest4005124-21ssh10:45:382
3890919994231,6cyclictest0-21swapper/111:40:401
3890918994232,4cyclictest0-21swapper/009:25:420
389092099417,31cyclictest3930907-21ssh09:14:552
389092099413,8cyclictest3894798-21ntpq07:20:372
389092099413,6cyclictest4094263-21ntpq12:35:392
389092099412,7cyclictest3924872-21ntpq09:00:442
389092099412,5cyclictest3980902-21ntpq10:15:402
389092099412,3cyclictest3956828-21ntpq09:45:502
389092099412,17cyclictest4053858-21ssh11:45:412
389092099412,16cyclictest4013201-21ntpq10:55:362
389092099411,5cyclictest3920365-21ntpq08:45:442
389092099411,18cyclictest3977036-21ps10:10:452
389092099411,11cyclictest3938532-21ssh09:23:532
3890919994133,4cyclictest0-21swapper/110:05:381
3890919994131,7cyclictest3983753-21grep10:20:011
3890918994132,6cyclictest3076692-21cups-browsed11:15:310
3890918994127,11cyclictest4017898-21unixbench_multi11:00:510
3890918994123,12cyclictest12350irq/32-eth010:20:440
389091899412,29cyclictest4001091-21ssh10:40:380
389092099409,28cyclictest4028163-21ssh11:14:572
389092099409,28cyclictest4028163-21ssh11:14:572
389092099408,30cyclictest3999751-21ssh10:39:472
389092099404,9cyclictest421-21dbus-daemon11:40:052
389092099403,9cyclictest3076692-21cups-browsed09:25:382
3890920994031,5cyclictest3948053-21fschecks_count09:35:312
389092099402,4cyclictest3960694-21ntpq09:50:402
3890920994021,16cyclictest3076692-21cups-browsed10:00:302
389092099401,9cyclictest3899349-21ntpq07:35:442
389092099401,5cyclictest3917236-21ntpq08:35:372
389092099401,5cyclictest3897763-21ntpq07:30:362
389092099401,10cyclictest4087211-21ssh12:26:292
389092099401,10cyclictest4079921-21ssh12:18:012
389092099401,10cyclictest3993268-21ssh10:30:472
389092099401,10cyclictest3914283-21ntpq08:25:422
3890919994031,6cyclictest421-21dbus-daemon09:45:011
3890919994028,9cyclictest4062386-21timerwakeupswit11:55:511
3890919994028,8cyclictest3917886-21cut08:40:011
3890918994032,5cyclictest4028242-21latency_hist11:15:010
3890918994032,5cyclictest4028242-21latency_hist11:15:010
3890918994029,8cyclictest12350irq/32-eth010:10:370
389092099393,6cyclictest3900763-21ntpq07:40:382
389092099392,5cyclictest3912759-21ntpq08:20:372
389092099391,9cyclictest3984041-21ssh10:20:072
389092099391,9cyclictest3903772-21ntpq07:50:412
389092099391,5cyclictest4066011-21ntpq12:00:422
389092099391,5cyclictest3923359-21ntpq08:55:432
389092099391,4cyclictest3908249-21ntpq08:05:382
389092099391,3cyclictest4029535-21ntpq11:15:442
389092099391,13cyclictest4082341-21ntpq12:20:452
3890919993931,4cyclictest0-21swapper/112:30:321
389091899393,31cyclictest3952642-21ssh09:40:390
3890918993932,4cyclictest4072942-21latency_hist12:10:020
389091899392,20cyclictest659-21snmpd09:55:460
389092099382,5cyclictest3944398-21ntpq09:30:392
389092099382,5cyclictest3944398-21ntpq09:30:392
389092099381,9cyclictest4059275-21ssh11:52:182
389092099381,7cyclictest3892756-21aten_r2power_po07:15:282
389092099381,4cyclictest3905237-21ntpq07:55:372
3890919993832,3cyclictest0-21swapper/112:10:011
3890919993827,6cyclictest0-21swapper/111:30:411
389091999382,28cyclictest3989080-21ssh10:25:441
389091999382,28cyclictest3989080-21ssh10:25:441
3890918993831,4cyclictest4097143-21sendmail12:40:010
3890918993831,4cyclictest3907403-21latency_hist08:05:010
3890918993826,9cyclictest3989005-21sh10:25:420
3890918993826,9cyclictest3989005-21sh10:25:420
3890918993825,10cyclictest439-21systemd-logind09:50:010
389091899382,29cyclictest3968909-21ssh10:00:430
3890918993811,21cyclictest3980871-21ssh10:15:380
389092099372,4cyclictest3915772-21ntpq08:30:412
389092099371,4cyclictest3921752-21ntpq08:50:372
3890919993731,3cyclictest0-21swapper/111:30:021
3890919993730,4cyclictest4009840-21sed10:50:511
3890919993729,5cyclictest4024170-21cron11:10:011
3890919993729,5cyclictest4024170-21cron11:10:011
3890919993723,4cyclictest171rcu_preempt09:50:481
3890918993729,5cyclictest3908911-21systemd-run08:10:010
3890918993729,3cyclictest0-21swapper/009:10:480
3890918993727,4cyclictest0-21swapper/012:17:110
3890918993724,10cyclictest3911936-21systemd-run08:20:010
3890918993724,10cyclictest3911936-21systemd-run08:20:010
389091899372,27cyclictest3972829-21ssh10:05:400
3890919993629,4cyclictest3922393-21idleruntime-cro08:55:011
3890919993628,5cyclictest3915426-21fschecks_time08:30:351
3890919993628,5cyclictest0-21swapper/107:20:011
3890919993627,6cyclictest3923880-21idleruntime-cro09:00:011
3890919993627,5cyclictest1055-21mta-sts-daemon09:21:221
389091999362,22cyclictest4082159-21ssh12:20:381
389091999361,29cyclictest4025377-21ssh11:10:391
389091999361,29cyclictest4025377-21ssh11:10:391
3890919993529,3cyclictest0-21swapper/110:40:011
3890919993527,4cyclictest1055-21mta-sts-daemon09:55:441
3890919993526,6cyclictest421-21dbus-daemon11:15:531
3890918993527,5cyclictest4068840-21ssh12:05:010
3890918993523,9cyclictest3925385-21cron09:05:010
3890918993522,10cyclictest3960114-21aten_r2power_vo09:50:300
3890918993520,9cyclictest12450irq/33-eth010:30:390
3890919993428,3cyclictest0-21swapper/109:30:011
3890919993424,8cyclictest4045591-21needreboot11:35:371
3890919993424,8cyclictest3910404-21sh08:15:011
3890919993424,8cyclictest3910404-21sh08:15:011
3890919993424,7cyclictest3076692-21cups-browsed07:40:391
3890919993423,4cyclictest0-21swapper/111:23:591
3890919993422,9cyclictest4005812-21unixbench_singl10:45:511
3890918993429,2cyclictest0-21swapper/007:55:380
3890918993425,6cyclictest1051-21mta-sts-daemon09:20:360
3890918993421,2cyclictest0-21swapper/011:55:010
3890918993417,11cyclictest151ktimers/012:10:380
3890919993325,5cyclictest4012828-21ssh10:55:311
3890919993325,5cyclictest3947286-21munin-run09:35:011
3890919993325,5cyclictest3947286-21munin-run09:35:011
3890919993322,8cyclictest3993610-21unixbench_multi10:30:521
3890918993326,4cyclictest3899142-21missed_timers07:35:370
3890918993326,4cyclictest12350irq/32-eth011:45:380
3890918993326,4cyclictest0-21swapper/012:25:370
3890918993326,4cyclictest0-21swapper/010:45:510
3890918993324,4cyclictest0-21swapper/012:20:480
3890918993324,4cyclictest0-21swapper/011:05:430
3890918993324,4cyclictest0-21swapper/011:05:430
3890918993320,9cyclictest0-21swapper/011:40:290
3890918993320,8cyclictest3926889-21diskmemload11:25:410
389092099321,4cyclictest3890879-21munin-run07:10:012
3890919993225,3cyclictest0-21swapper/109:45:511
3890919993224,5cyclictest0-21swapper/110:00:401
3890919993211,4cyclictest28-21ksoftirqd/108:45:441
3890918993216,11cyclictest12350irq/32-eth012:30:410
3890918993213,14cyclictest4009311-21ssh10:50:430
3890919993123,5cyclictest3897425-21fschecks_count07:30:311
3890918993125,3cyclictest1055-21mta-sts-daemon07:45:430
3890918993124,4cyclictest421-21dbus-daemon07:15:010
3890918993124,4cyclictest4044473-21cut11:35:010
3890918993123,5cyclictest3932802-21grep09:15:510
3890918993122,7cyclictest548-21gdbus07:44:410
389091999309,5cyclictest28-21ksoftirqd/108:20:391
389091999307,10cyclictest28-21ksoftirqd/107:35:401
389091999303,21cyclictest4086230-21ssh12:25:431
3890919993023,5cyclictest4056954-21sh11:50:091
3890919993023,3cyclictest0-21swapper/112:35:451
3890919993022,5cyclictest3987765-21ssh10:25:011
3890919993022,5cyclictest3930954-21cron09:15:011
3890919993022,5cyclictest3925400-21cut09:05:011
3890919993020,7cyclictest1054-21mta-sts-daemon08:00:431
3890919993017,10cyclictest4074497-21sh12:10:501
3890919993011,5cyclictest28-21ksoftirqd/107:55:401
3890919992911,8cyclictest261rcuc/107:21:351
3890918992921,5cyclictest3076692-21cups-browsed09:38:510
3890918992918,8cyclictest261-21systemd-journal07:35:010
3890918992917,3cyclictest0-21swapper/007:20:030
3890918992915,9cyclictest151ktimers/008:20:400
3890919992820,5cyclictest3925919-21fschecks_count09:05:311
389091899282,21cyclictest3917002-21awk08:35:330
3890918992821,4cyclictest3921104-21latency_hist08:50:030
3890918992820,5cyclictest3076692-21cups-browsed08:55:120
3890918992818,7cyclictest3922376-21cron08:55:010
389091999279,5cyclictest28-21ksoftirqd/107:25:431
389091999279,4cyclictest28-21ksoftirqd/108:25:451
389091999273,6cyclictest3911317-21ntpq08:15:401
389091999273,6cyclictest3911317-21ntpq08:15:401
389091999273,10cyclictest28-21ksoftirqd/107:45:311
3890919992721,3cyclictest3892398-21sh07:15:001
3890918992722,2cyclictest0-21swapper/008:15:000
3890918992722,2cyclictest0-21swapper/008:15:000
389091999266,5cyclictest28-21ksoftirqd/108:05:431
3890919992610,4cyclictest28-21ksoftirqd/108:40:421
3890918992617,5cyclictest12450irq/33-eth008:25:410
3890918992518,4cyclictest3904502-21latency_hist07:55:020
3890918992514,8cyclictest3918803-21ntp_kernel_pll_08:40:380
3890918992417,5cyclictest516-21in:imuxsock08:35:010
389091899241,18cyclictest3891091-21latency_hist07:10:030
389091899232,16cyclictest3894870-21ntpq07:20:410
389091999221,3cyclictest3890930-21munin-run07:10:021
389091899214,11cyclictest428-21mta-sts-daemon07:25:420
389092199192,11cyclictest0-21swapper/312:15:453
389092199182,9cyclictest0-21swapper/312:25:513
389092199182,9cyclictest0-21swapper/311:45:393
389092199182,9cyclictest0-21swapper/310:30:413
389092199181,10cyclictest0-21swapper/311:35:393
389092199173,8cyclictest0-21swapper/312:20:403
389092199173,8cyclictest0-21swapper/312:00:403
389092199172,9cyclictest0-21swapper/312:05:443
389092199172,9cyclictest0-21swapper/311:50:353
389092199172,9cyclictest0-21swapper/310:40:403
389092199172,9cyclictest0-21swapper/310:20:473
389092199172,9cyclictest0-21swapper/309:25:423
389092199171,10cyclictest0-21swapper/309:45:433
389092199163,8cyclictest0-21swapper/310:25:393
389092199163,8cyclictest0-21swapper/310:25:393
389092199163,7cyclictest0-21swapper/311:20:403
389092199163,7cyclictest0-21swapper/310:15:383
389092199162,9cyclictest0-21swapper/312:10:383
389092199162,8cyclictest0-21swapper/312:30:393
389092199162,8cyclictest0-21swapper/309:20:383
389092199162,8cyclictest0-21swapper/309:10:363
389092199162,8cyclictest0-21swapper/307:40:393
389092199161,9cyclictest0-21swapper/311:30:393
389092199161,9cyclictest0-21swapper/311:05:383
389092199161,9cyclictest0-21swapper/311:05:383
389092199161,9cyclictest0-21swapper/310:05:453
389092199161,8cyclictest0-21swapper/310:10:393
389092199153,7cyclictest0-21swapper/312:35:403
389092199153,7cyclictest0-21swapper/309:50:433
389092199153,7cyclictest0-21swapper/308:35:023
389092199152,8cyclictest0-21swapper/311:25:453
389092199152,8cyclictest0-21swapper/311:10:403
389092199152,8cyclictest0-21swapper/311:10:403
389092199152,8cyclictest0-21swapper/310:35:383
389092199152,6cyclictest0-21swapper/309:05:393
389092199152,5cyclictest0-21swapper/309:00:393
389092199151,9cyclictest0-21swapper/309:40:433
389092199151,8cyclictest0-21swapper/310:45:393
389092199151,8cyclictest0-21swapper/308:05:013
389092199144,7cyclictest0-21swapper/311:55:393
389092199142,8cyclictest0-21swapper/311:40:333
389092199142,8cyclictest0-21swapper/310:50:383
389092199142,8cyclictest0-21swapper/310:00:303
389092199142,7cyclictest0-21swapper/309:55:363
389092199142,6cyclictest0-21swapper/308:15:363
389092199142,6cyclictest0-21swapper/308:15:363
389092199142,5cyclictest0-21swapper/308:40:363
389092199141,9cyclictest0-21swapper/311:15:423
389092199141,9cyclictest0-21swapper/309:35:333
389092199141,8cyclictest0-21swapper/310:55:403
389092199141,8cyclictest0-21swapper/309:30:293
389092199141,8cyclictest0-21swapper/309:30:293
389092199141,8cyclictest0-21swapper/309:15:343
389092199141,6cyclictest0-21swapper/307:45:413
389092199141,6cyclictest0-21swapper/307:30:363
389092199141,6cyclictest0-21swapper/307:10:403
389092199132,6cyclictest0-21swapper/308:10:483
389092199132,6cyclictest0-21swapper/308:10:483
389092199132,5cyclictest0-21swapper/307:20:383
389092199131,8cyclictest0-21swapper/311:00:283
389092199131,6cyclictest0-21swapper/308:35:383
389092199131,6cyclictest0-21swapper/308:25:443
389092199131,5cyclictest0-21swapper/307:55:323
389092199122,6cyclictest0-21swapper/308:25:013
389092199122,5cyclictest0-21swapper/307:50:403
389092199122,5cyclictest0-21swapper/307:35:403
389092199121,6cyclictest0-21swapper/308:55:433
389092199121,6cyclictest0-21swapper/308:50:403
389092199121,6cyclictest0-21swapper/308:45:393
389092199121,6cyclictest0-21swapper/308:05:393
389092199121,6cyclictest0-21swapper/307:25:403
389092199121,6cyclictest0-21swapper/307:15:363
389092199102,4cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional