You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 22:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Fri Jun 19, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
383583621350,2sleep00-21swapper/00
398523521330,4sleep03985232-21unin-run0
3625620213371,58sleep30-21swapper/33
36256692112103,6sleep10-21swapper/11
362573928677,6sleep20-21swapper/22
362586328357,7sleep00-21swapper/00
3626085992219,1cyclictest0-21swapper/33
362608599216,12cyclictest0-21swapper/33
362608599211,16cyclictest0-21swapper/33
362608599211,16cyclictest0-21swapper/33
40345222200,11sleep244-21ksoftirqd/22
38151252201,2sleep20-21swapper/22
3626085992011,6cyclictest3965578-21taskset3
362608599201,15cyclictest3681819-21turbostat3
362608599199,6cyclictest66950irq/132-enp1s0-TxRx-33
362608599198,8cyclictest4053525-21turbostat3
3626085991911,5cyclictest0-21swapper/33
362608599191,15cyclictest0-21swapper/33
362608599191,15cyclictest0-21swapper/33
362608599191,14cyclictest0-21swapper/33
362608599191,14cyclictest0-21swapper/33
362608599191,14cyclictest0-21swapper/33
362608599191,14cyclictest0-21swapper/33
362608599191,14cyclictest0-21swapper/33
362608599191,14cyclictest0-21swapper/33
362608599191,14cyclictest0-21swapper/33
362608599191,14cyclictest0-21swapper/33
362608599190,16cyclictest0-21swapper/33
362608599190,14cyclictest66950irq/132-enp1s0-TxRx-33
39586072181,2sleep00-21swapper/00
362608599185,2cyclictest0-21swapper/33
362608599181,5cyclictest0-21swapper/33
3626085991814,2cyclictest0-21swapper/33
3626085991812,4cyclictest0-21swapper/33
362608599181,14cyclictest0-21swapper/33
362608599181,14cyclictest0-21swapper/33
362608599181,14cyclictest0-21swapper/33
362608599181,14cyclictest0-21swapper/33
362608599181,14cyclictest0-21swapper/33
362608599181,14cyclictest0-21swapper/33
362608599181,14cyclictest0-21swapper/33
362608599181,14cyclictest0-21swapper/33
362608599181,13cyclictest0-21swapper/33
362608599181,13cyclictest0-21swapper/33
362608599181,13cyclictest0-21swapper/33
362608599181,13cyclictest0-21swapper/33
362608599180,16cyclictest0-21swapper/33
362608599180,15cyclictest66950irq/132-enp1s0-TxRx-33
362608599180,15cyclictest0-21swapper/33
41096242171,4sleep03756415-21diskmemload0
41029832171,3sleep20-21swapper/22
40434232171,2sleep00-21swapper/00
39674412170,3sleep13967455-21ssh1
39349732171,12sleep03935181-21if_enp1s00
39167432171,3sleep10-21swapper/11
38402832171,2sleep20-21swapper/22
362608599176,7cyclictest3724442-21sleep3
362608599171,4cyclictest0-21swapper/33
362608599171,3cyclictest0-21swapper/33
362608599171,13cyclictest3687136-21turbostat3
362608599171,13cyclictest0-21swapper/33
362608599171,13cyclictest0-21swapper/33
362608599170,14cyclictest0-21swapper/33
39685052160,6sleep10-21swapper/11
39346222161,8sleep23934887-21df2
39314182160,1sleep20-21swapper/22
39257232161,4sleep23756415-21diskmemload2
39178532162,8sleep03917986-21diskstats0
39084942161,3sleep13756415-21diskmemload1
39081312161,5sleep23908141-21ssh2
37872452160,3sleep10-21swapper/11
362608599161,11cyclictest37498492chrt3
362608599160,13cyclictest0-21swapper/33
362608599160,13cyclictest0-21swapper/33
362608599160,13cyclictest0-21swapper/33
362608599160,13cyclictest0-21swapper/33
362608599160,13cyclictest0-21swapper/33
362608599160,13cyclictest0-21swapper/33
362608599160,12cyclictest0-21swapper/33
41101262151,3sleep20-21swapper/22
40678832150,2sleep00-21swapper/00
40079202151,2sleep00-21swapper/00
38819002150,1sleep00-21swapper/00
37781092152,8sleep03778179-21apt-get0
362608599151,2cyclictest0-21swapper/33
3626085991511,3cyclictest0-21swapper/33
3626085991511,2cyclictest0-21swapper/33
362608599151,11cyclictest3756431-21turbostat3
362608599151,11cyclictest0-21swapper/33
362608599150,13cyclictest0-21swapper/33
362608599150,13cyclictest0-21swapper/33
362608599150,13cyclictest0-21swapper/33
362608599150,12cyclictest0-21swapper/33
41020712141,9sleep04102084-21sh0
40940782141,5sleep03756415-21diskmemload0
40689712141,8sleep23756415-21diskmemload2
40167132141,9sleep14016723-21ssh1
38982872141,8sleep13756415-21diskmemload1
38477432141,2sleep10-21swapper/11
38474162141,3sleep23756415-21diskmemload2
38201622141,7sleep03820199-21ntpq0
37987862141,9sleep03798800-21ssh0
37781862141,9sleep13778278-21sh1
37541292141,9sleep13755242-21memory1
36808452141,8sleep13680899-21ntpq1
36601172141,9sleep13660127-40gst-plugin-scan1
3626085991412,1cyclictest0-21swapper/33
362608599140,2cyclictest0-21swapper/33
362608599140,12cyclictest0-21swapper/33
40766922131,3sleep04076706-21ssh0
40196722132,5sleep14019704-21ssh1
39852352131,8sleep03985689-21echo0
39661732131,1sleep20-21swapper/22
38918032131,6sleep03756415-21diskmemload0
38714722131,5sleep13871727-21users1
37626312131,8sleep03756415-21diskmemload0
37078012134,5sleep23708074-21grep2
362608599130,11cyclictest0-21swapper/33
40936192121,7sleep14093629-21ssh1
40909752121,7sleep24087641-21/usr/sbin/munin2
40710662122,5sleep24071214-21head2
40622082121,7sleep14062219-21ssh1
40018362121,6sleep13756415-21diskmemload1
39738422120,5sleep23756415-21diskmemload2
39490392121,3sleep13756415-21diskmemload1
39313562121,3sleep13931367-21ssh1
39179872121,6sleep13918208-21hddtemp_smartct1
39149052121,4sleep00-21swapper/00
38912982121,7sleep03756415-21diskmemload0
38912982121,7sleep03756415-21diskmemload0
37902322121,8sleep03790246-21ssh0
37021352121,5sleep03702189-21ntpq0
36809012121,3sleep23680992-21ps2
36601062121,2sleep00-21swapper/00
36317212125,3sleep23632946-21grep2
40594222111,5sleep13756415-21diskmemload1
40475852112,5sleep04048088-21tr0
40450172112,4sleep13756415-21diskmemload1
40341712111,3sleep10-21swapper/11
40238722111,5sleep24023931-21ntpq2
40164422111,6sleep24016454-21ssh2
40161832110,1sleep00-21swapper/00
40016062111,2sleep23626079-21cyclictest2
39856902110,6sleep13985178-21apt-get1
39749062110,1sleep00-21swapper/00
39680592111,6sleep03968071-21ssh0
38819972111,3sleep10-21swapper/11
38719542111,3sleep00-21swapper/00
38652002110,2sleep00-21swapper/00
38638242111,3sleep10-21swapper/11
38542922111,5sleep13854351-21ntpq1
38149142111,3sleep10-21swapper/11
38052212111,6sleep13756415-21diskmemload1
38042792111,5sleep20-21swapper/22
37976672111,5sleep23756415-21diskmemload2
37648762111,2sleep10-21swapper/11
37288022111,5sleep13728855-21ntpq1
37181782111,6sleep23718217-21/usr/sbin/munin2
36860872111,5sleep13686138-21ntpq1
36701652111,6sleep13670395-21unixbench_multi1
41121912101,4sleep13756415-21diskmemload1
41012102101,3sleep10-21swapper/11
40863142100,2sleep10-21swapper/11
40841222101,3sleep20-21swapper/22
40597562101,4sleep23756415-21diskmemload2
40516672101,2sleep20-21swapper/22
40505292101,4sleep13756415-21diskmemload1
40086092101,4sleep23756415-21diskmemload2
40081612101,3sleep10-21swapper/11
40007362101,3sleep00-21swapper/00
39900272101,4sleep23990068-21ntpq2
39826082101,3sleep13756415-21diskmemload1
39591422101,4sleep13756415-21diskmemload1
39506862101,3sleep20-21swapper/22
39488302101,3sleep00-21swapper/00
39095342101,4sleep23909634-21diskstats2
39083352101,4sleep03756415-21diskmemload0
38915242101,3sleep10-21swapper/11
38915242101,3sleep10-21swapper/11
38910312101,2sleep20-21swapper/22
38910312101,2sleep20-21swapper/22
38794662101,4sleep23879519-21ntpq2
38674692101,5sleep23868169-21cut2
38563862101,4sleep03756415-21diskmemload0
38492992101,3sleep00-21swapper/00
38395172101,4sleep13756415-21diskmemload1
38312952100,2sleep10-21swapper/11
38296642101,5sleep03829677-21sh0
38202002101,4sleep13820253-21ntpq1
38129542101,4sleep03756415-21diskmemload0
38063012101,4sleep03756415-21diskmemload0
37986552101,4sleep13756415-21diskmemload1
37789002101,4sleep23756415-21diskmemload2
37706532101,3sleep00-21swapper/00
37682752101,4sleep23769281-21memory2
37640562101,4sleep13756415-21diskmemload1
37393242101,4sleep03739412-21ntpq0
37355592101,4sleep23735642-21diskstats2
37140722105,2sleep00-21swapper/00
37087172101,4sleep13708736-21apt-get1
37074552101,4sleep03707505-21ntpq0
36729252101,5sleep23674720-21tr2
36729252101,5sleep23674720-21tr2
36716822101,6sleep13672924-21kthreadcore1
36716822101,6sleep13672924-21kthreadcore1
36496142101,4sleep13649633-21apt-get1
36493532101,5sleep23649591-21latency_hist2
36445642101,5sleep13644688-21perl1
4085428290,4sleep03756415-21diskmemload0
4072337291,4sleep14073970-21ssh1
4041942291,2sleep20-21swapper/22
4034120291,2sleep00-21swapper/00
4026428291,4sleep04026442-21ssh0
3982195291,2sleep20-21swapper/22
3957908291,3sleep23756415-21diskmemload2
3942457291,2sleep10-21swapper/11
3931681291,3sleep03756415-21diskmemload0
3900207292,4sleep20-21swapper/22
3858369291,3sleep23756415-21diskmemload2
3822664291,4sleep23822678-21ssh2
3786683290,4sleep23786744-21ntpq2
3762373291,3sleep23756415-21diskmemload2
3752412291,4sleep03754127-21cut0
3745192290,1sleep20-21swapper/22
3740702290,6sleep03740936-21cstates0
3735376291,4sleep13735558-21cstates1
3670158291,4sleep03670164-21sensors0
3660988291,5sleep23661037-21kthreadcore2
3654682291,4sleep03654859-21sort0
3635568291,4sleep13637376-21cut1
4060467281,2sleep00-21swapper/00
3858074280,1sleep20-21swapper/22
3829780281,3sleep23829791-21ssh2
3755516280,1sleep20-21swapper/22
3746010281,2sleep00-21swapper/00
3693260281,3sleep13693852-21kthreadcore1
3677909281,3sleep03676715-21/usr/sbin/munin0
3671610281,3sleep03671681-21irqstats0
3671610281,3sleep03671681-21irqstats0
3666289280,1sleep20-21swapper/22
3650549281,3sleep20-21swapper/22
3645089281,3sleep03646750-21kthreadcore0
3643043281,3sleep03643093-21ntpq0
3626840281,3sleep23628333-21cut2
3626259280,3sleep10-21swapper/11
3750725270,1sleep10-21swapper/11
3734859270,1sleep20-21swapper/22
3725305271,3sleep03726795-21kthreadcore0
3719561270,4sleep20-21swapper/22
3719374271,3sleep13719560-21cstates1
3718123271,3sleep13718171-21ntpq1
3709583271,3sleep23711266-21kthreadcore2
3698036270,2sleep20-21swapper/22
3693853271,3sleep23695573-21cut2
3692260270,1sleep20-21swapper/22
3687059270,1sleep20-21swapper/22
3643988270,1sleep10-21swapper/11
3639379271,2sleep20-21swapper/22
3748022261,2sleep20-21swapper/22
3730028260,1sleep10-21swapper/11
3719352260,2sleep00-21swapper/00
3713688260,4sleep00-21swapper/00
3705039261,2sleep10-21swapper/11
3697483260,1sleep00-21swapper/00
3691763261,2sleep00-21swapper/00
3687383260,2sleep10-21swapper/11
3681858260,3sleep00-21swapper/00
3660042260,2sleep20-21swapper/22
3634189261,2sleep00-21swapper/00
3630462260,3sleep03630549-21proc_pri0
3734476250,1sleep00-21swapper/00
3729768250,1sleep20-21swapper/22
3702758250,1sleep10-21swapper/11
3664363250,1sleep10-21swapper/11
3663386250,2sleep00-21swapper/00
3740463240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional