You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 09:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sun Apr 26, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2762465212650,71sleep30-21swapper/33
276224028758,10sleep10-21swapper/11
276256528262,14sleep00-21swapper/00
276256728063,10sleep20-21swapper/22
32137822231,16sleep22893076-21diskmemload2
2762692992319,2cyclictest0-21swapper/33
2762692992317,4cyclictest0-21swapper/33
276269299211,17cyclictest0-21swapper/33
276269299211,16cyclictest0-21swapper/33
276269299211,16cyclictest0-21swapper/33
276269299207,9cyclictest2910568-21turbostat3
2762692992012,5cyclictest2960296-21taskset3
276269299201,15cyclictest66950irq/132-enp1s0-TxRx-33
276269299201,15cyclictest0-21swapper/33
276269299200,18cyclictest0-21swapper/33
28937832191,13sleep22894000-21hddtemp_smartct2
2762692991914,3cyclictest0-21swapper/33
276269299191,15cyclictest66950irq/132-enp1s0-TxRx-33
276269299191,15cyclictest0-21swapper/33
276269299191,14cyclictest0-21swapper/33
276269299191,14cyclictest0-21swapper/33
276269299191,14cyclictest0-21swapper/33
276269299191,14cyclictest0-21swapper/33
276269299191,14cyclictest0-21swapper/33
276269299191,14cyclictest0-21swapper/33
276269299190,2cyclictest0-21swapper/33
32401002180,13sleep20-21swapper/22
31357882181,4sleep02893076-21diskmemload0
29025062182,11sleep02902639-21/usr/sbin/munin0
276269299189,5cyclictest30560262sleep33
276269299187,8cyclictest2812581-21turbostat3
276269299181,4cyclictest0-21swapper/33
276269299181,4cyclictest0-21swapper/33
276269299181,14cyclictest66950irq/132-enp1s0-TxRx-33
276269299181,14cyclictest0-21swapper/33
276269299181,14cyclictest0-21swapper/33
276269299181,14cyclictest0-21swapper/33
276269299181,14cyclictest0-21swapper/33
276269299181,14cyclictest0-21swapper/33
276269299181,13cyclictest0-21swapper/33
276269299181,13cyclictest0-21swapper/33
276269299181,13cyclictest0-21swapper/33
276269299181,13cyclictest0-21swapper/33
276269299180,14cyclictest66950irq/132-enp1s0-TxRx-33
276269299180,14cyclictest0-21swapper/33
30927992171,10sleep00-21swapper/00
30055952171,2sleep00-21swapper/00
276269299174,11cyclictest0-21swapper/33
2762692991715,1cyclictest0-21swapper/33
2762692991713,2cyclictest0-21swapper/33
276269299171,14cyclictest0-21swapper/33
276269299171,13cyclictest0-21swapper/33
276269299171,13cyclictest0-21swapper/33
276269299170,14cyclictest66950irq/132-enp1s0-TxRx-33
276269299170,14cyclictest0-21swapper/33
276269299170,14cyclictest0-21swapper/33
32485482161,8sleep20-21swapper/22
32428472161,2sleep00-21swapper/00
31885082161,4sleep02893076-21diskmemload0
29609972161,2sleep20-21swapper/22
29336522161,4sleep02893076-21diskmemload0
276269299168,5cyclictest2871232-21taskset3
276269299161,3cyclictest0-21swapper/33
276269299161,12cyclictest0-21swapper/33
276269299161,12cyclictest0-21swapper/33
2762692991610,4cyclictest0-21swapper/33
276269299160,13cyclictest0-21swapper/33
276269299160,13cyclictest0-21swapper/33
276269299160,13cyclictest0-21swapper/33
276269299160,13cyclictest0-21swapper/33
276269299160,13cyclictest0-21swapper/33
276269299160,13cyclictest0-21swapper/33
276269299160,11cyclictest0-21swapper/33
32518962151,10sleep23251915-21apt-get2
31696132150,1sleep00-21swapper/00
31625582151,10sleep23162572-21ssh2
30931652151,9sleep12893076-21diskmemload1
30664012152,8sleep02893076-21diskmemload0
30048482151,4sleep13004863-21ssh1
29368052151,3sleep12893076-21diskmemload1
29088962151,9sleep12893076-21diskmemload1
28934982151,9sleep02893764-21df0
276269299151,12cyclictest0-21swapper/33
276269299151,11cyclictest0-21swapper/33
276269299151,10cyclictest2807523-21chrt3
276269299150,12cyclictest0-21swapper/33
31798322141,3sleep12893076-21diskmemload1
31351562141,9sleep23135171-21ssh2
30763662141,4sleep23076377-21ssh2
30654172141,3sleep20-21swapper/22
30328312141,3sleep22893076-21diskmemload2
30309512146,4sleep13030963-21sh1
30298732140,2sleep0221rcuc/00
30203392141,8sleep22893076-21diskmemload2
30081492141,9sleep03009734-21grep0
29961432141,3sleep12893076-21diskmemload1
29601562140,2sleep00-21swapper/00
29202742143,6sleep12920309-21/usr/sbin/munin1
29063852141,8sleep22906441-21ntpq2
276269299147,5cyclictest2791303-21sleep3
2762692991411,2cyclictest0-21swapper/33
276269299140,2cyclictest0-21swapper/33
276269299140,11cyclictest0-21swapper/33
32239632131,3sleep23147816-21kworker/u16:1+rpciod2
31981202131,8sleep12893076-21diskmemload1
31785012131,7sleep02893076-21diskmemload0
30513302131,8sleep23051559-21ssh2
30202262131,6sleep03020277-21ntpq0
29636692132,5sleep22963698-21apt-get2
29170952130,2sleep20-21swapper/22
28336632130,1sleep20-21swapper/22
28184622131,8sleep02818558-21fschecks_time0
28115472131,7sleep22811585-21ntpq2
276269299131,8cyclictest2828831-21chrt3
32565482121,7sleep13256800-21/usr/sbin/munin1
32497192121,5sleep10-21swapper/11
32211502121,5sleep13221251-21ntpq1
32064562121,5sleep20-21swapper/22
31983332121,5sleep20-21swapper/22
31533272121,7sleep23153341-21ssh2
31436862121,7sleep02893076-21diskmemload0
31285212121,7sleep03128532-21ssh0
31265152121,6sleep12893076-21diskmemload1
30832012121,7sleep12893076-21diskmemload1
30489902121,7sleep12893076-21diskmemload1
29890712121,3sleep12989085-21ssh1
29523282121,7sleep22952331-21ssh2
29450452120,2sleep20-21swapper/22
29425182121,7sleep12942529-21ssh1
27841262121,7sleep22784837-21sed2
276269299120,1cyclictest0-21swapper/33
32041102110,5sleep10-21swapper/11
31720442111,3sleep10-21swapper/11
31715072111,3sleep20-21swapper/22
31634902112,6sleep02893076-21diskmemload0
31512292111,5sleep13151381-21ntpq1
31459132111,6sleep23145927-21ssh2
31385222111,3sleep10-21swapper/11
31143392111,6sleep13116042-21sed1
31097842111,7sleep23109798-21ssh2
30842712111,3sleep20-21swapper/22
30725432110,2sleep00-21swapper/00
30596292111,4sleep12893076-21diskmemload1
30590612111,3sleep00-21swapper/00
30403742111,3sleep00-21swapper/00
30386212111,6sleep12893076-21diskmemload1
30335412110,2sleep20-21swapper/22
30161212111,5sleep13016393-21df_inode1
29877272111,4sleep22893076-21diskmemload2
29852502111,5sleep02985308-21ntpq0
29457852111,5sleep02893076-21diskmemload0
29200092111,5sleep02920273-21df0
28885922111,6sleep02890092-21ps0
28663812112,4sleep02866399-21apt-get0
28551902111,7sleep20-21swapper/22
27823462111,6sleep12784125-21cut1
27741442111,5sleep12774255-21ntpq1
27741442111,5sleep12774255-21ntpq1
32585002101,5sleep02893076-21diskmemload0
32503622101,4sleep02893076-21diskmemload0
32406432100,1sleep10-21swapper/11
32331982100,2sleep10-21swapper/11
32316422101,3sleep20-21swapper/22
32231202101,3sleep00-21swapper/00
32125762101,4sleep13212636-21ntpq1
31898282101,4sleep12893076-21diskmemload1
31873702101,4sleep22893076-21diskmemload2
31773402101,4sleep22893076-21diskmemload2
31642552101,4sleep12893076-21diskmemload1
31367592101,2sleep10-21swapper/11
31259482101,5sleep23125959-21ssh2
31201462100,1sleep20-21swapper/22
31095282101,4sleep12893076-21diskmemload1
31013072101,4sleep02893076-21diskmemload0
30911532101,4sleep22893076-21diskmemload2
30833352101,3sleep00-21swapper/00
30757952100,1sleep10-21swapper/11
30574572101,4sleep12893076-21diskmemload1
30481532101,5sleep22929889-21kworker/u16:2+rpciod2
30428262101,5sleep03043342-21tr0
30142322101,3sleep22893076-21diskmemload2
30069242101,3sleep10-21swapper/11
29986102101,4sleep22998870-21df2
29953462100,4sleep20-21swapper/22
29804852101,2sleep00-21swapper/00
29712432100,4sleep10-21swapper/11
29624472101,5sleep12962457-21ssh1
29511802100,1sleep00-21swapper/00
29504922101,5sleep12946186-21/usr/sbin/munin1
29286672101,3sleep20-21swapper/22
29153832100,6sleep02915441-21ntpq0
29017392101,3sleep10-21swapper/11
28882902101,4sleep22888500-21hddtemp_smartct2
28717082100,5sleep22871727-21apt-get2
28613572101,6sleep02862868-21/usr/sbin/munin0
28560022101,5sleep02856616-21cut0
28448182101,4sleep02845075-21df_inode0
28327162101,4sleep12832856-21ntp_192.168.1141
28138132101,6sleep10-21swapper/11
27849362101,4sleep02784974-21ntpq0
27705042101,5sleep02770627-21tune2fs0
27705042101,5sleep02770627-21tune2fs0
27668132100,6sleep02762879-21/usr/sbin/munin0
3225814290,1sleep00-21swapper/00
3215067290,1sleep00-21swapper/00
3206590291,3sleep02893076-21diskmemload0
3197049291,3sleep02893076-21diskmemload0
3154117291,3sleep02893076-21diskmemload0
3111679291,2sleep00-21swapper/00
3103271290,2sleep00-21swapper/00
3100068291,2sleep10-21swapper/11
3094578291,2sleep20-21swapper/22
2995932291,4sleep02995943-21ssh0
2977648291,2sleep20-21swapper/22
2976594290,3sleep10-21swapper/11
2969901290,4sleep02893076-21diskmemload0
2924712291,3sleep20-21swapper/22
2917340291,2sleep10-21swapper/11
2887695290,1sleep10-21swapper/11
2882061290,1sleep20-21swapper/22
2866363290,3sleep22865851-21apt-get2
2838483291,5sleep22838521-21switchtime2
2834545290,1sleep00-21swapper/00
2795598291,4sleep22795647-21ntpq2
2780026291,4sleep22780243-21users2
2767975291,4sleep22768425-21users2
2767975291,4sleep22768425-21users2
2762878291,3sleep10-21swapper/11
2887641281,5sleep20-21swapper/22
2875840281,3sleep02875892-21open_files0
2859518281,3sleep12859573-21modprobe1
2849640281,3sleep12849641-21pool-udisksd1
2844379280,3sleep10-21swapper/11
2843837281,4sleep22844041-21sed2
2834695281,3sleep12835564-21cut1
2812826281,2sleep00-21swapper/00
2802755281,4sleep02804266-21kthreadcore0
2802463281,3sleep22802174-21/usr/sbin/munin2
2801862280,2sleep20-21swapper/22
2796589281,3sleep12796605-21latency_hist1
2788180281,3sleep02790024-21grep0
2776123281,3sleep02777470-21grep0
2892751270,1sleep10-21swapper/11
2887571270,2sleep00-21swapper/00
2877576271,3sleep12878063-21cut1
2865550270,1sleep20-21swapper/22
2862869271,3sleep10-21swapper/11
2859797272,2sleep22859993-21unixbench-2d2
2852327271,3sleep00-21swapper/00
2827863270,1sleep20-21swapper/22
2823527272,2sleep10-21swapper/11
2823150270,1sleep10-21swapper/11
2818181270,2sleep20-21swapper/22
2817763270,1sleep20-21swapper/22
2801731270,1sleep10-21swapper/11
2777471271,3sleep12779293-21kthreadcore1
2767640270,1sleep20-21swapper/22
2867687261,2sleep10-21swapper/11
2848759260,2sleep20-21swapper/22
2839465260,1sleep00-21swapper/00
2823505260,2sleep00-21swapper/00
2812535260,1sleep10-21swapper/11
2812308260,2sleep00-21swapper/00
2806301260,2sleep10-21swapper/11
2801771260,1sleep00-21swapper/00
2791824260,2sleep00-21swapper/00
2786497261,2sleep12786706-21hddtemp_smartct1
2786207260,2sleep22786226-21apt-get2
2882072250,1sleep00-21swapper/00
2876411250,1sleep10-21swapper/11
2854659250,1sleep10-21swapper/11
2831647250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional