You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 09:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Thu Jun 04, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2801688212163,53sleep30-21swapper/33
280181628554,10sleep20-21swapper/22
279693727848,17sleep00-21swapper/00
280182527658,12sleep10-21swapper/11
2802097992624,1cyclictest0-21swapper/33
2802097992623,1cyclictest0-21swapper/33
2802097992522,1cyclictest0-21swapper/33
2802097992511,11cyclictest0-21swapper/33
2802097992321,1cyclictest0-21swapper/33
2802097992321,1cyclictest0-21swapper/33
2802097992321,1cyclictest0-21swapper/33
2802097992321,1cyclictest0-21swapper/33
2802097992321,1cyclictest0-21swapper/33
2802097992220,1cyclictest0-21swapper/33
2802097992220,1cyclictest0-21swapper/33
2802097992220,1cyclictest0-21swapper/33
2802097992219,1cyclictest0-21swapper/33
2802097992218,2cyclictest0-21swapper/33
31939942211,8sleep22876378-21diskmemload2
29295962211,6sleep00-21swapper/00
280209799219,1cyclictest0-21swapper/33
280209799218,1cyclictest0-21swapper/33
280209799215,13cyclictest2973853-21taskset3
2802097992119,1cyclictest0-21swapper/33
2802097992119,1cyclictest0-21swapper/33
2802097992119,1cyclictest0-21swapper/33
2802097992119,1cyclictest0-21swapper/33
2802097992118,1cyclictest0-21swapper/33
2802097992118,1cyclictest0-21swapper/33
2802097992118,1cyclictest0-21swapper/33
2802097992117,2cyclictest0-21swapper/33
2802097992117,2cyclictest0-21swapper/33
31074232202,8sleep22876378-21diskmemload2
30658842201,5sleep03067537-21grep0
29938792201,9sleep12876378-21diskmemload1
2802097992018,1cyclictest0-21swapper/33
2802097992018,1cyclictest0-21swapper/33
2802097992017,1cyclictest0-21swapper/33
2802097992017,1cyclictest0-21swapper/33
2802097992017,1cyclictest0-21swapper/33
2802097992017,1cyclictest0-21swapper/33
2802097992016,2cyclictest0-21swapper/33
2802097992016,2cyclictest0-21swapper/33
2802097992016,2cyclictest0-21swapper/33
2802097992016,2cyclictest0-21swapper/33
2802097992016,2cyclictest0-21swapper/33
2802097992016,2cyclictest0-21swapper/33
2802097992016,2cyclictest0-21swapper/33
2802097992015,2cyclictest0-21swapper/33
280209799201,14cyclictest0-21swapper/33
2802097992010,6cyclictest0-21swapper/33
30264732191,3sleep10-21swapper/11
280209799198,2cyclictest541ktimers/33
2802097991917,1cyclictest0-21swapper/33
2802097991917,1cyclictest0-21swapper/33
2802097991917,1cyclictest0-21swapper/33
2802097991916,1cyclictest0-21swapper/33
2802097991916,1cyclictest0-21swapper/33
2802097991916,1cyclictest0-21swapper/33
2802097991916,1cyclictest0-21swapper/33
2802097991916,1cyclictest0-21swapper/33
2802097991916,1cyclictest0-21swapper/33
2802097991916,1cyclictest0-21swapper/33
2802097991916,1cyclictest0-21swapper/33
2802097991916,1cyclictest0-21swapper/33
2802097991915,2cyclictest0-21swapper/33
2802097991911,1cyclictest0-21swapper/33
280209799190,15cyclictest28202872sleep33
32034352181,2sleep00-21swapper/00
30543332181,8sleep23054344-21ssh2
29179152181,12sleep02919642-21cut0
2802097991816,1cyclictest0-21swapper/33
2802097991816,1cyclictest0-21swapper/33
2802097991815,1cyclictest0-21swapper/33
2802097991815,1cyclictest0-21swapper/33
2802097991815,1cyclictest0-21swapper/33
2802097991815,1cyclictest0-21swapper/33
2802097991815,1cyclictest0-21swapper/33
2802097991815,1cyclictest0-21swapper/33
2802097991810,5cyclictest0-21swapper/33
30456092171,3sleep02876378-21diskmemload0
30360242171,3sleep10-21swapper/11
29237922171,3sleep20-21swapper/22
32392322161,3sleep10-21swapper/11
30886312161,2sleep10-21swapper/11
30808842161,7sleep13080895-21ssh1
30645642161,4sleep13064578-21ssh1
30389562162,10sleep20-21swapper/22
29024702161,6sleep12902524-21ntpq1
2802097991614,1cyclictest0-21swapper/33
32295772151,6sleep02876378-21diskmemload0
32216452151,8sleep00-21swapper/00
32096842151,9sleep23209743-21ntpq2
31876622151,3sleep03187730-21latency_hist0
31876622151,3sleep03187730-21latency_hist0
31774552151,4sleep22876378-21diskmemload2
31251002151,4sleep03125112-21ssh0
31151002151,10sleep23115111-21ssh2
29911062151,3sleep20-21swapper/22
29683112152,8sleep02968420-21tr0
29560272151,4sleep22876378-21diskmemload2
28968222151,3sleep12876378-21diskmemload1
32132502141,3sleep02876378-21diskmemload0
32124672141,8sleep12876378-21diskmemload1
31868282141,3sleep22876378-21diskmemload2
31868282141,3sleep22876378-21diskmemload2
30964552141,8sleep02876378-21diskmemload0
30369632141,2sleep00-21swapper/00
30203302141,2sleep00-21swapper/00
30116482140,3sleep10-21swapper/11
28894852142,6sleep22889508-21apt-get2
28824672145,2sleep00-21swapper/00
28644042145,5sleep02864474-21tr0
28579282141,6sleep22857991-21/usr/sbin/munin2
31931172131,8sleep02876378-21diskmemload0
31782252132,7sleep02876378-21diskmemload0
31659042131,6sleep03166008-21ps0
31578082133,5sleep0151ktimers/00
31151432131,7sleep12876378-21diskmemload1
31008332131,6sleep10-21swapper/11
30604452131,7sleep13060700-21uptime1
30545522131,6sleep00-21swapper/00
30097132131,5sleep00-21swapper/00
29820452131,3sleep22982062-21ssh2
29813622131,8sleep12981459-21wc1
29728682131,8sleep22973134-21awk2
29410422130,4sleep12941053-21ssh1
29204362131,7sleep12920695-21uptime1
28322472131,6sleep22832295-21ntpq2
32283462124,4sleep23228357-21ssh2
31787962121,3sleep13178883-21latency_hist1
31672602121,5sleep20-21swapper/22
31508332121,7sleep13150844-21ssh1
31420642121,6sleep12876378-21diskmemload1
31330292121,6sleep00-21swapper/00
31311582121,7sleep23131244-21/usr/sbin/munin2
30898692121,6sleep02876378-21diskmemload0
30876242121,8sleep23087639-21ssh2
30757192121,7sleep23077338-21/usr/sbin/munin2
30638432122,6sleep20-21swapper/22
30621612120,8sleep03062174-21ssh0
30272762121,7sleep03027291-21ssh0
30189892120,2sleep20-21swapper/22
30018932121,7sleep23001906-21sh2
28848952121,1sleep10-21swapper/11
28763082121,4sleep072-21kcompactd00
28357362121,7sleep22837494-21cut2
32368902111,4sleep00-21swapper/00
32316732111,6sleep23231918-21expr2
32201052111,5sleep22876378-21diskmemload2
31970432111,3sleep10-21swapper/11
31846362110,7sleep13184651-21ssh1
31846362110,7sleep13184651-21ssh1
31615552111,5sleep13161581-21apt-get1
31585422112,4sleep22876378-21diskmemload2
31230372113,4sleep1331ktimers/11
31059192110,1sleep00-21swapper/00
30987892110,5sleep10-21swapper/11
30728992111,3sleep20-21swapper/22
30262812111,6sleep22876378-21diskmemload2
30103942111,3sleep20-21swapper/22
29701602112,5sleep12971897-21cut1
29507622111,3sleep10-21swapper/11
29496732111,3sleep20-21swapper/22
29399862111,4sleep20-21swapper/22
29318402110,5sleep20-21swapper/22
28621862111,5sleep12862233-21ntpq1
28227692111,6sleep12822839-21kthreadcore1
28102662111,7sleep22810415-21seq2
32284262101,4sleep12876378-21diskmemload1
32025412101,4sleep22876378-21diskmemload2
31512232101,4sleep02876378-21diskmemload0
31502462101,3sleep20-21swapper/22
31353402101,4sleep23135362-21apt-get2
31352592100,1sleep00-21swapper/00
31346512104,3sleep10-21swapper/11
31168872100,6sleep02876378-21diskmemload0
30991452101,3sleep20-21swapper/22
30775072101,4sleep03077572-21ntpq0
30423892101,4sleep13042516-21ntpq1
30186722101,2sleep10-21swapper/11
30000372101,4sleep02876378-21diskmemload0
29921482101,4sleep02992162-21ssh0
29845242101,6sleep02984533-21ssh0
29668272100,6sleep02876378-21diskmemload0
29652432100,1sleep20-21swapper/22
29381832101,3sleep00-21swapper/00
29262982103,4sleep1301irq_work/11
29138812101,3sleep00-21swapper/00
29122322101,4sleep22876378-21diskmemload2
29040772101,4sleep22876378-21diskmemload2
29023732101,5sleep02902469-21ssh0
28871932101,6sleep22876378-21diskmemload2
28822392101,3sleep20-21swapper/22
28744412101,6sleep22874490-21awk2
28282522101,4sleep02828513-21df_inode0
28165562101,5sleep22816777-21if_enp1s02
28105372101,4sleep02810747-21hddtemp_smartct0
3219499291,3sleep10-21swapper/11
3190662291,3sleep12876378-21diskmemload1
3160793291,4sleep13160801-21ssh1
3123290291,3sleep22876378-21diskmemload2
3053355291,2sleep10-21swapper/11
3033746291,4sleep23033808-21ntpq2
3000242291,4sleep12876378-21diskmemload1
2966733291,2sleep10-21swapper/11
2956557290,1sleep00-21swapper/00
2947269291,4sleep02876378-21diskmemload0
2946313291,4sleep12946372-21ssh1
2914157291,3sleep12876378-21diskmemload1
2893745291,4sleep02893809-21ntp_states0
2885701291,4sleep12876378-21diskmemload1
2859203291,4sleep22860906-21grep2
2822258291,4sleep22822508-21expr2
2817251291,4sleep02818955-21grep0
2815983291,4sleep02816080-21grep0
2885493281,3sleep02875367-21kworker/u16:0+rpciod0
2856402281,3sleep2559-21polkitd2
2850284280,3sleep22850332-21ntpq2
2846191281,2sleep12846210-21apt-get1
2840953281,3sleep02842629-21kthreadcore0
2839691280,1sleep00-21swapper/00
2809928281,4sleep12809929-21sshd1
2807587281,3sleep02809100-21kthreadcore0
2806385280,1sleep10-21swapper/11
2802816280,5sleep02804272-21kthreadcore0
2876190270,1sleep20-21swapper/22
2875309270,1sleep10-21swapper/11
2869978270,2sleep10-21swapper/11
2869769270,1sleep00-21swapper/00
2857096271,4sleep02809479-21kworker/0:1+events0
2845721271,2sleep22807504-21munin-node2
2838215271,2sleep00-21swapper/00
2834225270,2sleep10-21swapper/11
2833855270,1sleep10-21swapper/11
2827708270,1sleep20-21swapper/22
2821614270,2sleep20-21swapper/22
2816514270,4sleep12816555-21fschecks_count1
2810244270,2sleep10-21swapper/11
2809597270,1sleep20-21swapper/22
2876038260,1sleep10-21swapper/11
2870664261,2sleep00-21swapper/00
2869283260,1sleep10-21swapper/11
2864043260,4sleep00-21swapper/00
2858199260,1sleep00-21swapper/00
2857856260,1sleep10-21swapper/11
2857242260,1sleep10-21swapper/11
2851782260,1sleep10-21swapper/11
2851782260,1sleep10-21swapper/11
2851631260,1sleep00-21swapper/00
2851631260,1sleep00-21swapper/00
2851613260,4sleep20-21swapper/22
2851613260,4sleep20-21swapper/22
2839864260,1sleep10-21swapper/11
2833881262,2sleep22802090-21cyclictest2
2833726260,1sleep00-21swapper/00
2833010260,4sleep10-21swapper/11
2827976260,2sleep00-21swapper/00
2827694260,1sleep10-21swapper/11
2827107260,1sleep00-21swapper/00
2845545250,1sleep10-21swapper/11
2844995250,3sleep20-21swapper/22
2839580250,3sleep20-21swapper/22
2821730250,3sleep00-21swapper/00
2869703240,1sleep20-21swapper/22
2864672240,1sleep20-21swapper/22
2863775240,1sleep10-21swapper/11
2863243240,1sleep20-21swapper/22
2857334240,1sleep00-21swapper/00
2846486240,1sleep00-21swapper/00
2845534240,2sleep00-21swapper/00
2840395240,1sleep10-21swapper/11
2821565240,1sleep10-21swapper/11
2815855240,1sleep10-21swapper/11
2815520240,1sleep20-21swapper/22
2802536240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional