You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 16:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Sun May 03, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
177987121320,2sleep10-21swapper/11
1530122211759,54sleep30-21swapper/33
152969128156,16sleep00-21swapper/00
153015628049,11sleep20-21swapper/22
152518728050,11sleep10-21swapper/11
16617132250,11sleep014-21ksoftirqd/00
1530685992519,2cyclictest0-21swapper/33
153068599241,20cyclictest0-21swapper/33
153068599231,3cyclictest0-21swapper/33
153068599231,3cyclictest0-21swapper/33
1530685992220,1cyclictest0-21swapper/33
153068599221,3cyclictest0-21swapper/33
153068599221,17cyclictest66950irq/132-enp1s0-TxRx-33
153068599211,15cyclictest0-21swapper/33
153068599201,8cyclictest1998861-21chrt3
1530685992015,2cyclictest0-21swapper/33
1530685992015,2cyclictest0-21swapper/33
153068599201,3cyclictest0-21swapper/33
1530685992012,6cyclictest0-21swapper/33
1530685992012,5cyclictest0-21swapper/33
1530685992011,6cyclictest1728129-21taskset3
153068599200,16cyclictest66950irq/132-enp1s0-TxRx-33
153068599193,13cyclictest0-21swapper/33
1530685991916,1cyclictest0-21swapper/33
1530685991915,2cyclictest0-21swapper/33
1530685991912,5cyclictest0-21swapper/33
153068599191,15cyclictest0-21swapper/33
153068599191,15cyclictest0-21swapper/33
153068599191,14cyclictest0-21swapper/33
153068599191,14cyclictest0-21swapper/33
153068599190,2cyclictest0-21swapper/33
153068599190,15cyclictest0-21swapper/33
18053662181,3sleep10-21swapper/11
153068599189,6cyclictest1634580-21taskset3
153068599187,7cyclictest1837119-21taskset3
153068599182,11cyclictest1581227-21taskset3
153068599182,11cyclictest1581227-21taskset3
153068599181,4cyclictest1634205-21chrt3
1530685991812,3cyclictest0-21swapper/33
153068599181,15cyclictest0-21swapper/33
153068599181,14cyclictest0-21swapper/33
153068599181,14cyclictest0-21swapper/33
153068599181,14cyclictest0-21swapper/33
153068599181,14cyclictest0-21swapper/33
153068599181,14cyclictest0-21swapper/33
153068599181,14cyclictest0-21swapper/33
153068599181,13cyclictest0-21swapper/33
153068599181,13cyclictest0-21swapper/33
153068599181,13cyclictest0-21swapper/33
153068599181,12cyclictest1861773-21chrt3
153068599180,16cyclictest0-21swapper/33
153068599180,15cyclictest0-21swapper/33
17171682171,9sleep11717227-21ntpq1
15781502170,4sleep0221rcuc/00
153068599179,4cyclictest1625426-21taskset3
153068599176,8cyclictest1644964-21turbostat3
153068599171,4cyclictest0-21swapper/33
1530685991713,2cyclictest0-21swapper/33
153068599171,13cyclictest0-21swapper/33
153068599171,13cyclictest0-21swapper/33
1530685991710,4cyclictest0-21swapper/33
153068599170,14cyclictest0-21swapper/33
153068599170,14cyclictest0-21swapper/33
153068599170,14cyclictest0-21swapper/33
153068599170,13cyclictest0-21swapper/33
19917922164,2sleep2171rcu_preempt2
18813992160,1sleep10-21swapper/11
17598202161,4sleep01759878-21ntpq0
17097542161,3sleep20-21swapper/22
153068599161,14cyclictest0-21swapper/33
153068599161,12cyclictest0-21swapper/33
153068599160,13cyclictest0-21swapper/33
153068599160,13cyclictest0-21swapper/33
19741092150,1sleep10-21swapper/11
19555402151,2sleep00-21swapper/00
19484542151,3sleep20-21swapper/22
18998992151,4sleep01899925-21apt-get0
18703542151,4sleep01661240-21diskmemload0
18218292150,11sleep11821843-21ssh1
18134082151,7sleep20-21swapper/22
16766842151,2sleep10-21swapper/11
15602262152,9sleep11561231-21kthreadcore1
153068599159,4cyclictest0-21swapper/33
1530685991511,2cyclictest0-21swapper/33
153068599150,13cyclictest0-21swapper/33
153068599150,12cyclictest0-21swapper/33
153068599150,12cyclictest0-21swapper/33
20071182141,3sleep21661240-21diskmemload2
19503552141,4sleep11950369-21ssh1
19470062141,4sleep11947019-21ssh1
16840702141,9sleep11684084-21ssh1
16621462141,8sleep21662224-21irqstats2
16333032140,2sleep10-21swapper/11
15537572141,9sleep21553760-21sshd2
1530685991412,1cyclictest0-21swapper/33
1530685991412,1cyclictest0-21swapper/33
153068599140,2cyclictest0-21swapper/33
153068599140,12cyclictest0-21swapper/33
20063882131,3sleep11661240-21diskmemload1
19921632131,4sleep172-21kcompactd01
19815412132,2sleep1171rcu_preempt1
19587972135,3sleep21661240-21diskmemload2
19386242131,8sleep01938634-21ssh0
18714092131,7sleep20-21swapper/22
18567702131,8sleep21856781-21ssh2
18560912135,4sleep11661240-21diskmemload1
18217902131,5sleep20-21swapper/22
17879532131,8sleep01787966-21sh0
17867962131,7sleep11661240-21diskmemload1
17864672131,7sleep21661240-21diskmemload2
16385562131,6sleep01638636-21ntpq0
16333512131,6sleep21633400-21ntpq2
20102082121,3sleep02011840-21kthreadcore0
19840382121,6sleep090-21kswapd00
19827702121,5sleep00-21swapper/00
19405662121,5sleep20-21swapper/22
19304542121,7sleep01930466-21rm0
19304542121,7sleep01930466-21rm0
18973772121,7sleep11897391-21ssh1
18575132121,6sleep11661240-21diskmemload1
18478932121,2sleep2171rcu_preempt2
18315292120,5sleep20-21swapper/22
17896212121,7sleep01789640-21apt-get0
17616342121,6sleep20-21swapper/22
17450812121,6sleep21661240-21diskmemload2
17450812121,6sleep21661240-21diskmemload2
16957662121,2sleep11695777-21ssh1
15962642121,7sleep21596485-21/usr/sbin/munin2
15906222122,4sleep11590670-21ntpq1
15855802121,7sleep01581206-21/usr/sbin/munin0
15855802121,7sleep01581206-21/usr/sbin/munin0
15717462121,7sleep11573480-21cut1
15477182121,5sleep01547759-21ntpq0
19982252111,5sleep01661240-21diskmemload0
19729202111,2sleep20-21swapper/22
19641792110,3sleep11661240-21diskmemload1
19588752111,3sleep00-21swapper/00
19313262111,3sleep20-21swapper/22
19313262111,3sleep20-21swapper/22
19219632111,3sleep00-21swapper/00
19056262111,3sleep10-21swapper/11
18872342110,2sleep00-21swapper/00
18810872111,5sleep21661240-21diskmemload2
18483692111,5sleep01848816-21apt-config0
18287522110,7sleep11828759-21ssh1
18128432111,5sleep11661240-21diskmemload1
18061702111,1sleep00-21swapper/00
17983412111,5sleep290-21kswapd02
17732522111,5sleep01773361-21irqstats0
17543402111,3sleep20-21swapper/22
17442632111,3sleep00-21swapper/00
17442632111,3sleep00-21swapper/00
17306452111,6sleep21731807-21kthreadcore2
17289712111,5sleep11661240-21diskmemload1
16974802111,7sleep01699146-21cut0
16945452111,3sleep20-21swapper/22
16838392111,6sleep21683849-21ssh2
16707082111,6sleep21671714-21kthreadcore2
16454942112,4sleep20-21swapper/22
16025172112,4sleep21602536-21apt-get2
16011252111,5sleep11601238-21ntpq1
15699212110,1sleep20-21swapper/22
15692192111,5sleep01569267-21ntpq0
20160722101,2sleep20-21swapper/22
20098372102,4sleep10-21swapper/11
20013482101,4sleep02001562-21diskstats0
19999542101,3sleep20-21swapper/22
19913462101,4sleep11661240-21diskmemload1
19756322101,4sleep21661240-21diskmemload2
19582552101,2sleep20-21swapper/22
19306132101,3sleep10-21swapper/11
19306132101,3sleep10-21swapper/11
19216892101,3sleep10-21swapper/11
19140272101,2sleep20-21swapper/22
19138002101,4sleep11661240-21diskmemload1
19063632101,2sleep20-21swapper/22
18968002101,3sleep00-21swapper/00
18956802101,5sleep21895736-21ntpq2
18880462101,4sleep11661240-21diskmemload1
18871002101,4sleep21887161-21ntpq2
18642842101,4sleep01661240-21diskmemload0
18632872101,2sleep20-21swapper/22
18559772101,3sleep00-21swapper/00
18468302101,2sleep10-21swapper/11
18384682100,1sleep00-21swapper/00
18238622101,5sleep01824007-21grep0
18230502101,2sleep00-21swapper/00
18134932101,5sleep01813504-21ssh0
17951162101,3sleep10-21swapper/11
17784672101,4sleep21778479-21ssh2
17685412101,5sleep01768674-21gzip0
17531042101,3sleep00-21swapper/00
17530152101,4sleep11661240-21diskmemload1
17449382101,3sleep10-21swapper/11
17449382101,3sleep10-21swapper/11
17374142101,4sleep01661240-21diskmemload0
17340592101,4sleep11734191-21ntpq1
17270582100,3sleep20-21swapper/22
17266702100,5sleep01661240-21diskmemload0
17129502101,4sleep21661240-21diskmemload2
17011562101,4sleep21661240-21diskmemload2
16926932100,1sleep00-21swapper/00
16609642101,3sleep272-21kcompactd02
16494532101,4sleep11649542-21ntpq1
16127442100,1sleep20-21swapper/22
16084402101,6sleep11609446-21cut1
15812052102,4sleep21581224-21apt-get2
15812052102,4sleep21581224-21apt-get2
15386442101,5sleep11538821-21sed1
1971438291,2sleep00-21swapper/00
1946234291,4sleep01946300-21collect20
1934954291,4sleep11936606-21tr1
1924435291,4sleep21661240-21diskmemload2
1913606291,2sleep00-21swapper/00
1880825291,3sleep01661240-21diskmemload0
1871135291,3sleep10-21swapper/11
1839571290,2sleep20-21swapper/22
1837603291,2sleep10-21swapper/11
1789500290,3sleep21661240-21diskmemload2
1769672290,1sleep20-21swapper/22
1764184291,3sleep10-21swapper/11
1718805291,3sleep01661240-21diskmemload0
1708613291,3sleep01708677-21ntpq0
1704667291,2sleep10-21swapper/11
1692457291,4sleep11692468-21ssh1
1683864290,4sleep01661240-21diskmemload0
1676331291,2sleep00-21swapper/00
1666938291,3sleep11661240-21diskmemload1
1638637291,4sleep11638693-21grep1
1634322291,4sleep01634332-21apt0
1615095291,4sleep01616919-21cut0
1593249291,4sleep11594996-21tr1
1570526291,3sleep01570545-21apt-get0
1554333291,3sleep00-21swapper/00
1547761291,4sleep11547808-21ntpq1
1530988291,3sleep21531007-21apt-get2
1759880280,3sleep10-21swapper/11
1661230281,4sleep01661255-21latency_hist0
1656165281,2sleep10-21swapper/11
1655734280,1sleep10-21swapper/11
1654745281,3sleep00-21swapper/00
1645459281,3sleep01644965-21apt-get0
1644805281,4sleep11644806-21sshd1
1641147282,2sleep01642876-21kthreadcore0
1618575280,2sleep20-21swapper/22
1617399281,3sleep21-21systemd2
1586553281,3sleep01586575-21apt-get0
1580660280,1sleep10-21swapper/11
1563890281,3sleep01564032-31find0
1554068280,4sleep01554263-21latency_hist0
1640288270,4sleep21640463-21if_err_enp1s02
1627940270,2sleep20-21swapper/22
1623489270,1sleep10-21swapper/11
1622930271,3sleep00-21swapper/00
1607579270,1sleep10-21swapper/11
1600419270,3sleep01601124-21sed0
1591885270,2sleep00-21swapper/00
1586056270,1sleep10-21swapper/11
1586056270,1sleep10-21swapper/11
1575851270,2sleep21575873-21apt-get2
1573481270,3sleep20-21swapper/22
1565175271,3sleep10-21swapper/11
1559049270,1sleep10-21swapper/11
1543324270,1sleep00-21swapper/00
1543123270,1sleep20-21swapper/22
1536095270,1sleep00-21swapper/00
1535938270,1sleep10-21swapper/11
1652149261,2sleep20-21swapper/22
1638694260,2sleep20-21swapper/22
1628631260,4sleep00-21swapper/00
1607863260,4sleep00-21swapper/00
1602796261,2sleep01602845-21grep0
1601244260,2sleep20-21swapper/22
1561233260,3sleep21563027-21grep2
1553952260,1sleep10-21swapper/11
1546094260,3sleep21543686-21/usr/sbin/munin2
1627818250,1sleep10-21swapper/11
1616922250,1sleep10-21swapper/11
1591399250,3sleep20-21swapper/22
1558707250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional