You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 21:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sat Jun 20, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
219120521300,1sleep30-21swapper/33
1894823211961,54sleep30-21swapper/33
1894897210790,10sleep20-21swapper/22
189493628252,12sleep10-21swapper/11
189395827855,15sleep00-21swapper/00
189518199252,9cyclictest2107140-21taskset3
189518199211,17cyclictest0-21swapper/33
21672382201,3sleep00-21swapper/00
1895181992017,2cyclictest0-21swapper/33
189518199201,16cyclictest0-21swapper/33
189518199200,1cyclictest0-21swapper/33
189518199191,3cyclictest0-21swapper/33
1895181991911,4cyclictest0-21swapper/33
189518199191,14cyclictest0-21swapper/33
189518199191,14cyclictest0-21swapper/33
189518199191,14cyclictest0-21swapper/33
189518199191,14cyclictest0-21swapper/33
189518199191,14cyclictest0-21swapper/33
189518199191,14cyclictest0-21swapper/33
189518199191,14cyclictest0-21swapper/33
189518199191,14cyclictest0-21swapper/33
189518199190,17cyclictest0-21swapper/33
20904812183,10sleep12090484-21apt-get1
20546442180,2sleep02022890-21diskmemload0
20385812181,2sleep00-21swapper/00
1895181991814,2cyclictest0-21swapper/33
1895181991812,3cyclictest0-21swapper/33
189518199181,14cyclictest0-21swapper/33
189518199181,14cyclictest0-21swapper/33
189518199181,14cyclictest0-21swapper/33
189518199181,14cyclictest0-21swapper/33
189518199181,14cyclictest0-21swapper/33
189518199181,14cyclictest0-21swapper/33
189518199181,14cyclictest0-21swapper/33
189518199181,14cyclictest0-21swapper/33
189518199181,13cyclictest0-21swapper/33
189518199181,13cyclictest0-21swapper/33
189518199181,13cyclictest0-21swapper/33
189518199181,13cyclictest0-21swapper/33
189518199181,13cyclictest0-21swapper/33
189518199181,13cyclictest0-21swapper/33
189518199181,12cyclictest0-21swapper/33
189518199180,15cyclictest0-21swapper/33
189518199180,15cyclictest0-21swapper/33
189518199180,14cyclictest0-21swapper/33
23518292171,5sleep12353007-21memory1
224722721711,3sleep1321rcuc/11
22320172171,2sleep20-21swapper/22
20823312171,5sleep22022890-21diskmemload2
20624322171,2sleep20-21swapper/22
189518199172,9cyclictest2031731-21taskset3
189518199171,14cyclictest0-21swapper/33
189518199171,14cyclictest0-21swapper/33
189518199171,13cyclictest0-21swapper/33
189518199171,13cyclictest0-21swapper/33
189518199171,12cyclictest0-21swapper/33
189518199170,4cyclictest1996389-21taskset3
189518199170,14cyclictest66950irq/132-enp1s0-TxRx-33
189518199170,14cyclictest0-21swapper/33
189518199170,13cyclictest0-21swapper/33
189518199170,12cyclictest0-21swapper/33
22879422161,5sleep22287953-21ssh2
21991022161,4sleep22199124-21apt-get2
21383352160,1sleep20-21swapper/22
20875342161,2sleep10-21swapper/11
189518199162,11cyclictest2148570-21sleep3
189518199161,3cyclictest0-21swapper/33
1895181991612,2cyclictest0-21swapper/33
1895181991611,3cyclictest0-21swapper/33
189518199160,14cyclictest0-21swapper/33
189518199160,13cyclictest0-21swapper/33
23223722150,10sleep00-21swapper/00
20903992151,3sleep02022890-21diskmemload0
20589602151,10sleep12060665-21grep1
19342092151,10sleep11934211-21cpu1
189518199151,2cyclictest0-21swapper/33
1895181991511,2cyclictest0-21swapper/33
189518199151,11cyclictest1944204-21taskset3
189518199151,11cyclictest0-21swapper/33
189518199150,13cyclictest0-21swapper/33
189518199150,12cyclictest0-21swapper/33
189518199150,11cyclictest0-21swapper/33
23117792141,7sleep22311846-21ntp_states2
22638292141,9sleep12263843-21ssh1
22555012141,9sleep12255515-21ssh1
22403812141,9sleep02240403-21apt0
22043832141,7sleep10-21swapper/11
21978322141,9sleep02197834-21ssh0
21722582141,2sleep10-21swapper/11
21478942141,4sleep22147905-21ssh2
21338902144,2sleep1171rcu_preempt1
20552852141,7sleep12022890-21diskmemload1
20371482140,2sleep20-21swapper/22
189518199141,8cyclictest1902618-21taskset3
189518199141,2cyclictest0-21swapper/33
1895181991410,2cyclictest0-21swapper/33
189518199140,2cyclictest0-21swapper/33
23564022131,3sleep22356416-21ssh2
23048952131,7sleep22022890-21diskmemload2
23007562131,7sleep12302353-21tr1
22966302131,8sleep12296642-21rm1
22899952131,8sleep12290009-21ssh1
22565332131,8sleep02256543-21ssh0
22220552130,8sleep02022890-21diskmemload0
22176322131,8sleep12219273-21tr1
21957642131,8sleep22022890-21diskmemload2
21898632131,8sleep22189877-21ssh2
21149192131,3sleep02114930-21ssh0
20779062130,6sleep22078035-21gzip2
20275942131,3sleep02027749-21ssh0
19541312131,8sleep21954352-21/usr/sbin/munin2
19387792130,1sleep20-21swapper/22
1895181991311,1cyclictest0-21swapper/33
23473902120,3sleep02347400-21ssh0
23384802121,7sleep12338484-21ssh1
23350352121,6sleep02022890-21diskmemload0
23274662122,5sleep20-21swapper/22
22798002121,7sleep22279813-21ssh2
22750012121,6sleep12276488-21grep1
22716172121,7sleep22271628-21ssh2
22710052123,2sleep0171rcu_preempt0
22653112121,7sleep22265324-21ssh2
22399142121,5sleep12022890-21diskmemload1
22300812121,2sleep00-21swapper/00
22138132120,1sleep00-21swapper/00
21981252121,7sleep12198135-21ssh1
21883492121,8sleep10-21swapper/11
21822412121,4sleep00-21swapper/00
21759922121,7sleep22177665-21cut2
21626362121,6sleep12022890-21diskmemload1
21369612121,8sleep02022890-21diskmemload0
21225202121,7sleep02122534-21ssh0
21137222121,6sleep22022890-21diskmemload2
20953682121,7sleep22095379-21ssh2
20638112121,5sleep02022890-21diskmemload0
20311902121,7sleep12031201-21ssh1
19708382121,5sleep00-21swapper/00
19078682122,4sleep11907887-21apt-get1
23641232110,1sleep00-21swapper/00
23626182111,6sleep22022890-21diskmemload2
23573162111,6sleep12022890-21diskmemload1
23558002111,6sleep02355810-21ssh0
22986902111,3sleep00-21swapper/00
22800122111,1sleep00-21swapper/00
22483762111,4sleep02022890-21diskmemload0
22439812111,4sleep22022890-21diskmemload2
22303872111,6sleep22022890-21diskmemload2
22126912111,4sleep12022890-21diskmemload1
21811722111,3sleep10-21swapper/11
21805352111,3sleep00-21swapper/00
21620782111,3sleep20-21swapper/22
21302712110,7sleep12130282-21sh1
21296692110,4sleep20-21swapper/22
20890612111,6sleep02089071-21ssh0
20781952111,5sleep02078450-21users0
20700442111,3sleep10-21swapper/11
20457162110,1sleep00-21swapper/00
20449292111,4sleep22022890-21diskmemload2
20179362111,5sleep12018195-21df_inode1
20123992111,7sleep12012543-21cat1
19975452115,2sleep0251irq_work/00
19763562110,3sleep01976068-21/usr/sbin/munin0
19652982112,5sleep21965354-21systemd-run2
19550972112,4sleep01955116-21apt-get0
19499112111,5sleep01950170-21df_inode0
19186602111,5sleep21918872-21hddtemp_smartct2
18994232111,6sleep11899509-21tr1
23705112101,3sleep20-21swapper/22
23654592101,2sleep00-21swapper/00
23461382100,1sleep20-21swapper/22
23456312100,2sleep12345641-21ssh1
23374592101,3sleep20-21swapper/22
23302502101,4sleep12022890-21diskmemload1
23289502101,4sleep02022890-21diskmemload0
23236772101,4sleep12022890-21diskmemload1
23162262101,4sleep22022890-21diskmemload2
23139492101,4sleep02022890-21diskmemload0
23076412101,3sleep12307676-21ssh1
23041792101,4sleep02022890-21diskmemload0
22903622100,1sleep00-21swapper/00
22711072101,4sleep12022890-21diskmemload1
22653342101,4sleep02022890-21diskmemload0
22497792101,5sleep22250950-21cat2
22293312101,4sleep12022890-21diskmemload1
21655462101,2sleep20-21swapper/22
21644612101,4sleep02022890-21diskmemload0
21554882100,4sleep10-21swapper/11
21539032101,3sleep20-21swapper/22
21538282101,3sleep00-21swapper/00
21483492101,3sleep00-21swapper/00
21457622101,4sleep12022890-21diskmemload1
21303992101,4sleep02022890-21diskmemload0
21057462101,4sleep12022890-21diskmemload1
21024032101,5sleep22102786-21missed_timers2
20737402101,3sleep10-21swapper/11
20722402100,1sleep00-21swapper/00
20707642101,3sleep20-21swapper/22
20400632100,2sleep10-21swapper/11
20356682101,4sleep12035819-21ntp_192.168.1141
20274002101,4sleep22027464-21ntp_states2
19917932101,3sleep11991811-21apt-get1
19854952101,5sleep01985670-21wc0
19813092101,4sleep21981320-21apt-get2
19391512101,4sleep01939207-21munin-run0
19189612101,5sleep01920460-21cut0
19185472101,5sleep11918659-21/usr/sbin/munin1
19119012101,5sleep21911959-21grep2
18953552101,3sleep00-21swapper/00
2371133291,5sleep10-21swapper/11
2291527291,4sleep22291671-21grep2
2216087291,4sleep22022890-21diskmemload2
2213695291,4sleep22213709-21ssh2
2204087290,2sleep00-21swapper/00
2121333291,2sleep20-21swapper/22
2115474291,3sleep12022890-21diskmemload1
2112417291,4sleep12112431-21ssh1
2104149291,4sleep02104160-21ssh0
2053616291,2sleep20-21swapper/22
2018475291,4sleep02019256-21cut0
2013207291,4sleep02013276-21kernelversion0
1991488291,4sleep11991596-21cut1
1976067291,2sleep11976086-21apt-get1
1969592290,1sleep20-21swapper/22
1969478290,1sleep10-21swapper/11
1944660290,2sleep20-21swapper/22
1908155291,4sleep01908319-21head0
2014687281,3sleep22016439-21kthreadcore2
2007968281,3sleep22008279-21kthreadcore2
2002211280,2sleep20-21swapper/22
1997264280,1sleep20-21swapper/22
1996699281,3sleep01996723-21latency_hist0
1976636281,3sleep21976672-21kthreadcore2
1971742281,3sleep11973374-21kthreadcore1
1965128281,4sleep1116341-21sshd1
1944256280,1sleep10-21swapper/11
1941024281,4sleep01942687-21kthreadcore0
1928939280,2sleep20-21swapper/22
1913733281,3sleep21915312-21cut2
2022740270,1sleep20-21swapper/22
2013279271,2sleep10-21swapper/11
2012038270,1sleep00-21swapper/00
2007024270,1sleep10-21swapper/11
2003804271,2sleep00-21swapper/00
1996645270,1sleep20-21swapper/22
1986537270,2sleep20-21swapper/22
1981300271,2sleep10-21swapper/11
1971107271,2sleep20-21swapper/22
1970340270,1sleep00-21swapper/00
1960360271,2sleep00-21swapper/00
1955119270,3sleep11955229-21tr1
1950171271,3sleep11950292-21grep1
1944930271,3sleep01944653-21/usr/sbin/munin0
1933832270,1sleep00-21swapper/00
1924139271,3sleep01924514-21cut0
1923881271,2sleep20-21swapper/22
1913102270,2sleep10-21swapper/11
1904258271,2sleep00-21swapper/00
1900787271,2sleep10-21swapper/11
1900288270,1sleep20-21swapper/22
2001714260,1sleep10-21swapper/11
1991233260,4sleep00-21swapper/00
1954884260,2sleep20-21swapper/22
1949272260,1sleep10-21swapper/11
1939419260,1sleep20-21swapper/22
1902596260,1sleep20-21swapper/22
1932735250,1sleep10-21swapper/11
1923627250,1sleep10-21swapper/11
1913401250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional