You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-08 - 11:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Wed Apr 08, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1751099212457,62sleep30-21swapper/33
175083028567,12sleep20-21swapper/22
175095428062,11sleep10-21swapper/11
175108627656,14sleep00-21swapper/00
190542122910,7sleep01905432-21ssh0
21707182261,21sleep21882179-21diskmemload2
175135599221,17cyclictest1888681-21taskset3
175135599212,16cyclictest0-21swapper/33
175135599211,7cyclictest2110087-21taskset3
1751355992111,7cyclictest0-21swapper/33
175135599211,16cyclictest66950irq/132-enp1s0-TxRx-33
175135599211,16cyclictest0-21swapper/33
175135599210,19cyclictest0-21swapper/33
1751355992011,6cyclictest2180107-21taskset3
1751355992011,5cyclictest0-21swapper/33
175135599201,15cyclictest0-21swapper/33
175135599201,15cyclictest0-21swapper/33
1751355992010,7cyclictest1802536-21taskset3
1751355991913,3cyclictest0-21swapper/33
175135599191,15cyclictest66950irq/132-enp1s0-TxRx-33
175135599191,14cyclictest2090261-21taskset3
175135599191,14cyclictest2090261-21taskset3
175135599191,14cyclictest0-21swapper/33
175135599191,14cyclictest0-21swapper/33
175135599191,14cyclictest0-21swapper/33
175135599191,14cyclictest0-21swapper/33
175135599190,16cyclictest0-21swapper/33
213415921810,5sleep21882179-21diskmemload2
175135599184,2cyclictest0-21swapper/33
1751355991816,1cyclictest0-21swapper/33
1751355991812,4cyclictest0-21swapper/33
175135599181,14cyclictest2195376-21taskset3
175135599181,14cyclictest0-21swapper/33
175135599181,14cyclictest0-21swapper/33
175135599181,14cyclictest0-21swapper/33
175135599181,14cyclictest0-21swapper/33
175135599181,14cyclictest0-21swapper/33
175135599181,14cyclictest0-21swapper/33
175135599181,14cyclictest0-21swapper/33
175135599181,14cyclictest0-21swapper/33
175135599181,13cyclictest0-21swapper/33
175135599181,13cyclictest0-21swapper/33
175135599181,12cyclictest0-21swapper/33
175135599181,12cyclictest0-21swapper/33
175135599180,14cyclictest66950irq/132-enp1s0-TxRx-33
175135599180,14cyclictest66950irq/132-enp1s0-TxRx-33
21265892171,3sleep10-21swapper/11
21263832171,3sleep00-21swapper/00
20559032170,2sleep20-21swapper/22
175135599178,6cyclictest1866175-21turbostat3
175135599178,5cyclictest0-21swapper/33
1751355991713,2cyclictest0-21swapper/33
175135599171,13cyclictest66950irq/132-enp1s0-TxRx-33
175135599171,13cyclictest0-21swapper/33
175135599171,13cyclictest0-21swapper/33
175135599171,13cyclictest0-21swapper/33
175135599171,13cyclictest0-21swapper/33
175135599171,12cyclictest0-21swapper/33
175135599171,12cyclictest0-21swapper/33
175135599170,14cyclictest66950irq/132-enp1s0-TxRx-33
21974162161,4sleep11882179-21diskmemload1
21712032161,2sleep10-21swapper/11
21177172161,4sleep21882179-21diskmemload2
19412732161,2sleep00-21swapper/00
18992092161,4sleep21882179-21diskmemload2
18992092161,4sleep21882179-21diskmemload2
18882512161,4sleep11882179-21diskmemload1
175135599167,6cyclictest1854084-21taskset3
175135599167,5cyclictest1764199-21turbostat3
175135599163,8cyclictest0-21swapper/33
175135599160,14cyclictest0-21swapper/33
175135599160,14cyclictest0-21swapper/33
22485322151,2sleep10-21swapper/11
22155202152,5sleep0171rcu_preempt0
21086572150,3sleep10-21swapper/11
20974242151,7sleep00-21swapper/00
20551232151,9sleep11882179-21diskmemload1
20294952154,2sleep1171rcu_preempt1
19339632151,9sleep11882179-21diskmemload1
19153232151,9sleep11882179-21diskmemload1
175135599150,13cyclictest0-21swapper/33
175135599150,13cyclictest0-21swapper/33
175135599150,13cyclictest0-21swapper/33
175135599150,13cyclictest0-21swapper/33
175135599150,13cyclictest0-21swapper/33
175135599150,12cyclictest0-21swapper/33
175135599150,11cyclictest0-21swapper/33
21634342145,5sleep02163468-21sh0
21290712141,8sleep02130650-21cut0
19935362141,3sleep20-21swapper/22
19935362141,3sleep20-21swapper/22
19601042141,9sleep01960115-21ssh0
18395732142,8sleep01839799-21cat0
18170082141,9sleep11812871-21/usr/sbin/munin1
17722272141,9sleep21773700-21/usr/sbin/munin2
175135599149,3cyclictest0-21swapper/33
175135599149,3cyclictest0-21swapper/33
175135599145,6cyclictest1860878-21turbostat3
175135599144,6cyclictest0-21swapper/33
1751355991411,2cyclictest0-21swapper/33
175135599140,12cyclictest0-21swapper/33
175135599140,12cyclictest0-21swapper/33
175135599140,12cyclictest0-21swapper/33
22223472131,3sleep02222361-21ssh0
21959972131,3sleep02196008-21ssh0
21887782131,2sleep20-21swapper/22
21767682131,7sleep11882179-21diskmemload1
21412872131,6sleep02141345-21ntpq0
21151062131,6sleep02115157-21ntpq0
20408232131,6sleep22041043-21hddtemp_smartct2
20192342131,7sleep01882179-21diskmemload0
20120692131,5sleep20-21swapper/22
19852002131,6sleep11882179-21diskmemload1
19663892131,3sleep11966400-21ssh1
19425152131,8sleep21882179-21diskmemload2
19176322130,2sleep20-21swapper/22
18811482136,4sleep00-21swapper/00
17793542131,6sleep11779410-21ntpq1
22151252121,5sleep11882179-21diskmemload1
22051662121,3sleep12205177-21ssh1
21525042121,3sleep22152515-21ssh2
21334932121,5sleep11882179-21diskmemload1
21014682121,7sleep12101817-21latency_hist1
20981612121,3sleep22098171-21ssh2
20899312120,2sleep10-21swapper/11
20899312120,2sleep10-21swapper/11
20384192121,6sleep11882179-21diskmemload1
19778002121,7sleep11882179-21diskmemload1
19681052121,7sleep01968119-21ssh0
19602772121,7sleep21882179-21diskmemload2
19429672121,6sleep11882179-21diskmemload1
19178232121,5sleep01917956-21diskstats0
18867582122,5sleep00-21swapper/00
18665082121,5sleep11866711-21diskstats1
18622942121,7sleep01864045-21cut0
22431892111,3sleep02243275-21uname0
22428302111,5sleep22242963-21diskstats2
22317272111,4sleep21882179-21diskmemload2
22236862111,5sleep21882179-21diskmemload2
22045452110,2sleep00-21swapper/00
21872212114,4sleep0151ktimers/00
21613142111,6sleep11882179-21diskmemload1
21527522111,4sleep11882179-21diskmemload1
21433082111,4sleep11882179-21diskmemload1
21424952111,4sleep21882179-21diskmemload2
21236332111,4sleep22123756-21strings2
20894612111,3sleep00-21swapper/00
20894612111,3sleep00-21swapper/00
20817502111,3sleep20-21swapper/22
20756142111,6sleep02075783-21/usr/sbin/munin0
20484792111,6sleep02048493-21ssh0
20483842111,4sleep11882179-21diskmemload1
20322742111,6sleep02032455-21cut0
20199162111,3sleep20-21swapper/22
20141852111,2sleep1171rcu_preempt1
20135732111,5sleep00-21swapper/00
20028922112,3sleep21882179-21diskmemload2
19990102111,6sleep12000638-21cat1
19931512110,7sleep01993165-21ssh0
19931512110,7sleep01993165-21ssh0
19758572110,4sleep00-21swapper/00
19674712111,5sleep21882179-21diskmemload2
19583852111,4sleep11882179-21diskmemload1
19505572111,2sleep21950568-21ssh2
19491192111,4sleep11882179-21diskmemload1
19266712111,5sleep01926892-21hddtemp_smartct0
19018492111,6sleep21903559-21kthreadcore2
18812032110,1sleep10-21swapper/11
18522552111,6sleep21854083-21kthreadcore2
18491682111,5sleep11849216-21ntpq1
18439312111,5sleep01843985-21ntpq0
18291792111,5sleep11829198-21apt-get1
18238262111,4sleep20-21swapper/22
18016732111,5sleep21801721-21munin-run2
17859932111,5sleep21786196-21diskstats2
22396152101,2sleep10-21swapper/11
22313592101,3sleep11882179-21diskmemload1
22250782100,1sleep00-21swapper/00
22205752100,5sleep12220635-21ntpq1
21991642101,4sleep21882179-21diskmemload2
21965242101,3sleep20-21swapper/22
21813142101,4sleep11882179-21diskmemload1
21777612101,2sleep20-21swapper/22
21166862101,2sleep10-21swapper/11
21094032100,1sleep00-21swapper/00
21070882101,4sleep21882179-21diskmemload2
20831142101,4sleep11882179-21diskmemload1
20743522101,2sleep00-21swapper/00
20724322101,3sleep21882179-21diskmemload2
20678452101,6sleep12069465-21cut1
20646922101,2sleep20-21swapper/22
20621942101,5sleep12062251-21ntpq1
20569842101,2sleep00-21swapper/00
20311632101,3sleep00-21swapper/00
20111432101,3sleep11882179-21diskmemload1
19878982101,4sleep11987919-21apt-get1
19878982101,4sleep11987919-21apt-get1
19131432101,6sleep21913202-21/usr/sbin/munin2
19042392101,5sleep11904299-21kworker/u16:31
18971852101,2sleep00-21swapper/00
18971852101,2sleep00-21swapper/00
18971462100,2sleep10-21swapper/11
18971462100,2sleep10-21swapper/11
18902472101,3sleep20-21swapper/22
18401152101,5sleep11840226-21fschecks_time1
18264692101,4sleep01827731-21memory0
18176752100,1sleep20-21swapper/22
18169552100,1sleep00-21swapper/00
18059732101,4sleep21806024-21ntpq2
17644652101,4sleep01764484-21apt-get0
2239114290,1sleep20-21swapper/22
2238060290,5sleep00-21swapper/00
2208196291,4sleep22208679-21grep2
2159854290,1sleep20-21swapper/22
2147940291,4sleep00-21swapper/00
2090099290,1sleep20-21swapper/22
2090099290,1sleep20-21swapper/22
2064665291,2sleep00-21swapper/00
2031347290,2sleep20-21swapper/22
2027977290,1sleep20-21swapper/22
2000874291,4sleep02000934-21ntpq0
1985310291,2sleep20-21swapper/22
1978828291,2sleep00-21swapper/00
1970459291,4sleep21970678-21hddtemp_smartct2
1950945291,3sleep01882179-21diskmemload0
1934486291,3sleep21882179-21diskmemload2
1923243290,1sleep10-21swapper/11
1914703291,4sleep0802-21snmpd0
1875858291,4sleep11875912-21ntpq1
1856759291,4sleep11858473-21kthreadcore1
1855804291,3sleep00-21swapper/00
1821974291,5sleep11822489-21needreboot1
1807806291,4sleep11807956-21cat1
1807320291,4sleep21807542-21expr2
1800573291,4sleep11800662-21ntpq1
1791271290,6sleep01791418-21wc0
1764487291,4sleep11764701-21expr1
1763017291,4sleep01763157-21/usr/sbin/munin0
1760061290,5sleep21761773-21kthreadcore2
2171802281,3sleep02171979-21ssh0
2160071281,3sleep02160085-21ssh0
1876811281,3sleep21876813-21modprobe2
1870488281,2sleep00-21swapper/00
1866161281,3sleep2255-21systemd-journal2
1849943280,1sleep00-21swapper/00
1835317281,3sleep21837029-21cut2
1834505281,2sleep10-21swapper/11
1820181281,3sleep00-21swapper/00
1781417281,3sleep21783126-21kthreadcore2
1780565280,3sleep10-21swapper/11
1779299281,3sleep01779346-21ntpq0
1769806280,3sleep01769825-21apt-get0
1764183282,3sleep11764227-21idleruntime-cro1
1752213281,3sleep21752311-21grep2
1752213281,3sleep21752311-21grep2
1876704270,1sleep00-21swapper/00
1865468271,3sleep11865517-21switchtime1
1860761270,1sleep20-21swapper/22
1854445270,3sleep00-21swapper/00
1845182271,3sleep20-21swapper/22
1807287270,2sleep00-21swapper/00
1807015271,3sleep11807086-21unin-run1
1804118271,3sleep01805820-21latency0
1796314270,1sleep20-21swapper/22
1784753271,2sleep01784894-21mailstats0
1780016270,1sleep20-21swapper/22
1769508270,1sleep20-21swapper/22
1882029260,4sleep20-21swapper/22
1870145260,2sleep21870436-21strings2
1851069261,2sleep11852252-21kthreadcore1
1834154260,1sleep00-21swapper/00
1829201260,2sleep20-21swapper/22
1828761260,1sleep10-21swapper/11
1823495260,1sleep20-21swapper/22
1800782260,3sleep00-21swapper/00
1796003260,4sleep10-21swapper/11
1790931260,1sleep10-21swapper/11
1770536261,2sleep10-21swapper/11
1756659260,1sleep00-21swapper/00
1756659260,1sleep00-21swapper/00
1756533260,1sleep10-21swapper/11
1756533260,1sleep10-21swapper/11
1843740250,1sleep20-21swapper/22
1790748250,3sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional