You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 09:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Thu Apr 02, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17102322390,1sleep0170620-21latency_hist0
54559721430,3sleep2421rcuc/22
88610212263,55sleep30-21swapper/33
9327128466,11sleep20-21swapper/22
9339928354,10sleep10-21swapper/11
9343727555,14sleep00-21swapper/00
93833993228,2cyclictest0-21swapper/33
9383399239,10cyclictest321961-21turbostat3
9383399221,17cyclictest0-21swapper/33
9383399221,16cyclictest0-21swapper/33
4419252220,2sleep1224530-21diskmemload1
9383399211,16cyclictest0-21swapper/33
9383399210,17cyclictest389438-21taskset3
9383399201,16cyclictest0-21swapper/33
9383399201,16cyclictest0-21swapper/33
9383399201,15cyclictest0-21swapper/33
9383399201,14cyclictest0-21swapper/33
3292662201,4sleep0329281-21ssh0
2381872201,5sleep2238243-21ntpq2
9383399192,12cyclictest224548-21turbostat3
9383399191,6cyclictest531107-21taskset3
9383399191,6cyclictest0-21swapper/33
9383399191,15cyclictest0-21swapper/33
9383399191,15cyclictest0-21swapper/33
9383399191,15cyclictest0-21swapper/33
9383399191,14cyclictest0-21swapper/33
9383399191,14cyclictest0-21swapper/33
9383399191,14cyclictest0-21swapper/33
9383399191,14cyclictest0-21swapper/33
9383399191,14cyclictest0-21swapper/33
9383399191,14cyclictest0-21swapper/33
93833991910,6cyclictest259030-21taskset3
9383399190,8cyclictest0-21swapper/33
9383399190,16cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,14cyclictest0-21swapper/33
9383399181,13cyclictest99014-21taskset3
9383399181,13cyclictest99014-21taskset3
9383399181,13cyclictest0-21swapper/33
9383399181,13cyclictest0-21swapper/33
9383399181,13cyclictest0-21swapper/33
9383399180,15cyclictest0-21swapper/33
9383399180,15cyclictest0-21swapper/33
9383399180,14cyclictest0-21swapper/33
4009752181,5sleep00-21swapper/00
9383399171,13cyclictest0-21swapper/33
9383399171,13cyclictest0-21swapper/33
9383399171,13cyclictest0-21swapper/33
9383399170,13cyclictest0-21swapper/33
9383399170,13cyclictest0-21swapper/33
4776212171,2sleep00-21swapper/00
2306522171,3sleep20-21swapper/22
1288092171,12sleep1129744-21kthreadcore1
9383399160,4cyclictest1812432sleep33
9383399160,13cyclictest0-21swapper/33
9383399160,13cyclictest0-21swapper/33
9383399160,13cyclictest0-21swapper/33
9383399160,13cyclictest0-21swapper/33
9383399160,13cyclictest0-21swapper/33
9383399160,13cyclictest0-21swapper/33
9383399160,12cyclictest0-21swapper/33
53078221610,2sleep2505701-21kworker/2:0+events_freezable2
4762612160,1sleep10-21swapper/11
4579712161,8sleep1458036-21ntpq1
4420102161,2sleep00-21swapper/00
3849652161,2sleep2386621-21kthreadcore2
3633972161,3sleep00-21swapper/00
1711672161,3sleep20-21swapper/22
9383399150,13cyclictest0-21swapper/33
9383399150,13cyclictest0-21swapper/33
9383399150,13cyclictest0-21swapper/33
9383399150,12cyclictest0-21swapper/33
9383399150,12cyclictest0-21swapper/33
9383399150,12cyclictest0-21swapper/33
9383399150,12cyclictest0-21swapper/33
5300452151,2sleep00-21swapper/00
5202102151,2sleep20-21swapper/22
3120952151,3sleep20-21swapper/22
1091282150,4sleep2421rcuc/22
9383399147,4cyclictest186605-21chrt3
93833991412,1cyclictest0-21swapper/33
9383399140,2cyclictest0-21swapper/33
5753722144,2sleep2171rcu_preempt2
5065062141,9sleep0506707-21grep0
3985212141,6sleep10-21swapper/11
3707422141,4sleep10-21swapper/11
2675382147,4sleep1267550-21rm1
2509242141,7sleep1224530-21diskmemload1
1965372141,4sleep1196594-21ntpq1
1429272142,8sleep1142980-21ldconfig1
1056032141,8sleep2105653-21ntpq2
986062131,8sleep298801-21/usr/sbin/munin2
986062131,8sleep298801-21/usr/sbin/munin2
5765912130,2sleep20-21swapper/22
5576802131,8sleep0557692-21ssh0
5391862131,3sleep2539200-21ssh2
5380192131,6sleep1224530-21diskmemload1
5331052131,6sleep0533325-21hddtemp_smartct0
4954312131,7sleep2224530-21diskmemload2
4675232131,3sleep0467537-21ssh0
4270282133,3sleep00-21swapper/00
3788822131,6sleep00-21swapper/00
3725022131,7sleep2224530-21diskmemload2
3652532131,5sleep20-21swapper/22
3565882131,3sleep0356598-21ssh0
3388812131,2sleep20-21swapper/22
3370422131,7sleep1224530-21diskmemload1
3135342131,8sleep1224530-21diskmemload1
2939592131,3sleep0293973-21ssh0
2303312131,8sleep1230341-21ssh1
2128452130,2sleep00-21swapper/00
1859042131,6sleep0185953-21ntpq0
941852121,6sleep094204-21apt-get0
941852121,6sleep094204-21apt-get0
5931512121,6sleep1224530-21diskmemload1
5583642121,7sleep2558375-21ssh2
5565362121,6sleep1224530-21diskmemload1
5473572121,5sleep1224530-21diskmemload1
5050082121,6sleep00-21swapper/00
4445742125,3sleep2444680-21latency_hist2
4152132120,3sleep0415224-21ssh0
4012052122,4sleep2224530-21diskmemload2
3891492120,3sleep0389156-21ssh0
3652372121,7sleep1365252-21ssh1
3549632121,5sleep10-21swapper/11
3425012121,6sleep1224530-21diskmemload1
3398902121,7sleep0340153-21df0
3292382120,3sleep2329252-21ssh2
3103822121,8sleep0310395-21sh0
2689622121,7sleep0268984-21latency_hist0
2129492121,7sleep2212998-21sed2
1642552121,6sleep2164306-21ntpq2
1445192121,5sleep2144729-21hddtemp_smartct2
5416912111,5sleep0541832-21diskstats0
5110002111,4sleep10-21swapper/11
4949992110,7sleep0495006-21ssh0
4862372112,2sleep0171rcu_preempt0
4718082111,4sleep20-21swapper/22
4187502111,5sleep1419016-21df_inode1
3837082111,4sleep1383729-21apt-get1
3749332111,4sleep1374958-21apt-get1
3492582111,5sleep2350104-21cat2
3008152111,2sleep20-21swapper/22
2942382111,4sleep2224530-21diskmemload2
2858072111,4sleep20-21swapper/22
2557332111,5sleep2224530-21diskmemload2
2514122111,6sleep1251422-21ssh1
2484392111,3sleep20-21swapper/22
2482042111,3sleep00-21swapper/00
2235472110,2sleep20-21swapper/22
2022242111,7sleep2202482-21grep2
1819242111,5sleep1182127-21diskstats1
1598522110,1sleep00-21swapper/00
1495752111,5sleep2149594-21apt-get2
1483052111,5sleep0148351-21ntpq0
1434132110,5sleep00-21swapper/00
1335922111,4sleep2133848-21df2
997352103,3sleep1100617-21cat1
5905422101,2sleep20-21swapper/22
5895062101,4sleep0589565-21ntpq0
5834672100,7sleep1583469-21ssh1
5750582101,4sleep1575069-21ssh1
5730772100,1sleep00-21swapper/00
5645302101,3sleep20-21swapper/22
5641992100,1sleep00-21swapper/00
5634102101,4sleep1224530-21diskmemload1
5232802101,4sleep0523395-21munin-run0
5219312101,2sleep10-21swapper/11
5045652101,4sleep1224530-21diskmemload1
5024502101,2sleep20-21swapper/22
4939992101,2sleep10-21swapper/11
4842472101,4sleep2484307-21ntpq2
4841852101,4sleep1484238-21ntpq1
4681142101,5sleep2802-21snmpd2
4599052101,2sleep20-21swapper/22
4495252101,5sleep0449713-21ls0
4449042101,3sleep10-21swapper/11
4334112101,3sleep10-21swapper/11
4334112101,3sleep10-21swapper/11
4329812101,2sleep20-21swapper/22
4329812101,2sleep20-21swapper/22
4274752101,4sleep0427500-21apt-get0
4274752101,4sleep0427500-21apt-get0
4265432101,3sleep20-21swapper/22
4168002101,2sleep20-21swapper/22
4147612100,1sleep10-21swapper/11
4077062101,4sleep1224530-21diskmemload1
3992862101,2sleep20-21swapper/22
3988082101,5sleep0398821-21ssh0
3748902101,3sleep20-21swapper/22
3470572101,4sleep2224530-21diskmemload2
3266242101,4sleep1326689-21ntp_states1
3205572101,5sleep0320569-21ssh0
3123442101,3sleep10-21swapper/11
3037832100,1sleep00-21swapper/00
2934152101,4sleep1224530-21diskmemload1
2842022101,4sleep0224530-21diskmemload0
2771932101,4sleep2224530-21diskmemload2
2747132101,3sleep00-21swapper/00
2746742101,4sleep1224530-21diskmemload1
2682722101,3sleep20-21swapper/22
2345412101,4sleep1224530-21diskmemload1
2194702101,3sleep10-21swapper/11
2128932101,4sleep1212947-21ntpq1
2072972102,4sleep0207351-21ntpq0
1554732101,5sleep2156646-21cut2
1374552101,4sleep0137596-21ntpq0
1175872101,5sleep0117848-21df_inode0
576886291,4sleep0577005-21sort0
528319291,3sleep1528382-21ssh1
511627290,1sleep20-21swapper/22
466828291,4sleep11-21systemd1
453753291,4sleep0453865-21tr0
450132291,1sleep20-21swapper/22
366083291,4sleep0366094-21ssh0
320227291,3sleep20-21swapper/22
283146291,3sleep1224530-21diskmemload1
238928291,3sleep0224530-21diskmemload0
225179291,4sleep0225208-21ssh0
203500291,4sleep1203603-21perl1
197842291,2sleep00-21swapper/00
187740291,5sleep0189215-21kthreadcore0
177077293,2sleep2178134-21cut2
162407290,6sleep0164119-21cat0
132341290,4sleep2132395-21ntpq2
127020291,4sleep2127074-21open_files2
117849291,4sleep1117913-21fschecks_time1
110982291,4sleep1111028-21ntpq1
331103281,3sleep0331372-21ssh0
302631280,2sleep1224530-21diskmemload1
251506280,2sleep00-21swapper/00
192027280,1sleep20-21swapper/22
186776280,1sleep20-21swapper/22
179942281,4sleep0180523-21grep0
175083280,1sleep10-21swapper/11
134120281,3sleep1134664-21cut1
112403280,4sleep10-21swapper/11
106903280,3sleep0106922-21apt-get0
218875270,1sleep00-21swapper/00
218856270,1sleep10-21swapper/11
214142270,2sleep20-21swapper/22
208199270,1sleep20-21swapper/22
199092271,3sleep10-21swapper/11
193099271,2sleep2192515-21/usr/sbin/munin2
181205270,4sleep10-21swapper/11
171023270,4sleep00-21swapper/00
165123270,1sleep10-21swapper/11
122888270,2sleep00-21swapper/00
101845271,3sleep00-21swapper/00
94207261,1sleep10-21swapper/11
94207261,1sleep10-21swapper/11
224220260,1sleep00-21swapper/00
196358260,2sleep0196531-21strings0
191906260,4sleep10-21swapper/11
170295260,1sleep10-21swapper/11
170286260,1sleep20-21swapper/22
155403260,2sleep10-21swapper/11
153771261,2sleep10-21swapper/11
142981260,2sleep20-21swapper/22
122910262,1sleep10-21swapper/11
122431260,1sleep20-21swapper/22
112252261,2sleep00-21swapper/00
112228260,1sleep20-21swapper/22
148353250,2sleep10-21swapper/11
149565240,1sleep00-21swapper/00
128242240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional