You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-31 - 08:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Fri Jul 31, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
64309622240,2sleep0643094-21ssh0
207500213357,70sleep30-21swapper/33
20270528354,11sleep20-21swapper/22
20743627959,14sleep00-21swapper/00
20737727760,11sleep10-21swapper/11
5604192341,22sleep1335419-21diskmemload1
5167732251,8sleep2335419-21diskmemload2
20769999241,18cyclictest3709332sleep33
20769999231,19cyclictest0-21swapper/33
5259422210,8sleep0525957-21ssh0
20769999211,16cyclictest0-21swapper/33
20769999211,15cyclictest0-21swapper/33
20769999210,19cyclictest0-21swapper/33
5413062201,8sleep2335419-21diskmemload2
20769999209,8cyclictest0-21swapper/33
20769999202,13cyclictest288555-21taskset3
207699992016,2cyclictest0-21swapper/33
20769999201,16cyclictest0-21swapper/33
20769999201,15cyclictest0-21swapper/33
5593662190,3sleep00-21swapper/00
207699991915,2cyclictest0-21swapper/33
20769999191,15cyclictest0-21swapper/33
20769999191,15cyclictest0-21swapper/33
20769999191,15cyclictest0-21swapper/33
20769999191,15cyclictest0-21swapper/33
20769999191,14cyclictest272259-21taskset3
20769999191,14cyclictest0-21swapper/33
20769999190,16cyclictest0-21swapper/33
20769999190,14cyclictest0-21swapper/33
6228552182,4sleep2622850-21apt-get2
20769999188,8cyclictest335421-21turbostat3
20769999181,5cyclictest0-21swapper/33
20769999181,4cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999181,14cyclictest0-21swapper/33
20769999180,15cyclictest66950irq/132-enp1s0-TxRx-33
20769999180,14cyclictest0-21swapper/33
20769999180,14cyclictest0-21swapper/33
20769999180,14cyclictest0-21swapper/33
20769999180,14cyclictest0-21swapper/33
20769999180,14cyclictest0-21swapper/33
6923722170,1sleep1692371-21ssh1
5230352171,8sleep1523262-21users1
4925732171,5sleep0492798-21df_inode0
4311492171,11sleep0431217-21munin-run0
3509412171,3sleep00-21swapper/00
20769999171,4cyclictest0-21swapper/33
20769999171,13cyclictest0-21swapper/33
20769999171,13cyclictest0-21swapper/33
20769999171,13cyclictest0-21swapper/33
20769999171,12cyclictest0-21swapper/33
20769999171,12cyclictest0-21swapper/33
20769999170,14cyclictest66950irq/132-enp1s0-TxRx-33
20769999170,14cyclictest0-21swapper/33
20769999170,14cyclictest0-21swapper/33
20769999170,14cyclictest0-21swapper/33
20769999170,13cyclictest0-21swapper/33
20769999170,13cyclictest0-21swapper/33
6994952161,4sleep1335419-21diskmemload1
5511892160,6sleep1551202-21ssh1
5234722161,3sleep20-21swapper/22
3860812161,2sleep00-21swapper/00
3091772161,4sleep2309394-21latency_hist2
207699991612,3cyclictest0-21swapper/33
20769999161,13cyclictest0-21swapper/33
20769999160,14cyclictest0-21swapper/33
20769999160,13cyclictest0-21swapper/33
20769999160,13cyclictest0-21swapper/33
20769999160,13cyclictest0-21swapper/33
20769999160,13cyclictest0-21swapper/33
6853482151,2sleep00-21swapper/00
5967352151,5sleep1596759-21ssh1
5767252151,4sleep1335419-21diskmemload1
5086812151,3sleep0335419-21diskmemload0
20769999153,1cyclictest0-21swapper/33
207699991510,3cyclictest0-21swapper/33
20769999150,12cyclictest0-21swapper/33
20769999150,12cyclictest0-21swapper/33
20769999150,12cyclictest0-21swapper/33
6921812141,3sleep2335419-21diskmemload2
6104352140,2sleep20-21swapper/22
5886132141,2sleep20-21swapper/22
5870182141,6sleep2587031-21sh2
5691292141,8sleep2335419-21diskmemload2
5680932141,3sleep0568103-21ssh0
4743692141,8sleep1335419-21diskmemload1
4315062141,8sleep0335419-21diskmemload0
3854382141,9sleep1385452-21ssh1
3783422141,2sleep20-21swapper/22
3688902141,4sleep1368901-21ssh1
3593662141,8sleep2335419-21diskmemload2
3451742141,9sleep2346451-21cut2
20769999141,8cyclictest246643-21taskset3
207699991411,1cyclictest0-21swapper/33
20769999140,11cyclictest212647-21chrt3
6995432131,8sleep0699557-21ssh0
6054862130,6sleep1605497-21ssh1
5970562131,6sleep0597274-21hddtemp_smartct0
5675952132,7sleep1321rcuc/11
5327802131,7sleep1335419-21diskmemload1
4733512135,4sleep2473364-21ssh2
4572192131,6sleep00-21swapper/00
4472482130,3sleep1447263-21ssh1
4026272131,6sleep2116341-21sshd2
3967622131,7sleep1396917-21/usr/sbin/munin1
3961222131,2sleep10-21swapper/11
3792152131,8sleep2335419-21diskmemload2
3785322131,7sleep1335419-21diskmemload1
3100072131,3sleep1310738-21kthreadcore1
2823692131,7sleep2282613-21users2
20769999130,2cyclictest0-21swapper/33
6854722121,5sleep2335419-21diskmemload2
6746542121,6sleep1674668-21sh1
6741312121,7sleep0674143-21ssh0
6649362121,5sleep10-21swapper/11
6643222121,7sleep0664543-21ssh0
6337232121,2sleep00-21swapper/00
6218552121,2sleep1621870-21ssh1
6034502120,1sleep20-21swapper/22
5962912120,2sleep1596369-21sendmail-msp1
5783252121,6sleep2335419-21diskmemload2
5440102120,8sleep0544023-21ssh0
4947562120,8sleep1496426-21ssh1
4889092121,2sleep00-21swapper/00
4818582120,2sleep20-21swapper/22
4483942120,3sleep2448408-21ssh2
2576172121,7sleep1258045-21grep1
2357122120,1sleep20-21swapper/22
2129372121,5sleep2213249-21hddtemp_smartct2
6588462111,6sleep0658858-21ssh0
6583542110,7sleep1658365-21ssh1
6500922111,6sleep1650105-21ssh1
6199542111,5sleep0335419-21diskmemload0
5946442111,4sleep0335419-21diskmemload0
5868912111,4sleep0335419-21diskmemload0
5422322114,4sleep116708-21ntpd1
5010792111,3sleep2335419-21diskmemload2
4890302110,1sleep2335419-21diskmemload2
4754152111,5sleep0475634-21hddtemp_smartct0
4634622111,3sleep10-21swapper/11
4556502110,7sleep2335419-21diskmemload2
4544012111,4sleep1335419-21diskmemload1
4405472111,5sleep0440773-21hddtemp_smartct0
4390892111,3sleep10-21swapper/11
4228722112,4sleep1422875-21apt-get1
4182602110,7sleep2418322-21ssh2
4103002110,1sleep00-21swapper/00
3879142111,6sleep2387663-21sendmail-msp2
3700162110,5sleep20-21swapper/22
3686162111,3sleep00-21swapper/00
3199632110,1sleep00-21swapper/00
2676232111,6sleep2267687-21/usr/sbin/munin2
2559002112,6sleep1255968-21/usr/sbin/munin1
2505972111,5sleep2250667-21ntp_states2
6927192100,1sleep00-21swapper/00
6654082101,3sleep2335419-21diskmemload2
6578062101,3sleep20-21swapper/22
6489342101,2sleep20-21swapper/22
6387882101,4sleep1631322-40find1
6285602101,4sleep1335419-21diskmemload1
6215112101,2sleep20-21swapper/22
6116842101,3sleep00-21swapper/00
5809442101,2sleep10-21swapper/11
5535832105,2sleep20-21swapper/22
5497042101,3sleep2335419-21diskmemload2
5496782101,3sleep00-21swapper/00
5346202100,1sleep00-21swapper/00
5341842100,6sleep2534197-21ssh2
5098432101,4sleep1510111-21df1
5098292101,3sleep0509826-21apt-get0
4905692101,4sleep1335419-21diskmemload1
4824062101,6sleep1335419-21diskmemload1
4716542101,3sleep00-21swapper/00
4658872101,2sleep20-21swapper/22
4222572101,4sleep0335419-21diskmemload0
4214092101,4sleep1335419-21diskmemload1
4133482101,4sleep2335419-21diskmemload2
4020402100,6sleep0402051-21ssh0
3935382101,3sleep00-21swapper/00
3754312100,2sleep00-21swapper/00
3582162101,2sleep10-21swapper/11
3581052101,2sleep00-21swapper/00
3493642101,2sleep10-21swapper/11
3412992101,2sleep00-21swapper/00
3304672101,4sleep2330724-21df2
3304622101,4sleep1330464-21apt-get1
3212632101,5sleep2322948-21kthreadcore2
3069952101,4sleep0308097-21memory0
2992732101,4sleep2299482-21hddtemp_smartct2
2987092101,5sleep1298730-21latency_hist1
2833212101,5sleep0283298-21/usr/sbin/munin0
2570682100,6sleep0257069-21/usr/sbin/munin0
2413592101,4sleep2241129-21sendmail-msp2
2349952101,5sleep1235125-21sensors_temp1
2261112101,4sleep2226182-21irqstats2
2151372101,4sleep1215156-21apt-get1
2078892101,3sleep0207908-21apt-get0
701459291,3sleep2335419-21diskmemload2
677212291,3sleep1335419-21diskmemload1
668847291,4sleep2668980-21ssh2
638048291,4sleep2431ktimers/22
622805291,3sleep0622819-21ssh0
576510290,2sleep00-21swapper/00
505132291,4sleep1505280-21/usr/sbin/munin1
499857291,4sleep2499871-21ssh2
465916291,2sleep00-21swapper/00
437357291,2sleep20-21swapper/22
422889291,4sleep2423133-21cstates2
405508291,4sleep1405721-21diskstats1
341325291,2sleep20-21swapper/22
335713291,3sleep1335419-21diskmemload1
314728291,2sleep00-21swapper/00
288554290,3sleep20-21swapper/22
278768291,5sleep0280366-21kthreadcore0
273372290,5sleep1273407-21kthreadcore1
266944290,1sleep00-21swapper/00
251832291,3sleep0251851-21apt-get0
241651291,3sleep1241860-21hddtemp_smartct1
241527291,4sleep0241650-21diskstats0
236112291,3sleep1235828-21apt-get1
220663291,4sleep2220765-21grep2
329940280,1sleep20-21swapper/22
325230281,3sleep0106713-21snmpd0
303116280,4sleep1298982-21/usr/sbin/munin1
303014281,3sleep00-21swapper/00
267601280,2sleep10-21swapper/11
265888281,3sleep2266313-21strings2
258046281,3sleep2259657-21kthreadcore2
236125280,2sleep00-21swapper/00
236119280,2sleep20-21swapper/22
227360282,2sleep014-21ksoftirqd/00
220364280,1sleep10-21swapper/11
214765280,1sleep00-21swapper/00
319565272,3sleep2431ktimers/22
310740271,3sleep20-21swapper/22
304216271,2sleep10-21swapper/11
293753270,1sleep00-21swapper/00
288532271,2sleep1288551-21apt-get1
288522270,4sleep00-21swapper/00
287444271,3sleep1287592-21switchtime1
277396270,1sleep00-21swapper/00
272395270,4sleep00-21swapper/00
262108271,2sleep10-21swapper/11
251267270,1sleep10-21swapper/11
231442271,3sleep00-21swapper/00
230541270,1sleep10-21swapper/11
208452270,3sleep1321rcuc/11
207877270,1sleep20-21swapper/22
335108260,4sleep00-21swapper/00
320534261,2sleep10-21swapper/11
319449260,4sleep10-21swapper/11
298464260,4sleep20-21swapper/22
283600260,2sleep20-21swapper/22
278023260,1sleep10-21swapper/11
274499261,2sleep20-21swapper/22
220940261,2sleep00-21swapper/00
325231250,1sleep10-21swapper/11
309450250,1sleep00-21swapper/00
251825250,1sleep20-21swapper/22
250783250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional