You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 08:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Fri May 15, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
165971721290,4sleep11659721-21kernelversion1
1409584212059,57sleep30-21swapper/33
140950429170,10sleep10-21swapper/11
140976028768,12sleep20-21swapper/22
140916927652,14sleep00-21swapper/00
1409984992118,2cyclictest0-21swapper/33
1409984992118,2cyclictest0-21swapper/33
1409984992118,1cyclictest0-21swapper/33
1409984992117,2cyclictest0-21swapper/33
1409984992112,6cyclictest0-21swapper/33
140998499210,17cyclictest1476280-21taskset3
140998499201,2cyclictest0-21swapper/33
140998499201,16cyclictest0-21swapper/33
140998499201,16cyclictest0-21swapper/33
140998499201,15cyclictest66950irq/132-enp1s0-TxRx-33
140998499201,15cyclictest1851154-21taskset3
140998499201,15cyclictest1718150-21chrt3
140998499201,14cyclictest0-21swapper/33
140998499201,14cyclictest0-21swapper/33
1409984991916,2cyclictest0-21swapper/33
140998499191,5cyclictest0-21swapper/33
140998499191,15cyclictest1732626-21taskset3
140998499191,15cyclictest0-21swapper/33
140998499191,14cyclictest0-21swapper/33
140998499191,14cyclictest0-21swapper/33
140998499191,14cyclictest0-21swapper/33
1409984991910,6cyclictest0-21swapper/33
140998499190,15cyclictest66950irq/132-enp1s0-TxRx-33
18759892181,3sleep20-21swapper/22
18439222181,3sleep01843933-21ssh0
15457672180,13sleep10-21swapper/11
140998499182,13cyclictest1616218-21turbostat3
1409984991815,1cyclictest0-21swapper/33
140998499181,14cyclictest0-21swapper/33
140998499181,14cyclictest0-21swapper/33
140998499181,14cyclictest0-21swapper/33
140998499181,14cyclictest0-21swapper/33
140998499181,13cyclictest0-21swapper/33
140998499181,13cyclictest0-21swapper/33
140998499181,13cyclictest0-21swapper/33
140998499181,13cyclictest0-21swapper/33
140998499180,15cyclictest0-21swapper/33
140998499180,14cyclictest0-21swapper/33
140998499180,14cyclictest0-21swapper/33
18741162171,4sleep01540228-21diskmemload0
15463212171,3sleep00-21swapper/00
140998499171,4cyclictest0-21swapper/33
140998499171,13cyclictest1826956-21sleep3
140998499171,13cyclictest1616532-21chrt3
140998499171,13cyclictest0-21swapper/33
140998499171,13cyclictest0-21swapper/33
140998499170,14cyclictest0-21swapper/33
140998499170,14cyclictest0-21swapper/33
140998499170,14cyclictest0-21swapper/33
140998499170,14cyclictest0-21swapper/33
140998499170,14cyclictest0-21swapper/33
140998499170,14cyclictest0-21swapper/33
140998499170,13cyclictest0-21swapper/33
140998499170,13cyclictest0-21swapper/33
140998499170,13cyclictest0-21swapper/33
18403402161,10sleep11840400-21ntpq1
17672982161,3sleep10-21swapper/11
17170082161,3sleep20-21swapper/22
16828132161,10sleep11540228-21diskmemload1
15899052161,4sleep20-21swapper/22
15811692161,2sleep00-21swapper/00
14818892161,10sleep01482145-21df0
140998499161,3cyclictest0-21swapper/33
140998499161,12cyclictest0-21swapper/33
140998499160,12cyclictest0-21swapper/33
18939532151,4sleep01893964-21ssh0
18930562150,3sleep10-21swapper/11
18437602152,9sleep21843773-21sh2
18022262151,10sleep01802458-21ssh0
17923292151,9sleep01540228-21diskmemload0
17769962151,4sleep01540228-21diskmemload0
17769962151,4sleep01540228-21diskmemload0
17664562151,2sleep20-21swapper/22
17155272151,4sleep01540228-21diskmemload0
16913752151,2sleep20-21swapper/22
16548812151,5sleep11654950-21collect21
16315962151,2sleep00-21swapper/00
15822852151,4sleep21582299-21ssh2
14712962151,9sleep11471503-21diskstats1
1409984991511,2cyclictest0-21swapper/33
140998499151,10cyclictest1513885-21chrt3
140998499150,13cyclictest0-21swapper/33
140998499150,11cyclictest0-21swapper/33
18841822141,10sleep21540228-21diskmemload2
18618132141,8sleep21862029-21hddtemp_smartct2
18183792140,2sleep00-21swapper/00
17574452141,4sleep11757459-21ssh1
17246792141,7sleep01540228-21diskmemload0
16710982141,9sleep11671579-21grep1
16482842141,8sleep21540228-21diskmemload2
16209382141,2sleep20-21swapper/22
140998499145,6cyclictest0-21swapper/33
140998499143,8cyclictest0-21swapper/33
140998499143,8cyclictest0-21swapper/33
140998499141,11cyclictest0-21swapper/33
1409984991410,2cyclictest0-21swapper/33
1409984991410,2cyclictest0-21swapper/33
140998499140,12cyclictest0-21swapper/33
140998499140,12cyclictest0-21swapper/33
140998499140,12cyclictest0-21swapper/33
140998499140,12cyclictest0-21swapper/33
16982182131,8sleep21698228-21ssh2
16912622131,8sleep01691272-21ssh0
16380662131,7sleep21638128-21ntp_offset2
16075672131,8sleep11607631-21systemd-run1
15917322130,3sleep21591839-21ps2
15178992131,6sleep21517937-21ntpq2
14752802131,6sleep01475329-21ntpq0
14601702130,1sleep20-21swapper/22
14429592131,2sleep00-21swapper/00
14365942131,9sleep21437467-21missed_timers2
14181022131,4sleep01419569-21grep0
140998499130,1cyclictest0-21swapper/33
140998499130,10cyclictest0-21swapper/33
18243082121,7sleep01824313-21ssh0
18156832121,7sleep21815697-21ssh2
17852662122,4sleep21785294-21apt-get2
17579882121,5sleep20-21swapper/22
17503992121,3sleep21750410-21ssh2
17325242121,7sleep11732536-21sh1
17138622121,6sleep11713921-21ntpq1
17054012121,7sleep01705444-21ntpq0
16750942121,3sleep21675105-21ssh2
16676272121,6sleep01667840-21diskstats0
16633202122,2sleep2171rcu_preempt2
16484102121,3sleep11648422-21ssh1
16395902121,6sleep11639602-21ssh1
16138772121,7sleep11540228-21diskmemload1
15664602121,7sleep21567660-21cut2
15351682121,7sleep21535284-21cpuspeed_turbos2
14645932121,5sleep01464631-21ntpq0
14335742121,5sleep11433830-21df1
14267962121,7sleep01426920-21grep0
18849602111,4sleep01540228-21diskmemload0
18571802111,5sleep00-21swapper/00
18547332111,5sleep21856431-21tr2
18317132112,4sleep11540228-21diskmemload1
17752452111,6sleep21775259-21ssh2
17727292111,5sleep11772790-21ntpq1
17684572111,4sleep01540228-21diskmemload0
17600832111,4sleep01760107-21ssh0
17590792111,4sleep01540228-21diskmemload0
17342102111,3sleep00-21swapper/00
17336352111,3sleep20-21swapper/22
17235582111,7sleep21540228-21diskmemload2
17184352111,5sleep11718655-21hddtemp_smartct1
17082292111,2sleep11708239-21ssh1
16981942111,1sleep0171rcu_preempt0
16810662111,6sleep21540228-21diskmemload2
16632132111,5sleep01663257-21ntpq0
16422462111,2sleep00-21swapper/00
16412182111,6sleep01540228-21diskmemload0
16252062111,5sleep11624662-21apt-get1
16141442111,5sleep01540228-21diskmemload0
16046672111,3sleep00-21swapper/00
15878192111,3sleep10-21swapper/11
15699142111,4sleep11570012-21ntpq1
15559262110,2sleep01540228-21diskmemload0
15406022112,4sleep21540631-21apt-get2
15338982111,5sleep01533947-21ntpq0
15336502112,5sleep21533835-21strings2
15299332111,5sleep11530111-21diskstats1
14646332111,5sleep11464687-21ntpq1
14456802111,6sleep01447438-21cut0
14456802111,6sleep01447438-21cut0
14107362111,6sleep11410155-21/usr/sbin/munin1
18935502101,4sleep21540228-21diskmemload2
18826842101,5sleep11882808-21grep1
18675632100,1sleep00-21swapper/00
18668902101,4sleep11540228-21diskmemload1
18597622101,4sleep11540228-21diskmemload1
18491262100,1sleep20-21swapper/22
18343642101,4sleep01540228-21diskmemload0
18234942101,3sleep10-21swapper/11
18194762101,5sleep21819634-21grep2
18169642100,4sleep10-21swapper/11
18088342101,3sleep21540228-21diskmemload2
18017922101,4sleep21540228-21diskmemload2
17997772101,4sleep01540228-21diskmemload0
17911082101,4sleep11540228-21diskmemload1
17812512101,4sleep21540228-21diskmemload2
17812512101,4sleep21540228-21diskmemload2
17502832101,2sleep00-21swapper/00
17494662101,2sleep10-21swapper/11
17426182101,4sleep21540228-21diskmemload2
17359682101,5sleep11737608-21kthreadcore1
17067092101,3sleep20-21swapper/22
16915562101,3sleep10-21swapper/11
16825102102,3sleep01540228-21diskmemload0
16573892101,4sleep01540228-21diskmemload0
16505672101,5sleep21540228-21diskmemload2
16314282101,4sleep21540228-21diskmemload2
16240032101,3sleep10-21swapper/11
16155312101,4sleep21540228-21diskmemload2
15999962101,5sleep21600073-21irqstats2
15968512101,3sleep10-21swapper/11
15658232100,1sleep00-21swapper/00
15640012101,4sleep11540228-21diskmemload1
15638672101,4sleep21540228-21diskmemload2
15494962101,5sleep21549679-21kthreadcore2
15294582100,1sleep00-21swapper/00
15285682101,5sleep11528622-21ntpq1
15193432100,4sleep21519466-21diskstats2
14809892101,4sleep21481233-21users2
14661782101,4sleep11466388-21hddtemp_smartct1
14602982100,6sleep11460312-21latency_hist1
14322862100,2sleep10-21swapper/11
1851748291,4sleep11851758-21ssh1
1848933290,3sleep00-21swapper/00
1834112291,2sleep20-21swapper/22
1809588291,3sleep10-21swapper/11
1799977291,2sleep10-21swapper/11
1781941291,2sleep10-21swapper/11
1781941291,2sleep10-21swapper/11
1739405291,3sleep01739432-21ssh0
1698816291,2sleep10-21swapper/11
1616545291,2sleep00-21swapper/00
1597163291,2sleep00-21swapper/00
1587888291,2sleep00-21swapper/00
1580614291,2sleep10-21swapper/11
1553485293,3sleep11553696-21unixbench-2d1
1519177291,3sleep01519196-21apt-get0
1513851291,3sleep072-21kcompactd00
1492547291,3sleep01492566-21apt-get0
1485983291,4sleep11486043-21ld1
1476558291,4sleep01476605-21grep0
1450171291,5sleep01450296-21gltestperf0
1450171291,5sleep01450296-21gltestperf0
1434107291,4sleep01434850-21tr0
1432337291,4sleep21432396-21ld2
1432191291,3sleep01432279-21ntpq0
1423578291,4sleep21425258-21tr2
1422687290,1sleep10-21swapper/11
1419570291,4sleep11421382-21tr1
1414306291,4sleep21414363-21ntp_states2
1874861281,2sleep10-21swapper/11
1557999281,3sleep01559283-21kthreadcore0
1529313280,1sleep20-21swapper/22
1507226281,3sleep11507255-21ntpq1
1501973280,2sleep20-21swapper/22
1491061281,3sleep01491217-21needreboot0
1465884280,3sleep21465903-21apt-get2
1535146270,2sleep10-21swapper/11
1518736270,1sleep10-21swapper/11
1513442270,1sleep10-21swapper/11
1510394271,3sleep01512216-21kthreadcore0
1508536270,3sleep21508555-21apt-get2
1497417270,1sleep10-21swapper/11
1496648271,2sleep20-21swapper/22
1487242271,3sleep20-21swapper/22
1487220270,2sleep10-21swapper/11
1474765270,3sleep21475228-21strings2
1459222270,4sleep00-21swapper/00
1450453271,2sleep20-21swapper/22
1450453271,2sleep20-21swapper/22
1449319270,1sleep20-21swapper/22
1449319270,1sleep20-21swapper/22
1449190270,1sleep10-21swapper/11
1449190270,1sleep10-21swapper/11
1443253271,3sleep11443393-21cat1
1410154271,2sleep01410173-21apt-get0
1540132260,1sleep00-21swapper/00
1524037260,4sleep10-21swapper/11
1503178260,1sleep20-21swapper/22
1497879261,2sleep10-21swapper/11
1497857260,2sleep00-21swapper/00
1486663260,1sleep20-21swapper/22
1478270261,2sleep10-21swapper/11
1470636260,1sleep00-21swapper/00
1449873260,3sleep10-21swapper/11
1449873260,3sleep10-21swapper/11
1443911260,1sleep20-21swapper/22
1415301261,2sleep20-21swapper/22
1507636250,1sleep00-21swapper/00
1460843251,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional