You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 04:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Tue Jul 28, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
157507121460,3sleep21575047-21ssh2
157507121460,3sleep21575047-21ssh2
1377190212657,64sleep30-21swapper/33
137471021120,2sleep1331ktimers/11
137748428467,11sleep20-21swapper/22
137262428059,14sleep00-21swapper/00
15188772250,2sleep1331ktimers/11
15979002240,2sleep10-21swapper/11
137767899232,14cyclictest1749485-21chrt3
137767899221,18cyclictest0-21swapper/33
137767899221,17cyclictest1426561-21taskset3
137767899221,17cyclictest1421506-21taskset3
137767899212,16cyclictest0-21swapper/33
1377678992117,2cyclictest0-21swapper/33
14742612201,7sleep21474038-21sendmail-msp2
137767899202,15cyclictest0-21swapper/33
1377678992017,1cyclictest0-21swapper/33
1377678992012,5cyclictest0-21swapper/33
137767899201,1cyclictest0-21swapper/33
137767899201,15cyclictest0-21swapper/33
137767899201,15cyclictest0-21swapper/33
137767899201,15cyclictest0-21swapper/33
1377678992010,7cyclictest1834744-21taskset3
18641972191,12sleep20-21swapper/22
17937312191,12sleep21795244-21grep2
137767899194,13cyclictest0-21swapper/33
1377678991916,1cyclictest0-21swapper/33
1377678991911,5cyclictest66950irq/132-enp1s0-TxRx-33
137767899191,15cyclictest0-21swapper/33
137767899191,15cyclictest0-21swapper/33
137767899191,14cyclictest0-21swapper/33
137767899191,14cyclictest0-21swapper/33
137767899191,14cyclictest0-21swapper/33
137767899191,14cyclictest0-21swapper/33
137767899191,14cyclictest0-21swapper/33
137767899191,14cyclictest0-21swapper/33
137767899191,14cyclictest0-21swapper/33
137767899191,14cyclictest0-21swapper/33
137767899191,13cyclictest0-21swapper/33
137767899190,17cyclictest0-21swapper/33
137767899190,16cyclictest0-21swapper/33
18144272181,11sleep01816102-21cut0
16096132181,3sleep20-21swapper/22
16011472181,12sleep21601246-21cat2
15903582181,2sleep00-21swapper/00
137767899185,11cyclictest0-21swapper/33
137767899181,5cyclictest0-21swapper/33
1377678991812,3cyclictest0-21swapper/33
1377678991812,2cyclictest0-21swapper/33
1377678991812,2cyclictest0-21swapper/33
137767899181,14cyclictest0-21swapper/33
137767899181,14cyclictest0-21swapper/33
137767899181,14cyclictest0-21swapper/33
137767899181,14cyclictest0-21swapper/33
137767899181,13cyclictest0-21swapper/33
137767899181,13cyclictest0-21swapper/33
137767899181,13cyclictest0-21swapper/33
137767899181,13cyclictest0-21swapper/33
137767899180,17cyclictest66950irq/132-enp1s0-TxRx-33
137767899180,16cyclictest0-21swapper/33
137767899180,14cyclictest0-21swapper/33
18378502171,2sleep10-21swapper/11
15835252171,2sleep20-21swapper/22
137767899178,6cyclictest16623792chrt3
137767899171,4cyclictest0-21swapper/33
137767899171,13cyclictest0-21swapper/33
137767899171,13cyclictest0-21swapper/33
137767899171,12cyclictest0-21swapper/33
137767899171,12cyclictest0-21swapper/33
137767899170,14cyclictest0-21swapper/33
137767899170,13cyclictest0-21swapper/33
137767899170,12cyclictest0-21swapper/33
18040162161,5sleep11801383-40find1
17903852161,4sleep21505420-21diskmemload2
17903852161,4sleep21505420-21diskmemload2
17798982160,2sleep20-21swapper/22
17798982160,2sleep20-21swapper/22
17567132160,1sleep10-21swapper/11
17284152160,1sleep10-21swapper/11
16345612161,4sleep21505420-21diskmemload2
16158382160,7sleep00-21swapper/00
16071822161,4sleep11607190-21ssh1
137767899161,3cyclictest0-21swapper/33
137767899161,13cyclictest0-21swapper/33
137767899161,12cyclictest0-21swapper/33
137767899161,12cyclictest0-21swapper/33
137767899161,11cyclictest0-21swapper/33
137767899160,13cyclictest0-21swapper/33
137767899160,13cyclictest0-21swapper/33
137767899160,12cyclictest0-21swapper/33
17381722151,4sleep11505420-21diskmemload1
17125582151,3sleep11505420-21diskmemload1
15470772150,3sleep10-21swapper/11
1377678991510,3cyclictest0-21swapper/33
137767899150,3cyclictest0-21swapper/33
137767899150,12cyclictest0-21swapper/33
18036142141,8sleep01801240-21sendmail-msp0
17724682141,2sleep00-21swapper/00
17286412141,8sleep21505420-21diskmemload2
17228592141,7sleep11722976-21sed1
16588842141,4sleep01658898-21ssh0
15300832141,4sleep01530097-21ssh0
137767899141,8cyclictest1448065-21taskset3
137767899140,12cyclictest0-21swapper/33
18213312131,5sleep11821342-21ssh1
18129132131,7sleep11813133-21hddtemp_smartct1
17997502130,2sleep00-21swapper/00
16867612131,3sleep01686775-21ssh0
16863072131,3sleep11686321-21ssh1
16717702131,2sleep10-21swapper/11
16517542131,8sleep11651765-21ssh1
16492312131,7sleep21505420-21diskmemload2
16340412131,8sleep11634055-21ssh1
16115902131,8sleep11613180-21kthreadcore1
15920242131,6sleep21505420-21diskmemload2
15754602131,7sleep11505420-21diskmemload1
15750362131,4sleep11575069-21sh1
15750362131,4sleep11575069-21sh1
14640812131,8sleep21464293-21smartctl2
137767899131,2cyclictest0-21swapper/33
18718462120,2sleep00-21swapper/00
18290612121,6sleep01505420-21diskmemload0
18266522121,5sleep21505420-21diskmemload2
17891952121,7sleep11789213-21ssh1
17891952121,7sleep11789213-21ssh1
17837922121,7sleep11783962-21latency_hist1
17837922121,7sleep11783962-21latency_hist1
17588212121,7sleep21758148-21/usr/sbin/munin2
17494522121,3sleep20-21swapper/22
17407572121,1sleep10-21swapper/11
17028112121,6sleep11505420-21diskmemload1
16018572121,7sleep01601897-21uniq0
15978882121,5sleep072-21kcompactd00
15470892121,7sleep21547104-21ssh2
15380052121,5sleep20-21swapper/22
15308842121,2sleep20-21swapper/22
15287652121,6sleep11505420-21diskmemload1
15110942121,7sleep11511106-21ssh1
14902542121,7sleep11490325-21fschecks_time1
14381802121,7sleep10-21swapper/11
18528092110,2sleep20-21swapper/22
18523232111,3sleep00-21swapper/00
18479942111,7sleep11849160-21kthreadcore1
18459812111,5sleep01505420-21diskmemload0
18433512111,3sleep20-21swapper/22
18361072111,4sleep21505420-21diskmemload2
18355182111,5sleep01505420-21diskmemload0
17550072111,2sleep01755009-21ssh0
17372352111,5sleep00-21swapper/00
17314112111,7sleep01731425-21ssh0
17152392111,6sleep01716728-21cut0
17135222111,5sleep01505420-21diskmemload0
16975712111,4sleep01697783-21hddtemp_smartct0
16851842110,7sleep21505420-21diskmemload2
16353572111,4sleep01505420-21diskmemload0
16176572110,1sleep20-21swapper/22
15907892110,1sleep10-21swapper/11
15667462111,4sleep01566771-21apt-get0
15667462111,4sleep01566771-21apt-get0
15572962111,5sleep11505420-21diskmemload1
15545542110,2sleep00-21swapper/00
15361162111,3sleep10-21swapper/11
15319302112,4sleep01531952-21apt-get0
15198302111,5sleep01505420-21diskmemload0
14895342117,2sleep21382675-21kworker/2:1+events_freezable2
14271112111,5sleep01427130-21apt-get0
13901082112,5sleep2555-21dbus-daemon2
13783472111,7sleep10-21swapper/11
18730522101,5sleep11505420-21diskmemload1
18729052101,3sleep20-21swapper/22
18559242101,5sleep11855938-21ssh1
18110992100,6sleep21505420-21diskmemload2
18003222100,1sleep10-21swapper/11
17921532101,5sleep01792167-21ssh0
17921532101,5sleep01792167-21ssh0
17776842101,4sleep01779396-21grep0
17776842101,4sleep01779396-21grep0
17733232100,1sleep20-21swapper/22
17656572101,4sleep01505420-21diskmemload0
17425952101,5sleep21744236-21kthreadcore2
17226082101,3sleep20-21swapper/22
17095272101,5sleep21709960-21needreboot2
16971462101,5sleep21696887-21sendmail-msp2
16696622101,4sleep11505420-21diskmemload1
16691172101,3sleep21505420-21diskmemload2
16627722101,4sleep01662909-21perl0
16607912101,2sleep10-21swapper/11
16438062101,2sleep00-21swapper/00
16429472101,3sleep21505420-21diskmemload2
16411872101,4sleep11505420-21diskmemload1
16256772101,3sleep20-21swapper/22
16246862101,6sleep11624699-21rm1
16240582100,1sleep00-21swapper/00
15632632101,3sleep10-21swapper/11
15579792101,4sleep01557990-21ssh0
15542172101,4sleep21505420-21diskmemload2
15144822101,4sleep21505420-21diskmemload2
15124672101,4sleep01505420-21diskmemload0
15112572101,2sleep20-21swapper/22
14858972101,5sleep01487532-21kthreadcore0
14432892101,4sleep01443380-21irqstats0
14330692101,5sleep01434643-21cut0
14323432101,4sleep20-21swapper/22
14167302101,4sleep21416932-21diskstats2
14163742100,4sleep11416430-21munin-run1
13909532101,5sleep11391118-21tr1
13784372101,2sleep20-21swapper/22
1863606291,4sleep01863621-21ssh0
1844873290,3sleep10-21swapper/11
1813134292,3sleep20-21swapper/22
1772941291,4sleep11772956-21ssh1
1764350290,1sleep10-21swapper/11
1748162290,1sleep00-21swapper/00
1737318291,3sleep21505420-21diskmemload2
1693980291,3sleep11505420-21diskmemload1
1693505291,3sleep01693516-21ssh0
1688475291,2sleep20-21swapper/22
1676352291,3sleep21676363-21ssh2
1671139291,4sleep01671161-21apt-get0
1658552291,3sleep21505420-21diskmemload2
1645029291,3sleep01645056-21apt-get0
1575425290,3sleep01575456-21apt-get0
1558046291,4sleep290-21kswapd02
1546237291,4sleep01546246-21sh0
1474259291,3sleep039542-21systemd-journal0
1472061290,5sleep11472906-21missed_timers1
1457213291,4sleep11457325-21ntp_192.168.1141
1427133291,4sleep11427310-21cstates1
1411406291,3sleep21411183-21sendmail-msp2
1408405291,4sleep11409996-21latency1
1400648291,3sleep01400707-21munin-run0
1394285290,2sleep0171rcu_preempt0
1497355281,3sleep01499045-21sed0
1495226281,2sleep20-21swapper/22
1487536280,4sleep10-21swapper/11
1474286281,4sleep1205-21jbd2/sda2-81
1469019281,2sleep21469038-21apt-get2
1468747280,2sleep10-21swapper/11
1458539280,2sleep10-21swapper/11
1448043280,2sleep20-21swapper/22
1444504280,4sleep21446236-21grep2
1437576281,2sleep01437595-21apt-get0
1432337280,1sleep10-21swapper/11
1429256281,3sleep21430915-21sed2
1416644281,2sleep00-21swapper/00
1391119280,4sleep21392610-21cut2
1505367270,1sleep20-21swapper/22
1500444270,2sleep10-21swapper/11
1499918270,1sleep10-21swapper/11
1483784271,2sleep01483981-21/usr/sbin/munin0
1480073271,3sleep20-21swapper/22
1479495270,1sleep10-21swapper/11
1468396270,1sleep00-21swapper/00
1463477271,2sleep00-21swapper/00
1451217270,3sleep11451923-21/usr/sbin/munin1
1446843270,3sleep10-21swapper/11
1442192270,3sleep21442193-21sshd-session2
1426737270,4sleep10-21swapper/11
1422482271,3sleep01423982-21kthreadcore0
1420686271,3sleep10-21swapper/11
1416039270,1sleep00-21swapper/00
1410912270,1sleep20-21swapper/22
1405734270,1sleep20-21swapper/22
1401526270,4sleep11403060-21kthreadcore1
1400922270,2sleep00-21swapper/00
1395706270,2sleep10-21swapper/11
1389950270,1sleep00-21swapper/00
1385148271,2sleep11385167-21apt-get1
1505270260,4sleep00-21swapper/00
1491128261,2sleep00-21swapper/00
1470386261,2sleep00-21swapper/00
1459143261,2sleep20-21swapper/22
1453874261,2sleep20-21swapper/22
1453298261,1sleep00-21swapper/00
1449520261,2sleep00-21swapper/00
1425689260,2sleep21425862-21strings2
1410821260,1sleep00-21swapper/00
1381684260,2sleep01381857-21strings0
1489973250,1sleep20-21swapper/22
1395471250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional