You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-30 - 01:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Mon Mar 30, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
173388722820,2sleep10-21swapper/11
1304946212058,57sleep30-21swapper/33
130486428454,7sleep00-21swapper/00
130484028266,10sleep10-21swapper/11
130503527860,12sleep20-21swapper/22
1305229992320,1cyclictest0-21swapper/33
130522999221,3cyclictest0-21swapper/33
130522999221,17cyclictest0-21swapper/33
130522999210,18cyclictest0-21swapper/33
130522999203,14cyclictest66950irq/132-enp1s0-TxRx-33
1305229992018,1cyclictest0-21swapper/33
1305229992012,4cyclictest0-21swapper/33
130522999201,15cyclictest1761209-21taskset3
130522999201,14cyclictest0-21swapper/33
130522999201,14cyclictest0-21swapper/33
130522999201,14cyclictest0-21swapper/33
14444642190,2sleep20-21swapper/22
130522999191,5cyclictest66950irq/132-enp1s0-TxRx-33
130522999191,5cyclictest1716752-21chrt3
1305229991915,2cyclictest0-21swapper/33
1305229991911,6cyclictest0-21swapper/33
130522999191,15cyclictest0-21swapper/33
130522999191,15cyclictest0-21swapper/33
130522999191,14cyclictest66950irq/132-enp1s0-TxRx-33
130522999191,14cyclictest1366297-21turbostat3
130522999191,14cyclictest0-21swapper/33
130522999191,14cyclictest0-21swapper/33
130522999191,14cyclictest0-21swapper/33
1305229991910,6cyclictest1556264-21taskset3
16881442181,3sleep00-21swapper/00
16366612181,3sleep10-21swapper/11
15939412181,5sleep10-21swapper/11
13865742181,5sleep01386623-21ntpq0
130522999187,8cyclictest1430489-21turbostat3
130522999181,15cyclictest1602591-21taskset3
130522999181,14cyclictest1487097-21taskset3
130522999181,14cyclictest0-21swapper/33
130522999181,14cyclictest0-21swapper/33
130522999181,14cyclictest0-21swapper/33
130522999181,14cyclictest0-21swapper/33
130522999181,14cyclictest0-21swapper/33
130522999181,14cyclictest0-21swapper/33
130522999181,14cyclictest0-21swapper/33
130522999181,13cyclictest1494392-21taskset3
130522999181,13cyclictest0-21swapper/33
130522999181,13cyclictest0-21swapper/33
130522999181,13cyclictest0-21swapper/33
130522999180,15cyclictest0-21swapper/33
130522999180,14cyclictest0-21swapper/33
130522999180,14cyclictest0-21swapper/33
17779262171,2sleep20-21swapper/22
17481792171,10sleep01748322-21ntpq0
16975702170,1sleep20-21swapper/22
16692732171,10sleep11435812-21diskmemload1
15309692177,5sleep2431ktimers/22
14960762171,3sleep01485629-21kworker/u16:1+rpciod0
130522999171,13cyclictest0-21swapper/33
130522999171,13cyclictest0-21swapper/33
130522999171,13cyclictest0-21swapper/33
130522999171,13cyclictest0-21swapper/33
130522999170,14cyclictest0-21swapper/33
130522999170,14cyclictest0-21swapper/33
130522999170,14cyclictest0-21swapper/33
17785182161,5sleep11435812-21diskmemload1
17052392161,4sleep11705252-21rm1
16381232160,12sleep00-21swapper/00
16021792161,3sleep20-21swapper/22
15559712164,8sleep01555986-21ssh0
15311072161,2sleep10-21swapper/11
14419062161,2sleep10-21swapper/11
13493862161,5sleep11349439-21ntp_states1
130522999163,8cyclictest0-21swapper/33
130522999161,12cyclictest1376926-21turbostat3
130522999161,12cyclictest0-21swapper/33
130522999160,14cyclictest0-21swapper/33
130522999160,13cyclictest0-21swapper/33
130522999160,13cyclictest0-21swapper/33
130522999160,13cyclictest0-21swapper/33
130522999160,13cyclictest0-21swapper/33
130522999160,12cyclictest0-21swapper/33
17570392151,8sleep01757096-21ntpq0
17174162151,10sleep01717433-21cron0
16263952150,1sleep10-21swapper/11
16019062151,3sleep11435812-21diskmemload1
15815352151,3sleep21435812-21diskmemload2
13546332151,8sleep11354671-21ntpq1
130522999159,3cyclictest1355666-21turbostat3
130522999150,3cyclictest0-21swapper/33
130522999150,12cyclictest0-21swapper/33
130522999150,12cyclictest0-21swapper/33
17955152141,8sleep21435812-21diskmemload2
16973362141,8sleep01435812-21diskmemload0
16709292141,9sleep01670943-21ssh0
16646802141,8sleep01435812-21diskmemload0
14933242141,8sleep21493463-21/usr/sbin/munin2
14781282141,4sleep01478142-21ssh0
14774302141,8sleep10-21swapper/11
14688502141,3sleep01468860-21ssh0
14594812141,4sleep21459495-21ssh2
130522999148,3cyclictest1382250-21turbostat3
130522999148,3cyclictest1350267-21chrt3
130522999140,2cyclictest0-21swapper/33
130522999140,12cyclictest0-21swapper/33
130522999140,12cyclictest0-21swapper/33
130522999140,12cyclictest0-21swapper/33
130522999140,12cyclictest0-21swapper/33
17841132131,8sleep21784342-21users2
17692262131,6sleep20-21swapper/22
17658572131,6sleep01765912-21ntpq0
17398942137,1sleep00-21swapper/00
16804942131,8sleep01680508-21ssh0
16804942131,8sleep01680508-21ssh0
16552452131,8sleep01655259-21ssh0
16477162131,3sleep21435812-21diskmemload2
16449642131,9sleep00-21swapper/00
16038502131,8sleep01603864-21ssh0
15842322131,6sleep00-21swapper/00
14591042131,8sleep01459118-21ssh0
13709312131,8sleep01371179-21grep0
130522999130,11cyclictest0-21swapper/33
17681472120,2sleep11435812-21diskmemload1
17587432121,7sleep21435812-21diskmemload2
17073652120,3sleep01707379-21ssh0
16812652121,6sleep11435812-21diskmemload1
16812652121,6sleep11435812-21diskmemload1
16444242121,7sleep11644436-21ssh1
16347572121,6sleep21634812-21wc2
16118192120,2sleep20-21swapper/22
16099892121,6sleep11435812-21diskmemload1
16009842121,3sleep00-21swapper/00
15156272121,6sleep11515650-21ssh1
14860392121,7sleep11486050-21ssh1
14694152121,3sleep11469430-21ssh1
14294352121,5sleep11429486-21ntpq1
14096562121,7sleep21409753-21ps2
14088902120,4sleep01409069-21latency_hist0
13383942121,7sleep11338445-21ld1
17794492111,6sleep01779716-21/usr/sbin/munin0
17518772111,4sleep10-21swapper/11
17132632111,5sleep11713323-21ntpq1
16811742111,3sleep20-21swapper/22
16811742111,3sleep20-21swapper/22
16411212111,6sleep21642858-21cut2
16122672111,3sleep20-21swapper/22
15682592111,3sleep20-21swapper/22
15670802111,3sleep10-21swapper/11
15506862112,4sleep11550710-21apt-get1
15476402110,4sleep10-21swapper/11
15386982110,2sleep00-21swapper/00
15069022111,3sleep20-21swapper/22
15036432111,2sleep10-21swapper/11
14876982111,3sleep20-21swapper/22
14686672111,3sleep20-21swapper/22
14524762111,3sleep20-21swapper/22
14242142112,2sleep2171rcu_preempt2
14187202111,5sleep11418809-21ntpq1
14033062110,1sleep10-21swapper/11
13918932111,5sleep21391940-21ntpq2
13812302111,5sleep21381281-21ntpq2
13612432112,4sleep01361262-21apt-get0
18029022101,3sleep00-21swapper/00
18013742100,5sleep20-21swapper/22
17950562100,5sleep10-21swapper/11
17847302101,2sleep10-21swapper/11
17528342101,5sleep11752844-21ssh1
17513442101,2sleep20-21swapper/22
17406782101,4sleep11435812-21diskmemload1
17399552101,4sleep21435812-21diskmemload2
17331582101,3sleep00-21swapper/00
17265132101,4sleep21726538-21apt-get2
17230672101,4sleep11435812-21diskmemload1
17182442101,3sleep20-21swapper/22
17149692101,3sleep20-21swapper/22
16898492101,2sleep20-21swapper/22
16878022101,4sleep11435812-21diskmemload1
16704362101,3sleep20-21swapper/22
16628032100,1sleep10-21swapper/11
16621512100,2sleep21662165-21ssh2
16287212101,3sleep00-21swapper/00
16175962101,4sleep11435812-21diskmemload1
15933192101,5sleep01593333-21ssh0
15769792101,4sleep11577014-21ssh1
15762252101,3sleep11435812-21diskmemload1
15681302101,3sleep00-21swapper/00
15569092101,3sleep20-21swapper/22
15419212101,4sleep21542185-21df2
15394502101,4sleep21435812-21diskmemload2
15381582101,4sleep11435812-21diskmemload1
15321102100,1sleep00-21swapper/00
15218652101,4sleep21435812-21diskmemload2
15123742101,4sleep11435812-21diskmemload1
15045962101,3sleep00-21swapper/00
14946732101,3sleep11435812-21diskmemload1
14867122101,3sleep00-21swapper/00
14597782101,4sleep11435812-21diskmemload1
14533062101,3sleep00-21swapper/00
14481012101,4sleep11448983-21memory1
14324562101,5sleep10-21swapper/11
14295982101,5sleep01429722-21grep0
13931892101,4sleep01393443-21df0
13328422101,5sleep11332899-21ntp_states1
13328422101,5sleep11332899-21ntp_states1
13287522101,5sleep01328853-21expr0
13287522101,5sleep01328853-21expr0
13221732101,4sleep01322227-21ntpq0
13209962101,4sleep11321981-21memory1
13115032101,5sleep01310720-21munin-node0
13093652101,4sleep11309453-21ntpq1
1802305291,3sleep11435812-21diskmemload1
1794691290,1sleep00-21swapper/00
1774579291,3sleep01774642-21ntpq0
1722843291,2sleep00-21swapper/00
1700283291,5sleep21700427-21cpuspeed_turbos2
1699119290,1sleep10-21swapper/11
1653226290,5sleep11435812-21diskmemload1
1626713290,5sleep21626724-21ssh2
1619419291,4sleep01619431-21ssh0
1591079291,2sleep20-21swapper/22
1560004291,3sleep20-21swapper/22
1559584291,4sleep01559797-21diskstats0
1549002291,2sleep00-21swapper/00
1512165290,1sleep00-21swapper/00
1503494291,2sleep20-21swapper/22
1477701291,4sleep21477715-21ssh2
1442153291,2sleep00-21swapper/00
1431306291,4sleep01432454-21cut0
1426014291,4sleep21427519-21tr2
1424158291,4sleep11424212-21ntpq1
1397074291,4sleep21397214-21ntp_192.168.1142
1370585291,4sleep21370626-21ntpq2
1350608291,4sleep21350864-21df2
1349329291,4sleep01349384-21ntpq0
1340450294,3sleep11340555-21kthreadcore1
1338335291,3sleep01338392-21ntpq0
1328471291,4sleep21328483-21sh2
1327497291,4sleep11327546-21ntpq1
1323134290,1sleep20-21swapper/22
1519640280,2sleep00-21swapper/00
1435789280,2sleep20-21swapper/22
1403532280,1sleep00-21swapper/00
1402585281,3sleep21402639-21ntpq2
1387465280,2sleep172-21kcompactd01
1382227280,4sleep0255-21systemd-journal0
1376750280,1sleep20-21swapper/22
1360961281,4sleep11361065-21date1
1340556281,3sleep21342038-21kthreadcore2
1323405281,3sleep01323423-21apt-get0
1314046281,3sleep21315820-21kthreadcore2
1420078270,2sleep01420097-21apt-get0
1419645270,1sleep20-21swapper/22
1418924270,3sleep01419009-21ps0
1413609271,2sleep10-21swapper/11
1408557270,1sleep20-21swapper/22
1407957270,3sleep11408009-21open_files1
1398137270,1sleep10-21swapper/11
1388120271,2sleep01388221-21/usr/sbin/munin0
1377177270,2sleep10-21swapper/11
1373360271,3sleep0221rcuc/00
1372137270,4sleep10-21swapper/11
1371476270,1sleep10-21swapper/11
1366021270,1sleep20-21swapper/22
1365299271,2sleep10-21swapper/11
1357181271,2sleep00-21swapper/00
1350320270,4sleep20-21swapper/22
1332489270,2sleep21332732-21strings2
1332489270,2sleep21332732-21strings2
1312720270,2sleep10-21swapper/11
1305967271,2sleep20-21swapper/22
1413260260,2sleep00-21swapper/00
1392684260,1sleep10-21swapper/11
1387505260,4sleep20-21swapper/22
1360645260,4sleep20-21swapper/22
1354782261,2sleep00-21swapper/00
1344978260,1sleep00-21swapper/00
1310224260,1sleep00-21swapper/00
1333789250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional