You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 12:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Sun Jun 14, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1232403212562,58sleep30-21swapper/33
123233328759,16sleep10-21swapper/11
123177528558,14sleep00-21swapper/00
123227028154,16sleep20-21swapper/22
15934622271,21sleep11363032-21diskmemload1
123258899220,18cyclictest0-21swapper/33
123258899210,18cyclictest0-21swapper/33
16762032207,7sleep01363032-21diskmemload0
1232588992018,1cyclictest0-21swapper/33
1232588992011,6cyclictest1394324-21taskset3
123258899201,16cyclictest0-21swapper/33
123258899201,15cyclictest1632207-21taskset3
123258899201,15cyclictest1370267-21taskset3
123258899201,15cyclictest0-21swapper/33
123258899201,15cyclictest0-21swapper/33
123258899201,15cyclictest0-21swapper/33
123258899200,17cyclictest0-21swapper/33
16435592191,7sleep11363032-21diskmemload1
1232588991915,2cyclictest0-21swapper/33
1232588991911,6cyclictest66950irq/132-enp1s0-TxRx-33
123258899191,15cyclictest66950irq/132-enp1s0-TxRx-33
123258899191,14cyclictest0-21swapper/33
123258899191,14cyclictest0-21swapper/33
16264332181,3sleep00-21swapper/00
123258899187,8cyclictest1695143-21turbostat3
123258899184,11cyclictest0-21swapper/33
123258899181,4cyclictest0-21swapper/33
1232588991812,5cyclictest0-21swapper/33
123258899181,14cyclictest0-21swapper/33
123258899181,14cyclictest0-21swapper/33
123258899181,14cyclictest0-21swapper/33
123258899181,14cyclictest0-21swapper/33
123258899181,14cyclictest0-21swapper/33
123258899181,14cyclictest0-21swapper/33
123258899181,14cyclictest0-21swapper/33
123258899181,13cyclictest0-21swapper/33
123258899181,13cyclictest0-21swapper/33
123258899181,13cyclictest0-21swapper/33
123258899181,13cyclictest0-21swapper/33
123258899181,12cyclictest0-21swapper/33
123258899180,15cyclictest0-21swapper/33
123258899180,14cyclictest0-21swapper/33
123258899180,14cyclictest0-21swapper/33
17101172171,3sleep10-21swapper/11
15538132171,12sleep21555537-21grep2
14722822171,3sleep00-21swapper/00
14122892171,3sleep20-21swapper/22
12659972171,4sleep21266051-21ntpq2
123258899171,4cyclictest0-21swapper/33
123258899171,3cyclictest1256176-21taskset3
123258899171,14cyclictest0-21swapper/33
123258899171,13cyclictest0-21swapper/33
123258899171,13cyclictest0-21swapper/33
123258899171,13cyclictest0-21swapper/33
123258899171,12cyclictest1287908-21taskset3
123258899171,12cyclictest0-21swapper/33
123258899170,14cyclictest0-21swapper/33
123258899170,13cyclictest0-21swapper/33
123258899170,13cyclictest0-21swapper/33
166929221610,2sleep21669306-21ssh2
16499522161,2sleep20-21swapper/22
16385472161,11sleep01640194-21cat0
16264952161,4sleep21626509-21ssh2
15405952161,3sleep00-21swapper/00
14639352161,4sleep01363032-21diskmemload0
14388812161,3sleep20-21swapper/22
123258899165,1cyclictest0-21swapper/33
123258899161,3cyclictest0-21swapper/33
1232588991612,2cyclictest0-21swapper/33
123258899160,13cyclictest0-21swapper/33
123258899160,12cyclictest0-21swapper/33
17023382151,3sleep20-21swapper/22
16941222151,3sleep20-21swapper/22
16871152152,5sleep11687328-21diskstats1
15142822158,1sleep2171rcu_preempt2
13710802151,3sleep01363032-21diskmemload0
13681092151,9sleep21368336-21users2
123258899156,5cyclictest12566192sleep33
123258899150,13cyclictest0-21swapper/33
123258899150,13cyclictest0-21swapper/33
123258899150,13cyclictest0-21swapper/33
123258899150,13cyclictest0-21swapper/33
123258899150,12cyclictest0-21swapper/33
123258899150,12cyclictest0-21swapper/33
16940572141,3sleep01363032-21diskmemload0
15674832142,7sleep11363032-21diskmemload1
15473992141,2sleep00-21swapper/00
15328792141,3sleep11363032-21diskmemload1
15253892141,8sleep01363032-21diskmemload0
14620762141,3sleep11363032-21diskmemload1
14215142144,5sleep014-21ksoftirqd/00
14050122141,4sleep11405026-21ssh1
13951512141,8sleep01363032-21diskmemload0
13790772141,8sleep21363032-21diskmemload2
13474082141,9sleep21347561-21diskstats2
123258899148,3cyclictest12774802sleep33
1232588991411,2cyclictest0-21swapper/33
1232588991411,2cyclictest0-21swapper/33
123258899140,2cyclictest0-21swapper/33
123258899140,2cyclictest0-21swapper/33
123258899140,12cyclictest0-21swapper/33
123258899140,12cyclictest0-21swapper/33
123258899140,12cyclictest0-21swapper/33
123258899140,11cyclictest0-21swapper/33
16658792131,6sleep11665938-21ntpq1
16445132131,7sleep01363032-21diskmemload0
16181472131,7sleep01363032-21diskmemload0
16088532131,8sleep11608865-21ssh1
15159652131,1sleep10-21swapper/11
14989022136,4sleep01498912-21ssh0
14458012131,7sleep11363032-21diskmemload1
14217242131,8sleep21421738-21ssh2
13965982131,5sleep10-21swapper/11
13862052131,3sleep21386215-21ssh2
13094782130,1sleep10-21swapper/11
17042882121,8sleep20-21swapper/22
16785252121,6sleep11363032-21diskmemload1
16573212121,6sleep172-21kcompactd01
16002412121,5sleep20-21swapper/22
15777082121,7sleep21579000-21cut2
15062442121,3sleep11506255-21ssh1
14654942121,7sleep21465507-21ssh2
14550132121,4sleep20-21swapper/22
14494422121,7sleep11449570-21tune2fs1
14486402121,6sleep01363032-21diskmemload0
14405902121,3sleep21440606-21ssh2
12624502121,7sleep11263460-21cut1
12609632121,7sleep01261234-21awk0
17169572111,7sleep11717000-21sh1
17098862111,5sleep01363032-21diskmemload0
16763592111,4sleep11363032-21diskmemload1
16487332113,4sleep11648780-21ntpq1
16280722111,5sleep21628284-21hddtemp_smartct2
15938832111,5sleep01363032-21diskmemload0
15919962111,7sleep21592007-21ssh2
15902872111,5sleep01590516-21users0
15751562110,2sleep10-21swapper/11
15601472111,3sleep20-21swapper/22
15588492111,5sleep00-21swapper/00
15583002111,3sleep10-21swapper/11
15326692115,1sleep20-21swapper/22
15231972111,7sleep11523210-21sh1
15067032110,7sleep21506710-21sh2
14986812110,2sleep20-21swapper/22
14789632111,5sleep21479024-21ntp_states2
14712472111,4sleep11363032-21diskmemload1
14143772111,3sleep00-21swapper/00
14042852111,4sleep20-21swapper/22
13881162112,4sleep11363032-21diskmemload1
13801092110,2sleep0171rcu_preempt0
13630202111,4sleep21363050-21munin-run2
13567262111,5sleep11356775-21ntpq1
13103722111,7sleep01310530-21grep0
12872182111,5sleep01287269-21ntpq0
17124472101,4sleep01712471-21apt-get0
16829172101,6sleep01683065-21basename0
16753422100,2sleep20-21swapper/22
16683822101,4sleep01363032-21diskmemload0
16591792101,4sleep01363032-21diskmemload0
16362562101,4sleep21636280-21apt-get2
16337552101,2sleep00-21swapper/00
16326752101,3sleep10-21swapper/11
16243752101,4sleep11363032-21diskmemload1
16158292101,4sleep21363032-21diskmemload2
16024012101,4sleep21363032-21diskmemload2
16013552101,3sleep10-21swapper/11
15843982100,1sleep00-21swapper/00
15731152101,3sleep00-21swapper/00
15661552101,4sleep01363032-21diskmemload0
15495732101,4sleep11363032-21diskmemload1
15486932101,4sleep21363032-21diskmemload2
15390152101,5sleep11539072-21ssh1
15347152101,4sleep21363032-21diskmemload2
15262992101,3sleep00-21swapper/00
15235452101,5sleep21363032-21diskmemload2
15006292101,3sleep00-21swapper/00
14996632101,2sleep10-21swapper/11
14910152101,3sleep20-21swapper/22
14904372101,4sleep11363032-21diskmemload1
14898742101,4sleep01363032-21diskmemload0
14565382101,3sleep00-21swapper/00
14398862101,4sleep11363032-21diskmemload1
14296782101,3sleep20-21swapper/22
14278012101,3sleep00-21swapper/00
14276082101,4sleep11427668-21ntpq1
13962262101,3sleep21363032-21diskmemload2
13883502100,3sleep00-21swapper/00
13706642101,3sleep10-21swapper/11
13523342101,4sleep01352405-21munin-run0
13460762101,4sleep01346129-21ntp_states0
13370652101,4sleep11337136-21irqstats1
13312252101,4sleep01331244-21apt-get0
13302452101,5sleep21330317-21xargs2
12997082102,4sleep11300116-21kthreadcore1
12941562103,3sleep11294298-21gltestperf1
12820242101,4sleep11282072-21ntp_states1
12556632101,5sleep01255908-21/usr/sbin/munin0
12512202101,3sleep11232583-21cyclictest1
12455652101,4sleep01245624-21sendmail-msp0
12407232101,4sleep21240932-21hddtemp_smartct2
12327662101,3sleep10-21swapper/11
1714583290,6sleep21716344-21tr2
1701903291,2sleep10-21swapper/11
1658262291,4sleep21658273-21ssh2
1615908291,2sleep10-21swapper/11
1581525291,3sleep11363032-21diskmemload1
1573961291,3sleep21363032-21diskmemload2
1509288290,1sleep00-21swapper/00
1481357291,2sleep00-21swapper/00
1476101291,4sleep11477767-21cut1
1471690291,4sleep21471702-21ssh2
1436319291,2sleep00-21swapper/00
1420805290,3sleep11363032-21diskmemload1
1413669291,4sleep11413682-21sh1
1404500290,1sleep00-21swapper/00
1376609291,3sleep10-21swapper/11
1341933291,3sleep21341952-21apt-get2
1324577291,4sleep11324619-21ld1
1313965290,6sleep11314014-21ldconfig1
1304492291,3sleep01304511-21apt-get0
1300117291,4sleep21301793-21cut2
1295337291,4sleep01297093-21grep0
1281971291,4sleep01282017-21ntpq0
1700437281,3sleep01700449-21ssh0
1678855280,1sleep20-21swapper/22
1607531281,3sleep01607542-21ssh0
1358542281,3sleep01360047-21ps0
1352894281,4sleep20-21swapper/22
1346735280,1sleep10-21swapper/11
1325286280,4sleep00-21swapper/00
1315867281,3sleep01317493-21kthreadcore0
1288831281,3sleep00-21swapper/00
1288831281,3sleep00-21swapper/00
1267256281,3sleep01267512-21df0
1257121281,3sleep20-21swapper/22
1246136281,3sleep11246343-21tail1
1352613271,2sleep00-21swapper/00
1347842272,2sleep11347877-21kthreadcore1
1341324270,1sleep00-21swapper/00
1337137271,2sleep21337170-21kthreadcore2
1335376270,4sleep20-21swapper/22
1325857271,2sleep01325876-21apt-get0
1319086270,2sleep21319199-21strings2
1298838270,1sleep20-21swapper/22
1285310271,3sleep21287032-21latency2
1282602270,1sleep20-21swapper/22
1274194271,2sleep00-21swapper/00
1266598270,1sleep00-21swapper/00
1261261270,1sleep10-21swapper/11
1255975270,2sleep20-21swapper/22
1250858270,1sleep00-21swapper/00
1237809272,3sleep0151ktimers/00
1237630270,1sleep20-21swapper/22
1331924261,2sleep10-21swapper/11
1326430261,2sleep10-21swapper/11
1324620261,2sleep20-21swapper/22
1320047260,1sleep10-21swapper/11
1314831260,1sleep20-21swapper/22
1309343260,1sleep20-21swapper/22
1292512261,2sleep11292595-21gcc1
1292512261,2sleep11292595-21gcc1
1289111261,2sleep20-21swapper/22
1289111261,2sleep20-21swapper/22
1283205260,1sleep10-21swapper/11
1277552260,1sleep20-21swapper/22
1277369260,4sleep10-21swapper/11
1249912260,2sleep20-21swapper/22
1245364260,1sleep10-21swapper/11
1362979251,1sleep10-21swapper/11
1303058250,1sleep00-21swapper/00
1272254250,3sleep10-21swapper/11
1271828250,2sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional