You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 20:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sat Aug 08, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
142385322780,1sleep11423854-21sendmail-msp1
1278645212259,55sleep30-21swapper/33
127859228662,7sleep00-21swapper/00
127843028657,11sleep10-21swapper/11
127838728052,12sleep20-21swapper/22
1278885992725,1cyclictest0-21swapper/33
1278885992713,12cyclictest0-21swapper/33
1278885992523,1cyclictest0-21swapper/33
1278885992523,1cyclictest0-21swapper/33
1278885992422,1cyclictest0-21swapper/33
1278885992422,1cyclictest0-21swapper/33
1278885992321,1cyclictest0-21swapper/33
1278885992321,1cyclictest0-21swapper/33
1278885992320,1cyclictest0-21swapper/33
1278885992319,2cyclictest0-21swapper/33
1278885992220,1cyclictest0-21swapper/33
1278885992220,1cyclictest0-21swapper/33
1278885992220,1cyclictest0-21swapper/33
1278885992219,1cyclictest0-21swapper/33
1278885992219,1cyclictest0-21swapper/33
1278885992219,1cyclictest0-21swapper/33
1278885992219,1cyclictest0-21swapper/33
1278885992218,2cyclictest0-21swapper/33
127888599221,15cyclictest0-21swapper/33
1278885992119,1cyclictest0-21swapper/33
1278885992119,1cyclictest0-21swapper/33
1278885992119,1cyclictest0-21swapper/33
1278885992117,2cyclictest0-21swapper/33
1278885992117,2cyclictest0-21swapper/33
1278885992117,2cyclictest0-21swapper/33
1278885992117,2cyclictest0-21swapper/33
14835592201,4sleep01483573-21ssh0
127888599205,13cyclictest0-21swapper/33
1278885992018,1cyclictest0-21swapper/33
1278885992018,1cyclictest0-21swapper/33
1278885992018,1cyclictest0-21swapper/33
1278885992017,1cyclictest0-21swapper/33
1278885992017,1cyclictest0-21swapper/33
1278885992016,2cyclictest0-21swapper/33
1278885992016,2cyclictest0-21swapper/33
1278885992011,6cyclictest66950irq/132-enp1s0-TxRx-33
127888599201,16cyclictest1317147-21chrt3
17660952191,8sleep01406168-21diskmemload0
17321412191,3sleep11732155-21ssh1
15881302191,5sleep11406168-21diskmemload1
15361792191,4sleep11406168-21diskmemload1
127888599196,1cyclictest0-21swapper/33
1278885991917,1cyclictest0-21swapper/33
1278885991917,1cyclictest0-21swapper/33
1278885991917,1cyclictest0-21swapper/33
1278885991917,1cyclictest0-21swapper/33
1278885991916,1cyclictest0-21swapper/33
127888599191,5cyclictest0-21swapper/33
1278885991914,2cyclictest0-21swapper/33
127888599191,14cyclictest0-21swapper/33
127888599191,14cyclictest0-21swapper/33
14243112181,7sleep20-21swapper/22
127888599185,2cyclictest0-21swapper/33
1278885991816,1cyclictest0-21swapper/33
1278885991816,1cyclictest0-21swapper/33
1278885991816,1cyclictest0-21swapper/33
1278885991814,2cyclictest0-21swapper/33
1278885991814,2cyclictest0-21swapper/33
127888599181,14cyclictest0-21swapper/33
127888599181,14cyclictest0-21swapper/33
127888599181,13cyclictest0-21swapper/33
16865722171,4sleep21686583-21ssh2
16855342171,3sleep00-21swapper/00
15544362177,4sleep2431ktimers/22
15096672171,3sleep20-21swapper/22
14912922171,3sleep00-21swapper/00
14301702171,3sleep10-21swapper/11
127888599172,12cyclictest0-21swapper/33
1278885991715,1cyclictest0-21swapper/33
1278885991713,2cyclictest0-21swapper/33
1278885991713,2cyclictest0-21swapper/33
1278885991713,2cyclictest0-21swapper/33
1278885991713,2cyclictest0-21swapper/33
1278885991713,2cyclictest0-21swapper/33
1278885991713,2cyclictest0-21swapper/33
1278885991713,2cyclictest0-21swapper/33
127888599170,13cyclictest0-21swapper/33
17468882161,3sleep20-21swapper/22
17319742161,2sleep00-21swapper/00
16686842161,2sleep00-21swapper/00
14095752162,7sleep21410537-21memory2
1278885991613,1cyclictest0-21swapper/33
16888842151,7sleep11688916-21ssh1
16618672151,4sleep01661916-21munin-run0
16540002150,1sleep20-21swapper/22
16409412151,4sleep21640955-21ssh2
15205882151,3sleep10-21swapper/11
17676062141,8sleep21767617-21ssh2
17139852141,9sleep21714000-21ssh2
16704522140,5sleep11670465-21ssh1
16093402141,9sleep21609354-21sshd-auth2
15556992141,8sleep11406168-21diskmemload1
14415142141,8sleep21441585-21munin-run2
13895292140,2sleep20-21swapper/22
17569672131,5sleep10-21swapper/11
16424602131,3sleep11642472-21ssh1
16339392131,8sleep01633953-21ssh0
16328022130,8sleep21406168-21diskmemload2
15973572132,3sleep2171rcu_preempt2
15653702132,6sleep11565007-21sendmail-msp1
15364342131,4sleep01536448-21ssh0
15292682131,7sleep01406168-21diskmemload0
15174722131,2sleep21406168-21diskmemload2
15032472131,7sleep21503324-21munin-run2
14584932131,2sleep00-21swapper/00
14467632131,2sleep11406168-21diskmemload1
14205112131,8sleep01420525-21ssh0
13386602132,5sleep01338679-21apt-get0
13336982131,7sleep01333800-21perl0
13179362131,8sleep21318081-21expr2
12863152131,8sleep00-21swapper/00
17666332121,7sleep11766645-21ssh1
17383282121,7sleep11406168-21diskmemload1
16977002121,7sleep21697968-21ssh2
16895162121,7sleep21690475-21cut2
16866052121,7sleep11406168-21diskmemload1
16499032121,7sleep01649918-21ssh0
15742752121,5sleep20-21swapper/22
15635512124,5sleep21563563-21diskmemload2
15454132121,3sleep10-21swapper/11
15040002120,8sleep01504034-21sh0
14767342121,6sleep11476772-21munin-run1
14724472121,5sleep21472514-21ntp_states2
14379272121,7sleep01437939-21ssh0
14292592121,7sleep01429270-21ssh0
14223992121,7sleep21406168-21diskmemload2
13438532122,5sleep01343865-21apt-get0
13177152121,7sleep11317933-21df_inode1
13173782121,5sleep11317435-21munin-run1
13068612120,1sleep20-21swapper/22
17733622111,5sleep20-21swapper/22
17407822110,7sleep21740796-21ssh2
17404352111,4sleep00-21swapper/00
17156212111,5sleep01715845-21hddtemp_smartct0
17152942111,4sleep10-21swapper/11
17146142111,4sleep11406168-21diskmemload1
17144412110,7sleep01714454-21ssh0
17047582111,4sleep01406168-21diskmemload0
16972342111,6sleep01697493-21latency_hist0
16778762110,3sleep00-21swapper/00
16538142111,6sleep11653996-21sed1
16514002111,3sleep10-21swapper/11
16347572111,5sleep11406168-21diskmemload1
16257102111,4sleep01406168-21diskmemload0
16247772112,4sleep21406168-21diskmemload2
16163932110,5sleep00-21swapper/00
16005942111,5sleep01600630-21ssh0
16001552111,5sleep01600199-21munin-run0
15888862110,7sleep21588900-21ssh2
15793292111,3sleep00-21swapper/00
15792852111,6sleep11406168-21diskmemload1
15737752111,6sleep01573962-21latency_hist0
15621512111,5sleep01406168-21diskmemload0
15571932111,6sleep11558528-21ssh1
15518302111,7sleep01406168-21diskmemload0
15460782110,2sleep00-21swapper/00
15262572111,3sleep10-21swapper/11
15262142111,2sleep21526228-21ssh2
15035742111,4sleep11406168-21diskmemload1
14992122110,6sleep01406168-21diskmemload0
14926862111,4sleep11406168-21diskmemload1
14888972111,5sleep21489933-21memory2
14771842111,5sleep21477454-21df2
14771712112,4sleep11477169-21apt-get1
14746502110,7sleep01474664-21ssh0
14671152111,3sleep00-21swapper/00
14662432111,3sleep20-21swapper/22
14601322112,5sleep10-21swapper/11
14561912110,4sleep20-21swapper/22
14464102111,5sleep21446665-21users2
14400522111,4sleep10-21swapper/11
14124862111,5sleep00-21swapper/00
13912792111,5sleep21391350-21irqstats2
13803742111,4sleep01380393-21apt-get0
13019972111,4sleep01302016-21apt-get0
17753192100,1sleep00-21swapper/00
17557332101,3sleep00-21swapper/00
17504392100,1sleep20-21swapper/22
17476212101,4sleep11406168-21diskmemload1
17243742102,4sleep21724430-21fschecks_count2
17221972100,2sleep20-21swapper/22
17036652101,4sleep11406168-21diskmemload1
16774822101,3sleep21406168-21diskmemload2
16688122101,4sleep21668826-21ssh2
16514692101,5sleep21406168-21diskmemload2
16413442101,4sleep01406168-21diskmemload0
16241272101,2sleep10-21swapper/11
16076032101,3sleep20-21swapper/22
16060032101,4sleep11406168-21diskmemload1
15979892101,2sleep10-21swapper/11
15715152101,5sleep21571530-21ssh2
15461572100,2sleep20-21swapper/22
15195642101,5sleep01519577-21rm0
14947822101,4sleep139542-21systemd-journal1
14473412100,1sleep00-21swapper/00
14012362101,4sleep11401489-21df1
13890722101,5sleep11389528-21cut1
13751412101,4sleep21374922-21sendmail-msp2
13699422101,4sleep21369960-21apt-get2
13595422101,4sleep01359561-21apt-get0
13438682101,4sleep11344121-21df1
13387452101,4sleep21338948-21diskstats2
13374272101,4sleep21337497-21ntp_states2
13281312101,4sleep21328143-21apt-get2
13228922101,4sleep21322911-21apt-get2
13225882101,4sleep01322656-21munin-run0
13172772101,3sleep00-21swapper/00
12929132101,5sleep01294537-21kthreadcore0
12790732101,2sleep00-21swapper/00
1772879290,1sleep10-21swapper/11
1671168291,4sleep11671186-21ssh1
1615305290,1sleep10-21swapper/11
1584321291,4sleep01585884-21ssh0
1450746291,4sleep11406168-21diskmemload1
1412068291,3sleep11412079-21ssh1
1396032291,4sleep21396213-21ls2
1396004290,1sleep00-21swapper/00
1390311291,4sleep01390314-21sshd-auth0
1375171291,4sleep11374922-21sendmail-msp1
1353766290,2sleep10-21swapper/11
1324601291,4sleep01326324-21cut0
1306641290,2sleep10-21swapper/11
1291972291,3sleep21291332-21sendmail-msp2
1287845290,5sleep11289537-21kthreadcore1
1742347281,3sleep00-21swapper/00
1530124281,2sleep20-21swapper/22
1401221281,2sleep00-21swapper/00
1385128280,4sleep20-21swapper/22
1380978281,3sleep11382461-21ps1
1380101281,3sleep01380347-21latency_hist0
1369038281,3sleep21369285-21/usr/sbin/munin2
1349584281,3sleep01350462-21kthreadcore0
1339229281,3sleep11340192-21grep1
1333414280,2sleep10-21swapper/11
1332943280,1sleep00-21swapper/00
1323182280,4sleep11323345-21gltestperf1
1312095280,1sleep20-21swapper/22
1311928281,3sleep1106713-21snmpd1
1297380281,3sleep11298778-21cut1
1296807280,2sleep00-21swapper/00
1296417280,1sleep10-21swapper/11
1405871270,1sleep20-21swapper/22
1396010271,2sleep188-1kworker/1:1H+kblockd1
1374621270,1sleep10-21swapper/11
1365545271,3sleep01367152-21kthreadcore0
1359138270,1sleep10-21swapper/11
1354250271,2sleep00-21swapper/00
1311877270,1sleep00-21swapper/00
1291266271,2sleep21-21systemd2
1395716260,2sleep10-21swapper/11
1395591260,1sleep00-21swapper/00
1374576260,1sleep00-21swapper/00
1369316260,2sleep10-21swapper/11
1364134260,1sleep20-21swapper/22
1359564261,2sleep10-21swapper/11
1358981260,1sleep20-21swapper/22
1353892260,3sleep20-21swapper/22
1332756260,1sleep10-21swapper/11
1312420261,2sleep20-21swapper/22
1301406260,1sleep20-21swapper/22
1283974260,1sleep10-21swapper/11
1348462250,1sleep20-21swapper/22
1279335240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional