You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-06 - 08:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Thu Aug 06, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1579280212652,66sleep30-21swapper/33
157909828653,9sleep10-21swapper/11
157939428465,12sleep20-21swapper/22
157922127858,14sleep00-21swapper/00
1579560992927,1cyclictest0-21swapper/33
1579560992616,1cyclictest0-21swapper/33
1579560992523,1cyclictest0-21swapper/33
1579560992422,1cyclictest0-21swapper/33
1579560992422,1cyclictest0-21swapper/33
1579560992320,1cyclictest0-21swapper/33
1579560992320,1cyclictest0-21swapper/33
157956099227,13cyclictest0-21swapper/33
1579560992220,1cyclictest0-21swapper/33
1579560992220,1cyclictest0-21swapper/33
1579560992218,2cyclictest0-21swapper/33
157956099221,16cyclictest0-21swapper/33
157956099221,15cyclictest1582074-21taskset3
157956099219,10cyclictest541ktimers/33
157956099216,12cyclictest0-21swapper/33
1579560992119,1cyclictest0-21swapper/33
1579560992119,1cyclictest0-21swapper/33
157956099211,6cyclictest1701919-21taskset3
1579560992111,7cyclictest66950irq/132-enp1s0-TxRx-33
157956099211,16cyclictest0-21swapper/33
157956099211,15cyclictest0-21swapper/33
157956099211,15cyclictest0-21swapper/33
19338612201,3sleep20-21swapper/22
17827212201,7sleep11707449-21diskmemload1
1579560992018,1cyclictest0-21swapper/33
1579560992018,1cyclictest0-21swapper/33
1579560992017,1cyclictest0-21swapper/33
1579560992016,2cyclictest0-21swapper/33
157956099201,15cyclictest0-21swapper/33
157956099201,13cyclictest0-21swapper/33
1579560992010,6cyclictest1930280-21taskset3
20728742191,4sleep01707449-21diskmemload0
1579560991917,1cyclictest0-21swapper/33
1579560991917,1cyclictest0-21swapper/33
1579560991917,1cyclictest0-21swapper/33
1579560991916,1cyclictest0-21swapper/33
1579560991915,2cyclictest0-21swapper/33
157956099191,15cyclictest0-21swapper/33
157956099191,15cyclictest0-21swapper/33
157956099191,14cyclictest0-21swapper/33
20484772181,7sleep21707449-21diskmemload2
19917872180,9sleep21707449-21diskmemload2
16706372180,9sleep11670640-21sshd-auth1
1579560991816,1cyclictest0-21swapper/33
1579560991816,1cyclictest0-21swapper/33
1579560991816,1cyclictest0-21swapper/33
1579560991816,1cyclictest0-21swapper/33
1579560991814,2cyclictest0-21swapper/33
1579560991811,4cyclictest0-21swapper/33
157956099181,14cyclictest0-21swapper/33
157956099181,13cyclictest0-21swapper/33
157956099181,13cyclictest0-21swapper/33
157956099181,13cyclictest0-21swapper/33
157956099181,13cyclictest0-21swapper/33
157956099181,13cyclictest0-21swapper/33
157956099181,12cyclictest15884632sleep33
20231332171,3sleep22023162-21apt-get2
20146182171,3sleep20-21swapper/22
20030932172,8sleep02003189-21/usr/sbin/munin0
19651192171,3sleep00-21swapper/00
19559152171,6sleep21956135-21head2
18542692171,2sleep00-21swapper/00
17653732171,3sleep21707449-21diskmemload2
17570802171,3sleep20-21swapper/22
17294142171,5sleep11729475-21ntp_states1
157956099172,12cyclictest0-21swapper/33
1579560991715,1cyclictest0-21swapper/33
1579560991715,1cyclictest0-21swapper/33
1579560991715,1cyclictest0-21swapper/33
1579560991713,2cyclictest0-21swapper/33
157956099171,14cyclictest0-21swapper/33
157956099170,13cyclictest0-21swapper/33
20398412161,7sleep02039856-21ssh0
20002162161,4sleep22000224-21ssh2
199443621610,3sleep1331ktimers/11
18908512161,5sleep11890912-21ssh1
18005532160,3sleep10-21swapper/11
17861232161,3sleep01786137-21ssh0
17325062161,5sleep21732521-21ssh2
17321192161,7sleep01732134-21ssh0
157956099169,4cyclictest16756892sleep33
1579560991613,1cyclictest0-21swapper/33
1579560991612,2cyclictest0-21swapper/33
157956099160,13cyclictest0-21swapper/33
157956099160,13cyclictest0-21swapper/33
20409132151,9sleep10-21swapper/11
20057012151,5sleep12003351-40find1
19906382151,4sleep01990896-21users0
19838662151,4sleep21707449-21diskmemload2
19152382151,9sleep11707449-21diskmemload1
18726772158,3sleep21872687-21ssh2
18542402151,2sleep20-21swapper/22
17558562151,7sleep11756082-21users1
17501022151,9sleep01707449-21diskmemload0
16345602151,10sleep11634680-21tune2fs1
1579560991511,2cyclictest0-21swapper/33
157956099150,12cyclictest0-21swapper/33
157956099150,12cyclictest0-21swapper/33
20726972141,4sleep22072711-21ssh2
20214102141,9sleep02021424-21ssh0
18714352140,1sleep10-21swapper/11
18705012140,2sleep00-21swapper/00
18451432141,9sleep01845155-21ssh0
18371082141,9sleep01837115-21ssh0
17948442140,2sleep00-21swapper/00
17405312141,8sleep21707449-21diskmemload2
16290502140,6sleep00-21swapper/00
16080662141,2sleep21608085-21apt-get2
157956099140,12cyclictest0-21swapper/33
157956099140,12cyclictest0-21swapper/33
20731372131,8sleep11707449-21diskmemload1
20485032131,8sleep01707449-21diskmemload0
20319982131,6sleep12032184-21df_inode1
18895872133,2sleep0171rcu_preempt0
18791362130,8sleep21707449-21diskmemload2
17416862131,8sleep11707449-21diskmemload1
17024822130,1sleep20-21swapper/22
16552332131,6sleep21654999-21sendmail-msp2
20562822121,7sleep02056297-21ssh0
20129882120,8sleep01707449-21diskmemload0
19672422120,7sleep21707449-21diskmemload2
19488432121,7sleep21948857-21ssh2
19045532121,7sleep01904564-21ssh0
18526352121,7sleep11852647-21rm1
18436162121,6sleep11707449-21diskmemload1
17747662121,7sleep21774780-21ssh2
17736682121,5sleep11707449-21diskmemload1
17131452121,5sleep21707449-21diskmemload2
16355502121,7sleep01637055-21tr0
16198482121,8sleep21621530-21kthreadcore2
15978652121,7sleep01597977-21fschecks_time0
20652252111,3sleep00-21swapper/00
20643872111,4sleep21707449-21diskmemload2
20575392111,3sleep20-21swapper/22
20555832111,6sleep12055598-21ssh1
20362362112,2sleep20-21swapper/22
20293362111,3sleep00-21swapper/00
20119842111,2sleep20-21swapper/22
20010132111,3sleep10-21swapper/11
19818532111,6sleep01982054-21/usr/sbin/munin0
19758302110,2sleep20-21swapper/22
19740642111,4sleep01707449-21diskmemload0
19736652110,2sleep10-21swapper/11
19599912111,4sleep11707449-21diskmemload1
19578322110,7sleep01957846-21ssh0
19331542111,3sleep00-21swapper/00
19232982111,3sleep10-21swapper/11
19232982111,3sleep10-21swapper/11
19143892111,3sleep20-21swapper/22
18876242111,3sleep20-21swapper/22
18787662111,3sleep10-21swapper/11
18432742111,3sleep20-21swapper/22
18356382111,5sleep11707449-21diskmemload1
18347302111,5sleep21707449-21diskmemload2
18287692110,3sleep00-21swapper/00
18178802111,3sleep10-21swapper/11
17991332111,4sleep21707449-21diskmemload2
17843722113,4sleep2431ktimers/22
17501842110,1sleep20-21swapper/22
17133232111,3sleep00-21swapper/00
16654622111,5sleep11665521-21munin-run1
16573452111,6sleep01658974-21cut0
16547262111,6sleep11654730-21modprobe1
16447582111,5sleep01644776-21apt-get0
16151602111,6sleep01616899-21cut0
20656522101,2sleep10-21swapper/11
20314332101,4sleep11707449-21diskmemload1
19585602101,2sleep10-21swapper/11
19502622101,3sleep10-21swapper/11
19426352100,3sleep00-21swapper/00
19345942101,5sleep11933593-21sendmail-msp1
19339582101,4sleep01933961-21apt-get0
19325272101,3sleep20-21swapper/22
19246412101,2sleep20-21swapper/22
19246412101,2sleep20-21swapper/22
19223232100,3sleep00-21swapper/00
19223232100,3sleep00-21swapper/00
19079342101,4sleep01908200-21df0
19065172101,3sleep20-21swapper/22
18984352101,3sleep20-21swapper/22
18905802101,4sleep01890850-21df0
18792192100,1sleep00-21swapper/00
18629692101,2sleep00-21swapper/00
18621012101,2sleep10-21swapper/11
18620222101,4sleep21707449-21diskmemload2
18280342101,3sleep20-21swapper/22
18263852101,3sleep11707449-21diskmemload1
18174652101,2sleep00-21swapper/00
18116382101,2sleep20-21swapper/22
18100302101,4sleep01707449-21diskmemload0
17915212101,2sleep20-21swapper/22
17853632101,4sleep01707449-21diskmemload0
17694192101,6sleep01770142-21cat0
17599582101,5sleep11759974-21ssh1
17599022101,3sleep00-21swapper/00
17577122101,3sleep00-21swapper/00
17492612100,1sleep10-21swapper/11
17392792101,3sleep00-21swapper/00
17239082101,2sleep20-21swapper/22
17219982101,2sleep10-21swapper/11
17127232101,3sleep10-21swapper/11
16888412101,5sleep11690545-21cat1
16814652102,3sleep01681484-21apt-get0
16755642101,4sleep20-21swapper/22
16647672102,4sleep01664984-21/usr/sbin/munin0
16595882101,5sleep11659855-21head1
16342692101,5sleep21634030-21sendmail-msp2
16291012101,4sleep11629320-21diskstats1
16238252101,4sleep01624081-21df0
16185812101,4sleep11618578-21apt-get1
16176012101,5sleep11617852-21tail1
2014526291,2sleep10-21swapper/11
1984369290,3sleep10-21swapper/11
1933071291,2sleep10-21swapper/11
1899165291,2sleep10-21swapper/11
1887398291,3sleep11707449-21diskmemload1
1809477291,2sleep10-21swapper/11
1792278291,4sleep11792290-21ssh1
1721304291,3sleep01707449-21diskmemload0
1697187291,2sleep00-21swapper/00
1640132291,4sleep11641048-21cut1
1618559291,3sleep00-21swapper/00
1613334291,3sleep20-21swapper/22
1596539290,1sleep00-21swapper/00
1580255291,4sleep11580464-21cut1
1816443281,3sleep21816536-21ps2
1697184281,3sleep21696961-21sendmail-msp2
1691956280,2sleep00-21swapper/00
1680838281,1sleep00-21swapper/00
1666304281,3sleep01667382-21kthreadcore0
1597301281,3sleep21-21systemd2
1592299281,3sleep11592321-21apt-get1
1587582281,3sleep21588461-21kthreadcore2
1579764281,3sleep21579971-21cstates2
1704377271,2sleep01706150-21kthreadcore0
1702219271,3sleep10-21swapper/11
1694166271,2sleep21695777-21latency2
1692597271,2sleep10-21swapper/11
1682047271,2sleep10-21swapper/11
1676215271,2sleep10-21swapper/11
1660512270,1sleep20-21swapper/22
1654925270,1sleep20-21swapper/22
1654908270,1sleep00-21swapper/00
1641049271,2sleep20-21swapper/22
1639565270,2sleep00-21swapper/00
1629321271,2sleep21629522-21head2
1627996271,2sleep10-21swapper/11
1612399271,3sleep10-21swapper/11
1604264271,3sleep11604925-21kthreadcore1
1598141271,2sleep20-21swapper/22
1597575270,2sleep10-21swapper/11
1587011270,2sleep10-21swapper/11
1691333260,1sleep20-21swapper/22
1683482261,2sleep20-21swapper/22
1676777261,2sleep20-21swapper/22
1672602261,2sleep00-21swapper/00
1649450260,4sleep20-21swapper/22
1607774260,1sleep20-21swapper/22
1603382261,2sleep00-21swapper/00
1702783250,1sleep10-21swapper/11
1687303250,1sleep00-21swapper/00
1674313250,1sleep10-21swapper/11
1665522250,1sleep20-21swapper/22
1645489250,2sleep10-21swapper/11
1624152250,1sleep20-21swapper/22
1610389250,1sleep00-21swapper/00
1591907250,1sleep00-21swapper/00
1584110250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional