You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 09:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Tue Mar 31, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3679492211957,57sleep30-21swapper/33
367937128554,12sleep20-21swapper/22
367963628355,6sleep00-21swapper/00
367931128054,6sleep10-21swapper/11
39973632280,2sleep00-21swapper/00
367978999241,19cyclictest0-21swapper/33
367978999211,15cyclictest0-21swapper/33
3679789992018,1cyclictest0-21swapper/33
367978999201,3cyclictest0-21swapper/33
367978999201,16cyclictest4134235-21chrt3
367978999201,16cyclictest0-21swapper/33
3679789992011,5cyclictest0-21swapper/33
367978999201,15cyclictest0-21swapper/33
367978999201,15cyclictest0-21swapper/33
367978999201,15cyclictest0-21swapper/33
39269802191,4sleep13928713-21grep1
3679789991912,4cyclictest0-21swapper/33
367978999191,15cyclictest0-21swapper/33
367978999191,14cyclictest3763463-21taskset3
367978999191,14cyclictest0-21swapper/33
367978999191,14cyclictest0-21swapper/33
367978999191,14cyclictest0-21swapper/33
367978999191,14cyclictest0-21swapper/33
367978999191,14cyclictest0-21swapper/33
367978999191,14cyclictest0-21swapper/33
367978999191,14cyclictest0-21swapper/33
367978999191,14cyclictest0-21swapper/33
367978999191,14cyclictest0-21swapper/33
367978999191,14cyclictest0-21swapper/33
3679789991910,5cyclictest66950irq/132-enp1s0-TxRx-33
40907552181,8sleep24090766-21ssh2
40904162181,3sleep00-21swapper/00
40184252181,12sleep13810887-21diskmemload1
38267102181,8sleep13810887-21diskmemload1
38154572183,4sleep23815508-21ntpq2
3679789991815,1cyclictest0-21swapper/33
3679789991815,1cyclictest0-21swapper/33
367978999181,14cyclictest0-21swapper/33
367978999181,14cyclictest0-21swapper/33
367978999181,14cyclictest0-21swapper/33
367978999181,14cyclictest0-21swapper/33
367978999181,14cyclictest0-21swapper/33
367978999181,14cyclictest0-21swapper/33
367978999181,14cyclictest0-21swapper/33
367978999181,13cyclictest0-21swapper/33
367978999181,13cyclictest0-21swapper/33
367978999181,13cyclictest0-21swapper/33
367978999180,16cyclictest0-21swapper/33
367978999180,15cyclictest0-21swapper/33
367978999180,15cyclictest0-21swapper/33
367978999180,14cyclictest0-21swapper/33
367978999180,13cyclictest3792475-21taskset3
39315042171,3sleep00-21swapper/00
38374772171,2sleep20-21swapper/22
3679789991712,3cyclictest0-21swapper/33
367978999171,13cyclictest0-21swapper/33
367978999171,13cyclictest0-21swapper/33
367978999171,12cyclictest0-21swapper/33
367978999171,12cyclictest0-21swapper/33
367978999170,14cyclictest0-21swapper/33
367978999170,13cyclictest0-21swapper/33
367978999170,12cyclictest0-21swapper/33
40821082161,2sleep00-21swapper/00
40819762161,4sleep13810887-21diskmemload1
39595782161,4sleep13810887-21diskmemload1
39398802160,1sleep00-21swapper/00
39147892161,3sleep00-21swapper/00
38791202161,4sleep13810887-21diskmemload1
367978999161,14cyclictest0-21swapper/33
367978999161,13cyclictest0-21swapper/33
367978999161,12cyclictest0-21swapper/33
367978999161,12cyclictest0-21swapper/33
367978999160,13cyclictest0-21swapper/33
367978999160,13cyclictest0-21swapper/33
367978999160,12cyclictest0-21swapper/33
40658342151,2sleep20-21swapper/22
40447182151,2sleep00-21swapper/00
39837632150,1sleep10-21swapper/11
38263382151,6sleep03826351-21ssh0
367978999151,2cyclictest0-21swapper/33
367978999151,12cyclictest0-21swapper/33
367978999150,3cyclictest0-21swapper/33
367978999150,12cyclictest0-21swapper/33
41611152141,3sleep13810887-21diskmemload1
41410382141,7sleep23810887-21diskmemload2
41192562141,3sleep20-21swapper/22
41061512140,3sleep20-21swapper/22
40925172141,8sleep04092738-21hddtemp_smartct0
39946922141,2sleep10-21swapper/11
39589982141,4sleep03959012-21ssh0
39404682141,9sleep23810887-21diskmemload2
38801362141,6sleep20-21swapper/22
37042862140,1sleep00-21swapper/00
367978999147,4cyclictest3800005-21chrt3
3679789991412,1cyclictest0-21swapper/33
3679789991411,2cyclictest0-21swapper/33
3679789991411,1cyclictest0-21swapper/33
41327252131,8sleep04132732-21ssh0
41315022131,6sleep24131556-21ntpq2
40375012131,8sleep04037515-21ssh0
40293672131,6sleep00-21swapper/00
39666642131,3sleep03966675-21ssh0
39487922131,7sleep13810887-21diskmemload1
38715162131,7sleep23810887-21diskmemload2
38620482131,8sleep03862061-21ssh0
38356252131,3sleep03835632-21ssh0
38155822131,7sleep03815642-21ls0
36799582133,5sleep03679976-21apt-get0
367978999130,1cyclictest0-21swapper/33
367978999130,1cyclictest0-21swapper/33
41504462121,7sleep04150460-21ssh0
41425692120,5sleep10-21swapper/11
40740352121,5sleep10-21swapper/11
40714962121,3sleep04071507-21ssh0
40211362121,7sleep23810887-21diskmemload2
39858492121,7sleep03985863-21ssh0
39304922121,4sleep23930507-21ssh2
39241222120,5sleep03810887-21diskmemload0
38778842121,7sleep03877899-21ssh0
38468802121,7sleep03847151-21cut0
38275392121,3sleep23827550-21ssh2
367978999128,3cyclictest3709187-21chrt3
41780812111,6sleep14178092-21ssh1
41666102111,3sleep20-21swapper/22
41602982111,4sleep00-21swapper/00
41591952111,4sleep23810887-21diskmemload2
41504732111,3sleep20-21swapper/22
41347682111,4sleep13810887-21diskmemload1
40974752111,4sleep13810887-21diskmemload1
40894722111,4sleep13810887-21diskmemload1
40799592111,3sleep20-21swapper/22
40649592111,4sleep00-21swapper/00
40553902111,4sleep13810887-21diskmemload1
40472402111,2sleep14047248-21ssh1
40377332111,3sleep20-21swapper/22
39925202111,4sleep23810887-21diskmemload2
39748902111,7sleep03974902-21ssh0
39164782111,4sleep13810887-21diskmemload1
39148152111,3sleep20-21swapper/22
39053582111,3sleep00-21swapper/00
38978102110,3sleep23897824-21ssh2
38943832111,5sleep13894438-21ntpq1
38704392111,5sleep13810887-21diskmemload1
38700162111,3sleep00-21swapper/00
38622232111,6sleep23810887-21diskmemload2
38365722111,3sleep20-21swapper/22
37778962111,5sleep03777937-21ntpq0
37602022111,5sleep03761508-21memory0
37225912111,6sleep03720290-21/usr/sbin/munin0
36917812111,5sleep03691831-21ntpq0
41710002101,4sleep23810887-21diskmemload2
41635012100,2sleep10-21swapper/11
41360072101,3sleep00-21swapper/00
41261902101,2sleep00-21swapper/00
41243892101,4sleep13810887-21diskmemload1
41177232100,4sleep13810887-21diskmemload1
41149692101,4sleep03810887-21diskmemload0
41097062100,1sleep20-21swapper/22
41060692101,2sleep00-21swapper/00
41059302101,4sleep13810887-21diskmemload1
40650742101,6sleep13810887-21diskmemload1
40612302101,4sleep24061353-21ntpq2
40550852101,3sleep00-21swapper/00
40534252101,4sleep23810887-21diskmemload2
40393822101,5sleep24039505-21tr2
40385722100,1sleep10-21swapper/11
40297342101,4sleep24029748-21ssh2
40190222100,6sleep04019032-21ssh0
40118332100,1sleep10-21swapper/11
40100222101,4sleep23810887-21diskmemload2
40035902101,4sleep13810887-21diskmemload1
40024862101,4sleep23810887-21diskmemload2
39814482101,4sleep23982147-21memory2
39748192101,4sleep23810887-21diskmemload2
39745492101,2sleep10-21swapper/11
39657742101,3sleep23810887-21diskmemload2
39505602100,1sleep00-21swapper/00
39408642101,3sleep13810887-21diskmemload1
39164902101,3sleep23916518-21apt-get2
39131172101,4sleep13810887-21diskmemload1
39043432101,3sleep13810887-21diskmemload1
38874802101,3sleep00-21swapper/00
38813032101,3sleep20-21swapper/22
38551922101,5sleep13855424-21df_inode1
38538822101,3sleep20-21swapper/22
38352332101,2sleep10-21swapper/11
38002052100,2sleep10-21swapper/11
37991932101,4sleep03799243-21ntpq0
37733032101,5sleep13773581-21apt-key1
37655572101,4sleep13766812-21memory1
37469452101,5sleep10-21swapper/11
37418842101,4sleep03742093-21hddtemp_smartct0
36857382101,6sleep23685309-21munin-node2
36840772103,3sleep13684131-21ntpq1
4174837291,2sleep00-21swapper/00
4162290291,2sleep00-21swapper/00
4151269290,1sleep10-21swapper/11
4092226291,4sleep24092349-21ssh2
4021771291,2sleep10-21swapper/11
4004958290,5sleep04005170-21kthreadcore0
3993413291,2sleep00-21swapper/00
3964576291,3sleep13810887-21diskmemload1
3956762291,3sleep23810887-21diskmemload2
3949013291,4sleep23949024-21ssh2
3890153291,3sleep00-21swapper/00
3888144291,3sleep13810887-21diskmemload1
3852172291,3sleep10-21swapper/11
3843014291,2sleep00-21swapper/00
3842569291,2sleep10-21swapper/11
3793876291,4sleep13793909-21ntpq1
3785360291,4sleep23787054-21grep2
3773584291,3sleep23773313-21apt-get2
3772255291,4sleep03772293-21ntpq0
3758136291,4sleep23758508-21tr2
3733034291,4sleep23734786-21sed2
3727059291,5sleep13728837-21kthreadcore1
3725900292,3sleep23726024-21perl2
3708660291,4sleep23704292-21/usr/sbin/munin2
3702741291,4sleep23702790-21sshd2
3693056291,4sleep23693081-21grep2
3903954280,3sleep23810887-21diskmemload2
3817001281,2sleep13810887-21diskmemload1
3790799281,3sleep23792474-21cut2
3784504281,2sleep10-21swapper/11
3783369280,1sleep20-21swapper/22
3777939281,4sleep13777993-21modprobe1
3756314281,3sleep03756363-21ntpq0
3742166281,2sleep20-21swapper/22
3720289280,2sleep20-21swapper/22
3719338281,3sleep03719441-21/usr/sbin/munin0
3704864281,4sleep13706301-21kthreadcore1
3697017281,3sleep10-21swapper/11
3680238281,1sleep20-21swapper/22
3810686270,1sleep10-21swapper/11
3810284272,1sleep20-21swapper/22
3806443271,3sleep03807963-21kthreadcore0
3805560270,1sleep20-21swapper/22
3779916271,2sleep00-21swapper/00
3770155271,2sleep20-21swapper/22
3767876270,1sleep00-21swapper/00
3757036270,4sleep20-21swapper/22
3754090271,3sleep10-21swapper/11
3751298270,3sleep23751460-21sed2
3746923270,2sleep03746942-21apt-get0
3746458270,1sleep10-21swapper/11
3741259270,1sleep10-21swapper/11
3741259270,1sleep10-21swapper/11
3735799270,4sleep00-21swapper/00
3725325270,1sleep10-21swapper/11
3714962270,2sleep20-21swapper/22
3714657271,2sleep2802-21snmpd2
3714324270,1sleep10-21swapper/11
3702600270,4sleep10-21swapper/11
3698623270,3sleep03698124-21apt-get0
3697846270,1sleep00-21swapper/00
3805350260,4sleep10-21swapper/11
3804607260,2sleep00-21swapper/00
3795438261,2sleep20-21swapper/22
3794861260,4sleep00-21swapper/00
3781593260,3sleep10-21swapper/11
3762509260,1sleep10-21swapper/11
3740338260,1sleep20-21swapper/22
3740338260,1sleep20-21swapper/22
3730323260,3sleep00-21swapper/00
3710486261,2sleep03712184-21kthreadcore0
3789280250,1sleep00-21swapper/00
3730732250,1sleep10-21swapper/11
3715257250,1sleep10-21swapper/11
3685090251,1sleep10-21swapper/11
3763241240,1sleep20-21swapper/22
3736055240,1sleep00-21swapper/00
3736055240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional