You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 22:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Thu Mar 19, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
376522721900,4sleep1321rcuc/11
3578205213159,67sleep30-21swapper/33
357817529564,12sleep20-21swapper/22
357806828554,12sleep10-21swapper/11
357295428159,14sleep00-21swapper/00
38461242410,2sleep10-21swapper/11
357832599221,18cyclictest0-21swapper/33
357832599220,20cyclictest0-21swapper/33
357832599211,16cyclictest3822805-21taskset3
357832599211,16cyclictest0-21swapper/33
357832599211,15cyclictest66950irq/132-enp1s0-TxRx-33
39679382201,8sleep23968002-21ntpq2
37316092201,2sleep20-21swapper/22
36084302201,14sleep13610134-21cut1
357832599203,12cyclictest4051623-21taskset3
38929362191,12sleep13708677-21diskmemload1
357832599192,5cyclictest0-21swapper/33
3578325991915,2cyclictest0-21swapper/33
3578325991912,5cyclictest66950irq/132-enp1s0-TxRx-33
357832599191,15cyclictest0-21swapper/33
357832599191,15cyclictest0-21swapper/33
357832599191,14cyclictest0-21swapper/33
357832599191,14cyclictest0-21swapper/33
357832599191,14cyclictest0-21swapper/33
357832599191,14cyclictest0-21swapper/33
3578325991910,5cyclictest36180982sleep33
357832599190,16cyclictest0-21swapper/33
36594952180,8sleep13659636-21ntpq1
357832599189,5cyclictest39724592sleep33
357832599188,6cyclictest66950irq/132-enp1s0-TxRx-33
357832599188,6cyclictest3708683-21turbostat3
357832599181,15cyclictest0-21swapper/33
357832599181,14cyclictest0-21swapper/33
357832599181,14cyclictest0-21swapper/33
357832599181,14cyclictest0-21swapper/33
357832599181,14cyclictest0-21swapper/33
357832599181,14cyclictest0-21swapper/33
357832599181,14cyclictest0-21swapper/33
357832599181,14cyclictest0-21swapper/33
357832599181,14cyclictest0-21swapper/33
357832599181,13cyclictest0-21swapper/33
357832599181,13cyclictest0-21swapper/33
357832599181,13cyclictest0-21swapper/33
357832599181,13cyclictest0-21swapper/33
357832599181,13cyclictest0-21swapper/33
357832599181,13cyclictest0-21swapper/33
357832599180,16cyclictest0-21swapper/33
357832599180,16cyclictest0-21swapper/33
37681592171,11sleep03708677-21diskmemload0
37233612171,3sleep20-21swapper/22
36772362171,10sleep23677445-21hddtemp_smartct2
357832599171,14cyclictest0-21swapper/33
357832599171,13cyclictest0-21swapper/33
357832599171,13cyclictest0-21swapper/33
357832599171,13cyclictest0-21swapper/33
357832599171,13cyclictest0-21swapper/33
357832599171,13cyclictest0-21swapper/33
357832599171,13cyclictest0-21swapper/33
357832599170,14cyclictest0-21swapper/33
357832599170,14cyclictest0-21swapper/33
357832599170,13cyclictest0-21swapper/33
40313282161,4sleep13708677-21diskmemload1
40064102160,4sleep10-21swapper/11
39807472161,4sleep03708677-21diskmemload0
39540042160,1sleep20-21swapper/22
39367282161,2sleep10-21swapper/11
39255052161,2sleep20-21swapper/22
38413552161,2sleep00-21swapper/00
38188422163,7sleep23818972-21/usr/sbin/munin2
37521152161,3sleep20-21swapper/22
37322982160,1sleep00-21swapper/00
357832599161,12cyclictest0-21swapper/33
357832599161,12cyclictest0-21swapper/33
357832599161,12cyclictest0-21swapper/33
357832599160,14cyclictest0-21swapper/33
357832599160,13cyclictest0-21swapper/33
40032152151,9sleep24003274-21ntpq2
39893472151,6sleep03989357-21ssh0
39858352151,8sleep23986090-21users2
38831452151,3sleep13708677-21diskmemload1
37872882151,3sleep13708677-21diskmemload1
37593752151,2sleep20-21swapper/22
37590022151,2sleep00-21swapper/00
357832599153,7cyclictest0-21swapper/33
3578325991512,1cyclictest0-21swapper/33
357832599150,13cyclictest0-21swapper/33
357832599150,12cyclictest0-21swapper/33
357832599150,12cyclictest0-21swapper/33
357832599150,12cyclictest0-21swapper/33
357832599150,12cyclictest0-21swapper/33
39169372141,4sleep23916951-21ssh2
39088042141,2sleep20-21swapper/22
38645822141,3sleep23708677-21diskmemload2
38387732141,2sleep03708677-21diskmemload0
38291292141,2sleep20-21swapper/22
38215572141,3sleep13708677-21diskmemload1
37935412141,3sleep13708677-21diskmemload1
37882642141,8sleep00-21swapper/00
37779252141,2sleep13708677-21diskmemload1
37400022141,6sleep13740197-21users1
35820852141,9sleep03582542-21cut0
357832599143,7cyclictest0-21swapper/33
357832599142,8cyclictest3692950-21taskset3
3578325991410,2cyclictest0-21swapper/33
357832599140,2cyclictest0-21swapper/33
357832599140,2cyclictest0-21swapper/33
357832599140,12cyclictest0-21swapper/33
357832599140,12cyclictest0-21swapper/33
357832599140,12cyclictest0-21swapper/33
40165032131,7sleep13708677-21diskmemload1
39791852131,7sleep13708677-21diskmemload1
39549732131,7sleep23955105-21grep2
39452902131,8sleep23945301-21ssh2
38811472131,8sleep03881160-21sh0
38731982131,6sleep20-21swapper/22
38135872131,8sleep03813601-21ssh0
37345952131,7sleep13708677-21diskmemload1
37221832131,8sleep03722271-21ssh0
36543372131,6sleep03654383-21ntpq0
40767002121,7sleep24076713-21ssh2
40592722121,6sleep13708677-21diskmemload1
40451572121,7sleep04046792-21sed0
40322142121,7sleep24032217-21ssh2
40226632121,5sleep20-21swapper/22
40147192121,3sleep14014730-21ssh1
39702222121,7sleep13970233-21ssh1
39590542121,6sleep13910873-21kworker/1:0+events1
39380642121,7sleep13939453-21grep1
39016102121,6sleep13708677-21diskmemload1
38924202121,2sleep00-21swapper/00
38747102121,8sleep13874725-21ssh1
38644582121,5sleep00-21swapper/00
38638132121,6sleep13708677-21diskmemload1
38320362122,5sleep23832059-21apt-get2
37854482121,6sleep03785460-21ssh0
37168212121,7sleep03716831-21ssh0
36827892121,7sleep23683046-21cut2
36766612121,7sleep03676732-21systemd-run0
36080492121,7sleep03608429-21cut0
36027872121,7sleep03604446-21cut0
40691282110,7sleep04069392-21expr0
40654922111,3sleep00-21swapper/00
40502592111,4sleep13708677-21diskmemload1
40399052111,3sleep00-21swapper/00
40333472111,3sleep00-21swapper/00
40234492111,7sleep04023459-21ssh0
40127522111,6sleep24012763-21ssh2
40043812111,7sleep04004395-21ssh0
39864792111,4sleep13708677-21diskmemload1
39552652111,5sleep03955485-21hddtemp_smartct0
39261432110,1sleep10-21swapper/11
38057772111,5sleep23806046-21df_inode2
38031882111,6sleep03803203-21sh0
38012582111,7sleep10-21swapper/11
37659342111,5sleep23766037-21ntpq2
37587152111,5sleep13708677-21diskmemload1
37521432111,5sleep03708677-21diskmemload0
37238642111,4sleep13708677-21diskmemload1
37076872111,5sleep03707737-21ntpq0
37032362116,2sleep10-21swapper/11
36932532110,7sleep13693461-21if_enp1s01
36713552111,5sleep13671404-21munin-run1
36170502111,5sleep23617098-21ntpq2
35965962111,5sleep03596854-21df_inode0
40745042102,3sleep13708677-21diskmemload1
40657122101,3sleep23708677-21diskmemload2
40596172101,3sleep00-21swapper/00
40415762101,2sleep20-21swapper/22
40382352101,4sleep14038295-21ssh1
40077922101,4sleep04007816-21apt-get0
39948762101,3sleep00-21swapper/00
39905202101,5sleep23991026-21ssh2
39729712101,4sleep23973053-21irqstats2
39699092100,1sleep00-21swapper/00
39518202100,1sleep10-21swapper/11
39504082101,4sleep03708677-21diskmemload0
39286862101,4sleep23928715-21apt-get2
39265322101,4sleep03708677-21diskmemload0
39176852101,3sleep10-21swapper/11
39031272102,4sleep10-21swapper/11
38995432101,6sleep23899554-21diskmemload2
38990232101,6sleep03899036-21ssh0
38903612101,3sleep20-21swapper/22
38822972101,4sleep23708677-21diskmemload2
38564962101,4sleep13708677-21diskmemload1
38564962101,4sleep13708677-21diskmemload1
38554362101,3sleep23708677-21diskmemload2
38554362101,3sleep23708677-21diskmemload2
38552102100,3sleep00-21swapper/00
38552102100,3sleep00-21swapper/00
38477112101,3sleep23708677-21diskmemload2
38398642101,3sleep13708677-21diskmemload1
38289962101,3sleep00-21swapper/00
38284532100,3sleep10-21swapper/11
38205342100,6sleep03820545-21ssh0
38140412101,3sleep10-21swapper/11
38037332101,2sleep20-21swapper/22
37836452101,2sleep20-21swapper/22
37785232100,2sleep03778537-21ssh0
37502332101,4sleep13708677-21diskmemload1
37432802101,2sleep00-21swapper/00
36277252101,4sleep13627771-21ntpq1
36224342101,5sleep03622488-21ntpq0
36126202101,4sleep23612625-40gst-plugin-scan2
36063132101,5sleep13606366-21ntp_states1
35858962104,3sleep1331ktimers/11
4065342290,1sleep10-21swapper/11
4056784291,2sleep23708677-21diskmemload2
4049164291,3sleep24049166-21ssh2
3989864291,2sleep10-21swapper/11
3937984291,2sleep00-21swapper/00
3936921290,2sleep00-21swapper/00
3916665291,4sleep03916679-21ssh0
3902433291,3sleep00-21swapper/00
3873227291,2sleep00-21swapper/00
3788154292,3sleep23788187-21apt-get2
3774671290,1sleep20-21swapper/22
3742543290,3sleep20-21swapper/22
3714684290,3sleep23708677-21diskmemload2
3713511291,3sleep13708677-21diskmemload1
3699731291,4sleep03701518-21tr0
3695601291,4sleep03696833-21memory0
3681012291,3sleep03681060-21ntpq0
3672184291,4sleep13673062-21kthreadcore1
3670405291,4sleep23670464-21ld2
3639663291,3sleep23639682-21apt-get2
3623141290,1sleep10-21swapper/11
3618707290,1sleep20-21swapper/22
3613298291,4sleep13613507-21hddtemp_smartct1
3599773291,4sleep13600533-21cat1
3591246291,3sleep23591265-21apt-get2
3585991291,4sleep23586162-21diskstats2
3692928281,2sleep20-21swapper/22
3687148280,1sleep10-21swapper/11
3665995280,1sleep10-21swapper/11
3660916280,2sleep20-21swapper/22
3660291280,1sleep20-21swapper/22
3655178280,1sleep10-21swapper/11
3650574281,3sleep23650788-21smartctl2
3649647280,1sleep10-21swapper/11
3644351280,1sleep10-21swapper/11
3639199280,1sleep00-21swapper/00
3639161282,4sleep2431ktimers/22
3629310280,1sleep20-21swapper/22
3614997281,3sleep03616837-21grep0
3601641281,3sleep20-21swapper/22
3708408270,1sleep10-21swapper/11
3692564270,1sleep00-21swapper/00
3692505270,1sleep20-21swapper/22
3688165271,3sleep13689039-21kthreadcore1
3686646271,3sleep00-21swapper/00
3678915271,2sleep10-21swapper/11
3676537270,1sleep20-21swapper/22
3671077270,1sleep00-21swapper/00
3660372270,1sleep00-21swapper/00
3649826270,1sleep00-21swapper/00
3649755270,1sleep20-21swapper/22
3633360270,1sleep10-21swapper/11
3596095270,1sleep10-21swapper/11
3595953271,1sleep00-21swapper/00
3579072272,2sleep20-21swapper/22
3638835260,2sleep10-21swapper/11
3633080260,2sleep03633134-21ld0
3628619260,4sleep00-21swapper/00
3625302261,2sleep20-21swapper/22
3606634260,2sleep20-21swapper/22
3590853260,1sleep00-21swapper/00
3583304260,1sleep10-21swapper/11
3707503250,1sleep20-21swapper/22
3703020250,3sleep20-21swapper/22
3665866250,1sleep00-21swapper/00
3642632250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional