You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 23:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Thu Jul 16, 2026 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
903136212953,71sleep30-21swapper/33
90336528755,12sleep10-21swapper/11
90338027960,12sleep20-21swapper/22
90325327656,14sleep00-21swapper/00
13944542300,3sleep10-21swapper/11
13069012229,2sleep20-21swapper/22
10658132221,8sleep11031344-21diskmemload1
90354599216,11cyclictest0-21swapper/33
90354599211,3cyclictest0-21swapper/33
903545992111,6cyclictest0-21swapper/33
90354599211,16cyclictest0-21swapper/33
90354599211,16cyclictest0-21swapper/33
13668792214,3sleep00-21swapper/00
90354599209,7cyclictest1194615-21chrt3
903545992015,4cyclictest66950irq/132-enp1s0-TxRx-33
90354599201,15cyclictest0-21swapper/33
90354599201,15cyclictest0-21swapper/33
90354599195,2cyclictest0-21swapper/33
903545991915,2cyclictest0-21swapper/33
903545991912,5cyclictest0-21swapper/33
903545991911,5cyclictest1230779-21taskset3
90354599191,15cyclictest0-21swapper/33
90354599191,15cyclictest0-21swapper/33
90354599191,15cyclictest0-21swapper/33
90354599191,15cyclictest0-21swapper/33
90354599191,14cyclictest0-21swapper/33
90354599190,2cyclictest0-21swapper/33
10106882191,6sleep244-21ksoftirqd/22
90354599189,6cyclictest942333-21turbostat3
90354599182,11cyclictest1041361-21taskset3
903545991815,1cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,14cyclictest0-21swapper/33
90354599181,13cyclictest0-21swapper/33
90354599181,13cyclictest0-21swapper/33
90354599181,13cyclictest0-21swapper/33
90354599181,13cyclictest0-21swapper/33
90354599181,13cyclictest0-21swapper/33
90354599181,13cyclictest0-21swapper/33
90354599180,15cyclictest66950irq/132-enp1s0-TxRx-33
90354599180,15cyclictest0-21swapper/33
12987032181,3sleep10-21swapper/11
12404582181,4sleep01240725-21df0
12050012180,9sleep21205012-21ssh2
11783892181,3sleep10-21swapper/11
90354599171,4cyclictest0-21swapper/33
90354599171,4cyclictest0-21swapper/33
903545991712,3cyclictest0-21swapper/33
903545991712,3cyclictest0-21swapper/33
90354599171,14cyclictest0-21swapper/33
90354599171,13cyclictest0-21swapper/33
90354599171,13cyclictest0-21swapper/33
90354599171,13cyclictest0-21swapper/33
90354599171,13cyclictest0-21swapper/33
90354599171,13cyclictest0-21swapper/33
90354599171,13cyclictest0-21swapper/33
90354599171,13cyclictest0-21swapper/33
90354599171,13cyclictest0-21swapper/33
90354599170,13cyclictest0-21swapper/33
11505222171,3sleep00-21swapper/00
10834942171,2sleep20-21swapper/22
90354599168,6cyclictest926558-21turbostat3
90354599162,10cyclictest1349904-21taskset3
90354599162,10cyclictest1349904-21taskset3
90354599160,13cyclictest0-21swapper/33
90354599160,13cyclictest0-21swapper/33
90354599160,13cyclictest0-21swapper/33
90354599160,13cyclictest0-21swapper/33
90354599160,13cyclictest0-21swapper/33
90354599160,13cyclictest0-21swapper/33
90354599160,13cyclictest0-21swapper/33
13589512160,1sleep00-21swapper/00
13267282161,4sleep11031344-21diskmemload1
12909512161,2sleep10-21swapper/11
12210472161,4sleep11031344-21diskmemload1
11436622160,3sleep10-21swapper/11
11357082162,7sleep11135775-21munin-run1
11013982160,3sleep00-21swapper/00
10978112161,4sleep01097824-21ssh0
9559302151,9sleep0956926-21memory0
903545991512,2cyclictest0-21swapper/33
903545991511,2cyclictest0-21swapper/33
90354599150,12cyclictest0-21swapper/33
90354599150,12cyclictest0-21swapper/33
13607122151,3sleep11031344-21diskmemload1
13504872150,1sleep10-21swapper/11
13504872150,1sleep10-21swapper/11
13182182150,1sleep00-21swapper/00
13089542151,3sleep01031344-21diskmemload0
11847902151,9sleep01031344-21diskmemload0
11085102151,3sleep2171rcu_preempt2
10489832151,7sleep00-21swapper/00
90354599141,2cyclictest0-21swapper/33
90354599140,11cyclictest0-21swapper/33
12723732141,9sleep01272387-21ssh0
12565712141,4sleep11256583-21ssh1
12138832141,2sleep20-21swapper/22
11873102141,2sleep21031344-21diskmemload2
11419332141,3sleep21031344-21diskmemload2
11349142140,2sleep20-21swapper/22
11330972140,2sleep00-21swapper/00
11181252141,4sleep21031344-21diskmemload2
10992042141,2sleep10-21swapper/11
10665592140,2sleep20-21swapper/22
10191042142,8sleep11019776-21/usr/sbin/munin1
9939132131,8sleep1994082-21sort1
9417192131,6sleep1941935-21users1
13431512130,9sleep11343162-21ssh1
13431512130,9sleep11343162-21ssh1
12900892130,2sleep20-21swapper/22
12738702131,3sleep21273884-21ssh2
12319252131,8sleep01232018-21entropy0
12114112130,1sleep00-21swapper/00
11941042131,7sleep21031344-21diskmemload2
11667222131,8sleep11166984-21grep1
11667222131,8sleep11166984-21grep1
11020142131,8sleep11102973-21grep1
10460002131,7sleep11031344-21diskmemload1
9742582121,7sleep1973981-21/usr/sbin/munin1
9601492121,7sleep00-21swapper/00
9365122121,6sleep2936669-21users2
9307772121,7sleep0930869-21/usr/sbin/munin0
13336862121,6sleep01031344-21diskmemload0
13083522121,6sleep11031344-21diskmemload1
12577532121,7sleep01257763-21ssh0
12566122121,6sleep01031344-21diskmemload0
12490162121,5sleep21031344-21diskmemload2
12290692121,5sleep00-21swapper/00
11975882121,6sleep11199089-21cut1
11262832121,3sleep10-21swapper/11
11152512121,5sleep10-21swapper/11
10409822122,5sleep21041359-21cat2
10051352120,1sleep20-21swapper/22
9999042111,6sleep21000049-21latency_hist2
9849992111,6sleep0985359-21cat0
9163532112,4sleep0916356-21apt-get0
13939652111,4sleep21031344-21diskmemload2
13790052111,4sleep20-21swapper/22
13684852110,1sleep10-21swapper/11
13586232111,3sleep20-21swapper/22
13324372111,4sleep21031344-21diskmemload2
13263132110,1sleep00-21swapper/00
13234112111,3sleep20-21swapper/22
13174592111,4sleep10-21swapper/11
13168542110,2sleep21316865-21ssh2
12992772111,4sleep21031344-21diskmemload2
12649162111,3sleep10-21swapper/11
12627832111,4sleep21031344-21diskmemload2
12366682111,4sleep11031344-21diskmemload1
12310732110,7sleep11231075-21ssh1
11949902111,6sleep01195005-21ssh0
11750132111,5sleep01175077-21ntp_states0
11583052111,5sleep01031344-21diskmemload0
11583052111,5sleep01031344-21diskmemload0
11538652111,5sleep11154088-21smartctl1
11538652111,5sleep11154088-21smartctl1
10983492110,1sleep20-21swapper/22
10806182111,3sleep10-21swapper/11
10665192110,2sleep00-21swapper/00
10618402111,4sleep21031344-21diskmemload2
10543592111,5sleep21054384-21apt-get2
9833032100,6sleep0983424-21gzip0
9694552101,5sleep0971036-21cut0
9425892101,4sleep0942608-21apt-get0
9376452100,6sleep0937715-21grep0
9322692101,4sleep0932356-21df_inode0
9215612101,4sleep0921580-21apt-get0
9204922101,5sleep2920649-21cat2
9091382101,5sleep2910512-21cat2
13942402101,4sleep01031344-21diskmemload0
13794722101,3sleep01379100-21apt-get0
13763082100,6sleep01376319-21ssh0
13685522101,2sleep20-21swapper/22
13512422100,1sleep00-21swapper/00
13512422100,1sleep00-21swapper/00
13447662102,3sleep21344772-21apt-get2
13447662102,3sleep21344772-21apt-get2
13359622101,4sleep21031344-21diskmemload2
13359622101,4sleep21031344-21diskmemload2
12981312100,1sleep00-21swapper/00
12902182101,3sleep01031344-21diskmemload0
12832992101,3sleep20-21swapper/22
12745722101,5sleep11274583-21ssh1
12485422100,2sleep10-21swapper/11
12316872101,3sleep21231708-21apt-get2
12286282101,2sleep20-21swapper/22
12215062101,4sleep01031344-21diskmemload0
12105922101,2sleep10-21swapper/11
12042152101,3sleep00-21swapper/00
11938832101,5sleep10-21swapper/11
11688142100,1sleep00-21swapper/00
11688142100,1sleep00-21swapper/00
11534342101,5sleep20-21swapper/22
11534342101,5sleep20-21swapper/22
11491952100,2sleep10-21swapper/11
10920172100,1sleep00-21swapper/00
10897422101,3sleep21031344-21diskmemload2
10649692101,4sleep01031344-21diskmemload0
10496542101,4sleep11051188-21cut1
10454252101,3sleep00-21swapper/00
10399032101,5sleep00-21swapper/00
10380442101,3sleep20-21swapper/22
10211422101,4sleep11021144-21apt-get1
10208262101,2sleep20-21swapper/22
995101290,4sleep0995236-21diskstats0
984128290,1sleep10-21swapper/11
958813291,4sleep2958875-21irqstats2
948392291,4sleep2949213-21cut2
928486291,4sleep2930207-21grep2
927407291,4sleep1927442-21kthreadcore1
926477290,2sleep20-21swapper/22
916327290,1sleep10-21swapper/11
908376290,2sleep10-21swapper/11
903737291,3sleep00-21swapper/00
1386074291,3sleep21031344-21diskmemload2
1384364291,4sleep11384367-21ssh1
1377800291,3sleep11031344-21diskmemload1
1341352291,3sleep01031344-21diskmemload0
1341352291,3sleep01031344-21diskmemload0
1334866291,2sleep10-21swapper/11
1281345291,2sleep00-21swapper/00
1275144291,3sleep11275168-21apt-get1
1247468290,3sleep20-21swapper/22
1214906291,4sleep21031344-21diskmemload2
1186322292,4sleep1331ktimers/11
1175815291,4sleep21175829-21ssh2
1162240291,3sleep21162448-21cstates2
1162240291,3sleep21162448-21cstates2
1144809291,3sleep21144844-21ssh2
1136103290,3sleep01031344-21diskmemload0
1124703291,2sleep20-21swapper/22
1123812291,3sleep01031344-21diskmemload0
1091874290,1sleep10-21swapper/11
1075173290,1sleep00-21swapper/00
1074441291,2sleep10-21swapper/11
1025724290,1sleep00-21swapper/00
1005437290,3sleep00-21swapper/00
991023281,3sleep0992677-21kthreadcore0
989708281,2sleep2989727-21apt-get2
979258280,3sleep20-21swapper/22
973980281,3sleep2973982-21/usr/sbin/munin2
973425281,3sleep2973427-21modprobe2
963558280,4sleep20-21swapper/22
963536280,2sleep10-21swapper/11
958296281,4sleep10-21swapper/11
953603281,3sleep2954279-21cat2
1110036280,3sleep01110033-21apt-get0
1031714280,2sleep10-21swapper/11
1026654281,3sleep21026864-21smartctl2
1017402281,3sleep01019102-21cut0
1010666281,2sleep11010685-21apt-get1
1006010282,2sleep11007365-21kthreadcore1
998134271,3sleep1998856-21missed_timers1
989060270,1sleep20-21swapper/22
974553271,3sleep00-21swapper/00
957721270,1sleep10-21swapper/11
952450270,1sleep10-21swapper/11
952041271,3sleep0952254-21sed0
915882270,1sleep10-21swapper/11
911643271,3sleep0912600-21kthreadcore0
908530270,1sleep20-21swapper/22
1026579271,3sleep10-21swapper/11
1026381270,3sleep01026578-21expr0
1015353270,1sleep00-21swapper/00
1010042270,1sleep20-21swapper/22
1004847270,1sleep10-21swapper/11
1000192271,2sleep00-21swapper/00
989093260,1sleep10-21swapper/11
968259260,1sleep00-21swapper/00
944219261,2sleep20-21swapper/22
943160260,2sleep10-21swapper/11
937907261,2sleep2937961-21kthreadcore2
932068260,1sleep10-21swapper/11
926177260,1sleep10-21swapper/11
1025750260,4sleep20-21swapper/22
969045250,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional