You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-09 - 20:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Tue Jun 09, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1096859211558,51sleep30-21swapper/33
110168029262,11sleep10-21swapper/11
110184828251,10sleep20-21swapper/22
110170527754,7sleep00-21swapper/00
14019072370,2sleep00-21swapper/00
13171182280,3sleep20-21swapper/22
1101998992320,1cyclictest0-21swapper/33
110199899231,18cyclictest1168376-21taskset3
110199899211,16cyclictest66950irq/132-enp1s0-TxRx-33
110199899211,16cyclictest0-21swapper/33
110199899211,16cyclictest0-21swapper/33
110199899211,16cyclictest0-21swapper/33
1101998992017,1cyclictest0-21swapper/33
1101998992016,2cyclictest0-21swapper/33
110199899201,15cyclictest0-21swapper/33
110199899201,15cyclictest0-21swapper/33
110199899201,15cyclictest0-21swapper/33
110199899201,15cyclictest0-21swapper/33
110199899201,14cyclictest0-21swapper/33
1101998991916,1cyclictest0-21swapper/33
1101998991912,4cyclictest0-21swapper/33
110199899191,15cyclictest13898102sleep33
110199899191,15cyclictest0-21swapper/33
110199899191,15cyclictest0-21swapper/33
110199899191,14cyclictest0-21swapper/33
110199899191,14cyclictest0-21swapper/33
110199899191,14cyclictest0-21swapper/33
110199899191,13cyclictest0-21swapper/33
110199899190,17cyclictest0-21swapper/33
110199899190,16cyclictest66950irq/132-enp1s0-TxRx-33
15182222181,3sleep20-21swapper/22
110199899181,6cyclictest1454570-21taskset3
1101998991816,1cyclictest0-21swapper/33
1101998991814,2cyclictest1449928-21taskset3
1101998991812,3cyclictest0-21swapper/33
110199899181,14cyclictest0-21swapper/33
110199899181,14cyclictest0-21swapper/33
110199899181,14cyclictest0-21swapper/33
110199899181,14cyclictest0-21swapper/33
110199899181,14cyclictest0-21swapper/33
110199899181,13cyclictest0-21swapper/33
110199899181,13cyclictest0-21swapper/33
110199899180,14cyclictest0-21swapper/33
15533372171,5sleep21553364-21diskstats2
15163822171,1sleep00-21swapper/00
14745742171,3sleep00-21swapper/00
14188812171,5sleep21418938-21fschecks_count2
13858672171,4sleep21386271-21grep2
12338572171,11sleep21234632-21cut2
110199899172,10cyclictest1418527-21taskset3
1101998991714,1cyclictest0-21swapper/33
1101998991712,3cyclictest0-21swapper/33
110199899171,13cyclictest0-21swapper/33
110199899171,13cyclictest0-21swapper/33
110199899171,13cyclictest0-21swapper/33
110199899171,12cyclictest0-21swapper/33
110199899171,12cyclictest0-21swapper/33
1101998991710,5cyclictest0-21swapper/33
110199899170,14cyclictest0-21swapper/33
110199899170,14cyclictest0-21swapper/33
110199899170,13cyclictest0-21swapper/33
14069142161,8sleep10-21swapper/11
12932082161,3sleep20-21swapper/22
12459272161,5sleep21245992-21ntp_states2
12424902161,7sleep11244075-21grep1
11793632161,10sleep01179581-21df_inode0
110199899161,4cyclictest0-21swapper/33
1101998991612,3cyclictest0-21swapper/33
110199899160,14cyclictest0-21swapper/33
110199899160,13cyclictest0-21swapper/33
110199899160,13cyclictest0-21swapper/33
110199899160,13cyclictest0-21swapper/33
110199899160,13cyclictest0-21swapper/33
110199899160,13cyclictest0-21swapper/33
15409692151,10sleep11541238-21head1
14401672151,3sleep20-21swapper/22
14401672151,3sleep20-21swapper/22
13745052151,2sleep10-21swapper/11
13407832151,9sleep21232874-21diskmemload2
12647042151,3sleep10-21swapper/11
11677212151,4sleep11167996-21unixbench_singl1
110199899159,4cyclictest0-21swapper/33
110199899150,13cyclictest0-21swapper/33
110199899150,13cyclictest0-21swapper/33
110199899150,12cyclictest0-21swapper/33
110199899150,12cyclictest0-21swapper/33
15365052140,2sleep00-21swapper/00
15249522142,8sleep0802-21snmpd0
15029502142,8sleep21232874-21diskmemload2
13748262141,8sleep01232874-21diskmemload0
13557082141,7sleep01355758-21ntpq0
13413292141,3sleep01232874-21diskmemload0
13327182141,4sleep21332728-21ssh2
13226352141,9sleep01305143-21kworker/u16:2+rpciod0
13061172141,8sleep01232874-21diskmemload0
12733442141,9sleep11273359-21ssh1
110199899141,12cyclictest0-21swapper/33
110199899141,10cyclictest1227575-21turbostat3
110199899140,2cyclictest0-21swapper/33
110199899140,12cyclictest0-21swapper/33
15848292131,7sleep21232874-21diskmemload2
15766542131,4sleep11576665-21ssh1
15529642131,7sleep11232874-21diskmemload1
15503752131,8sleep11550385-21ssh1
15348012131,8sleep11534812-21ssh1
15348012131,8sleep11534812-21ssh1
14990112131,8sleep11232874-21diskmemload1
14981302131,7sleep01498191-21ntpq0
14591082131,2sleep20-21swapper/22
14572792131,8sleep01457293-21ssh0
14332482131,2sleep10-21swapper/11
14165822131,8sleep21416596-21ssh2
13410052131,8sleep11341019-21ssh1
13157852131,8sleep11315797-21ssh1
12984102131,3sleep11298421-21ssh1
12918882131,8sleep11291898-21ssh1
12830632131,8sleep21283068-21ssh2
12177372131,8sleep11219331-21tr1
12020092131,8sleep01203719-21tr0
11953092131,8sleep11195419-21sed1
11739892131,6sleep21174241-21df2
110199899131,8cyclictest1158017-21chrt3
110199899130,2cyclictest0-21swapper/33
15234072121,7sleep21232874-21diskmemload2
15186432120,2sleep10-21swapper/11
15089142121,6sleep01232874-21diskmemload0
14913652121,7sleep01232874-21diskmemload0
14895032121,6sleep11489653-21ntp_192.168.1141
14771312122,5sleep11477395-21df1
14770972122,5sleep01477121-21apt-get0
14676942121,5sleep20-21swapper/22
14270062121,5sleep20-21swapper/22
13661472121,7sleep21366158-21ssh2
13430992121,5sleep11232874-21diskmemload1
13341232121,4sleep00-21swapper/00
13085692120,8sleep11232874-21diskmemload1
12806292120,7sleep11232874-21diskmemload1
12639182121,3sleep01263929-21ssh0
12566532121,7sleep01256664-21ssh0
11359712121,7sleep01135949-31find0
11021922121,7sleep11102429-21cstates1
110199899120,1cyclictest0-21swapper/33
15850992111,4sleep11232874-21diskmemload1
15658662111,6sleep11566082-21sed1
15353092110,2sleep00-21swapper/00
15353092110,2sleep00-21swapper/00
15005782111,4sleep21232874-21diskmemload2
14841922110,5sleep20-21swapper/22
14730122111,5sleep21473062-21ntpq2
14486262111,7sleep01232874-21diskmemload0
14477852111,5sleep21447831-21ntpq2
14244252111,5sleep01232874-21diskmemload0
14178132111,6sleep01417823-21ssh0
14169092111,6sleep11232874-21diskmemload1
13912332111,6sleep11391243-21ssh1
13754182111,5sleep21232874-21diskmemload2
13675142111,5sleep01232874-21diskmemload0
13643932111,5sleep11364451-21ntpq1
13584532111,4sleep21232874-21diskmemload2
13327572110,7sleep11332767-21ssh1
13218722111,5sleep21321906-21ntpq2
13093322111,6sleep01309493-21cstates0
12723402110,1sleep20-21swapper/22
12659322111,4sleep20-21swapper/22
12559382111,4sleep10-21swapper/11
12230172111,3sleep2421rcuc/22
12118292111,5sleep21212084-21df2
12104152113,3sleep21210528-21ntpq2
11623572111,7sleep01162460-21timerandwakeup0
11526492111,4sleep11152668-21apt-get1
11360302111,6sleep11135949-31find1
15747522101,3sleep01232874-21diskmemload0
15618702101,6sleep21561337-21/usr/sbin/munin2
15580082101,4sleep01232874-21diskmemload0
15512522101,3sleep00-21swapper/00
15496872101,4sleep21232874-21diskmemload2
15321152101,5sleep21532174-21ntpq2
15321152101,5sleep21532174-21ntpq2
15246772101,3sleep10-21swapper/11
15078842101,4sleep11232874-21diskmemload1
14737962101,4sleep11232874-21diskmemload1
14674092101,3sleep01232874-21diskmemload0
14647942101,6sleep11464853-21open_inodes1
14596612101,4sleep11232874-21diskmemload1
14509122101,5sleep11450923-21ssh1
14412132101,4sleep11441224-21ssh1
14412132101,4sleep11441224-21ssh1
14403942101,3sleep00-21swapper/00
14403942101,3sleep00-21swapper/00
14082072101,4sleep21232874-21diskmemload2
13990642101,3sleep20-21swapper/22
13831932101,2sleep00-21swapper/00
13823092101,4sleep21232874-21diskmemload2
13588642101,5sleep11358874-21ssh1
13226482101,3sleep10-21swapper/11
13059992101,4sleep21232874-21diskmemload2
12997582101,4sleep01232874-21diskmemload0
12891032101,3sleep01232874-21diskmemload0
12753122101,4sleep00-21swapper/00
12570702101,4sleep21232874-21diskmemload2
12473592101,3sleep00-21swapper/00
12372612101,4sleep11237412-21ntp_192.168.1141
12227652101,4sleep01222868-21perl0
11983212101,4sleep11199416-21memory1
11943272101,4sleep21194377-21ntpq2
11742672100,1sleep00-21swapper/00
11513642101,4sleep11151407-21ntpq1
11419942101,4sleep21142013-21apt-get2
11189512101,4sleep01119008-21ntp_states0
11153512101,4sleep21115422-21irqstats2
1573753290,4sleep21573897-21ntpq2
1561305291,3sleep01232874-21diskmemload0
1541371291,4sleep21541378-21ssh2
1491831290,1sleep20-21swapper/22
1432122291,4sleep01432136-21ssh0
1425907291,3sleep11425917-21ssh1
1398845291,2sleep10-21swapper/11
1398597290,1sleep00-21swapper/00
1391416290,3sleep00-21swapper/00
1384014291,2sleep10-21swapper/11
1349840291,3sleep01232874-21diskmemload0
1348416291,3sleep21232874-21diskmemload2
1271097291,3sleep01271190-21ntpq0
1233134291,3sleep00-21swapper/00
1232859290,2sleep20-21swapper/22
1219332291,4sleep21221032-21cat2
1212356291,4sleep11212900-21tr1
1189981291,4sleep21190172-21sed2
1184897291,3sleep10-21swapper/11
1183265291,4sleep21183344-21ntp_192.168.1142
1140795291,5sleep11140844-21open_files1
1130723291,3sleep21130564-21apt-get2
1109786291,4sleep21109929-21grep2
1578171281,2sleep00-21swapper/00
1284476281,3sleep21284556-21ssh2
1231993281,3sleep11232133-21grep1
1217134281,2sleep00-21swapper/00
1196597281,3sleep01198320-21cut0
1173967281,2sleep10-21swapper/11
1141492280,1sleep20-21swapper/22
1125225281,3sleep21125411-21latency_hist2
1120163281,3sleep10-21swapper/11
1205681270,1sleep10-21swapper/11
1205603270,1sleep20-21swapper/22
1190544271,3sleep00-21swapper/00
1189693270,1sleep00-21swapper/00
1168636270,2sleep20-21swapper/22
1168230270,1sleep00-21swapper/00
1156840271,3sleep00-21swapper/00
1152075270,1sleep20-21swapper/22
1146890270,1sleep00-21swapper/00
1140741271,2sleep00-21swapper/00
1130224270,1sleep20-21swapper/22
1125024270,1sleep00-21swapper/00
1114328270,1sleep10-21swapper/11
1102170270,2sleep00-21swapper/00
1232830260,1sleep00-21swapper/00
1227244260,4sleep10-21swapper/11
1215713260,3sleep00-21swapper/00
1208065261,2sleep00-21swapper/00
1179582261,2sleep10-21swapper/11
1168814261,2sleep00-21swapper/00
1157995260,2sleep20-21swapper/22
1153937261,2sleep20-21swapper/22
1151226261,2sleep01151362-21netstat0
1146175261,2sleep10-21swapper/11
1126057261,2sleep10-21swapper/11
1118507260,2sleep11118842-21strings1
1114345260,1sleep00-21swapper/00
1107106260,1sleep20-21swapper/22
1211509250,1sleep10-21swapper/11
1168929250,1sleep10-21swapper/11
1167998250,1sleep20-21swapper/22
1162779250,3sleep10-21swapper/11
1195420240,1sleep20-21swapper/22
1125486240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional