You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-25 - 21:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Wed Mar 25, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
143136021160,2sleep30-21swapper/33
1185206211655,56sleep30-21swapper/33
12197792860,4sleep2421rcuc/22
118507328365,12sleep20-21swapper/22
118492428164,11sleep10-21swapper/11
118494628061,13sleep00-21swapper/00
118534299241,3cyclictest0-21swapper/33
118534299233,16cyclictest1604309-21turbostat3
118534299231,19cyclictest0-21swapper/33
118534299231,15cyclictest0-21swapper/33
1185342992211,7cyclictest1383862-21taskset3
118534299220,12cyclictest0-21swapper/33
118534299214,13cyclictest66950irq/132-enp1s0-TxRx-33
1185342992118,1cyclictest0-21swapper/33
118534299211,16cyclictest66950irq/132-enp1s0-TxRx-33
118534299211,16cyclictest66950irq/132-enp1s0-TxRx-33
118534299211,13cyclictest0-21swapper/33
118534299210,12cyclictest0-21swapper/33
16802492201,14sleep11315973-21diskmemload1
14719652201,4sleep21315973-21diskmemload2
118534299209,3cyclictest0-21swapper/33
118534299208,9cyclictest1190631-21turbostat3
118534299204,13cyclictest66950irq/132-enp1s0-TxRx-33
1185342992017,1cyclictest0-21swapper/33
118534299201,3cyclictest0-21swapper/33
118534299201,3cyclictest0-21swapper/33
118534299201,3cyclictest0-21swapper/33
118534299201,3cyclictest0-21swapper/33
1185342992013,6cyclictest0-21swapper/33
1185342992013,4cyclictest66950irq/132-enp1s0-TxRx-33
118534299201,16cyclictest0-21swapper/33
118534299200,11cyclictest0-21swapper/33
118534299200,10cyclictest0-21swapper/33
16722682191,3sleep20-21swapper/22
16650282193,2sleep2421rcuc/22
15406492191,2sleep20-21swapper/22
13746982191,4sleep11315973-21diskmemload1
118534299193,13cyclictest1399416-21taskset3
118534299192,14cyclictest0-21swapper/33
118534299192,12cyclictest1347521-21taskset3
1185342991914,3cyclictest0-21swapper/33
1185342991914,3cyclictest0-21swapper/33
118534299191,3cyclictest0-21swapper/33
1185342991912,4cyclictest0-21swapper/33
118534299191,15cyclictest0-21swapper/33
118534299191,15cyclictest0-21swapper/33
118534299191,15cyclictest0-21swapper/33
118534299191,14cyclictest0-21swapper/33
118534299190,16cyclictest0-21swapper/33
118534299190,11cyclictest0-21swapper/33
14720562181,2sleep10-21swapper/11
14376552181,8sleep11315973-21diskmemload1
118534299182,10cyclictest1386864-21taskset3
118534299181,5cyclictest0-21swapper/33
118534299181,5cyclictest0-21swapper/33
118534299181,3cyclictest0-21swapper/33
118534299181,2cyclictest0-21swapper/33
118534299181,14cyclictest0-21swapper/33
118534299181,14cyclictest0-21swapper/33
118534299181,14cyclictest0-21swapper/33
118534299181,14cyclictest0-21swapper/33
118534299181,13cyclictest1289362-21turbostat3
118534299181,13cyclictest1283329-21taskset3
118534299181,13cyclictest0-21swapper/33
118534299180,15cyclictest0-21swapper/33
16384342171,3sleep10-21swapper/11
15575092171,3sleep10-21swapper/11
15487702171,3sleep10-21swapper/11
15069592171,10sleep11315973-21diskmemload1
13083492171,11sleep01309472-21memory0
12936382170,2sleep10-21swapper/11
118534299172,11cyclictest0-21swapper/33
118534299171,6cyclictest1294667-21turbostat3
118534299171,6cyclictest1273051-21turbostat3
118534299171,4cyclictest1197988-21turbostat3
118534299171,2cyclictest0-21swapper/33
118534299170,9cyclictest0-21swapper/33
118534299170,13cyclictest0-21swapper/33
118534299170,10cyclictest0-21swapper/33
15864712161,4sleep11315973-21diskmemload1
15673862160,1sleep20-21swapper/22
15656152161,2sleep10-21swapper/11
14950762160,1sleep00-21swapper/00
14527492161,3sleep20-21swapper/22
13339602161,2sleep20-21swapper/22
13244712161,6sleep01324482-21ssh0
118534299161,5cyclictest1310647-21turbostat3
118534299161,3cyclictest1224812-21turbostat3
1185342991613,2cyclictest0-21swapper/33
118534299160,14cyclictest0-21swapper/33
16458302150,1sleep20-21swapper/22
16115302151,2sleep20-21swapper/22
15939572151,10sleep11593968-21ssh1
15727172151,9sleep11573819-21memory1
15542222151,10sleep20-21swapper/22
13868512151,8sleep20-21swapper/22
118534299150,13cyclictest0-21swapper/33
118534299150,12cyclictest0-21swapper/33
16089292141,7sleep11608993-21ntpq1
15562652141,9sleep01556325-21/usr/sbin/munin0
15330052141,8sleep20-21swapper/22
15137322141,2sleep01315973-21diskmemload0
15137322141,2sleep01315973-21diskmemload0
14881392141,2sleep00-21swapper/00
14611992141,8sleep01315973-21diskmemload0
14392082141,9sleep01439357-21head0
14094072140,9sleep01315973-21diskmemload0
13918272140,3sleep11315973-21diskmemload1
13850462141,6sleep20-21swapper/22
118534299140,12cyclictest0-21swapper/33
16396542131,6sleep01639869-21diskstats0
15848452130,1sleep00-21swapper/00
15246102131,6sleep01315973-21diskmemload0
14301212131,5sleep21430154-21ssh2
14272032131,8sleep21427218-21ssh2
14218702131,6sleep10-21swapper/11
14015182131,2sleep11315973-21diskmemload1
13933222131,4sleep01393332-21ssh0
13856922131,8sleep11385705-21ssh1
13722382131,6sleep21373470-21memory2
13667232131,7sleep01315973-21diskmemload0
13430872130,1sleep00-21swapper/00
12568452130,1sleep20-21swapper/22
12453542131,8sleep10-21swapper/11
16652102121,3sleep11665297-21apt-config1
16482382120,5sleep10-21swapper/11
16283492121,7sleep11315973-21diskmemload1
16199992120,4sleep20-21swapper/22
15354092121,6sleep11536688-21grep1
15070652120,1sleep20-21swapper/22
14865072121,7sleep21486561-21ld2
14799532121,7sleep01479968-21ssh0
14566432121,6sleep21456862-21hddtemp_smartct2
14389122121,7sleep21438972-21/usr/sbin/munin2
14291432121,6sleep00-21swapper/00
14182962121,3sleep11418309-21ssh1
14115432121,5sleep10-21swapper/11
13583472121,6sleep21315973-21diskmemload2
13427342121,7sleep01342749-21rm0
13249612121,6sleep21324963-21ssh2
12427592121,7sleep21244517-21tr2
12192612120,4sleep20-21swapper/22
12039302121,7sleep21203583-21/usr/sbin/munin2
16815622111,3sleep20-21swapper/22
16725602111,4sleep11315973-21diskmemload1
16556332110,5sleep00-21swapper/00
16482102110,7sleep20-21swapper/22
16447112111,4sleep11315973-21diskmemload1
16350052111,5sleep01635064-21ntpq0
16291962112,3sleep01315973-21diskmemload0
16126902111,3sleep00-21swapper/00
15961382111,5sleep21596271-21diskstats2
15948672111,4sleep20-21swapper/22
15872112113,4sleep0151ktimers/00
15740272111,5sleep01574082-21ntpq0
15408302110,1sleep00-21swapper/00
15324402110,4sleep10-21swapper/11
15255752111,3sleep20-21swapper/22
15157852111,4sleep11315973-21diskmemload1
15157852111,4sleep11315973-21diskmemload1
15080102111,5sleep01315973-21diskmemload0
14965212111,4sleep11315973-21diskmemload1
14919582111,6sleep21493340-21kthreadcore2
14819362111,3sleep20-21swapper/22
14630752111,4sleep11315973-21diskmemload1
14443102111,4sleep11315973-21diskmemload1
14127592111,4sleep01413021-21df0
14061442111,5sleep21407900-21tr2
14004622110,2sleep00-21swapper/00
13694042111,6sleep01369417-21sh0
13675332110,4sleep20-21swapper/22
13588352111,4sleep01315973-21diskmemload0
13506562111,3sleep10-21swapper/11
13500342111,4sleep21315973-21diskmemload2
13405472110,4sleep10-21swapper/11
13148592111,5sleep01314948-21ntpq0
13046852111,5sleep01304929-21users0
12989852111,5sleep21299038-21ntpq2
12665772111,6sleep21266690-21ntpq2
12252252111,4sleep21225348-21diskstats2
12192602111,5sleep11219176-31find1
12192392110,6sleep01219176-31find0
12022802111,5sleep21202328-21ntpq2
16747842102,4sleep01672017-21kworker/0:0+events0
16640932101,3sleep00-21swapper/00
16291142101,3sleep20-21swapper/22
16199312100,2sleep10-21swapper/11
16187132101,3sleep01315973-21diskmemload0
16001502101,5sleep11600206-21ntpq1
15852702101,2sleep20-21swapper/22
15673092101,4sleep00-21swapper/00
15486072101,2sleep00-21swapper/00
15434742101,4sleep21543496-21apt-get2
15259992101,3sleep01315973-21diskmemload0
15236042101,4sleep11315973-21diskmemload1
14896522100,2sleep11489663-21ssh1
14795742101,4sleep11315973-21diskmemload1
14649212101,4sleep01315973-21diskmemload0
14527232101,4sleep00-21swapper/00
14304262100,2sleep00-21swapper/00
14130862101,5sleep21413304-21/usr/sbin/munin2
13781832101,5sleep01378217-21sh0
13654992101,4sleep11315973-21diskmemload1
13518592101,3sleep11315973-21diskmemload1
13398982101,3sleep20-21swapper/22
13312392101,4sleep01315973-21diskmemload0
13166992101,4sleep11315973-21diskmemload1
13159182100,4sleep20-21swapper/22
13149492101,5sleep11314996-21ntpq1
12883242101,4sleep21288378-21ntpq2
12842502101,4sleep11284269-21apt-get1
12812402101,4sleep01282527-21memory0
12735842101,5sleep21273765-21head2
12572882101,4sleep21257307-21apt-get2
12525332101,5sleep01253755-21cut0
12520332101,4sleep11252249-21diskstats1
12506542101,4sleep11250702-21ntpq1
12359662102,2sleep20-21swapper/22
12256322101,5sleep11226978-21kthreadcore1
12038612101,5sleep11203928-21fschecks_count1
12036052101,4sleep01203860-21df0
11894572100,6sleep21189597-21awk2
1671086291,2sleep00-21swapper/00
1635950291,2sleep20-21swapper/22
1601625291,2sleep00-21swapper/00
1574801291,2sleep20-21swapper/22
1514058291,2sleep20-21swapper/22
1514058291,2sleep20-21swapper/22
1451894290,4sleep11451952-21ntpq1
1395753291,4sleep21395973-21hddtemp_smartct2
1331299290,1sleep10-21swapper/11
1304256291,3sleep11304304-21ntpq1
1278622291,2sleep20-21swapper/22
1251413290,3sleep20-21swapper/22
1220278291,4sleep11221370-21cut1
1197022291,4sleep11197076-21ntpq1
1305589281,2sleep20-21swapper/22
1305004280,2sleep20-21swapper/22
1271993281,3sleep21272044-21ntpq2
1261442281,3sleep01261487-21/usr/sbin/munin0
1251703281,3sleep01251821-21latency_hist0
1234742281,3sleep21234866-21timerandwakeup2
1231727280,1sleep10-21swapper/11
1223701281,2sleep00-21swapper/00
1209174281,3sleep11209288-21fschecks_time1
1197929280,1sleep00-21swapper/00
1197725281,1sleep20-21swapper/22
1190503280,2sleep10-21swapper/11
1294918271,2sleep10-21swapper/11
1294469270,1sleep00-21swapper/00
1294407270,1sleep20-21swapper/22
1288437271,3sleep00-21swapper/00
1278300270,3sleep00-21swapper/00
1275033271,3sleep10-21swapper/11
1270160271,3sleep00-21swapper/00
1268261270,3sleep10-21swapper/11
1262926271,3sleep11263042-21fschecks_time1
1257282270,1sleep10-21swapper/11
1235936270,3sleep10-21swapper/11
1230388270,2sleep00-21swapper/00
1225060270,2sleep00-21swapper/00
1213705270,1sleep20-21swapper/22
1210291271,3sleep00-21swapper/00
1186430271,2sleep01188125-21kthreadcore0
1310350260,1sleep10-21swapper/11
1295218260,2sleep01295425-21smartctl0
1283646260,4sleep10-21swapper/11
1264054261,2sleep00-21swapper/00
1203255260,4sleep10-21swapper/11
1203229260,1sleep00-21swapper/00
1240131250,1sleep00-21swapper/00
1241330240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional