You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-03 - 01:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Thu Jul 02, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
116792721270,3sleep0221rcuc/00
988124212064,52sleep30-21swapper/33
988124212064,52sleep30-21swapper/33
98812828857,12sleep20-21swapper/22
98812828857,12sleep20-21swapper/22
98811228160,14sleep00-21swapper/00
98811228160,14sleep00-21swapper/00
98828227858,12sleep10-21swapper/11
98828227858,12sleep10-21swapper/11
988548992415,7cyclictest1116294-21sleep3
13041462240,9sleep134-21ksoftirqd/11
98854899211,16cyclictest1287552-21chrt3
98854899211,16cyclictest0-21swapper/33
98854899211,15cyclictest0-21swapper/33
98854899202,5cyclictest1436199-21taskset3
98854899201,15cyclictest0-21swapper/33
98854899201,15cyclictest0-21swapper/33
13734462206,10sleep11373467-21unix_chkpwd1
98854899199,6cyclictest10107972sleep33
98854899197,1cyclictest0-21swapper/33
988548991916,1cyclictest0-21swapper/33
988548991916,1cyclictest0-21swapper/33
98854899191,2cyclictest0-21swapper/33
98854899191,14cyclictest0-21swapper/33
98854899191,14cyclictest0-21swapper/33
98854899191,14cyclictest0-21swapper/33
98854899191,14cyclictest0-21swapper/33
98854899191,14cyclictest0-21swapper/33
98854899190,17cyclictest0-21swapper/33
13306332191,2sleep20-21swapper/22
98854899188,6cyclictest1105833-21turbostat3
98854899187,8cyclictest1105833-21turbostat3
98854899182,11cyclictest1090413-21taskset3
98854899181,7cyclictest1247307-21chrt3
988548991816,1cyclictest0-21swapper/33
988548991815,1cyclictest0-21swapper/33
988548991814,3cyclictest0-21swapper/33
988548991814,2cyclictest0-21swapper/33
988548991814,2cyclictest0-21swapper/33
988548991813,2cyclictest0-21swapper/33
988548991812,3cyclictest0-21swapper/33
988548991811,5cyclictest0-21swapper/33
98854899181,14cyclictest1439615-21chrt3
98854899181,14cyclictest0-21swapper/33
98854899181,14cyclictest0-21swapper/33
98854899181,14cyclictest0-21swapper/33
98854899181,14cyclictest0-21swapper/33
98854899181,14cyclictest0-21swapper/33
98854899181,14cyclictest0-21swapper/33
98854899181,14cyclictest0-21swapper/33
98854899181,13cyclictest0-21swapper/33
14072222181,12sleep01407568-21tr0
98854899178,4cyclictest13331412sleep33
98854899173,11cyclictest0-21swapper/33
98854899172,12cyclictest0-21swapper/33
98854899171,4cyclictest0-21swapper/33
988548991713,2cyclictest0-21swapper/33
98854899171,13cyclictest0-21swapper/33
98854899170,14cyclictest0-21swapper/33
98854899170,13cyclictest0-21swapper/33
98854899170,13cyclictest0-21swapper/33
98854899170,13cyclictest0-21swapper/33
14641322171,7sleep01464273-21latency_hist0
14281522171,3sleep00-21swapper/00
98854899162,10cyclictest988724-21taskset3
98854899160,13cyclictest0-21swapper/33
98854899160,13cyclictest0-21swapper/33
98854899160,13cyclictest0-21swapper/33
98854899160,13cyclictest0-21swapper/33
98854899160,13cyclictest0-21swapper/33
14363172161,4sleep11116283-21diskmemload1
14221672161,2sleep00-21swapper/00
13139672161,2sleep00-21swapper/00
13130652161,4sleep11116283-21diskmemload1
12735152161,2sleep20-21swapper/22
10589302161,4sleep21059187-21df2
98854899156,6cyclictest1053391-21turbostat3
988548991512,2cyclictest0-21swapper/33
98854899150,13cyclictest0-21swapper/33
98854899150,12cyclictest0-21swapper/33
14047852151,3sleep20-21swapper/22
13906362151,4sleep11116283-21diskmemload1
13872022151,9sleep01116283-21diskmemload0
13513742151,10sleep11349036-21/usr/sbin/munin1
12329502151,7sleep21233163-21diskstats2
12153942151,3sleep20-21swapper/22
11826462151,1sleep10-21swapper/11
11337042151,6sleep11133928-21if_enp1s01
10798882151,4sleep21079886-21apt-get2
98854899141,9cyclictest1032688-21chrt3
988548991412,1cyclictest0-21swapper/33
98854899141,1cyclictest0-21swapper/33
988548991411,1cyclictest0-21swapper/33
98854899140,3cyclictest0-21swapper/33
98854899140,3cyclictest0-21swapper/33
98854899140,2cyclictest0-21swapper/33
13460992141,3sleep01346109-21ssh0
13298722141,3sleep11329883-21ssh1
12826612141,8sleep01282928-21df0
12816412141,9sleep21281655-21ssh2
11890992141,9sleep21189109-21ssh2
11215152141,8sleep01116283-21diskmemload0
10332142141,8sleep21033377-21tr2
14365792131,3sleep01436590-21ssh0
14300342130,8sleep21116283-21diskmemload2
13956952131,5sleep00-21swapper/00
13651272131,7sleep21116283-21diskmemload2
13398842131,8sleep21116283-21diskmemload2
12988822131,8sleep01299162-21sed0
12715512132,6sleep11116283-21diskmemload1
12574202131,8sleep21257454-21munin-run2
12406452131,6sleep11116283-21diskmemload1
12328412131,5sleep01232874-21ssh0
12323922131,8sleep21232406-21ssh2
11583202132,4sleep01116283-21diskmemload0
11562682130,2sleep00-21swapper/00
11315052131,2sleep10-21swapper/11
11314932131,4sleep01131504-21ssh0
11008812132,5sleep01100900-21apt-get0
10032452131,8sleep21005016-21grep2
14563082121,5sleep11456341-21ssh1
14307162121,7sleep11430727-21ssh1
14048622122,4sleep11116283-21diskmemload1
13555802121,3sleep21355594-21ssh2
13463512121,2sleep20-21swapper/22
13224212121,3sleep11322432-21ssh1
12742602121,5sleep01274524-21df0
12053202121,6sleep01116283-21diskmemload0
11749352122,5sleep21174959-21apt-get2
11589732121,7sleep11160489-21cut1
11445302121,6sleep11145643-21memory1
11420822121,7sleep21142263-21ssh2
11395732121,7sleep01139583-21ssh0
11014552121,7sleep11102765-21cut1
10536832122,5sleep01053680-21apt-get0
14483012111,5sleep01448521-21hddtemp_smartct0
14310262111,6sleep21431298-21ssh2
14135922110,6sleep21116283-21diskmemload2
13980032111,6sleep21-21systemd2
13872242111,6sleep21387235-21ssh2
13791732111,3sleep20-21swapper/22
13738322112,4sleep01373858-21apt-get0
13726512111,7sleep01372665-21ssh0
13549562111,4sleep01116283-21diskmemload0
13327332111,5sleep01332839-21/usr/sbin/munin0
13082382111,6sleep21309687-21tr2
12954652111,5sleep11295720-21users1
12635692111,5sleep21116283-21diskmemload2
12635452111,6sleep01263556-21ssh0
12633182111,6sleep11116283-21diskmemload1
12524412111,5sleep01253434-21memory0
12489262111,3sleep00-21swapper/00
12160092111,4sleep11116283-21diskmemload1
12081432111,4sleep01208168-21apt-get0
11918112111,6sleep21191925-21fschecks_time2
11915402111,5sleep11191809-21df_inode1
11719432111,4sleep21116283-21diskmemload2
11631532111,5sleep21116283-21diskmemload2
11473322111,7sleep01147345-21rm0
11412562111,5sleep21141291-21munin-run2
10946292111,7sleep11094854-21grep1
10734042111,5sleep01073453-21ntp_states0
10170292114,4sleep2431ktimers/22
14541432101,4sleep21116283-21diskmemload2
14479912101,4sleep11448004-21apt-get1
14448412101,5sleep11444853-21rm1
14440152101,5sleep21439616-21/usr/sbin/munin2
14213142101,3sleep10-21swapper/11
14204392100,3sleep20-21swapper/22
13983872101,4sleep01398410-21apt-get0
13873532101,5sleep11387364-21ssh1
13656822101,3sleep20-21swapper/22
13644632100,2sleep10-21swapper/11
13621582101,4sleep00-21swapper/00
13468752101,4sleep11116283-21diskmemload1
13384142100,4sleep10-21swapper/11
13212872101,2sleep00-21swapper/00
13158202102,3sleep21315819-21apt-get2
13042992101,3sleep20-21swapper/22
12960362101,2sleep20-21swapper/22
12900952101,4sleep21116283-21diskmemload2
12792442101,3sleep11116283-21diskmemload1
12665592100,6sleep01266759-21kthreadcore0
12542272101,3sleep11116283-21diskmemload1
12466432101,2sleep20-21swapper/22
12458982104,1sleep10-21swapper/11
12251372101,4sleep11225219-21irqstats1
12241552101,3sleep20-21swapper/22
12232782101,4sleep01116283-21diskmemload0
12077622100,2sleep10-21swapper/11
12026112101,4sleep21203759-21memory2
11904582100,1sleep10-21swapper/11
11833002101,4sleep01183523-21df_inode0
11760682101,5sleep01177648-21kthreadcore0
11716722101,6sleep11171671-21ssh1
11555282101,3sleep11116283-21diskmemload1
11277562101,4sleep21128847-21memory2
11228212100,6sleep11122832-21sh1
11172602104,3sleep21117847-21kthreadcore2
11113152101,3sleep11111334-21apt-get1
10999512101,5sleep11100195-21users1
10851662101,3sleep01085178-21apt-get0
10641652101,4sleep01064167-21apt-get0
10589252101,4sleep11058927-21apt-get1
10431732100,3sleep00-21swapper/00
10064822101,4sleep0221rcuc/00
1461486291,4sleep21461497-21sh2
1444645291,2sleep00-21swapper/00
1373795291,4sleep11373807-21ssh1
1331555291,3sleep01116283-21diskmemload0
1305700291,2sleep00-21swapper/00
1288328291,3sleep11116283-21diskmemload1
1213496291,2sleep10-21swapper/11
1196547291,2sleep00-21swapper/00
1156488291,2sleep20-21swapper/22
1111885291,4sleep21111921-21kthreadcore2
1095644291,3sleep21095646-21apt-get2
1048404291,4sleep11048422-21apt1
1038205291,5sleep01038409-21gltestperf0
1037914290,3sleep11037933-21apt-get1
1028908291,4sleep01030593-21kthreadcore0
1015903291,4sleep01016097-21wc0
1011716291,2sleep20-21swapper/22
995963281,2sleep10-21swapper/11
994350281,3sleep00-21swapper/00
988721280,2sleep20-21swapper/22
1413967281,3sleep11413977-21ssh1
1231523281,3sleep01231534-21ssh0
1108101281,3sleep21109868-21cut2
1095018280,2sleep00-21swapper/00
1090392280,2sleep20-21swapper/22
1079867281,3sleep10-21swapper/11
1079487281,3sleep20-21swapper/22
1064731281,3sleep11066179-21kthreadcore1
1059210281,3sleep01059427-21idleruntime0
1038486281,3sleep21039600-21grep2
1032108281,1sleep20-21swapper/22
1021616280,1sleep10-21swapper/11
1016098280,5sleep11016281-21vmstat1
1010540281,3sleep10-21swapper/11
996570271,3sleep20-21swapper/22
993030271,3sleep1331ktimers/11
1106690270,3sleep1321rcuc/11
1106107270,2sleep00-21swapper/00
1102767271,3sleep21104505-21kthreadcore2
1100238270,1sleep00-21swapper/00
1085828271,3sleep10-21swapper/11
1084843270,1sleep00-21swapper/00
1079520270,1sleep00-21swapper/00
1074269270,1sleep20-21swapper/22
1074098270,4sleep10-21swapper/11
1068236271,2sleep21068390-21systemctl2
1058288270,1sleep10-21swapper/11
1053163270,1sleep00-21swapper/00
1048400270,4sleep20-21swapper/22
1043195271,2sleep10-21swapper/11
1022767271,2sleep10-21swapper/11
1022196270,2sleep00-21swapper/00
1022187270,3sleep20-21swapper/22
1016928270,2sleep00-21swapper/00
1011291270,1sleep20-21swapper/22
1006133270,1sleep10-21swapper/11
1116077260,1sleep00-21swapper/00
1058450260,2sleep20-21swapper/22
1037540260,1sleep00-21swapper/00
1032655260,1sleep10-21swapper/11
1001227260,2sleep00-21swapper/00
1085014250,1sleep20-21swapper/22
1078512250,1sleep10-21swapper/11
1042959250,1sleep20-21swapper/22
1030597250,2sleep10-21swapper/11
988459240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional