You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-05 - 09:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Fri Jun 05, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137100122550,1sleep01370999-21ssh0
931083212454,65sleep30-21swapper/33
98318921170,2sleep244-21ksoftirqd/22
93575928462,16sleep10-21swapper/11
93573528363,12sleep20-21swapper/22
93582027755,17sleep00-21swapper/00
93629899251,20cyclictest1143756-21chrt3
936298992211,8cyclictest0-21swapper/33
936298992210,8cyclictest66950irq/132-enp1s0-TxRx-33
936298992210,8cyclictest66950irq/132-enp1s0-TxRx-33
93629899215,13cyclictest0-21swapper/33
93629899211,15cyclictest66950irq/132-enp1s0-TxRx-33
93629899201,15cyclictest66950irq/132-enp1s0-TxRx-33
93629899200,16cyclictest12141022sleep33
936298991916,2cyclictest0-21swapper/33
936298991915,2cyclictest0-21swapper/33
93629899191,15cyclictest66950irq/132-enp1s0-TxRx-33
93629899191,15cyclictest0-21swapper/33
93629899191,15cyclictest0-21swapper/33
93629899191,15cyclictest0-21swapper/33
93629899191,14cyclictest0-21swapper/33
93629899191,14cyclictest0-21swapper/33
12429712191,13sleep11243156-21expr1
93629899186,9cyclictest1286293-21turbostat3
936298991814,3cyclictest66950irq/132-enp1s0-TxRx-33
936298991814,2cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,14cyclictest0-21swapper/33
93629899181,13cyclictest0-21swapper/33
93629899181,13cyclictest0-21swapper/33
93629899181,13cyclictest0-21swapper/33
93629899181,13cyclictest0-21swapper/33
93629899181,13cyclictest0-21swapper/33
93629899180,16cyclictest0-21swapper/33
12470622181,5sleep01247162-21ntpq0
11979942181,2sleep00-21swapper/00
11806322181,3sleep00-21swapper/00
936298991713,2cyclictest0-21swapper/33
93629899171,14cyclictest0-21swapper/33
93629899171,13cyclictest0-21swapper/33
93629899171,13cyclictest0-21swapper/33
93629899171,13cyclictest0-21swapper/33
93629899171,13cyclictest0-21swapper/33
93629899171,13cyclictest0-21swapper/33
93629899171,13cyclictest0-21swapper/33
93629899171,13cyclictest0-21swapper/33
93629899171,10cyclictest1018992-21taskset3
93629899170,4cyclictest986921-21chrt3
93629899170,14cyclictest0-21swapper/33
93629899170,14cyclictest0-21swapper/33
93629899170,13cyclictest0-21swapper/33
14003962171,4sleep01400475-21systemctl0
13916762171,5sleep11391732-21true1
11709252171,3sleep20-21swapper/22
11439842170,7sleep11066680-21diskmemload1
11361842171,3sleep00-21swapper/00
10461232173,7sleep11046695-21cut1
10016282171,11sleep21001717-21ntpq2
93629899162,9cyclictest954665-21taskset3
936298991613,1cyclictest0-21swapper/33
93629899161,12cyclictest992349-21taskset3
93629899161,12cyclictest0-21swapper/33
936298991610,3cyclictest0-21swapper/33
936298991610,3cyclictest0-21swapper/33
93629899160,13cyclictest0-21swapper/33
93629899160,13cyclictest0-21swapper/33
93629899160,13cyclictest0-21swapper/33
13611102160,1sleep10-21swapper/11
13359802161,3sleep00-21swapper/00
12142242161,6sleep11214235-21ssh1
12076522161,4sleep01066680-21diskmemload0
936298991512,1cyclictest0-21swapper/33
936298991511,3cyclictest0-21swapper/33
93629899150,3cyclictest0-21swapper/33
93629899150,12cyclictest0-21swapper/33
93629899150,12cyclictest0-21swapper/33
93629899150,12cyclictest0-21swapper/33
93629899150,12cyclictest0-21swapper/33
14235232150,2sleep10-21swapper/11
13535752151,2sleep00-21swapper/00
12747592151,10sleep11274771-21ssh1
12298562151,3sleep00-21swapper/00
11286312151,9sleep21066680-21diskmemload2
10902872151,2sleep10-21swapper/11
93629899141,9cyclictest1054948-21taskset3
93629899140,2cyclictest0-21swapper/33
93629899140,2cyclictest0-21swapper/33
13722362141,3sleep10-21swapper/11
13632902141,2sleep20-21swapper/22
13202442141,2sleep0171rcu_preempt0
12918842141,6sleep00-21swapper/00
11463432141,8sleep21146549-21diskstats2
11269012141,4sleep01126912-21ssh0
11188222147,3sleep2431ktimers/22
11026452141,9sleep11102753-21ssh1
10733212141,4sleep11066680-21diskmemload1
9407892131,6sleep2940843-21ntpq2
9406272131,7sleep0940740-21ntpq0
93629899130,2cyclictest0-21swapper/33
14276232131,8sleep21428852-21cut2
13757772131,8sleep21377437-21cut2
13650782131,7sleep01365119-21munin-run0
13283962131,6sleep10-21swapper/11
12685882130,2sleep20-21swapper/22
12446742132,7sleep21246360-21kthreadcore2
12219432131,7sleep01221957-21ssh0
12214702130,3sleep21066680-21diskmemload2
12040182131,7sleep21066680-21diskmemload2
11833962131,8sleep01183477-21kthreadcore0
11833962131,8sleep01183477-21kthreadcore0
11785252130,2sleep10-21swapper/11
11721802131,8sleep11172424-21cat1
11600532131,7sleep11066680-21diskmemload1
11455632130,4sleep21145574-21ssh2
10349282130,2sleep20-21swapper/22
10177312131,7sleep01017778-21ntpq0
10177312131,7sleep01017778-21ntpq0
9856602120,2sleep10-21swapper/11
14168722121,4sleep10-21swapper/11
14082252121,7sleep11408239-21ssh1
13962582121,5sleep21396318-21ntpq2
13882732121,8sleep01388285-21ssh0
13345622121,5sleep11334621-21ntpq1
13083492121,6sleep01308408-21ntpq0
12922462121,8sleep11292260-21ssh1
12756892121,5sleep20-21swapper/22
12756622121,7sleep01275673-21sh0
12654032120,5sleep11265416-21rm1
11005982121,7sleep21100612-21ssh2
10739662121,3sleep00-21swapper/00
10083342121,7sleep01008565-21cstates0
9913042111,2sleep10-21swapper/11
9875022111,1sleep00-21swapper/00
9601372111,5sleep2960260-21diskstats2
14310792111,5sleep11431169-21ntpq1
14234672112,6sleep01423479-21sh0
14145302110,1sleep20-21swapper/22
14084142111,7sleep01408429-21ssh0
14076852111,5sleep21066680-21diskmemload2
13726022111,6sleep21372613-21ssh2
13517772110,2sleep20-21swapper/22
13355522111,3sleep20-21swapper/22
13170152111,6sleep290-21kswapd02
13084102111,6sleep11300777-21kworker/u16:2+rpciod1
12866682111,3sleep20-21swapper/22
12596712110,3sleep20-21swapper/22
12064172110,7sleep01066680-21diskmemload0
11950582111,5sleep11195071-21ssh1
11813072112,4sleep21181331-21apt-get2
11813072112,4sleep21181331-21apt-get2
11712142110,1sleep00-21swapper/00
11621012111,6sleep01162112-21ssh0
11504472110,1sleep10-21swapper/11
11192572112,5sleep01119476-21ssh0
10975822111,6sleep01097644-21gcc0
10924342111,5sleep20-21swapper/22
10771152111,6sleep21078772-21kthreadcore2
10548602111,5sleep21054941-21ntpq2
10254012111,6sleep21027104-21cut2
9700162101,5sleep2970276-21sed2
9590162101,5sleep0959222-21grep0
9372072101,5sleep1937250-21kthreadcore1
14348812101,5sleep01434887-21ssh0
14242362101,2sleep20-21swapper/22
13912372101,4sleep21066680-21diskmemload2
13820622101,4sleep01066680-21diskmemload0
13554482101,4sleep11066680-21diskmemload1
13432292101,4sleep01343283-21ntpq0
13394012101,4sleep21339620-21hddtemp_smartct2
13392632101,4sleep11339400-21diskstats1
13271472101,4sleep21066680-21diskmemload2
13215962101,4sleep01066680-21diskmemload0
13032652101,3sleep11066680-21diskmemload1
12953142101,3sleep20-21swapper/22
12859332101,4sleep01066680-21diskmemload0
12859332101,4sleep01066680-21diskmemload0
12782312101,4sleep11278451-21hddtemp_smartct1
12782312101,4sleep11278451-21hddtemp_smartct1
12687692101,2sleep00-21swapper/00
12567312101,3sleep10-21swapper/11
12345142101,5sleep11234818-21tr1
12312032100,2sleep11231215-21ssh1
12293892101,4sleep21229453-21ntpq2
12166812101,5sleep11216821-21tr1
12148432101,4sleep21066680-21diskmemload2
11977602101,4sleep21066680-21diskmemload2
11627782101,2sleep20-21swapper/22
11380262101,4sleep01066680-21diskmemload0
11257962101,5sleep21125811-21ssh2
11242382101,5sleep11124309-21ld1
11109022101,4sleep11111164-21df1
11021262101,4sleep21102386-21df2
10988762101,2sleep10-21swapper/11
10919022101,3sleep00-21swapper/00
10666712101,4sleep21066724-21munin-run2
10562682101,4sleep01056526-21df_inode0
10030072101,2sleep00-21swapper/00
976588291,4sleep0976769-21gltestperf0
969686290,1sleep00-21swapper/00
965682291,3sleep1965885-21diskstats1
954644290,2sleep20-21swapper/22
954168290,2sleep00-21swapper/00
1415815290,1sleep00-21swapper/00
1396180290,1sleep10-21swapper/11
1374362290,4sleep11066680-21diskmemload1
1319403291,5sleep11319414-21sh1
1309445291,4sleep21309459-21ssh2
1300421291,3sleep00-21swapper/00
1283087291,3sleep21066680-21diskmemload2
1283087291,3sleep21066680-21diskmemload2
1251610291,4sleep01251621-21ssh0
1239914291,3sleep21066680-21diskmemload2
1237874291,3sleep01066680-21diskmemload0
1205775291,2sleep10-21swapper/11
1187241291,4sleep11187252-21ssh1
1187241291,4sleep11187252-21ssh1
1172766290,2sleep20-21swapper/22
1146235291,4sleep01146270-21ssh0
1134430291,3sleep11066680-21diskmemload1
1107529291,4sleep01066680-21diskmemload0
1083037291,4sleep11083051-21ssh1
1081097290,1sleep00-21swapper/00
1072375291,2sleep20-21swapper/22
1055964291,4sleep01056138-21cat0
1040803291,5sleep21042002-21kthreadcore2
1040232290,1sleep10-21swapper/11
1028472291,4sleep01028615-21mailstats0
1024299291,2sleep10-21swapper/11
1019246291,4sleep01019278-21fschecks_count0
996675281,3sleep0583-21gdbus0
982039280,1sleep00-21swapper/00
976870281,3sleep2978083-21grep2
975040281,3sleep0975089-21ntpq0
971262280,1sleep10-21swapper/11
959972281,3sleep0959991-21apt-get0
944702281,3sleep0946377-21kthreadcore0
941806281,3sleep2942148-21kthreadcore2
1066586280,2sleep00-21swapper/00
1065983280,1sleep10-21swapper/11
1039975280,1sleep00-21swapper/00
1034611280,1sleep10-21swapper/11
1030012280,1sleep00-21swapper/00
1007881281,3sleep20-21swapper/22
997204270,1sleep20-21swapper/22
991871270,1sleep20-21swapper/22
975901270,1sleep20-21swapper/22
964785270,1sleep10-21swapper/11
959393270,2sleep172-21kcompactd01
954295270,4sleep10-21swapper/11
1061104270,1sleep20-21swapper/22
1061005272,3sleep1331ktimers/11
1055736270,1sleep10-21swapper/11
1050435270,1sleep00-21swapper/00
1050355270,1sleep20-21swapper/22
1039832270,5sleep10-21swapper/11
1023802270,1sleep10-21swapper/11
1018970270,2sleep20-21swapper/22
1013043270,1sleep10-21swapper/11
1004923271,2sleep10-21swapper/11
1002651270,1sleep10-21swapper/11
1002492270,1sleep00-21swapper/00
981322261,2sleep1981324-21modprobe1
954155260,4sleep20-21swapper/22
1018446260,4sleep10-21swapper/11
1018446260,4sleep10-21swapper/11
1013969261,2sleep20-21swapper/22
1013969261,2sleep20-21swapper/22
1012764260,1sleep20-21swapper/22
996690250,1sleep10-21swapper/11
948089250,1sleep10-21swapper/11
1029673250,2sleep20-21swapper/22
1035094240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional