You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 01:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Wed Jun 24, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3893233212067,49sleep30-21swapper/33
389350029276,10sleep20-21swapper/22
389326028354,11sleep10-21swapper/11
389329428049,7sleep00-21swapper/00
3893731992419,2cyclictest0-21swapper/33
389373199221,17cyclictest165304-21taskset3
389373199221,16cyclictest0-21swapper/33
475162211,8sleep047530-21ssh0
3893731992118,1cyclictest0-21swapper/33
3893731992118,1cyclictest0-21swapper/33
3893731992117,2cyclictest0-21swapper/33
389373199211,14cyclictest0-21swapper/33
389373199210,19cyclictest0-21swapper/33
3893731992017,1cyclictest0-21swapper/33
3893731992016,2cyclictest0-21swapper/33
3893731992015,2cyclictest0-21swapper/33
389373199201,15cyclictest964212chrt3
389373199201,15cyclictest0-21swapper/33
389373199200,18cyclictest0-21swapper/33
40270042191,2sleep20-21swapper/22
3893731991916,1cyclictest0-21swapper/33
3893731991914,2cyclictest0-21swapper/33
3893731991913,3cyclictest66950irq/132-enp1s0-TxRx-33
3893731991912,5cyclictest0-21swapper/33
3893731991912,4cyclictest0-21swapper/33
389373199191,15cyclictest0-21swapper/33
389373199191,14cyclictest3916538-21taskset3
389373199191,14cyclictest0-21swapper/33
389373199191,14cyclictest0-21swapper/33
389373199191,14cyclictest0-21swapper/33
389373199191,14cyclictest0-21swapper/33
41116252180,7sleep20-21swapper/22
41037152182,11sleep24021471-21diskmemload2
40725492181,3sleep04074100-21grep0
3893731991815,1cyclictest0-21swapper/33
3893731991814,2cyclictest0-21swapper/33
3893731991814,2cyclictest0-21swapper/33
3893731991813,2cyclictest0-21swapper/33
389373199181,14cyclictest0-21swapper/33
389373199181,14cyclictest0-21swapper/33
389373199181,14cyclictest0-21swapper/33
389373199181,14cyclictest0-21swapper/33
389373199181,13cyclictest0-21swapper/33
389373199180,14cyclictest0-21swapper/33
389373199180,14cyclictest0-21swapper/33
994882171,5sleep04021471-21diskmemload0
41361442171,3sleep20-21swapper/22
40780142171,2sleep10-21swapper/11
40700912170,5sleep04021471-21diskmemload0
40700912170,5sleep04021471-21diskmemload0
389373199173,12cyclictest0-21swapper/33
3893731991714,1cyclictest0-21swapper/33
3893731991714,1cyclictest0-21swapper/33
3893731991713,2cyclictest0-21swapper/33
3893731991713,2cyclictest0-21swapper/33
3893731991713,2cyclictest0-21swapper/33
389373199171,13cyclictest0-21swapper/33
389373199170,14cyclictest0-21swapper/33
389373199170,13cyclictest0-21swapper/33
728862161,7sleep272897-21ssh2
41673352161,4sleep04021471-21diskmemload0
415992161,4sleep04021471-21diskmemload0
40614242161,3sleep24021471-21diskmemload2
40351292161,2sleep20-21swapper/22
3893731991613,1cyclictest0-21swapper/33
3893731991613,1cyclictest0-21swapper/33
3893731991613,1cyclictest0-21swapper/33
3893731991613,1cyclictest0-21swapper/33
3893731991611,3cyclictest0-21swapper/33
389373199161,12cyclictest0-21swapper/33
389373199160,14cyclictest0-21swapper/33
389373199160,13cyclictest0-21swapper/33
389373199160,13cyclictest0-21swapper/33
389373199160,13cyclictest0-21swapper/33
389373199160,12cyclictest0-21swapper/33
389373199160,12cyclictest0-21swapper/33
900722151,2sleep10-21swapper/11
41528722150,4sleep14021471-21diskmemload1
41433722151,4sleep04143377-21ssh0
41363412152,8sleep04021471-21diskmemload0
3893731991513,1cyclictest0-21swapper/33
389373199151,2cyclictest0-21swapper/33
3893731991512,1cyclictest0-21swapper/33
3893731991512,1cyclictest0-21swapper/33
3893731991511,3cyclictest0-21swapper/33
3893731991511,2cyclictest0-21swapper/33
389373199151,12cyclictest0-21swapper/33
389373199151,12cyclictest0-21swapper/33
389373199150,13cyclictest0-21swapper/33
389373199150,12cyclictest0-21swapper/33
341352151,4sleep14021471-21diskmemload1
220102152,10sleep10-21swapper/11
159502151,3sleep10-21swapper/11
1497482151,2sleep2171rcu_preempt2
1342542151,2sleep10-21swapper/11
655252140,2sleep00-21swapper/00
41755202141,7sleep20-21swapper/22
41611942141,4sleep24161202-21ssh2
41591232140,2sleep0221rcuc/00
40364432141,9sleep14036457-21ssh1
40167042140,1sleep10-21swapper/11
39797892141,9sleep03979961-21cstates0
389373199140,11cyclictest0-21swapper/33
1588432141,2sleep00-21swapper/00
908362131,3sleep090846-21ssh0
891392131,5sleep20-21swapper/22
573492131,8sleep14021471-21diskmemload1
424842131,8sleep242506-21acpi2
41426902131,6sleep14021471-21diskmemload1
41355222131,8sleep14021471-21diskmemload1
41250252131,8sleep04125191-21systemctl0
40834152131,8sleep14083553-21/usr/sbin/munin1
39902792137,3sleep13990401-21tr1
39890322131,3sleep23989087-21/usr/sbin/munin2
39279692131,8sleep03928265-21tr0
314442131,8sleep131455-21ssh1
1688352132,5sleep1168832-21apt-get1
1495022131,7sleep04021471-21diskmemload0
981312121,6sleep24021471-21diskmemload2
41918902121,6sleep14021471-21diskmemload1
41703822121,5sleep04170393-21ssh0
41091762121,5sleep10-21swapper/11
41025082121,7sleep14102519-21ssh1
40936072120,1sleep10-21swapper/11
40708122121,2sleep14070824-21ssh1
40708122121,2sleep14070824-21ssh1
40603842121,6sleep04060394-21ssh0
40493772121,5sleep14050393-21memory1
40459892121,3sleep00-21swapper/00
40361342121,3sleep04036146-21ssh0
39691102121,7sleep23969096-21sendmail-msp2
39600792124,4sleep03961737-21kthreadcore0
39104752121,7sleep13910551-21sshd-session1
39017512121,3sleep23902190-21cut2
38968712121,5sleep13897846-21memory1
176662121,3sleep00-21swapper/00
1760492121,3sleep0176063-21ssh0
158842120,7sleep04021471-21diskmemload0
1518052121,5sleep14021471-21diskmemload1
1420912121,7sleep2142105-21ssh2
1176882121,6sleep04021471-21diskmemload0
1109912121,7sleep1111136-21gltestperf1
1078092121,5sleep04021471-21diskmemload0
1011612121,7sleep1102666-21cut1
812112111,2sleep24021471-21diskmemload2
803332111,4sleep04021471-21diskmemload0
759922111,4sleep10-21swapper/11
660012111,5sleep10-21swapper/11
58172111,3sleep20-21swapper/22
560212111,6sleep20-21swapper/22
41866312111,5sleep04185203-21kworker/0:2+events0
41854472112,5sleep20-21swapper/22
41621052112,4sleep14162126-21apt-get1
41589822111,7sleep14158996-21ssh1
41458332111,4sleep04146060-21df_inode0
41445022110,1sleep20-21swapper/22
41264122111,4sleep10-21swapper/11
41208172112,4sleep24120840-21apt-get2
40947832111,4sleep24021471-21diskmemload2
40876032111,4sleep04021471-21diskmemload0
40869972111,4sleep24021471-21diskmemload2
40783022111,3sleep20-21swapper/22
40605372110,7sleep14060550-21ssh1
40531792111,2sleep0171rcu_preempt0
40230852111,7sleep14024322-21ssh1
39958162111,6sleep23996004-21gltestperf2
39902572111,5sleep03990276-21apt-get0
39788942111,5sleep13979111-21users1
39637692110,1sleep20-21swapper/22
39489802111,6sleep13950003-21grep1
235712111,4sleep24021471-21diskmemload2
1700052111,2sleep20-21swapper/22
1673542111,5sleep04021471-21diskmemload0
1663842110,2sleep20-21swapper/22
1544252111,6sleep1156165-21grep1
13922111,3sleep10-21swapper/11
1338192111,4sleep04021471-21diskmemload0
1327882111,3sleep20-21swapper/22
1197572112,1sleep00-21swapper/00
1194012111,4sleep2119534-21diskstats2
1157182111,4sleep24021471-21diskmemload2
987612100,3sleep10-21swapper/11
86512101,4sleep04021471-21diskmemload0
741342101,3sleep00-21swapper/00
651712101,4sleep24021471-21diskmemload2
41928432101,3sleep20-21swapper/22
41852912101,4sleep14021471-21diskmemload1
41745772101,3sleep14021471-21diskmemload1
41690962100,1sleep20-21swapper/22
41180552100,1sleep00-21swapper/00
41041672101,3sleep04021471-21diskmemload0
41010282101,3sleep00-21swapper/00
408902101,3sleep24021471-21diskmemload2
40849822101,4sleep04021471-21diskmemload0
40527082101,3sleep20-21swapper/22
40443222101,4sleep14021471-21diskmemload1
40380112101,3sleep20-21swapper/22
40266212101,3sleep04021471-21diskmemload0
40214522101,5sleep24021597-21latency_hist2
40159912101,6sleep24015992-21sshd-session2
40103182101,4sleep24010562-21users2
39746472100,1sleep00-21swapper/00
39644672100,1sleep20-21swapper/22
39444552101,5sleep23945252-21kthreadcore2
39207182101,4sleep13920785-21memory1
39207182101,4sleep13920785-21memory1
333292100,1sleep20-21swapper/22
1666972101,4sleep1166709-21ssh1
1572992101,4sleep24021471-21diskmemload2
145742100,1sleep20-21swapper/22
1434962100,6sleep1143530-21apt1
1426342101,4sleep04021471-21diskmemload0
1253322101,2sleep10-21swapper/11
73015291,3sleep173025-21ssh1
50680291,2sleep00-21swapper/00
48076291,2sleep10-21swapper/11
4186987291,4sleep04187001-21ssh0
4151782290,3sleep24021471-21diskmemload2
4118037291,3sleep24021471-21diskmemload2
4112494291,3sleep14112504-21ssh1
4063276291,2sleep20-21swapper/22
4063276291,2sleep20-21swapper/22
3995549291,4sleep03995609-21grep0
3979765291,4sleep23979769-21find2
3963475290,1sleep10-21swapper/11
3953691291,4sleep03953919-21df0
3953631290,4sleep20-21swapper/22
3948424291,4sleep23948624-21cat2
3937931291,4sleep13937953-21apt-get1
3932303290,2sleep10-21swapper/11
3921672290,2sleep20-21swapper/22
3921672290,2sleep20-21swapper/22
3903848291,4sleep0106713-21snmpd0
102667291,4sleep2104292-21sed2
4016256281,3sleep00-21swapper/00
4010943281,3sleep10-21swapper/11
3984462280,1sleep10-21swapper/11
3979225280,1sleep20-21swapper/22
3974229280,5sleep1106713-21snmpd1
3974087280,1sleep00-21swapper/00
3943553281,3sleep03943726-21if_err_enp1s00
3937954281,2sleep20-21swapper/22
3937930280,2sleep00-21swapper/00
3922178280,2sleep20-21swapper/22
3911675280,3sleep13911693-21apt-get1
3911371280,1sleep00-21swapper/00
3908127280,4sleep23909846-21cut2
3895207280,4sleep03896869-21cut0
3893909281,2sleep23893928-21apt-get2
30848281,3sleep030858-21ssh0
4021098270,1sleep00-21swapper/00
4011260270,2sleep10-21swapper/11
4005712271,2sleep20-21swapper/22
4005495270,1sleep10-21swapper/11
4000477270,1sleep10-21swapper/11
3992145272,1sleep20-21swapper/22
3989962271,2sleep10-21swapper/11
3969030271,2sleep10-21swapper/11
3947917270,3sleep10-21swapper/11
3937433270,1sleep10-21swapper/11
3932065270,1sleep20-21swapper/22
3926855270,1sleep10-21swapper/11
3924192271,3sleep03925954-21kthreadcore0
3916904270,1sleep00-21swapper/00
3916904270,1sleep00-21swapper/00
3912434271,3sleep23914030-21tr2
4000784261,2sleep04000940-21cstates0
3989741260,4sleep00-21swapper/00
3965349261,2sleep00-21swapper/00
3937394260,1sleep20-21swapper/22
3937343260,4sleep00-21swapper/00
3905944260,1sleep10-21swapper/11
4006007250,1sleep00-21swapper/00
3953420250,1sleep10-21swapper/11
3948698250,1sleep00-21swapper/00
3911973250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional