You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 18:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Fri May 01, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
980929212349,68sleep30-21swapper/33
98063228667,12sleep20-21swapper/22
98077928063,10sleep10-21swapper/11
98077727551,17sleep00-21swapper/00
981112992217,3cyclictest0-21swapper/33
98111299221,17cyclictest66950irq/132-enp1s0-TxRx-33
98111299221,17cyclictest1230044-21taskset3
98111299220,18cyclictest1375106-21chrt3
117337322211,3sleep21175075-21kthreadcore2
98111299210,18cyclictest66950irq/132-enp1s0-TxRx-33
98111299201,16cyclictest1245041-21taskset3
98111299201,14cyclictest0-21swapper/33
14609672202,5sleep11460995-21apt-get1
14234232201,3sleep00-21swapper/00
14232542200,12sleep134-21ksoftirqd/11
98111299199,6cyclictest12874722sleep33
981112991914,3cyclictest66950irq/132-enp1s0-TxRx-33
981112991912,5cyclictest0-21swapper/33
98111299191,15cyclictest0-21swapper/33
98111299191,14cyclictest0-21swapper/33
98111299191,14cyclictest0-21swapper/33
98111299191,14cyclictest0-21swapper/33
98111299191,14cyclictest0-21swapper/33
98111299191,14cyclictest0-21swapper/33
98111299191,14cyclictest0-21swapper/33
98111299191,14cyclictest0-21swapper/33
98111299191,14cyclictest0-21swapper/33
12010282190,15sleep11201168-21ssh1
98111299181,5cyclictest0-21swapper/33
98111299181,5cyclictest0-21swapper/33
981112991814,2cyclictest0-21swapper/33
981112991812,4cyclictest0-21swapper/33
981112991811,4cyclictest66950irq/132-enp1s0-TxRx-33
981112991811,4cyclictest0-21swapper/33
98111299181,14cyclictest0-21swapper/33
98111299181,14cyclictest0-21swapper/33
98111299181,14cyclictest0-21swapper/33
98111299181,14cyclictest0-21swapper/33
98111299181,14cyclictest0-21swapper/33
98111299181,14cyclictest0-21swapper/33
98111299181,14cyclictest0-21swapper/33
98111299181,14cyclictest0-21swapper/33
98111299181,13cyclictest1147017-21taskset3
98111299181,13cyclictest0-21swapper/33
98111299181,13cyclictest0-21swapper/33
98111299181,12cyclictest1111790-21sleep3
98111299180,15cyclictest0-21swapper/33
98111299180,15cyclictest0-21swapper/33
12903742181,2sleep20-21swapper/22
98111299177,6cyclictest13083282sleep33
98111299171,13cyclictest1171301-21turbostat3
98111299171,13cyclictest0-21swapper/33
98111299171,13cyclictest0-21swapper/33
98111299171,13cyclictest0-21swapper/33
98111299171,13cyclictest0-21swapper/33
98111299171,12cyclictest1057716-21taskset3
98111299170,9cyclictest0-21swapper/33
98111299170,15cyclictest0-21swapper/33
14653032171,10sleep01465402-21ps0
14583062171,11sleep11111764-21diskmemload1
12722672170,8sleep01111764-21diskmemload0
12070362172,11sleep00-21swapper/00
10858602171,7sleep01086600-21grep0
981112991613,2cyclictest0-21swapper/33
98111299160,13cyclictest0-21swapper/33
98111299160,13cyclictest0-21swapper/33
98111299160,13cyclictest0-21swapper/33
98111299160,13cyclictest0-21swapper/33
98111299160,13cyclictest0-21swapper/33
98111299160,13cyclictest0-21swapper/33
98111299160,13cyclictest0-21swapper/33
98111299160,12cyclictest0-21swapper/33
13564992160,1sleep10-21swapper/11
13464092161,11sleep11111764-21diskmemload1
13388282161,3sleep20-21swapper/22
12275722162,4sleep01111764-21diskmemload0
12016282161,4sleep01201667-21sed0
11802362161,9sleep21180377-21diskstats2
98111299159,4cyclictest0-21swapper/33
98111299156,6cyclictest10311052sleep33
981112991512,2cyclictest0-21swapper/33
98111299150,13cyclictest0-21swapper/33
98111299150,12cyclictest0-21swapper/33
14347782150,1sleep10-21swapper/11
13389072151,9sleep01111764-21diskmemload0
12905402151,4sleep01111764-21diskmemload0
12367182151,3sleep01111764-21diskmemload0
11876912151,2sleep00-21swapper/00
11608872150,1sleep00-21swapper/00
11452342151,2sleep00-21swapper/00
981112991411,2cyclictest0-21swapper/33
981112991411,2cyclictest0-21swapper/33
98111299141,10cyclictest0-21swapper/33
14564202141,7sleep21456479-21ntpq2
14480442141,7sleep01448103-21ntpq0
14411622141,3sleep21111764-21diskmemload2
14337772140,2sleep00-21swapper/00
13666072141,5sleep11111764-21diskmemload1
12721072141,8sleep21111764-21diskmemload2
11758612141,6sleep11175915-21ssh1
11253082141,7sleep11125562-21users1
9891142131,8sleep2989209-21kthreadcore2
98111299131,8cyclictest1047629-21chrt3
981112991311,1cyclictest0-21swapper/33
14151342130,9sleep01415149-21ssh0
13152292131,8sleep11315239-21ssh1
12559082131,5sleep20-21swapper/22
12211582130,9sleep11221172-21ssh1
12087192131,8sleep11209618-21sed1
11542052131,7sleep11111764-21diskmemload1
11429882131,5sleep20-21swapper/22
11284072131,8sleep21128417-21ssh2
11186272131,8sleep01111764-21diskmemload0
10840352131,7sleep11084084-21ntpq1
10257802131,8sleep01025860-21cat0
9998412124,5sleep0999890-21kthreadcore0
9942392121,7sleep1994349-21fschecks_time1
14543882121,7sleep01456199-21cut0
14262382121,2sleep21111764-21diskmemload2
13898712121,7sleep01111764-21diskmemload0
13727132121,5sleep2802-21snmpd2
13508242121,7sleep01350969-21gltestperf0
13505422121,7sleep21350522-21/usr/sbin/munin2
13486222121,2sleep00-21swapper/00
13412302120,7sleep11111764-21diskmemload1
13315782121,1sleep10-21swapper/11
13149552121,3sleep21314966-21ssh2
13143112120,5sleep00-21swapper/00
12912502121,7sleep11291296-21cut1
12738062121,5sleep20-21swapper/22
12397672121,5sleep21239788-21apt-get2
11927992121,5sleep11192858-21ntpq1
11779932121,5sleep00-21swapper/00
11631732120,2sleep10-21swapper/11
11518622121,7sleep01111764-21diskmemload0
11337692121,7sleep11133884-21/usr/sbin/munin1
11053662121,6sleep11105414-21ntpq1
10797062121,7sleep21079785-21grep2
10534722121,7sleep21053666-21grep2
10517492121,5sleep01051806-21ntpq0
10086332121,5sleep21008681-21ntpq2
9867022111,5sleep0987488-21tr0
14691092110,7sleep21469363-21cat2
14484622111,6sleep21448544-21threads2
14445802115,3sleep10-21swapper/11
14429302111,5sleep01111764-21diskmemload0
14396782111,4sleep11439734-21ntpq1
13988412112,4sleep21111764-21diskmemload2
13898822111,3sleep10-21swapper/11
13815162111,4sleep11111764-21diskmemload1
13663912111,6sleep01366404-21ssh0
13589152112,3sleep21348740-21kworker/u16:1-kvfree_rcu_reclaim2
13464732111,5sleep21346537-21ntpq2
13166932111,5sleep11316925-21df_inode1
12912972111,4sleep20-21swapper/22
12647992111,5sleep00-21swapper/00
12637672111,7sleep21263780-21ssh2
12446602110,2sleep01244671-21ssh0
12023532111,5sleep21111764-21diskmemload2
11927372112,4sleep01192789-21ntpq0
11618262111,3sleep10-21swapper/11
11378632111,6sleep11138082-21head1
11344522111,4sleep21111764-21diskmemload2
11343292111,3sleep00-21swapper/00
11196902111,6sleep21119704-21ssh2
11016222111,5sleep01101831-21hddtemp_smartct0
10918412113,4sleep01093577-21grep0
10695672115,2sleep2421rcuc/22
10516552111,5sleep21051693-21ntpq2
10331812111,6sleep11035012-21grep1
10319322111,6sleep01033180-21grep0
10099402111,5sleep01009958-21apt-get0
9830882103,3sleep0984907-21grep0
14334252101,3sleep20-21swapper/22
14146722101,4sleep11111764-21diskmemload1
14144972101,4sleep21414507-21ssh2
14063822101,4sleep01111764-21diskmemload0
13878322104,3sleep21388329-21ssh2
13760182101,4sleep21376101-21irqstats2
13749562101,2sleep10-21swapper/11
13218042101,4sleep01111764-21diskmemload0
13047332101,2sleep10-21swapper/11
13044562101,2sleep20-21swapper/22
12968502101,4sleep01111764-21diskmemload0
12871252101,4sleep11111764-21diskmemload1
12734522101,4sleep11111764-21diskmemload1
12612012101,3sleep10-21swapper/11
12454052101,4sleep11111764-21diskmemload1
12365652101,2sleep10-21swapper/11
12362932101,3sleep20-21swapper/22
12294232101,2sleep20-21swapper/22
12209622101,4sleep01111764-21diskmemload0
12139312101,4sleep21111764-21diskmemload2
12106962101,4sleep21111764-21diskmemload2
11957722101,5sleep21195782-21ssh2
11861612101,3sleep10-21swapper/11
11705702101,5sleep21046919-21kworker/u16:1+xprtiod2
11611462100,2sleep20-21swapper/22
11530542101,2sleep20-21swapper/22
11274792101,2sleep00-21swapper/00
11183452101,3sleep10-21swapper/11
10971312101,5sleep21098882-21grep2
10945702101,4sleep21094711-21ntpq2
10687482101,5sleep01068986-21cat0
10451112101,5sleep21045958-21grep2
10260372101,4sleep21026056-21apt-get2
993980291,4sleep0994238-21df_inode0
1406679291,4sleep21111764-21diskmemload2
1401657290,5sleep11401765-21kthreadcore1
1397739291,3sleep01111764-21diskmemload0
1395931290,6sleep11396695-21cat1
1377379291,4sleep00-21swapper/00
1373371291,3sleep01111764-21diskmemload0
1327640291,3sleep20-21swapper/22
1325284291,2sleep00-21swapper/00
1324120291,4sleep21324133-21sh2
1300240291,4sleep01301861-21rm0
1280094291,3sleep11111764-21diskmemload1
1279856291,4sleep01279867-21ssh0
1252653290,4sleep01111764-21diskmemload0
1226919291,4sleep11226980-21ntpq1
1107294291,4sleep11108947-21cut1
1085328291,4sleep11085586-21df_inode1
1073936290,4sleep10-21swapper/11
1057762290,2sleep11057744-21fwupd1
1057761290,1sleep00-21swapper/00
1046873291,5sleep01046874-21pool-udisksd0
1022463291,4sleep11024287-21kthreadcore1
1014923291,3sleep11014863-31find1
1010222291,4sleep21010346-21perl2
986355280,3sleep2986386-21perf2
985460281,4sleep1985546-21grep1
1248387281,3sleep11111764-21diskmemload1
1168703281,3sleep01168710-21ssh0
1110726281,2sleep01110772-21ntpq0
1100938280,2sleep00-21swapper/00
1095988280,2sleep10-21swapper/11
1088343281,3sleep20-21swapper/22
1079987280,3sleep01080006-21apt-get0
1078656281,3sleep10-21swapper/11
1074360281,3sleep01074106-21apt-get0
1067567281,3sleep21063674-21/usr/sbin/munin2
1036715281,2sleep01036734-21apt-get0
1002360281,4sleep21003204-21sort2
998669270,1sleep20-21swapper/22
1111060271,3sleep21111252-21timerwakeupswit2
1106240270,1sleep20-21swapper/22
1084727270,1sleep20-21swapper/22
1062332271,1sleep00-21swapper/00
1062332271,1sleep00-21swapper/00
1046760270,1sleep10-21swapper/11
1041518270,1sleep20-21swapper/22
1036737271,2sleep11036969-21expr1
1026315270,3sleep01026370-21grep0
1025082270,3sleep21025122-21wc2
1020395270,1sleep10-21swapper/11
1004903271,3sleep11005113-21/usr/sbin/munin1
1004089270,1sleep10-21swapper/11
993636260,1sleep10-21swapper/11
1095042261,2sleep10-21swapper/11
1073754260,4sleep00-21swapper/00
1068590260,1sleep10-21swapper/11
1060085261,2sleep20-21swapper/22
1060085261,2sleep20-21swapper/22
1058903261,2sleep10-21swapper/11
1058903261,2sleep10-21swapper/11
1030960260,1sleep10-21swapper/11
1020422260,2sleep20-21swapper/22
1006530261,2sleep00-21swapper/00
1047591250,2sleep10-21swapper/11
1035013250,1sleep20-21swapper/22
1016297251,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional