You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-11 - 07:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Thu Jun 11, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2536621211748,64sleep30-21swapper/33
253670828355,10sleep10-21swapper/11
253651627860,12sleep20-21swapper/22
253596427855,16sleep00-21swapper/00
253697299271,13cyclictest0-21swapper/33
253697299270,18cyclictest0-21swapper/33
253697299240,2cyclictest0-21swapper/33
28918132231,10sleep12667046-21diskmemload1
28918132231,10sleep12667046-21diskmemload1
253697299231,19cyclictest0-21swapper/33
253697299231,18cyclictest0-21swapper/33
26566392222,5sleep22656658-21apt-get2
253697299227,11cyclictest2629466-21turbostat3
253697299220,3cyclictest27167212sleep33
27483212212,2sleep10-21swapper/11
2536972992118,2cyclictest0-21swapper/33
2536972992111,8cyclictest28757212sleep33
253697299211,16cyclictest0-21swapper/33
253697299211,16cyclictest0-21swapper/33
253697299211,16cyclictest0-21swapper/33
253697299210,1cyclictest0-21swapper/33
28684102201,4sleep02667046-21diskmemload0
253697299207,10cyclictest2608187-21turbostat3
253697299201,5cyclictest0-21swapper/33
253697299201,4cyclictest0-21swapper/33
253697299201,4cyclictest0-21swapper/33
253697299201,3cyclictest0-21swapper/33
253697299201,16cyclictest0-21swapper/33
253697299200,16cyclictest0-21swapper/33
253697299200,11cyclictest0-21swapper/33
29171432191,2sleep10-21swapper/11
253697299194,13cyclictest2541920-21taskset3
253697299191,5cyclictest0-21swapper/33
253697299191,14cyclictest2993911-21taskset3
253697299191,14cyclictest0-21swapper/33
253697299191,14cyclictest0-21swapper/33
253697299191,13cyclictest0-21swapper/33
253697299190,8cyclictest0-21swapper/33
29600962182,3sleep02536966-21cyclictest0
27216642181,4sleep02667046-21diskmemload0
253697299183,3cyclictest66950irq/132-enp1s0-TxRx-33
253697299183,3cyclictest66950irq/132-enp1s0-TxRx-33
253697299182,13cyclictest0-21swapper/33
253697299182,11cyclictest2755136-21taskset3
253697299181,7cyclictest2707934-21chrt3
253697299181,7cyclictest2707934-21chrt3
253697299181,6cyclictest2640121-21turbostat3
253697299181,6cyclictest2554901-21turbostat3
2536972991811,5cyclictest0-21swapper/33
253697299181,14cyclictest0-21swapper/33
253697299181,14cyclictest0-21swapper/33
253697299181,14cyclictest0-21swapper/33
253697299181,14cyclictest0-21swapper/33
253697299181,14cyclictest0-21swapper/33
253697299181,14cyclictest0-21swapper/33
253697299181,14cyclictest0-21swapper/33
253697299181,13cyclictest0-21swapper/33
253697299181,13cyclictest0-21swapper/33
253697299181,13cyclictest0-21swapper/33
253697299180,2cyclictest0-21swapper/33
27306622171,3sleep02730669-21ssh0
27015362172,11sleep12703070-21kthreadcore1
27015362172,11sleep12703070-21kthreadcore1
26894052170,4sleep00-21swapper/00
25485662171,11sleep02548615-21ntpq0
253697299172,10cyclictest2656661-21taskset3
253697299171,14cyclictest0-21swapper/33
2536972991711,3cyclictest0-21swapper/33
253697299171,13cyclictest0-21swapper/33
253697299171,12cyclictest0-21swapper/33
253697299170,13cyclictest0-21swapper/33
28657902161,11sleep12667046-21diskmemload1
26985972161,3sleep10-21swapper/11
26807232160,1sleep00-21swapper/00
253697299165,2cyclictest0-21swapper/33
253697299161,3cyclictest0-21swapper/33
2536972991613,1cyclictest0-21swapper/33
2536972991611,3cyclictest2586866-21turbostat3
253697299161,13cyclictest0-21swapper/33
253697299160,6cyclictest0-21swapper/33
30186932151,3sleep00-21swapper/00
30083152151,3sleep03008379-21ntpq0
29572432151,8sleep12957302-21ntpq1
29193382150,2sleep20-21swapper/22
28586552151,2sleep10-21swapper/11
28152422151,4sleep02815254-21rm0
253697299151,11cyclictest0-21swapper/33
253697299150,13cyclictest0-21swapper/33
253697299150,13cyclictest0-21swapper/33
253697299150,13cyclictest0-21swapper/33
253697299150,12cyclictest0-21swapper/33
29020412141,4sleep22902052-21ssh2
28930912141,3sleep02667046-21diskmemload0
28930912141,3sleep02667046-21diskmemload0
28528162141,3sleep00-21swapper/00
28088512140,2sleep20-21swapper/22
27740892141,2sleep10-21swapper/11
27640382141,5sleep12764050-21ssh1
27334102141,2sleep20-21swapper/22
26902902141,7sleep12667046-21diskmemload1
253697299141,2cyclictest0-21swapper/33
253697299140,2cyclictest0-21swapper/33
253697299140,12cyclictest0-21swapper/33
30102462131,2sleep20-21swapper/22
30025352131,8sleep12992888-21kworker/u16:2+rpciod1
30023522130,8sleep20-21swapper/22
29939252131,7sleep02667046-21diskmemload0
29844882131,3sleep02667046-21diskmemload0
29668552131,8sleep22966867-21ssh2
28500672131,8sleep12850077-21ssh1
28417272130,1sleep00-21swapper/00
27989962131,7sleep12667046-21diskmemload1
27252852131,6sleep20-21swapper/22
27115372130,2sleep0221rcuc/00
26838942131,7sleep22667046-21diskmemload2
26284862131,6sleep02628539-21ntpq0
26084462131,8sleep02608503-21sort0
25858372131,6sleep02585884-21ntpq0
29928922121,7sleep12992902-21ssh1
29445162121,5sleep22944744-21diskstats2
29328682121,3sleep22932878-21ssh2
29266422120,2sleep1171rcu_preempt1
29085982120,4sleep20-21swapper/22
28749972120,8sleep00-21swapper/00
28521602121,5sleep20-21swapper/22
27894792121,7sleep22789699-21ssh2
27741282121,7sleep02774138-21ssh0
27650002120,3sleep00-21swapper/00
27144722121,6sleep22667046-21diskmemload2
27142252120,2sleep1171rcu_preempt1
26993752121,6sleep02667046-21diskmemload0
26675672121,6sleep22667731-21diskstats2
26137752121,7sleep22613930-21cstates2
29934762110,4sleep20-21swapper/22
29854822111,4sleep12667046-21diskmemload1
29771282111,2sleep20-21swapper/22
29667402110,2sleep02966751-21ssh0
29423262113,3sleep22667046-21diskmemload2
29329772110,5sleep12667046-21diskmemload1
29171542111,5sleep22667046-21diskmemload2
29093262111,3sleep10-21swapper/11
28914002111,5sleep22667046-21diskmemload2
28914002111,5sleep22667046-21diskmemload2
28331302111,6sleep12833141-21ssh1
28310342110,5sleep02831104-21ssh0
28227942111,3sleep00-21swapper/00
27988692111,4sleep22667046-21diskmemload2
27837942111,3sleep20-21swapper/22
27569862111,3sleep10-21swapper/11
27456942111,5sleep02746545-21memory0
27322382111,6sleep12732249-21ssh1
26826672111,2sleep22682680-21ssh2
26731442111,3sleep10-21swapper/11
26674082111,4sleep00-21swapper/00
26622622110,4sleep10-21swapper/11
25714272111,5sleep12571636-21hddtemp_smartct1
25538752111,6sleep12553929-21ntpq1
25422442111,7sleep02542319-21latency_hist0
30205572101,2sleep20-21swapper/22
30196552101,4sleep12667046-21diskmemload1
30026142101,2sleep00-21swapper/00
29836222101,2sleep20-21swapper/22
29724282103,3sleep12974079-21latency1
29623022101,5sleep12963245-21kthreadcore1
29583722100,6sleep22958382-21ssh2
29514512101,4sleep02667046-21diskmemload0
29436352101,2sleep00-21swapper/00
29413872101,4sleep12667046-21diskmemload1
29269082101,2sleep00-21swapper/00
29103442101,3sleep00-21swapper/00
28773082101,4sleep12667046-21diskmemload1
28732062101,4sleep22873271-21ntp_states2
28696362101,6sleep12871239-21cut1
28606372101,4sleep22860850-21diskstats2
28488612100,1sleep20-21swapper/22
28487812101,4sleep02667046-21diskmemload0
28415592101,4sleep22667046-21diskmemload2
28253072101,4sleep22667046-21diskmemload2
28169212101,4sleep12667046-21diskmemload1
28081062101,3sleep10-21swapper/11
27933682101,5sleep02793635-21df_inode0
27924022100,6sleep02792415-21sh0
27904922101,4sleep12667046-21diskmemload1
27840152101,4sleep12667046-21diskmemload1
27834442101,3sleep00-21swapper/00
27731622101,3sleep20-21swapper/22
27580052101,3sleep00-21swapper/00
27550832101,4sleep22667046-21diskmemload2
27414682101,3sleep00-21swapper/00
27397232101,4sleep12667046-21diskmemload1
27051852101,4sleep02667046-21diskmemload0
27051852101,4sleep02667046-21diskmemload0
27049842101,4sleep22705051-21ntp_states2
27049842101,4sleep22705051-21ntp_states2
26670192100,4sleep00-21swapper/00
26443372101,4sleep12644426-21ntpq1
26406552101,5sleep02640804-21grep0
26352082101,4sleep02635331-21diskstats0
26297122100,3sleep22629731-21apt-get2
26282932101,4sleep22628434-21ntpq2
26190802101,4sleep22619338-21df_inode2
25927332101,5sleep12592946-21hddtemp_smartct1
25660962100,6sleep12565808-21/usr/sbin/munin1
25646612101,2sleep2171rcu_preempt2
3007673291,4sleep10-21swapper/11
2977579291,3sleep02667046-21diskmemload0
2949552291,3sleep12667046-21diskmemload1
2933733291,3sleep02667046-21diskmemload0
2917672291,3sleep02667046-21diskmemload0
2899153291,4sleep12899163-21ssh1
2898993290,2sleep00-21swapper/00
2883588294,1sleep20-21swapper/22
2877368291,4sleep02877341-21/usr/sbin/munin0
2832954291,3sleep22667046-21diskmemload2
2815761291,3sleep20-21swapper/22
2806483291,4sleep02667046-21diskmemload0
2764189291,4sleep22764200-21ssh2
2743163290,5sleep22744004-21kthreadcore2
2739644291,4sleep22739658-21ssh2
2724556291,2sleep10-21swapper/11
2696562291,4sleep22696628-21/usr/sbin/munin2
2680776291,3sleep12667046-21diskmemload1
2646229291,4sleep02646569-21cut0
2624670291,4sleep12624794-21perl1
2608712291,5sleep22608922-21/usr/sbin/munin2
2608504290,6sleep10-21swapper/11
2597791291,5sleep12597956-21cstates1
2597769291,3sleep02597788-21apt-get0
2596350291,4sleep22596490-21ntp_192.168.1142
2587117291,2sleep00-21swapper/00
2576755291,4sleep02576965-21hddtemp_smartct0
2573102291,4sleep02574952-21cut0
2571136291,3sleep22571155-21apt-get2
2561335291,4sleep02562987-21grep0
2560220291,4sleep12560247-21munin-run1
2555150291,3sleep02555169-21apt-get0
2542778291,4sleep22544276-21users2
2835236281,3sleep12819997-21kworker/u16:1+rpciod1
2822518280,2sleep10-21swapper/11
2655631280,4sleep00-21swapper/00
2640937281,3sleep20-21swapper/22
2640074280,1sleep10-21swapper/11
2634726280,1sleep00-21swapper/00
2634452280,1sleep10-21swapper/11
2619058280,3sleep10-21swapper/11
2591836280,2sleep20-21swapper/22
2582066281,3sleep22582128-21fschecks_count2
2581254280,2sleep20-21swapper/22
2569900280,3sleep00-21swapper/00
2565196280,1sleep10-21swapper/11
2542320281,3sleep12542772-21ps1
2661967270,2sleep20-21swapper/22
2656332270,1sleep10-21swapper/11
2623788270,1sleep00-21swapper/00
2607869270,1sleep00-21swapper/00
2603099270,2sleep10-21swapper/11
2592436270,1sleep00-21swapper/00
2591939270,1sleep10-21swapper/11
2551199271,2sleep20-21swapper/22
2542201270,1sleep20-21swapper/22
2661395260,4sleep10-21swapper/11
2658931261,2sleep00-21swapper/00
2650068260,2sleep20-21swapper/22
2646570261,2sleep10-21swapper/11
2635042261,1sleep22635061-21apt-get2
2617755260,2sleep10-21swapper/11
2617570261,2sleep00-21swapper/00
2604074261,2sleep22605763-21cut2
2601890260,2sleep20-21swapper/22
2582891261,2sleep10-21swapper/11
2570851260,4sleep20-21swapper/22
2651191250,1sleep20-21swapper/22
2576462250,1sleep12576481-21apt-get1
2559523250,1sleep20-21swapper/22
2549616240,1sleep00-21swapper/00
2537427240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional