You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 17:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Sun Jul 05, 2026 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7130621460,3sleep10-21swapper/11
3877386212052,62sleep30-21swapper/33
3877225211596,12sleep10-21swapper/11
387733528062,12sleep20-21swapper/22
387721927952,19sleep00-21swapper/00
41820062490,3sleep0221rcuc/00
387772399251,8cyclictest40298552sleep33
387772399235,14cyclictest66950irq/132-enp1s0-TxRx-33
387772399231,19cyclictest0-21swapper/33
387772399231,18cyclictest4191607-21taskset3
387772399221,17cyclictest0-21swapper/33
793092213,9sleep134-21ksoftirqd/11
41599032210,1sleep14078612-21kworker/u16:1+kvfree_rcu_reclaim1
387772399212,14cyclictest4054597-21chrt3
3877723992118,1cyclictest0-21swapper/33
3877723992118,1cyclictest0-21swapper/33
387772399201,3cyclictest0-21swapper/33
387772399201,16cyclictest66950irq/132-enp1s0-TxRx-33
387772399201,14cyclictest0-21swapper/33
387772399200,18cyclictest0-21swapper/33
41840932192,13sleep24004875-21diskmemload2
41681662191,7sleep04004875-21diskmemload0
387772399192,14cyclictest66950irq/132-enp1s0-TxRx-33
387772399192,12cyclictest4038153-21taskset3
3877723991911,5cyclictest66950irq/132-enp1s0-TxRx-33
387772399191,15cyclictest0-21swapper/33
387772399191,15cyclictest0-21swapper/33
387772399191,14cyclictest0-21swapper/33
387772399191,14cyclictest0-21swapper/33
387772399191,14cyclictest0-21swapper/33
387772399190,14cyclictest0-21swapper/33
387772399187,8cyclictest3973697-21turbostat3
3877723991815,1cyclictest0-21swapper/33
3877723991812,4cyclictest0-21swapper/33
3877723991812,3cyclictest0-21swapper/33
387772399181,15cyclictest0-21swapper/33
387772399181,14cyclictest0-21swapper/33
387772399181,14cyclictest0-21swapper/33
387772399181,14cyclictest0-21swapper/33
387772399181,14cyclictest0-21swapper/33
387772399181,14cyclictest0-21swapper/33
387772399181,14cyclictest0-21swapper/33
387772399181,14cyclictest0-21swapper/33
387772399181,13cyclictest0-21swapper/33
387772399181,13cyclictest0-21swapper/33
387772399181,13cyclictest0-21swapper/33
387772399180,14cyclictest0-21swapper/33
387772399180,14cyclictest0-21swapper/33
387772399180,14cyclictest0-21swapper/33
1237702181,2sleep00-21swapper/00
41102922171,3sleep00-21swapper/00
40858702171,4sleep24004875-21diskmemload2
387772399177,6cyclictest4004883-21turbostat3
3877723991714,2cyclictest0-21swapper/33
3877723991714,1cyclictest0-21swapper/33
3877723991713,2cyclictest0-21swapper/33
3877723991712,3cyclictest0-21swapper/33
387772399171,12cyclictest0-21swapper/33
387772399171,12cyclictest0-21swapper/33
387772399171,12cyclictest0-21swapper/33
387772399171,11cyclictest3963111-21taskset3
387772399170,14cyclictest0-21swapper/33
387772399170,14cyclictest0-21swapper/33
387772399170,14cyclictest0-21swapper/33
387772399170,13cyclictest0-21swapper/33
72172161,2sleep20-21swapper/22
41931142161,3sleep20-21swapper/22
41410422161,3sleep20-21swapper/22
40198462161,4sleep14004875-21diskmemload1
387772399162,9cyclictest3911414-21taskset3
387772399160,13cyclictest0-21swapper/33
387772399160,13cyclictest0-21swapper/33
387772399160,13cyclictest0-21swapper/33
387772399160,13cyclictest0-21swapper/33
387772399160,13cyclictest0-21swapper/33
387772399160,12cyclictest0-21swapper/33
371112161,4sleep04004875-21diskmemload0
173852161,9sleep072-21kcompactd00
1534972161,4sleep24004875-21diskmemload2
735112150,1sleep20-21swapper/22
633872151,2sleep10-21swapper/11
578032151,9sleep057817-21ssh0
41724052151,10sleep14174090-21cut1
41010822150,2sleep20-21swapper/22
40930512151,2sleep20-21swapper/22
40697842151,2sleep00-21swapper/00
40188592151,4sleep24004875-21diskmemload2
39947342155,2sleep2171rcu_preempt2
387772399159,3cyclictest0-21swapper/33
387772399158,4cyclictest3895252-21chrt3
387772399150,13cyclictest0-21swapper/33
387772399150,13cyclictest0-21swapper/33
387772399150,13cyclictest0-21swapper/33
387772399150,13cyclictest0-21swapper/33
387772399150,13cyclictest0-21swapper/33
387772399150,12cyclictest0-21swapper/33
1211282150,1sleep20-21swapper/22
41206042141,8sleep04004875-21diskmemload0
41029272141,2sleep10-21swapper/11
40781302141,3sleep14078142-21ssh1
40676082141,2sleep20-21swapper/22
387772399140,12cyclictest0-21swapper/33
221942141,8sleep14004875-21diskmemload1
1463872141,8sleep1146397-21ssh1
41832732131,8sleep14183287-21ssh1
41361112130,2sleep10-21swapper/11
41172672131,3sleep14117277-21ssh1
40545642132,5sleep24054586-21apt-get2
40264332131,8sleep04026437-21ssh0
39739592131,7sleep03973741-21sendmail-msp0
39469052131,8sleep13946997-21wc1
3877723991311,1cyclictest0-21swapper/33
30022131,8sleep13038-21kthreadcore1
993402121,5sleep00-21swapper/00
824152121,5sleep00-21swapper/00
797682122,6sleep0106713-21snmpd0
502002121,6sleep250463-21df2
497522121,7sleep049953-21latency_hist0
493452120,3sleep249358-21ssh2
489752121,5sleep10-21swapper/11
41603672120,6sleep00-21swapper/00
41452672122,4sleep24145295-21apt-get2
40780932121,3sleep24078104-21ssh2
40519822120,3sleep24051993-21ssh2
40458132121,7sleep04045815-21ssh0
40218752121,7sleep14022099-21smartctl1
39739922122,6sleep13974042-21cpuspeed_turbos1
39374842120,7sleep13937196-21apt-get1
39374842120,7sleep13937196-21apt-get1
39165932121,5sleep03916605-21apt-get0
39110962121,7sleep13911137-21munin-run1
39058312121,6sleep23905835-21cron2
157992120,2sleep20-21swapper/22
1524012120,2sleep00-21swapper/00
1137972120,1sleep272-21kcompactd02
911222111,5sleep191149-21ssh1
881732111,6sleep24004875-21diskmemload2
742032111,6sleep039542-21systemd-journal0
568732111,4sleep14004875-21diskmemload1
41689362110,2sleep20-21swapper/22
41609602111,4sleep24004875-21diskmemload2
41516242110,1sleep00-21swapper/00
41431702111,4sleep14004875-21diskmemload1
41205402111,3sleep20-21swapper/22
41094142111,5sleep14004875-21diskmemload1
40802812110,3sleep14081356-21kthreadcore1
40779732110,1sleep00-21swapper/00
40596152110,1sleep00-21swapper/00
40202952112,4sleep04004875-21diskmemload0
40116692111,5sleep04004875-21diskmemload0
39938442111,5sleep03994060-21users0
39635492111,4sleep23963568-21apt-get2
39598492111,6sleep03961541-21grep0
39427892111,5sleep03943007-21df_inode0
39319172111,5sleep13931993-21systemd-run1
39319172111,5sleep13931993-21systemd-run1
39154982111,6sleep03915641-21sed0
39061072111,3sleep00-21swapper/00
39055422110,1sleep10-21swapper/11
38797252111,6sleep13881464-21grep1
332612110,2sleep00-21swapper/00
163492111,4sleep14004875-21diskmemload1
1398292111,6sleep1139839-21ssh1
1310372111,2sleep20-21swapper/22
1304162111,4sleep14004875-21diskmemload1
1209082111,6sleep1120913-21rm1
1048262111,5sleep14004875-21diskmemload1
1046482110,1sleep20-21swapper/22
987672101,3sleep00-21swapper/00
634482101,3sleep24004875-21diskmemload2
41918592101,2sleep00-21swapper/00
41755072100,2sleep20-21swapper/22
41703352101,5sleep04170570-21expr0
41698812101,4sleep14004875-21diskmemload1
41511652100,3sleep10-21swapper/11
41369372101,2sleep00-21swapper/00
41256012101,3sleep14004875-21diskmemload1
41185602100,1sleep00-21swapper/00
41173582101,4sleep24117372-21ssh2
41023092101,4sleep04004875-21diskmemload0
40865622102,5sleep00-21swapper/00
40680662101,3sleep10-21swapper/11
40598362101,3sleep10-21swapper/11
40525612101,4sleep14004875-21diskmemload1
40463212100,3sleep04046338-21ssh0
40371802101,4sleep04004875-21diskmemload0
40355612100,2sleep14004875-21diskmemload1
40352892101,3sleep20-21swapper/22
40215542101,3sleep24021573-21apt-get2
40105782101,5sleep24010592-21ssh2
40055532100,2sleep10-21swapper/11
400112100,6sleep140022-21ssh1
40007102101,5sleep14002298-21kthreadcore1
39885252101,6sleep10-21swapper/11
39883802100,6sleep03988524-21mailstats0
39807962101,5sleep13982507-21kthreadcore1
39740552101,4sleep23974258-21diskstats2
39557782101,4sleep23956974-21memory2
39322482101,4sleep13932266-21apt-get1
39322482101,4sleep13932266-21apt-get1
39272702101,4sleep03927479-21hddtemp_smartct0
39272702101,4sleep03927479-21hddtemp_smartct0
38779112101,5sleep03877998-21tr0
331832101,2sleep10-21swapper/11
226442101,3sleep20-21swapper/22
156782100,1sleep00-21swapper/00
1540052101,3sleep10-21swapper/11
1476212101,2sleep00-21swapper/00
1454622101,4sleep24004875-21diskmemload2
1388652101,3sleep20-21swapper/22
1321142101,4sleep0132138-21apt-get0
1213262101,3sleep00-21swapper/00
91744291,4sleep20-21swapper/22
88672291,3sleep10-21swapper/11
80334291,3sleep24004875-21diskmemload2
65777291,2sleep00-21swapper/00
4193473291,2sleep10-21swapper/11
4134938291,2sleep00-21swapper/00
4134536290,2sleep20-21swapper/22
4110459291,2sleep20-21swapper/22
4091490291,4sleep14091628-21ssh1
4045688290,1sleep20-21swapper/22
4043503290,1sleep10-21swapper/11
40357290,1sleep20-21swapper/22
3999954291,3sleep03999966-21apt-get0
3999537290,2sleep10-21swapper/11
3990093291,5sleep13991004-21kthreadcore1
3962556291,4sleep13962781-21sed1
3953160291,3sleep03953163-21apt-get0
3952849291,3sleep13952928-21systemd-run1
3927001291,4sleep23927074-21grep2
3927001291,4sleep23927074-21grep2
3922348291,4sleep13923003-21cut1
3917465291,4sleep13919083-21cut1
3911392291,3sleep20-21swapper/22
3900279290,1sleep00-21swapper/00
3890410291,2sleep20-21swapper/22
3885697291,4sleep23886612-21cut2
3885130291,3sleep13885149-21apt-get1
3878178291,4sleep23878388-21hddtemp_smartct2
30937290,3sleep20-21swapper/22
1530291,3sleep00-21swapper/00
107443291,3sleep14004875-21diskmemload1
107392291,2sleep00-21swapper/00
4092848280,4sleep00-21swapper/00
3973010281,3sleep13973273-21unixbench_singl1
3954151281,3sleep13955775-21kthreadcore1
3921815280,1sleep20-21swapper/22
3901434281,4sleep03902187-21kthreadcore0
3882860281,3sleep03883216-21ntp_192.168.1140
3991006270,3sleep2421rcuc/22
3984364270,2sleep20-21swapper/22
3983369270,3sleep23983595-21unixbench-2d2
3968760270,2sleep20-21swapper/22
3968748270,1sleep00-21swapper/00
3952599270,1sleep00-21swapper/00
3949259271,2sleep20-21swapper/22
3942721270,2sleep23942740-21apt-get2
3942164270,1sleep00-21swapper/00
3942164270,1sleep00-21swapper/00
3936827270,1sleep00-21swapper/00
3936827270,1sleep00-21swapper/00
3910776270,1sleep20-21swapper/22
3900326270,1sleep20-21swapper/22
3894564271,3sleep03894763-21cat0
3894248270,4sleep10-21swapper/11
4004037261,2sleep20-21swapper/22
3994732260,1sleep00-21swapper/00
3965055261,1sleep00-21swapper/00
3942136260,1sleep20-21swapper/22
3942136260,1sleep20-21swapper/22
3936287260,1sleep20-21swapper/22
3936287260,1sleep20-21swapper/22
3926345260,4sleep00-21swapper/00
3921327260,4sleep20-21swapper/22
3911380260,2sleep10-21swapper/11
3895618260,4sleep10-21swapper/11
3979201250,1sleep00-21swapper/00
3966751250,1sleep10-21swapper/11
3958336250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional