You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 02:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sun Apr 19, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2929661211956,58sleep30-21swapper/33
292973928052,12sleep20-21swapper/22
292989527862,10sleep10-21swapper/11
292978527858,14sleep00-21swapper/00
312934723112,8sleep03129358-21ssh0
293020299231,18cyclictest0-21swapper/33
293020299231,14cyclictest0-21swapper/33
293020299218,1cyclictest0-21swapper/33
293020299211,17cyclictest0-21swapper/33
293020299211,16cyclictest0-21swapper/33
293020299211,15cyclictest0-21swapper/33
33043182201,7sleep13061045-21diskmemload1
293020299202,14cyclictest3280893-21taskset3
293020299202,12cyclictest3140565-21taskset3
2930202992017,1cyclictest0-21swapper/33
2930202992013,5cyclictest0-21swapper/33
293020299201,15cyclictest0-21swapper/33
2930202991916,2cyclictest0-21swapper/33
2930202991916,1cyclictest0-21swapper/33
2930202991915,2cyclictest0-21swapper/33
2930202991915,2cyclictest0-21swapper/33
293020299191,15cyclictest66950irq/132-enp1s0-TxRx-33
293020299191,15cyclictest0-21swapper/33
293020299191,15cyclictest0-21swapper/33
293020299191,15cyclictest0-21swapper/33
293020299191,15cyclictest0-21swapper/33
293020299191,14cyclictest3250991-21taskset3
293020299191,14cyclictest0-21swapper/33
293020299191,14cyclictest0-21swapper/33
293020299191,14cyclictest0-21swapper/33
293020299191,14cyclictest0-21swapper/33
293020299191,14cyclictest0-21swapper/33
293020299191,13cyclictest34029422sleep33
293020299190,13cyclictest0-21swapper/33
32700662181,2sleep20-21swapper/22
293020299189,6cyclictest2942968-21turbostat3
2930202991815,1cyclictest0-21swapper/33
2930202991811,4cyclictest3060813-21taskset3
293020299181,14cyclictest0-21swapper/33
293020299181,14cyclictest0-21swapper/33
293020299181,14cyclictest0-21swapper/33
293020299181,14cyclictest0-21swapper/33
293020299181,14cyclictest0-21swapper/33
293020299181,14cyclictest0-21swapper/33
293020299181,13cyclictest0-21swapper/33
293020299181,13cyclictest0-21swapper/33
293020299181,13cyclictest0-21swapper/33
293020299181,13cyclictest0-21swapper/33
293020299180,14cyclictest0-21swapper/33
34273092171,2sleep00-21swapper/00
32963502171,2sleep10-21swapper/11
32780722171,3sleep20-21swapper/22
32611322177,6sleep2431ktimers/22
32357252171,3sleep20-21swapper/22
32247852171,3sleep20-21swapper/22
31132902171,3sleep20-21swapper/22
293020299171,3cyclictest2980411-21taskset3
2930202991711,4cyclictest30503502sleep33
293020299171,13cyclictest0-21swapper/33
293020299171,13cyclictest0-21swapper/33
293020299171,13cyclictest0-21swapper/33
293020299171,12cyclictest0-21swapper/33
293020299170,14cyclictest66950irq/132-enp1s0-TxRx-33
293020299170,14cyclictest0-21swapper/33
293020299170,13cyclictest0-21swapper/33
293020299170,13cyclictest0-21swapper/33
34167732161,2sleep00-21swapper/00
34167732161,2sleep00-21swapper/00
33219342161,4sleep03321948-21ssh0
33115972161,2sleep10-21swapper/11
32451682161,3sleep00-21swapper/00
32159642161,2sleep20-21swapper/22
31227772161,2sleep20-21swapper/22
31042502161,3sleep00-21swapper/00
30854082160,1sleep10-21swapper/11
30715672162,7sleep00-21swapper/00
293020299161,14cyclictest0-21swapper/33
293020299161,13cyclictest0-21swapper/33
293020299161,13cyclictest0-21swapper/33
293020299161,12cyclictest0-21swapper/33
293020299161,12cyclictest0-21swapper/33
293020299160,14cyclictest0-21swapper/33
293020299160,13cyclictest0-21swapper/33
293020299160,13cyclictest0-21swapper/33
32947952151,3sleep23061045-21diskmemload2
31807382151,4sleep13061045-21diskmemload1
31348382152,7sleep13135811-21memory1
31105342151,2sleep03061045-21diskmemload0
29526102151,7sleep12952630-21ntpq1
293020299151,2cyclictest0-21swapper/33
293020299150,13cyclictest0-21swapper/33
293020299150,13cyclictest0-21swapper/33
293020299150,13cyclictest0-21swapper/33
293020299150,13cyclictest0-21swapper/33
293020299150,12cyclictest0-21swapper/33
293020299150,12cyclictest0-21swapper/33
33506312140,2sleep10-21swapper/11
31545312141,9sleep13154544-21ssh1
31047102141,8sleep13061045-21diskmemload1
293020299140,11cyclictest0-21swapper/33
293020299140,11cyclictest0-21swapper/33
34160412131,6sleep23416083-21ntpq2
34160412131,6sleep23416083-21ntpq2
34096722131,7sleep23061045-21diskmemload2
33763302130,8sleep23061045-21diskmemload2
33654332131,8sleep23365446-21sshd2
33489912131,5sleep20-21swapper/22
33334422131,7sleep23333548-21perl2
33133502131,2sleep00-21swapper/00
32598612131,2sleep10-21swapper/11
32405672131,5sleep03240717-21ntp_192.168.1140
32177052131,7sleep13061045-21diskmemload1
32016892131,5sleep10-21swapper/11
31483872131,3sleep13148401-21ssh1
31316762131,6sleep20-21swapper/22
31237042131,8sleep13124306-21cut1
30741152131,6sleep23074269-21ntpq2
30676862131,8sleep03067697-21ssh0
29903272132,8sleep22986168-21/usr/sbin/munin2
34179722121,6sleep13061045-21diskmemload1
34179722121,6sleep13061045-21diskmemload1
33849702121,7sleep23384982-21ssh2
33845552121,7sleep03384568-21sh0
33763692121,8sleep13061045-21diskmemload1
33515432121,7sleep23353145-21grep2
32860002121,7sleep23286011-21ssh2
32846852121,7sleep13284744-21ntpq1
32365552121,2sleep10-21swapper/11
32329122120,3sleep03232926-21ssh0
31981582121,6sleep13061045-21diskmemload1
31461712120,3sleep03146184-21ssh0
31398532121,5sleep00-21swapper/00
30957532121,7sleep03095766-21ssh0
30942942126,3sleep10-21swapper/11
30834132121,7sleep03083589-21ssh0
30771282121,6sleep13077138-21ssh1
30656662121,6sleep23065734-21/usr/sbin/munin2
30278602121,5sleep23027914-21ntpq2
30136662121,7sleep13014819-21cut1
34286842110,7sleep13061045-21diskmemload1
34213252111,5sleep23421405-21irqstats2
33826472111,4sleep13061045-21diskmemload1
33659212111,3sleep10-21swapper/11
33497232111,2sleep03349733-21ssh0
33302712111,4sleep13061045-21diskmemload1
33302712111,4sleep13061045-21diskmemload1
33157202111,6sleep23315859-21tr2
32776052111,5sleep13061045-21diskmemload1
32441342111,3sleep20-21swapper/22
32329822111,2sleep13232993-21ssh1
31983722111,4sleep20-21swapper/22
31735842111,3sleep00-21swapper/00
31730912111,4sleep13061045-21diskmemload1
31719352111,3sleep20-21swapper/22
31405312111,3sleep20-21swapper/22
31216242111,5sleep03061045-21diskmemload0
31112052111,5sleep13061045-21diskmemload1
30557542111,6sleep23055953-21latency_hist2
30497112111,3sleep20-21swapper/22
30329742111,7sleep03033114-21ntp_192.168.1140
30126322111,3sleep172-21kcompactd01
30077692111,4sleep03007788-21apt-get0
29702292111,5sleep22970455-21df_inode2
34073242101,4sleep03407377-21ntpq0
33992282101,3sleep20-21swapper/22
33946632101,5sleep03394742-21fschecks_time0
33898532101,4sleep03061045-21diskmemload0
33750282101,3sleep00-21swapper/00
33546622101,3sleep00-21swapper/00
33499302101,4sleep13061045-21diskmemload1
33403882101,2sleep00-21swapper/00
33318132100,2sleep20-21swapper/22
33318132100,2sleep20-21swapper/22
33220562101,4sleep13061045-21diskmemload1
33127062100,2sleep23312720-21ssh2
32994962100,5sleep03301150-21grep0
32966832101,3sleep00-21swapper/00
32792102101,2sleep00-21swapper/00
32679372101,3sleep13061045-21diskmemload1
32524232101,3sleep10-21swapper/11
32522892101,4sleep23061045-21diskmemload2
32258172101,4sleep13061045-21diskmemload1
32104092101,5sleep03210416-21ssh0
32078842101,3sleep00-21swapper/00
32012242101,3sleep00-21swapper/00
31922312101,3sleep20-21swapper/22
31754092101,2sleep20-21swapper/22
31663362100,6sleep03166488-21sort0
31491512101,4sleep03149163-21ssh0
31210982100,2sleep23121109-21ssh2
31021422100,3sleep20-21swapper/22
30925532101,3sleep20-21swapper/22
30859272101,3sleep20-21swapper/22
30665882101,3sleep10-21swapper/11
30547702101,4sleep13054819-21ntpq1
30223322101,5sleep13022473-21ntpq1
30211962101,4sleep03022331-21memory0
30063232101,4sleep13006464-21ntpq1
29975172101,4sleep22997633-21irqstats2
29961182101,6sleep02996347-21awk0
29642392100,2sleep10-21swapper/11
29499142101,6sleep20-21swapper/22
3407666291,3sleep13061045-21diskmemload1
3398960291,5sleep13399222-21unixbench_singl1
3391011291,2sleep20-21swapper/22
3385901291,4sleep13386123-21/usr/sbin/munin1
3364553290,1sleep00-21swapper/00
3337357291,4sleep13337410-21ntpq1
3328650291,3sleep03328710-21ntpq0
3328650291,3sleep03328710-21ntpq0
3303127291,2sleep20-21swapper/22
3280355291,2sleep00-21swapper/00
3268436291,2sleep00-21swapper/00
3259850291,2sleep00-21swapper/00
3227274291,2sleep00-21swapper/00
3202595291,3sleep23204201-21kthreadcore2
3189281291,4sleep03189295-21ssh0
3184198291,3sleep13061045-21diskmemload1
3181736290,3sleep00-21swapper/00
3162921291,3sleep13061045-21diskmemload1
3159818291,4sleep23161573-21cut2
3149888291,4sleep23061045-21diskmemload2
3118390290,1sleep10-21swapper/11
3056583291,4sleep23057816-21kthreadcore2
3056301291,4sleep03056486-21sed0
3033408291,4sleep13033420-21run-parts1
3030013291,4sleep23031719-21cut2
2980838290,2sleep20-21swapper/22
2980838290,2sleep20-21swapper/22
2965136291,4sleep22966178-21cut2
2958264291,4sleep22958427-21/usr/sbin/munin2
2957978291,3sleep12958025-21ntpq1
2943802291,4sleep22944730-21cut2
2935283290,2sleep10-21swapper/11
2930516290,5sleep02930578-21cpuspeed_turbos0
3061037281,3sleep13061056-21latency_hist1
3054711280,3sleep03054762-21ntpq0
3039754280,1sleep10-21swapper/11
3014820281,3sleep23015190-21kthreadcore2
2986779281,3sleep02988412-21kthreadcore0
2979886281,3sleep12980073-21sed1
2976249281,3sleep02977921-21grep0
2974527281,3sleep10-21swapper/11
2963185281,2sleep20-21swapper/22
2960752281,3sleep02962532-21kthreadcore0
2943236280,1sleep10-21swapper/11
2942786280,1sleep20-21swapper/22
3050097270,1sleep10-21swapper/11
3044772270,1sleep20-21swapper/22
3036517271,2sleep20-21swapper/22
3035241271,2sleep10-21swapper/11
3028692270,1sleep00-21swapper/00
3018971271,2sleep20-21swapper/22
3002415270,2sleep20-21swapper/22
3001799270,1sleep00-21swapper/00
2996476270,4sleep10-21swapper/11
2985814270,1sleep00-21swapper/00
2985814270,1sleep00-21swapper/00
2975224270,1sleep00-21swapper/00
2954400271,3sleep02954582-21grep0
2946510270,4sleep00-21swapper/00
2942922270,1sleep10-21swapper/11
2934378270,2sleep22934551-21strings2
3045339260,1sleep00-21swapper/00
3040581260,2sleep03041582-21kthreadcore0
3034670260,3sleep03034689-21apt-get0
3024324261,2sleep13025354-21cut1
3007200260,4sleep00-21swapper/00
2996808260,2sleep1802-21snmpd1
2991747260,2sleep22991766-21apt-get2
2991102260,1sleep10-21swapper/11
2985815260,1sleep10-21swapper/11
2985815260,1sleep10-21swapper/11
2979708261,2sleep20-21swapper/22
2969635260,2sleep10-21swapper/11
2964854261,2sleep00-21swapper/00
2953308260,4sleep00-21swapper/00
2938690261,2sleep02940389-21kthreadcore0
3013097250,1sleep00-21swapper/00
3012141250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional