You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 19:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Mon Jun 01, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3037469212855,69sleep30-21swapper/33
310625821150,3sleep2421rcuc/22
3037715210087,7sleep00-21swapper/00
303765728164,11sleep10-21swapper/11
303753128162,12sleep20-21swapper/22
34977072220,2sleep10-21swapper/11
303787499218,2cyclictest0-21swapper/33
33519252201,15sleep23352005-21cat2
33049852201,14sleep00-21swapper/00
303787499201,15cyclictest3042885-21taskset3
303787499201,15cyclictest0-21swapper/33
303787499201,14cyclictest3151915-21taskset3
3037874992010,7cyclictest3209653-21taskset3
34566722191,8sleep13168092-21diskmemload1
3037874991916,2cyclictest0-21swapper/33
3037874991912,6cyclictest0-21swapper/33
3037874991912,4cyclictest0-21swapper/33
303787499191,15cyclictest0-21swapper/33
303787499191,15cyclictest0-21swapper/33
303787499191,15cyclictest0-21swapper/33
303787499191,15cyclictest0-21swapper/33
303787499191,14cyclictest0-21swapper/33
303787499191,14cyclictest0-21swapper/33
303787499191,14cyclictest0-21swapper/33
303787499191,14cyclictest0-21swapper/33
303787499191,14cyclictest0-21swapper/33
303787499190,2cyclictest0-21swapper/33
303787499182,12cyclictest3510198-21chrt3
303787499181,15cyclictest3507401-21taskset3
3037874991811,4cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,14cyclictest0-21swapper/33
303787499181,13cyclictest0-21swapper/33
303787499180,2cyclictest0-21swapper/33
303787499180,16cyclictest0-21swapper/33
303787499180,14cyclictest0-21swapper/33
33191372171,12sleep03320841-21grep0
32882892171,2sleep10-21swapper/11
303787499172,10cyclictest3136140-21taskset3
303787499171,14cyclictest0-21swapper/33
303787499171,14cyclictest0-21swapper/33
303787499171,14cyclictest0-21swapper/33
303787499171,13cyclictest0-21swapper/33
303787499171,12cyclictest0-21swapper/33
303787499170,14cyclictest0-21swapper/33
303787499170,13cyclictest0-21swapper/33
34892102161,4sleep03168092-21diskmemload0
32020722161,3sleep00-21swapper/00
303787499161,3cyclictest0-21swapper/33
3037874991613,1cyclictest0-21swapper/33
3037874991611,3cyclictest0-21swapper/33
303787499161,13cyclictest0-21swapper/33
303787499161,12cyclictest0-21swapper/33
303787499161,12cyclictest0-21swapper/33
303787499160,14cyclictest0-21swapper/33
303787499160,13cyclictest0-21swapper/33
303787499160,13cyclictest0-21swapper/33
303787499160,13cyclictest0-21swapper/33
303787499159,4cyclictest3066387-21chrt3
303787499153,8cyclictest3156524-21taskset3
303787499150,13cyclictest0-21swapper/33
303787499150,12cyclictest0-21swapper/33
34703602141,7sleep03470424-21ntp_states0
34276752140,2sleep10-21swapper/11
34036042140,2sleep00-21swapper/00
33701312142,4sleep13370318-21sed1
32989602141,3sleep03168092-21diskmemload0
32538252141,2sleep00-21swapper/00
32263532141,4sleep23226367-21ssh2
32031102141,8sleep13168092-21diskmemload1
31841092141,9sleep03168092-21diskmemload0
30454252141,8sleep03045444-21apt-get0
303787499141,10cyclictest3168110-21turbostat3
3037874991410,2cyclictest0-21swapper/33
303787499140,2cyclictest0-21swapper/33
303787499140,2cyclictest0-21swapper/33
35247662131,8sleep23524779-21ssh2
35141052131,8sleep13514203-21ntpq1
35090872132,2sleep1171rcu_preempt1
34812592131,3sleep23481273-21ssh2
34737192131,9sleep13168092-21diskmemload1
34574522132,5sleep23457665-21diskstats2
34457682133,5sleep03445794-21apt-get0
34383302130,2sleep10-21swapper/11
34104602131,3sleep03410471-21ssh0
34049192131,8sleep13168092-21diskmemload1
33967852131,8sleep13397084-21cut1
33926802131,5sleep10-21swapper/11
33780792131,7sleep03168092-21diskmemload0
33670782131,7sleep13168092-21diskmemload1
33411652131,8sleep13341175-21ssh1
33241372131,2sleep13261553-21kworker/u16:0+xprtiod1
32622512131,8sleep00-21swapper/00
32561452131,5sleep20-21swapper/22
32458492131,3sleep03245862-21ssh0
31693272131,2sleep00-21swapper/00
30551702131,4sleep03055357-21/usr/sbin/munin0
303787499138,3cyclictest3114190-21chrt3
303787499130,2cyclictest0-21swapper/33
303787499130,2cyclictest0-21swapper/33
303787499130,2cyclictest0-21swapper/33
303787499130,1cyclictest0-21swapper/33
35327032121,5sleep20-21swapper/22
34615432121,5sleep13461602-21ntpq1
34353182121,7sleep23435375-21ntpq2
34223362120,3sleep03422349-21ssh0
34195252121,6sleep03168092-21diskmemload0
34178672121,5sleep23417927-21ntpq2
34128052121,3sleep20-21swapper/22
33511402120,3sleep03351147-21ssh0
33411282126,2sleep23168092-21diskmemload2
33332122124,1sleep1171rcu_preempt1
33140092121,6sleep13314021-21ssh1
32982232121,6sleep23168092-21diskmemload2
32953042121,5sleep13295362-21ntpq1
32681952121,6sleep23268953-21sort2
32197392121,7sleep23219752-21ssh2
31471022120,2sleep10-21swapper/11
31295432120,8sleep13129589-21ntpq1
30869962121,6sleep23087046-21ntpq2
30869962121,6sleep23087046-21ntpq2
35054082111,5sleep23505508-21ntpq2
34976962111,6sleep03168092-21diskmemload0
34554562111,5sleep03168092-21diskmemload0
34398942111,5sleep23440163-21df2
33861652110,2sleep10-21swapper/11
33860032110,8sleep03386015-21sh0
33577112111,5sleep20-21swapper/22
33341842111,2sleep20-21swapper/22
32561892112,2sleep10-21swapper/11
31950292111,5sleep23195058-21apt-get2
31865862111,5sleep03186797-21hddtemp_smartct0
31814662110,2sleep23181648-21strings2
31765342111,2sleep10-21swapper/11
31564282111,5sleep13156469-21ntpq1
31348632111,5sleep13134912-21ntpq1
31294842111,5sleep03129541-21ntpq0
30762592111,5sleep23076307-21ntpq2
30716252111,4sleep23071546-31find2
30715512112,4sleep13071546-31find1
30381682111,5sleep23038334-21diskstats2
35353832101,4sleep03168092-21diskmemload0
35328932101,2sleep10-21swapper/11
35255852101,2sleep00-21swapper/00
35180282101,5sleep23518038-21sh2
34954612101,4sleep23496589-21memory2
34874292101,5sleep272-21kcompactd02
34839862101,4sleep13484116-21diskstats1
34819812101,3sleep00-21swapper/00
34747702101,2sleep10-21swapper/11
34632042101,3sleep00-21swapper/00
34509112100,2sleep20-21swapper/22
34455232101,3sleep13168092-21diskmemload1
34314552100,6sleep03431567-21perl0
34265902101,5sleep23426639-21ntpq2
34195362100,1sleep10-21swapper/11
33902992101,5sleep23391448-21/usr/sbin/munin2
33880252101,6sleep03388654-21cut0
33852582101,2sleep20-21swapper/22
33681432101,3sleep20-21swapper/22
33668132101,3sleep00-21swapper/00
33580852101,2sleep00-21swapper/00
33575052101,3sleep10-21swapper/11
33495862100,3sleep10-21swapper/11
33421492101,5sleep03342162-21ssh0
33303382101,4sleep03330398-21ssh0
33179042101,4sleep23318028-21irqstats2
33160802101,5sleep03168092-21diskmemload0
33059462100,1sleep10-21swapper/11
32898112101,4sleep23168092-21diskmemload2
32826862101,4sleep03168092-21diskmemload0
32810602100,3sleep172-21kcompactd01
32787122101,4sleep03168092-21diskmemload0
32650052101,4sleep13265228-21df_inode1
32477602101,5sleep23247814-21tune2fs2
32442442100,1sleep20-21swapper/22
32432192101,4sleep13168092-21diskmemload1
32356552101,2sleep00-21swapper/00
32264762101,2sleep00-21swapper/00
32192172101,3sleep10-21swapper/11
32112512101,3sleep20-21swapper/22
32043612101,4sleep03205371-21grep0
31867982101,5sleep13186877-21ssh1
31824552101,3sleep10-21swapper/11
31440032101,4sleep13145324-21memory1
31362842101,4sleep03136407-21diskstats0
31182742101,5sleep13118816-21grep1
31152612101,5sleep23115377-21irqstats2
31149232100,1sleep00-21swapper/00
31029562101,4sleep23103009-21ntp_states2
30988742101,4sleep03098893-21apt-get0
30715982101,4sleep03071546-31find0
3524197290,1sleep10-21swapper/11
3514339291,4sleep03514431-21ps0
3466080290,5sleep20-21swapper/22
3369711290,1sleep20-21swapper/22
3311856291,5sleep23312702-21missed_timers2
3305506291,4sleep23168092-21diskmemload2
3279498291,2sleep20-21swapper/22
3265537291,4sleep03266504-21cut0
3247318291,4sleep13247325-21ssh1
3232271291,4sleep23233943-21cat2
3227100291,3sleep13168092-21diskmemload1
3212383291,3sleep00-21swapper/00
3205372291,5sleep13207071-21grep1
3186881291,4sleep23188221-21kthreadcore2
3175188291,4sleep23175201-21ssh2
3159399291,5sleep03161195-21kthreadcore0
3147589291,4sleep03148837-21grep0
3141181291,4sleep23141229-21munin-run2
3131067290,5sleep23131112-21fschecks_count2
3113828291,4sleep23113894-21sed2
3110048291,4sleep13111193-21cut1
3087960291,5sleep03088007-21/usr/sbin/munin0
3087960291,5sleep03088007-21/usr/sbin/munin0
3506259281,2sleep00-21swapper/00
3397086281,3sleep23168092-21diskmemload2
3229874281,3sleep13043281-21munin-node1
3162101281,3sleep13162338-21/usr/sbin/munin1
3145325280,1sleep20-21swapper/22
3130792280,3sleep00-21swapper/00
3097955281,3sleep03098192-21unixbench_singl0
3094111281,3sleep23095015-21kthreadcore2
3077844281,3sleep03077897-21fschecks_count0
3062195281,3sleep13063859-21tr1
3061443280,3sleep03061462-21apt-get0
3038068281,3sleep13038166-21tr1
3167941270,1sleep00-21swapper/00
3163586271,2sleep2421rcuc/22
3163014270,2sleep10-21swapper/11
3162748270,4sleep20-21swapper/22
3154064271,3sleep23155859-21cut2
3152361270,2sleep00-21swapper/00
3146397271,4sleep03026862-21kworker/0:0+events0
3129837271,3sleep23129923-21sort2
3124901270,1sleep00-21swapper/00
3124477271,3sleep23124492-21mailstats2
3093544270,3sleep13093549-21sort1
3092550270,1sleep00-21swapper/00
3092102270,1sleep20-21swapper/22
3090956270,2sleep10-21swapper/11
3083465271,3sleep13084716-21cut1
3083465271,3sleep13084716-21cut1
3066273270,1sleep20-21swapper/22
3061168270,2sleep00-21swapper/00
3056112270,2sleep10-21swapper/11
3050820271,2sleep20-21swapper/22
3050440270,1sleep10-21swapper/11
3043464271,3sleep20-21swapper/22
3139913260,3sleep10-21swapper/11
3124927260,1sleep10-21swapper/11
3104755261,2sleep10-21swapper/11
3104152261,1sleep03104171-21apt-get0
3099477261,2sleep13100992-21tr1
3077092260,1sleep00-21swapper/00
3056716261,2sleep20-21swapper/22
3050798260,3sleep10-21swapper/11
3038046260,2sleep03038065-21apt-get0
3151214251,1sleep20-21swapper/22
3109479250,1sleep00-21swapper/00
3081604250,1sleep20-21swapper/22
3077565250,1sleep10-21swapper/11
3076080250,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional