You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-02 - 14:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Sat May 02, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3351016211961,54sleep30-21swapper/33
335088528061,12sleep10-21swapper/11
335092727860,12sleep20-21swapper/22
335089927253,14sleep00-21swapper/00
3351228992417,5cyclictest66950irq/132-enp1s0-TxRx-33
335122899221,18cyclictest0-21swapper/33
37757842211,14sleep13775840-21ntpq1
37261832211,8sleep03726196-21ssh0
335122899211,7cyclictest3694311-21taskset3
335122899211,16cyclictest66950irq/132-enp1s0-TxRx-33
3351228992110,8cyclictest3582369-21chrt3
335122899201,15cyclictest0-21swapper/33
335122899201,14cyclictest0-21swapper/33
335122899200,6cyclictest36329322sleep33
335122899200,17cyclictest66950irq/132-enp1s0-TxRx-33
335122899200,17cyclictest0-21swapper/33
38307352190,1sleep10-21swapper/11
38125902191,15sleep23481786-21diskmemload2
38099802191,6sleep13481786-21diskmemload1
37353272191,2sleep00-21swapper/00
3351228991916,2cyclictest0-21swapper/33
3351228991916,2cyclictest0-21swapper/33
335122899191,15cyclictest66950irq/132-enp1s0-TxRx-33
335122899191,14cyclictest0-21swapper/33
335122899191,14cyclictest0-21swapper/33
335122899191,14cyclictest0-21swapper/33
335122899191,14cyclictest0-21swapper/33
335122899191,14cyclictest0-21swapper/33
335122899190,8cyclictest3608435-21taskset3
38283262181,8sleep23828337-21ssh2
37009112180,8sleep03481786-21diskmemload0
34902492181,13sleep13490398-21sort1
335122899189,6cyclictest3658469-21taskset3
335122899189,6cyclictest3658469-21taskset3
3351228991815,1cyclictest0-21swapper/33
335122899181,4cyclictest3751653-21taskset3
335122899181,4cyclictest3751653-21taskset3
335122899181,14cyclictest33957852chrt3
335122899181,14cyclictest0-21swapper/33
335122899181,14cyclictest0-21swapper/33
335122899181,14cyclictest0-21swapper/33
335122899181,14cyclictest0-21swapper/33
335122899181,14cyclictest0-21swapper/33
335122899181,14cyclictest0-21swapper/33
335122899181,14cyclictest0-21swapper/33
335122899181,14cyclictest0-21swapper/33
335122899181,14cyclictest0-21swapper/33
335122899181,13cyclictest0-21swapper/33
335122899180,15cyclictest0-21swapper/33
335122899180,15cyclictest0-21swapper/33
36678992171,3sleep10-21swapper/11
36247352171,3sleep00-21swapper/00
35427152171,8sleep13542800-21uname1
3351228991713,2cyclictest0-21swapper/33
335122899171,14cyclictest0-21swapper/33
335122899171,14cyclictest0-21swapper/33
335122899171,14cyclictest0-21swapper/33
335122899171,13cyclictest0-21swapper/33
335122899171,13cyclictest0-21swapper/33
335122899171,13cyclictest0-21swapper/33
335122899171,13cyclictest0-21swapper/33
335122899171,12cyclictest0-21swapper/33
335122899170,14cyclictest0-21swapper/33
335122899170,14cyclictest0-21swapper/33
335122899170,14cyclictest0-21swapper/33
38113862161,2sleep00-21swapper/00
37516612161,2sleep00-21swapper/00
37516612161,2sleep00-21swapper/00
36596752161,3sleep10-21swapper/11
36596752161,3sleep10-21swapper/11
36543182161,4sleep03655929-21ssh0
36543182161,4sleep03655929-21ssh0
36355802160,2sleep03635593-21ssh0
335122899161,3cyclictest0-21swapper/33
3351228991613,1cyclictest0-21swapper/33
335122899161,13cyclictest0-21swapper/33
335122899161,13cyclictest0-21swapper/33
335122899161,12cyclictest0-21swapper/33
335122899160,14cyclictest0-21swapper/33
335122899160,13cyclictest0-21swapper/33
335122899160,13cyclictest0-21swapper/33
38037502151,9sleep03803760-21ssh0
38013882151,4sleep13801449-21ntpq1
37603502150,2sleep20-21swapper/22
36914642151,4sleep03691528-21ntpq0
36455142151,10sleep23646072-21cut2
35995372151,4sleep13481786-21diskmemload1
35135942151,4sleep23481786-21diskmemload2
34663762150,1sleep30-21swapper/33
335122899150,13cyclictest0-21swapper/33
335122899150,13cyclictest0-21swapper/33
335122899150,13cyclictest0-21swapper/33
335122899150,12cyclictest0-21swapper/33
335122899150,12cyclictest0-21swapper/33
335122899150,12cyclictest0-21swapper/33
335122899150,12cyclictest0-21swapper/33
335122899150,12cyclictest0-21swapper/33
335122899150,12cyclictest0-21swapper/33
335122899150,12cyclictest0-21swapper/33
38212652140,2sleep10-21swapper/11
37849522140,2sleep10-21swapper/11
37020142140,2sleep20-21swapper/22
36846242141,9sleep03481786-21diskmemload0
36778082141,4sleep13677822-21ssh1
36421412141,2sleep10-21swapper/11
36270462141,3sleep23481786-21diskmemload2
36183522141,5sleep13618364-21ssh1
36083362141,8sleep23481786-21diskmemload2
35997162141,8sleep03481786-21diskmemload0
33905332140,1sleep20-21swapper/22
33588702141,9sleep13359074-21/usr/sbin/munin1
3351228991411,1cyclictest0-21swapper/33
335122899140,11cyclictest0-21swapper/33
37765952131,7sleep23481786-21diskmemload2
37516292131,5sleep10-21swapper/11
37516292131,5sleep10-21swapper/11
36359032131,7sleep23481786-21diskmemload2
35591932135,1sleep2171rcu_preempt2
35591932135,1sleep2171rcu_preempt2
35421092133,5sleep0151ktimers/00
35264602132,6sleep23528135-21cut2
34906472131,8sleep03490914-21ssh0
34670352131,8sleep13468743-21cut1
34447382131,7sleep03444756-21apt-get0
38283022121,5sleep00-21swapper/00
38183062121,6sleep03818369-21ntpq0
37851352121,6sleep03785149-21ssh0
37704832121,2sleep13770494-21ssh1
37372992121,5sleep10-21swapper/11
37191982121,5sleep00-21swapper/00
37183672121,5sleep20-21swapper/22
37116872121,7sleep03711697-21ssh0
37113232121,7sleep23711331-21ssh2
37106072120,1sleep10-21swapper/11
36853602121,6sleep23481786-21diskmemload2
36442272122,4sleep23481786-21diskmemload2
36336392120,1sleep10-21swapper/11
36163682121,7sleep23481786-21diskmemload2
35675972121,7sleep23567650-21grep2
35033642121,6sleep13503423-21ntpq1
33849042121,6sleep03384901-40gst-plugin-scan0
33734862121,5sleep23373529-21ntpq2
38211352111,4sleep23481786-21diskmemload2
37715782111,5sleep03771599-21apt-get0
37700722111,6sleep23770086-21ssh2
37635972111,4sleep03763724-21irqstats0
37513572111,5sleep23481786-21diskmemload2
37513572111,5sleep23481786-21diskmemload2
37428902111,4sleep23481786-21diskmemload2
37251792111,5sleep23725231-21ntpq2
36762002111,5sleep03481786-21diskmemload0
36589712112,4sleep23481786-21diskmemload2
36589712112,4sleep23481786-21diskmemload2
36524102111,3sleep00-21swapper/00
36364332111,4sleep03481786-21diskmemload0
36029872112,2sleep1171rcu_preempt1
35934642115,3sleep03593642-21latency_hist0
35908052111,3sleep20-21swapper/22
35827052110,4sleep20-21swapper/22
35670502110,2sleep00-21swapper/00
35670502110,2sleep00-21swapper/00
35668642111,6sleep13566878-21ssh1
35668642111,6sleep13566878-21ssh1
35335952110,2sleep20-21swapper/22
35298652111,3sleep10-21swapper/11
35150102111,6sleep13515021-21ssh1
34807932111,5sleep03480842-21ntpq0
34528512111,5sleep13453986-21memory1
34320922111,5sleep23432231-21ntp_192.168.1142
34282302111,5sleep03428483-21df0
34181172111,6sleep23419016-21kthreadcore2
33753392111,7sleep20-21swapper/22
33644302111,5sleep23364643-21hddtemp_smartct2
33588072111,6sleep03358785-21/usr/sbin/munin0
38385262101,4sleep03481786-21diskmemload0
37973702101,4sleep23797503-21diskstats2
37965262101,3sleep13481786-21diskmemload1
37914232101,4sleep03792530-21memory0
37890922101,5sleep23789785-21cat2
37430142101,6sleep13481786-21diskmemload1
37419412101,3sleep00-21swapper/00
37199122101,4sleep13481786-21diskmemload1
37031092101,3sleep10-21swapper/11
36940412101,6sleep13694052-21diskmemload1
36916242101,2sleep20-21swapper/22
36851142101,3sleep10-21swapper/11
36766942101,4sleep23481786-21diskmemload2
36659352101,4sleep23481786-21diskmemload2
36142132101,6sleep03615047-21ssh0
36027752101,6sleep03602986-21/usr/sbin/munin0
35989642101,4sleep23481786-21diskmemload2
35729692101,2sleep10-21swapper/11
35678882101,5sleep03568048-21ssh0
35585712101,4sleep13481786-21diskmemload1
35548752101,4sleep03481786-21diskmemload0
35470912101,3sleep23481786-21diskmemload2
35395442100,2sleep03476129-21kworker/u16:0+xprtiod0
35310642100,2sleep00-21swapper/00
35214202100,1sleep20-21swapper/22
35134242101,3sleep00-21swapper/00
35064572101,4sleep03481786-21diskmemload0
35023182101,4sleep23481786-21diskmemload2
34934252101,4sleep13494642-21memory1
34791282101,5sleep13480597-21cat1
34714642101,4sleep13471688-21df_inode1
34607352101,4sleep13460754-21apt-get1
34442022102,4sleep1555-21dbus-daemon1
34391562101,4sleep23439409-21df2
34234342101,5sleep00-21swapper/00
33733482100,6sleep13373483-21netstat1
33568852101,4sleep23357255-21diskstats2
3831262291,4sleep13831291-21apt-get1
3831183291,3sleep20-21swapper/22
3784639291,4sleep23784911-21grep2
3760174291,4sleep13760188-21ssh1
3759231291,4sleep03759369-21sed0
3730502291,4sleep23732158-21cut2
3728539291,4sleep13728551-21ssh1
3650770291,3sleep13481786-21diskmemload1
3619489291,4sleep13619504-21ssh1
3592381291,2sleep10-21swapper/11
3580563291,3sleep10-21swapper/11
3576343291,2sleep00-21swapper/00
3555914291,4sleep23555924-21ssh2
3533236291,3sleep13532945-21apt-get1
3523573291,2sleep00-21swapper/00
3522635291,3sleep13481786-21diskmemload1
3486456291,2sleep00-21swapper/00
3482117291,2sleep20-21swapper/22
3471689290,2sleep20-21swapper/22
3461551291,4sleep23463149-21cut2
3451135291,4sleep03452850-21tr0
3445021291,4sleep23445145-21perl2
3439460291,4sleep03439672-21if_enp1s00
3416245291,4sleep13416296-21gcc1
3416245291,4sleep13416296-21gcc1
3405670291,4sleep13405723-21ntp_states1
3394966291,4sleep23395010-21gcc2
3387265291,4sleep13389059-21tr1
3384912290,3sleep13384885-31find1
3355561291,4sleep23355612-21ldconfig2
3666358280,1sleep00-21swapper/00
3497581280,3sleep20-21swapper/22
3481776281,3sleep23481796-21latency_hist2
3459851281,3sleep03455407-21/usr/sbin/munin0
3430935281,3sleep13432091-21/usr/sbin/munin1
3424557282,3sleep10-21swapper/11
3397871281,3sleep23399657-21cut2
3378718281,3sleep13378798-21ntp_192.168.1141
3471420271,3sleep03471463-21grep0
3470963270,1sleep20-21swapper/22
3460136270,1sleep20-21swapper/22
3455401270,1sleep10-21swapper/11
3438549270,1sleep10-21swapper/11
3423152271,3sleep23423325-21gltestperf2
3422360270,1sleep10-21swapper/11
3421914271,3sleep00-21swapper/00
3413127271,3sleep20-21swapper/22
3413127271,3sleep20-21swapper/22
3411866270,1sleep00-21swapper/00
3402127271,3sleep03403511-21kthreadcore0
3400856270,2sleep00-21swapper/00
3396819272,2sleep1321rcuc/11
3390300270,4sleep00-21swapper/00
3384966270,2sleep20-21swapper/22
3379625270,1sleep00-21swapper/00
3374408270,1sleep00-21swapper/00
3369009270,1sleep10-21swapper/11
3368197271,2sleep03368251-21ntpq0
3351428270,2sleep10-21swapper/11
3466051260,2sleep00-21swapper/00
3465705260,1sleep00-21swapper/00
3446169261,2sleep10-21swapper/11
3438545260,3sleep20-21swapper/22
3416094260,2sleep03416239-21strings0
3416094260,2sleep03416239-21strings0
3411666260,1sleep20-21swapper/22
3405323260,2sleep23405556-21strings2
3395886260,4sleep10-21swapper/11
3355366260,2sleep00-21swapper/00
3454277250,1sleep20-21swapper/22
3433458250,1sleep00-21swapper/00
3411269250,1sleep10-21swapper/11
3391202250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional