You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-15 - 22:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Mon Jun 15, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2671327212451,68sleep30-21swapper/33
266654228658,17sleep00-21swapper/00
267136328453,12sleep10-21swapper/11
267120828060,12sleep20-21swapper/22
2671680992711,13cyclictest0-21swapper/33
2671680992622,2cyclictest0-21swapper/33
2671680992523,1cyclictest0-21swapper/33
2671680992513,1cyclictest0-21swapper/33
2671680992422,1cyclictest0-21swapper/33
267168099241,14cyclictest0-21swapper/33
267168099240,19cyclictest2699998-21taskset3
29298562231,18sleep02803426-21diskmemload0
2671680992321,1cyclictest0-21swapper/33
2671680992321,1cyclictest0-21swapper/33
2671680992319,2cyclictest0-21swapper/33
267168099231,17cyclictest3100051-21taskset3
28862452221,9sleep22803426-21diskmemload2
2671680992220,1cyclictest0-21swapper/33
2671680992218,2cyclictest0-21swapper/33
2671680992215,4cyclictest541ktimers/33
30320742211,9sleep02803426-21diskmemload0
267168099216,12cyclictest0-21swapper/33
2671680992119,1cyclictest0-21swapper/33
2671680992119,1cyclictest0-21swapper/33
2671680992118,1cyclictest0-21swapper/33
2671680992118,1cyclictest0-21swapper/33
2671680992118,1cyclictest0-21swapper/33
2671680992118,1cyclictest0-21swapper/33
267168099211,7cyclictest2711566-21chrt3
2671680992117,2cyclictest0-21swapper/33
2671680992116,2cyclictest0-21swapper/33
267168099211,15cyclictest0-21swapper/33
267168099210,17cyclictest0-21swapper/33
30581262200,3sleep20-21swapper/22
28792022200,10sleep12803426-21diskmemload1
267168099205,12cyclictest0-21swapper/33
267168099201,9cyclictest2733199-21taskset3
2671680992018,1cyclictest0-21swapper/33
2671680992017,1cyclictest0-21swapper/33
2671680992017,1cyclictest0-21swapper/33
2671680992016,2cyclictest0-21swapper/33
2671680992016,2cyclictest0-21swapper/33
267168099201,15cyclictest0-21swapper/33
267168099201,15cyclictest0-21swapper/33
267168099201,15cyclictest0-21swapper/33
267168099197,1cyclictest0-21swapper/33
267168099197,1cyclictest0-21swapper/33
267168099193,13cyclictest0-21swapper/33
267168099193,13cyclictest0-21swapper/33
2671680991917,1cyclictest0-21swapper/33
2671680991917,1cyclictest0-21swapper/33
2671680991917,1cyclictest0-21swapper/33
2671680991917,1cyclictest0-21swapper/33
2671680991917,1cyclictest0-21swapper/33
2671680991916,1cyclictest0-21swapper/33
2671680991915,2cyclictest0-21swapper/33
2671680991915,2cyclictest0-21swapper/33
31165592181,3sleep20-21swapper/22
30388272181,5sleep20-21swapper/22
30338482181,13sleep03033840-21/usr/sbin/munin0
29307932189,5sleep2431ktimers/22
267168099185,4cyclictest541ktimers/33
2671680991816,1cyclictest0-21swapper/33
2671680991816,1cyclictest0-21swapper/33
2671680991816,1cyclictest0-21swapper/33
2671680991815,2cyclictest0-21swapper/33
2671680991814,2cyclictest0-21swapper/33
2671680991813,3cyclictest26902072sleep33
267168099181,13cyclictest0-21swapper/33
267168099180,15cyclictest0-21swapper/33
29560022171,7sleep22956013-21ssh2
29560022171,7sleep22956013-21ssh2
28818192170,3sleep12882262-21sshd1
28785882171,3sleep00-21swapper/00
28412122171,3sleep20-21swapper/22
27805012171,10sleep02780550-21ntpq0
27533802171,7sleep02753749-21grep0
2671680991715,1cyclictest0-21swapper/33
2671680991715,1cyclictest0-21swapper/33
2671680991715,1cyclictest0-21swapper/33
2671680991715,1cyclictest0-21swapper/33
2671680991713,2cyclictest0-21swapper/33
2671680991713,2cyclictest0-21swapper/33
2671680991713,2cyclictest0-21swapper/33
2671680991713,2cyclictest0-21swapper/33
2671680991713,2cyclictest0-21swapper/33
267168099170,13cyclictest0-21swapper/33
30817362161,5sleep03081747-21ssh0
30750602161,3sleep20-21swapper/22
29797042161,3sleep20-21swapper/22
29490622161,4sleep02803426-21diskmemload0
29490622161,4sleep02803426-21diskmemload0
2671680991613,2cyclictest0-21swapper/33
267168099160,13cyclictest0-21swapper/33
30906692151,9sleep12803426-21diskmemload1
30829732151,2sleep10-21swapper/11
30822662150,2sleep2421rcuc/22
30670932150,1sleep20-21swapper/22
30660572151,7sleep10-21swapper/11
29988252151,8sleep02803426-21diskmemload0
29053692151,4sleep02803426-21diskmemload0
28758852150,3sleep00-21swapper/00
27431272152,3sleep12743167-21ntpq1
26896712151,10sleep12689693-21grep1
2671680991513,1cyclictest0-21swapper/33
31125912141,7sleep13113867-21memory1
30508142141,8sleep12803426-21diskmemload1
29471392148,3sleep12947153-21ssh1
29132462141,8sleep22803426-21diskmemload2
28948042141,5sleep22894814-21ssh2
31237052131,2sleep10-21swapper/11
31237052131,2sleep10-21swapper/11
30240012131,5sleep10-21swapper/11
30139252131,5sleep20-21swapper/22
29889642131,8sleep02988975-21ssh0
29151722131,6sleep12915427-21df1
29052062131,2sleep20-21swapper/22
28686772131,8sleep22803426-21diskmemload2
28511482131,7sleep02803426-21diskmemload0
28438392131,3sleep12803426-21diskmemload1
28325232130,8sleep02832589-21ntpq0
28224572131,2sleep1171rcu_preempt1
28175252131,8sleep02803426-21diskmemload0
28037622131,8sleep02803426-21diskmemload0
28037622131,8sleep02803426-21diskmemload0
26952582131,6sleep22695514-21df2
26904652131,8sleep12690800-21cut1
31067922121,7sleep23106806-21ssh2
31066752121,5sleep00-21swapper/00
30981712121,3sleep03098173-21ssh0
30869782121,7sleep03088610-21expr0
30723642121,5sleep03072620-21/usr/sbin/munin0
30251332121,7sleep22803426-21diskmemload2
30139132121,7sleep13013924-21ssh1
30051852121,7sleep13005195-21ssh1
29374942121,5sleep20-21swapper/22
28862192121,7sleep02803426-21diskmemload0
28670292120,7sleep02803426-21diskmemload0
28568032121,5sleep22857062-21df2
28567732122,5sleep12856798-21apt-get1
28342182121,5sleep20-21swapper/22
28173042121,5sleep22803426-21diskmemload2
27106022121,5sleep12710649-21ntpq1
27060712120,2sleep10-21swapper/11
31231742110,2sleep03123185-21ssh0
31231742110,2sleep03123185-21ssh0
31223752112,4sleep23122423-21ntpq2
31223752112,4sleep23122423-21ntpq2
31088162110,7sleep13108826-21ssh1
30990882111,2sleep20-21swapper/22
30649422111,4sleep02803426-21diskmemload0
30558042111,5sleep02803426-21diskmemload0
30292042111,5sleep12803426-21diskmemload1
30226032111,3sleep20-21swapper/22
30042712111,5sleep03004350-21ssh0
29887412111,4sleep12803426-21diskmemload1
29755452111,6sleep12977133-21cut1
29727692111,5sleep02803426-21diskmemload0
29620592111,5sleep12962112-21ntpq1
29620592111,5sleep12962112-21ntpq1
29466772110,2sleep2171rcu_preempt2
29311092111,6sleep12931118-21sh1
29122742110,1sleep00-21swapper/00
28973292111,6sleep02803426-21diskmemload0
28655142111,5sleep10-21swapper/11
28537812111,6sleep12853795-21ssh1
28499452111,4sleep22803426-21diskmemload2
28484212110,6sleep172-21kcompactd01
28369182112,4sleep02803426-21diskmemload0
28219682111,3sleep00-21swapper/00
27902412111,5sleep02791237-21memory0
27503392111,6sleep22751571-21grep2
27502672111,5sleep12750338-21irqstats1
27337582111,5sleep02733967-21hddtemp_smartct0
27171762112,4sleep02717195-21apt-get0
27106512111,5sleep22710695-21ntp_states2
26887292111,5sleep22688782-21ntp_states2
26792252110,2sleep10-21swapper/11
31171652101,4sleep02803426-21diskmemload0
30921842101,2sleep20-21swapper/22
30236062101,4sleep02803426-21diskmemload0
30124362101,5sleep03012495-21ntpq0
29981842101,3sleep22803426-21diskmemload2
29915592101,3sleep12803426-21diskmemload1
29894782101,5sleep22989490-21rm2
29805992101,2sleep00-21swapper/00
29737212101,2sleep20-21swapper/22
29662092101,4sleep12966235-21apt-get1
29578732102,4sleep00-21swapper/00
29578732102,4sleep00-21swapper/00
29533522101,4sleep12803426-21diskmemload1
29533522101,4sleep12803426-21diskmemload1
29459262101,2sleep00-21swapper/00
29388752101,4sleep02803426-21diskmemload0
29379292101,3sleep10-21swapper/11
29206652101,4sleep22803426-21diskmemload2
29199672100,1sleep00-21swapper/00
29035302100,2sleep10-21swapper/11
28779922101,4sleep22803426-21diskmemload2
28744492101,2sleep10-21swapper/11
28567052101,3sleep02803426-21diskmemload0
28540542101,2sleep00-21swapper/00
28529352101,3sleep20-21swapper/22
28408012101,4sleep02803426-21diskmemload0
28275282100,1sleep00-21swapper/00
28262232100,1sleep20-21swapper/22
28257782101,3sleep10-21swapper/11
28229702101,2sleep20-21swapper/22
28188682101,6sleep12818879-21ssh1
28087932101,3sleep10-21swapper/11
28087932101,3sleep10-21swapper/11
27968272101,5sleep02796878-21ntpq0
27873292100,6sleep12787330-21/usr/sbin/munin1
27444122101,4sleep22744431-21apt-get2
27332022101,4sleep12733251-21munin-run1
27264602100,5sleep22726556-21ntpq2
27174572101,5sleep22717560-21perl2
27060802100,7sleep02706006-31find0
27050352101,4sleep22705091-21ntpq2
26750462101,4sleep12675792-21memory1
3098947291,3sleep13098961-21ssh1
3067707291,3sleep10-21swapper/11
3049842291,4sleep10-21swapper/11
3048783291,4sleep03048796-21sh0
3047863291,2sleep20-21swapper/22
3040712291,2sleep10-21swapper/11
3000601291,4sleep23002056-21cat2
2957789291,2sleep20-21swapper/22
2957789291,2sleep20-21swapper/22
2912526291,2sleep10-21swapper/11
2894800291,4sleep12894803-21ssh1
2874913291,2sleep20-21swapper/22
2838136290,1sleep10-21swapper/11
2836944291,3sleep22803426-21diskmemload2
2828266291,4sleep12828276-21ssh1
2803788291,3sleep22803817-21apt-get2
2803788291,3sleep22803817-21apt-get2
2798335291,2sleep00-21swapper/00
2782035291,4sleep02782259-21expr0
2770034290,1sleep20-21swapper/22
2761450291,4sleep22763153-21grep2
2759276291,4sleep02759351-21/usr/sbin/munin0
2755668291,5sleep22757037-21kthreadcore2
2718412291,4sleep12720137-21cut1
2706388290,6sleep02706006-31find0
2684602290,5sleep02684621-21apt-get0
2677001291,4sleep02677471-21cut0
2797934281,3sleep22797803-21apt-get2
2797761280,2sleep10-21swapper/11
2787314281,4sleep21-21systemd2
2775786280,1sleep20-21swapper/22
2770102282,3sleep00-21swapper/00
2760426281,2sleep10-21swapper/11
2749473280,1sleep10-21swapper/11
2739051280,2sleep22738541-21apt-get2
2734040281,3sleep22735395-21grep2
2725781280,4sleep12726459-21sed1
2684320281,3sleep22684345-21munin-run2
2673300281,3sleep02675043-21kthreadcore0
2797571270,1sleep20-21swapper/22
2781471271,2sleep12781855-21latency_hist1
2775051271,2sleep10-21swapper/11
2770441270,2sleep10-21swapper/11
2765211270,1sleep00-21swapper/00
2737437271,2sleep10-21swapper/11
2716772270,1sleep00-21swapper/00
2695236270,2sleep10-21swapper/11
2694878272,3sleep2431ktimers/22
2694779270,1sleep00-21swapper/00
2782002260,1sleep10-21swapper/11
2781857261,2sleep22781483-21apt-get2
2781140260,1sleep20-21swapper/22
2759902260,1sleep10-21swapper/11
2749412260,4sleep00-21swapper/00
2733024260,1sleep00-21swapper/00
2728124260,2sleep20-21swapper/22
2716904260,1sleep10-21swapper/11
2715961261,2sleep20-21swapper/22
2676636260,1sleep20-21swapper/22
2802434250,1sleep10-21swapper/11
2775258250,2sleep00-21swapper/00
2739057250,1sleep00-21swapper/00
2699165251,1sleep00-21swapper/00
2722444240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional