You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-28 - 18:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Tue Apr 28, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2404959211861,53sleep30-21swapper/33
240573221100,2sleep20-21swapper/22
240513627951,10sleep10-21swapper/11
240520227857,6sleep00-21swapper/00
240026227858,12sleep20-21swapper/22
240540599220,18cyclictest2646396-21taskset3
27203292211,2sleep00-21swapper/00
25681252212,12sleep2802-21snmpd2
240540599211,17cyclictest2739260-21taskset3
240540599211,16cyclictest2857824-21taskset3
243192022011,5sleep12428896-21/usr/sbin/munin1
240540599204,13cyclictest0-21swapper/33
240540599201,16cyclictest0-21swapper/33
240540599201,16cyclictest0-21swapper/33
240540599201,15cyclictest66950irq/132-enp1s0-TxRx-33
240540599200,17cyclictest66950irq/132-enp1s0-TxRx-33
26390012191,13sleep02639138-21tr0
240540599192,14cyclictest66950irq/132-enp1s0-TxRx-33
2405405991914,3cyclictest0-21swapper/33
240540599191,15cyclictest66950irq/132-enp1s0-TxRx-33
240540599191,15cyclictest0-21swapper/33
240540599191,15cyclictest0-21swapper/33
240540599191,14cyclictest0-21swapper/33
240540599191,14cyclictest0-21swapper/33
240540599191,14cyclictest0-21swapper/33
240540599191,14cyclictest0-21swapper/33
240540599191,14cyclictest0-21swapper/33
240540599191,14cyclictest0-21swapper/33
240540599191,14cyclictest0-21swapper/33
240540599191,14cyclictest0-21swapper/33
240540599191,14cyclictest0-21swapper/33
240540599190,16cyclictest66950irq/132-enp1s0-TxRx-33
240540599190,16cyclictest66950irq/132-enp1s0-TxRx-33
275453421811,2sleep10-21swapper/11
26317702181,7sleep12633432-21tr1
2405405991813,2cyclictest0-21swapper/33
240540599181,14cyclictest0-21swapper/33
240540599181,14cyclictest0-21swapper/33
240540599181,14cyclictest0-21swapper/33
240540599181,14cyclictest0-21swapper/33
240540599181,14cyclictest0-21swapper/33
240540599181,14cyclictest0-21swapper/33
240540599181,14cyclictest0-21swapper/33
240540599181,14cyclictest0-21swapper/33
240540599181,13cyclictest0-21swapper/33
240540599181,13cyclictest0-21swapper/33
2405405991810,5cyclictest2541918-21chrt3
240540599180,14cyclictest0-21swapper/33
240540599180,14cyclictest0-21swapper/33
240540599180,14cyclictest0-21swapper/33
26885222170,2sleep20-21swapper/22
240540599171,14cyclictest0-21swapper/33
240540599171,14cyclictest0-21swapper/33
240540599171,13cyclictest0-21swapper/33
240540599170,14cyclictest0-21swapper/33
240540599170,14cyclictest0-21swapper/33
240540599170,14cyclictest0-21swapper/33
27720702161,2sleep20-21swapper/22
27118672160,1sleep10-21swapper/11
26387182161,4sleep22638757-21grep2
25774272161,7sleep02577438-21ssh0
25686222160,1sleep00-21swapper/00
240540599161,13cyclictest0-21swapper/33
240540599161,13cyclictest0-21swapper/33
240540599161,12cyclictest0-21swapper/33
240540599161,12cyclictest0-21swapper/33
240540599160,13cyclictest0-21swapper/33
240540599160,13cyclictest0-21swapper/33
28831752150,3sleep12536312-21diskmemload1
28651712151,9sleep20-21swapper/22
28325712151,2sleep00-21swapper/00
28149212151,2sleep20-21swapper/22
27211152151,3sleep20-21swapper/22
27127542150,2sleep02536312-21diskmemload0
26977382151,2sleep00-21swapper/00
240540599151,2cyclictest0-21swapper/33
240540599151,2cyclictest0-21swapper/33
2405405991512,2cyclictest0-21swapper/33
2405405991512,2cyclictest0-21swapper/33
2405405991512,1cyclictest0-21swapper/33
2405405991511,3cyclictest0-21swapper/33
2405405991511,2cyclictest2489129-21taskset3
2405405991510,3cyclictest0-21swapper/33
240540599150,13cyclictest0-21swapper/33
240540599150,13cyclictest0-21swapper/33
240540599150,12cyclictest0-21swapper/33
240540599150,12cyclictest0-21swapper/33
240540599150,12cyclictest0-21swapper/33
28733412141,9sleep02536312-21diskmemload0
28512412141,5sleep02852873-21kthreadcore0
28068982141,7sleep22536312-21diskmemload2
27657372141,2sleep00-21swapper/00
27453922141,4sleep12745402-21ssh1
26236792141,2sleep10-21swapper/11
26034442141,2sleep10-21swapper/11
25946092140,2sleep00-21swapper/00
25837952141,7sleep12583838-21ntpq1
25439302141,2sleep00-21swapper/00
240540599145,6cyclictest2445122-21turbostat3
240540599142,8cyclictest0-21swapper/33
240540599140,12cyclictest0-21swapper/33
240540599140,11cyclictest0-21swapper/33
28899212131,8sleep12536312-21diskmemload1
28823482131,8sleep02882362-21ssh0
28815192131,8sleep22536312-21diskmemload2
28570812131,6sleep20-21swapper/22
28489222131,2sleep22536312-21diskmemload2
28486652131,5sleep00-21swapper/00
27641892131,4sleep22764202-21rm2
27455612135,3sleep02536312-21diskmemload0
27301012131,7sleep02536312-21diskmemload0
26563072131,8sleep20-21swapper/22
26203982131,8sleep12620408-21ssh1
25454432131,7sleep02545662-21hddtemp_smartct0
25454432131,7sleep02545662-21hddtemp_smartct0
28339422121,7sleep02834472-21cut0
27914812121,7sleep02791560-21fschecks_time0
27879122121,2sleep20-21swapper/22
27810972122,6sleep22781108-21sh2
27452632121,7sleep22745274-21ssh2
27286012121,6sleep22536312-21diskmemload2
26959062121,8sleep22695920-21ssh2
26770882121,5sleep22677156-21ntpq2
26709052121,7sleep12670919-21ssh1
26194002121,6sleep22536312-21diskmemload2
25246372120,2sleep20-21swapper/22
25087712121,7sleep12508862-21ps1
24825712120,1sleep20-21swapper/22
24825712120,1sleep20-21swapper/22
24392752120,6sleep12439205-31find1
24279362121,7sleep12428010-21grep1
24143762121,7sleep02416140-21cut0
28913542111,4sleep20-21swapper/22
28739612112,2sleep10-21swapper/11
28658542111,4sleep12536312-21diskmemload1
28627842110,5sleep02862842-21ntpq0
28469212111,6sleep12846933-21ssh1
28392122111,4sleep22536312-21diskmemload2
28215032110,7sleep12821513-21ssh1
28170232111,5sleep22817104-21irqstats2
28135272110,2sleep10-21swapper/11
28134212111,3sleep00-21swapper/00
28064472111,5sleep02536312-21diskmemload0
28037042111,6sleep12803760-21ntpq1
27965232111,4sleep22536312-21diskmemload2
27868682111,6sleep02786911-21ntpq0
27798522111,4sleep02536312-21diskmemload0
27514252111,6sleep22752705-21memory2
27376212111,6sleep22737631-21ssh2
26734622111,6sleep12674631-21cut1
26716232111,3sleep20-21swapper/22
26696722111,5sleep02536312-21diskmemload0
26514542111,5sleep12651511-21ntpq1
26513962110,1sleep00-21swapper/00
26271782111,5sleep02536312-21diskmemload0
26133202111,5sleep02613585-21df0
26110932111,3sleep00-21swapper/00
25859982111,4sleep02536312-21diskmemload0
25796822111,5sleep22579897-21diskstats2
25773902111,6sleep12577401-21sshd1
25620602110,6sleep12562071-21ssh1
25593662111,7sleep12536312-21diskmemload1
24833002111,3sleep10-21swapper/11
24671032111,5sleep12467262-21diskstats1
24055902111,7sleep12405729-21cpuspeed_turbos1
28887112101,4sleep02536312-21diskmemload0
28564212101,3sleep10-21swapper/11
28388932100,2sleep12838896-21ssh1
28309382101,3sleep10-21swapper/11
28301992101,2sleep20-21swapper/22
28214992101,3sleep00-21swapper/00
27767202101,6sleep12536312-21diskmemload1
27650842101,4sleep12536312-21diskmemload1
27492912101,6sleep02749793-21cat0
27361622101,4sleep02536312-21diskmemload0
27359242101,4sleep12736024-21ntpq1
27291052102,4sleep12729120-21ssh1
27124612101,2sleep20-21swapper/22
27031622101,4sleep12536312-21diskmemload1
26886942101,4sleep12536312-21diskmemload1
26875972100,1sleep00-21swapper/00
26778412101,2sleep00-21swapper/00
26616242100,1sleep10-21swapper/11
26434132101,4sleep12643601-21users1
26368662101,3sleep00-21swapper/00
26296442101,4sleep22536312-21diskmemload2
26118082101,4sleep20-21swapper/22
25964812101,4sleep02536312-21diskmemload0
25922192101,4sleep12536312-21diskmemload1
25526532100,6sleep12536312-21diskmemload1
25526532100,6sleep12536312-21diskmemload1
25504052101,4sleep22536312-21diskmemload2
25504052101,4sleep22536312-21diskmemload2
25424642101,3sleep10-21swapper/11
25208612101,5sleep02520965-21cut0
25153372101,5sleep12515544-21diskstats1
25152752101,6sleep02515333-21cpuspeed_turbos0
25140072101,4sleep02514053-21ntpq0
24833652101,4sleep22483570-21diskstats2
24386932101,4sleep02438909-21users0
24382982101,4sleep22438349-21ntpq2
24131882101,6sleep10-21swapper/11
2872501291,3sleep22536312-21diskmemload2
2795208291,4sleep12795330-21ssh1
2787657291,3sleep12536312-21diskmemload1
2767661290,5sleep12769370-21kthreadcore1
2762605291,3sleep02536312-21diskmemload0
2720963291,3sleep10-21swapper/11
2703533291,4sleep22703536-21ssh2
2700762291,3sleep00-21swapper/00
2695993291,2sleep10-21swapper/11
2660702290,1sleep00-21swapper/00
2636419291,3sleep22536312-21diskmemload2
2612183290,1sleep10-21swapper/11
2601894290,1sleep20-21swapper/22
2593056291,2sleep20-21swapper/22
2578041290,2sleep20-21swapper/22
2560245291,5sleep02536312-21diskmemload0
2558798290,2sleep20-21swapper/22
2542671291,2sleep20-21swapper/22
2532086290,5sleep12533780-21cut1
2531235291,3sleep02531254-21apt-get0
2526210291,4sleep22526306-21tune2fs2
2514055290,4sleep12514108-21ntp_states1
2510218291,4sleep22510430-21sh2
2494240291,4sleep02494443-21gltestperf0
2483782291,4sleep02483853-21irqstats0
2481670291,4sleep02481718-21ntpq0
2481670291,4sleep02481718-21ntpq0
2472346291,4sleep22472530-21cstates2
2461364291,4sleep02461401-21perf0
2455020291,4sleep02455064-21modprobe0
2452440291,4sleep12454211-21cut1
2441763291,4sleep12443557-21cut1
2647563282,3sleep20-21swapper/22
2524517281,2sleep12524630-21ntpq1
2493469280,1sleep10-21swapper/11
2466996281,3sleep00-21swapper/00
2461111280,4sleep10-21swapper/11
2449820281,3sleep12449904-21ps1
2445098281,3sleep02445169-21munin-run0
2433675280,2sleep272-21kcompactd02
2422302281,2sleep00-21swapper/00
2419924281,3sleep12420341-21kthreadcore1
2417728280,4sleep20-21swapper/22
2409606281,3sleep02409654-21ntpq0
2536187270,1sleep20-21swapper/22
2527165271,3sleep10-21swapper/11
2525930270,2sleep00-21swapper/00
2519983270,1sleep20-21swapper/22
2508972271,3sleep00-21swapper/00
2503786270,1sleep00-21swapper/00
2499888271,2sleep20-21swapper/22
2498665270,1sleep20-21swapper/22
2493951270,2sleep10-21swapper/11
2493581270,1sleep20-21swapper/22
2489237271,3sleep00-21swapper/00
2472324270,2sleep10-21swapper/11
2461669270,1sleep10-21swapper/11
2461644270,1sleep20-21swapper/22
2423586271,3sleep00-21swapper/00
2508567260,2sleep20-21swapper/22
2499277260,1sleep10-21swapper/11
2478222260,3sleep12479533-21kthreadcore1
2478222260,3sleep12479533-21kthreadcore1
2477041260,1sleep00-21swapper/00
2461647260,2sleep02461666-21apt-get0
2456036260,2sleep20-21swapper/22
2447216260,3sleep0221rcuc/00
2439628260,2sleep20-21swapper/22
2467263250,1sleep20-21swapper/22
2460668250,2sleep20-21swapper/22
2450397250,1sleep20-21swapper/22
2428918250,1sleep00-21swapper/00
2423853250,1sleep20-21swapper/22
2422404250,2sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional