You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-27 - 22:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Mon Apr 27, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
50959211858,56sleep30-21swapper/33
5101528755,12sleep10-21swapper/11
1958792860,1sleep2195878-21ssh2
5099628366,11sleep20-21swapper/22
5104927959,14sleep00-21swapper/00
604162400,5sleep0471rcuog/20
51283992511,9cyclictest442324-21taskset3
5128399226,14cyclictest0-21swapper/33
51283992117,2cyclictest0-21swapper/33
5128399211,16cyclictest66950irq/132-enp1s0-TxRx-33
51283992013,4cyclictest509510-21taskset3
5128399201,15cyclictest66950irq/132-enp1s0-TxRx-33
5128399201,14cyclictest0-21swapper/33
5128399201,13cyclictest0-21swapper/33
3889032201,5sleep2388961-21ntpq2
5128399192,15cyclictest66950irq/132-enp1s0-TxRx-33
51283991911,6cyclictest0-21swapper/33
51283991911,5cyclictest255754-21taskset3
5128399191,15cyclictest0-21swapper/33
5128399191,15cyclictest0-21swapper/33
5128399191,14cyclictest272304-21taskset3
5128399191,14cyclictest196188-21taskset3
5128399191,14cyclictest0-21swapper/33
5128399191,14cyclictest0-21swapper/33
5128399191,14cyclictest0-21swapper/33
5128399191,13cyclictest0-21swapper/33
5005952191,12sleep00-21swapper/00
4189442191,3sleep2420146-21kthreadcore2
5128399181,5cyclictest0-21swapper/33
5128399181,4cyclictest117470-21taskset3
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,14cyclictest0-21swapper/33
5128399181,13cyclictest0-21swapper/33
5128399181,13cyclictest0-21swapper/33
5128399181,13cyclictest0-21swapper/33
5128399181,13cyclictest0-21swapper/33
5128399181,13cyclictest0-21swapper/33
5128399180,14cyclictest0-21swapper/33
5171272178,2sleep2171rcu_preempt2
5128399172,10cyclictest262112-21taskset3
51283991712,3cyclictest0-21swapper/33
5128399171,13cyclictest0-21swapper/33
5128399171,13cyclictest0-21swapper/33
5128399171,13cyclictest0-21swapper/33
5128399171,13cyclictest0-21swapper/33
5128399171,13cyclictest0-21swapper/33
5128399171,13cyclictest0-21swapper/33
5128399170,14cyclictest0-21swapper/33
5128399170,14cyclictest0-21swapper/33
1875862171,3sleep00-21swapper/00
5128399169,4cyclictest1497512sleep33
51283991613,1cyclictest0-21swapper/33
5128399160,13cyclictest0-21swapper/33
5128399160,13cyclictest0-21swapper/33
5128399160,13cyclictest0-21swapper/33
3847162161,5sleep1384789-21grep1
3496382161,4sleep2181976-21diskmemload2
2720712161,11sleep1272082-21ssh1
2652262162,9sleep0181976-21diskmemload0
2375082161,4sleep2237557-21ntpq2
2326442160,2sleep0232655-21ssh0
853482151,10sleep184777-31find1
5128399151,8cyclictest1180082sleep33
51283991512,2cyclictest0-21swapper/33
51283991512,2cyclictest0-21swapper/33
5128399151,12cyclictest0-21swapper/33
5128399150,13cyclictest0-21swapper/33
5128399150,13cyclictest0-21swapper/33
5128399150,13cyclictest0-21swapper/33
5128399150,12cyclictest0-21swapper/33
5128399150,12cyclictest0-21swapper/33
5128399150,12cyclictest0-21swapper/33
5128399150,12cyclictest0-21swapper/33
5128399150,12cyclictest0-21swapper/33
4510592150,3sleep00-21swapper/00
4488622151,9sleep1181976-21diskmemload1
4243862151,2sleep10-21swapper/11
2991392151,2sleep20-21swapper/22
2970222150,1sleep00-21swapper/00
2681002151,10sleep2269773-21grep2
2580092156,6sleep1181976-21diskmemload1
1759832159,3sleep00-21swapper/00
911062141,9sleep191133-21perf1
911062141,9sleep191133-21perf1
5256092141,9sleep1525620-21ssh1
51283991412,1cyclictest0-21swapper/33
5128399140,12cyclictest0-21swapper/33
4571362141,9sleep0457146-21ssh0
4140572141,9sleep0414121-21ntpq0
4088932141,3sleep0181976-21diskmemload0
3937082141,7sleep0394473-21grep0
3724032145,2sleep2171rcu_preempt2
3574882141,8sleep1181976-21diskmemload1
2483132141,6sleep20-21swapper/22
1953542141,2sleep00-21swapper/00
970412131,8sleep296693-21/usr/sbin/munin2
654642131,4sleep267196-21grep2
647562131,8sleep165463-21cut1
5128399130,1cyclictest0-21swapper/33
4840002131,3sleep0484008-21ssh0
3549782131,8sleep0181976-21diskmemload0
3010762131,8sleep0301681-21cut0
2483512131,7sleep1181976-21diskmemload1
2168632131,6sleep2216944-21irqstats2
1531742131,4sleep1153937-21sed1
853542121,5sleep284714-31sort2
554072121,5sleep255548-21ntpq2
5344012121,6sleep1181976-21diskmemload1
5277132120,1sleep00-21swapper/00
5189132120,5sleep10-21swapper/11
514782122,5sleep051733-21df0
4922192121,5sleep20-21swapper/22
4731622121,6sleep0473227-21ssh0
4731622121,6sleep0473227-21ssh0
4409302121,6sleep1181976-21diskmemload1
4358042121,5sleep0435863-21diskstats0
4157992121,7sleep1415810-21ssh1
3914062120,2sleep00-21swapper/00
3472612121,7sleep0347262-21ssh0
3339612121,5sleep2334226-21df2
3251342121,7sleep1325148-21ssh1
2603602121,6sleep2262111-21cut2
2541002121,5sleep2254249-21ntpq2
2497412121,7sleep0249849-21sed0
2214372121,7sleep1221448-21ssh1
1950832121,7sleep1195147-21modprobe1
1823812121,3sleep1182522-21expr1
1648282121,3sleep1164960-21/usr/sbin/munin1
1337522121,5sleep1133800-21munin-run1
1078602121,3sleep0108013-21tr0
5361542111,3sleep20-21swapper/22
5106872111,7sleep0510698-21sh0
5001412111,4sleep2181976-21diskmemload2
4912862111,7sleep1491293-21ssh1
4852932111,4sleep1485325-21munin-run1
4756372110,1sleep10-21swapper/11
4756372110,1sleep10-21swapper/11
4729572111,5sleep2473097-21ntpq2
4729572111,5sleep2473097-21ntpq2
4679532111,3sleep10-21swapper/11
4646552111,5sleep0464707-21ntpq0
4596532111,3sleep10-21swapper/11
4522502111,5sleep2452517-21df2
4313492111,6sleep2431403-21ntpq2
3997982110,2sleep20-21swapper/22
3747652111,3sleep1374779-21ssh1
3736752111,7sleep0373687-21sh0
3647822111,7sleep1364796-21ssh1
3586502111,6sleep2358848-21latency_hist2
3345192111,5sleep1334589-21ps1
3128522111,5sleep0312996-21ntpq0
2989872110,7sleep1181976-21diskmemload1
2808672111,3sleep00-21swapper/00
2807402111,3sleep20-21swapper/22
2383702111,6sleep0238380-21ssh0
2294092111,6sleep20-21swapper/22
2205732111,5sleep0220633-21ntpq0
2078992111,4sleep0207927-21apt-get0
2048642111,7sleep0204874-21ssh0
1662442112,4sleep0166263-21apt-get0
973792101,5sleep098773-21kthreadcore0
913822101,6sleep00-21swapper/00
853522101,4sleep084777-31find0
751632101,4sleep20-21swapper/22
5186122101,2sleep00-21swapper/00
5084482101,2sleep20-21swapper/22
5068932101,3sleep10-21swapper/11
4655652101,2sleep20-21swapper/22
4515092101,3sleep20-21swapper/22
4420692101,5sleep2442081-21ssh2
4275012101,6sleep0427130-21/usr/sbin/munin0
4152832101,4sleep2181976-21diskmemload2
4068912101,5sleep2406893-21ssh2
3980832101,4sleep1181976-21diskmemload1
3839302101,2sleep10-21swapper/11
3767582101,4sleep2376840-21irqstats2
3764132100,3sleep00-21swapper/00
3638292100,6sleep0363908-21find0
3637262101,3sleep2181976-21diskmemload2
3396402100,1sleep00-21swapper/00
3317192101,3sleep20-21swapper/22
3294542101,3sleep0181976-21diskmemload0
3255232101,4sleep1181976-21diskmemload1
3244572101,3sleep00-21swapper/00
3238212101,3sleep20-21swapper/22
3163362101,4sleep2181976-21diskmemload2
3059702101,4sleep1181976-21diskmemload1
3007922101,3sleep272-21kcompactd02
2900312101,4sleep2181976-21diskmemload2
2661272101,6sleep1181976-21diskmemload1
2314272101,3sleep10-21swapper/11
2079352101,5sleep1208101-21cstates1
1892852101,3sleep20-21swapper/22
1715702100,1sleep10-21swapper/11
1649612101,4sleep2164989-21ntpq2
1597552101,4sleep0159995-21users0
1497132101,5sleep2149740-21munin-run2
1496992101,6sleep0144415-21perf0
1312882101,5sleep0132593-21memory0
1129632101,5sleep2113115-21head2
533411291,2sleep00-21swapper/00
520032291,3sleep2181976-21diskmemload2
499697291,2sleep10-21swapper/11
492885291,3sleep0181976-21diskmemload0
482948291,2sleep20-21swapper/22
434996291,3sleep1181976-21diskmemload1
423368291,2sleep00-21swapper/00
401213291,2sleep10-21swapper/11
348476291,3sleep1181976-21diskmemload1
313859291,4sleep1313869-21ssh1
289297291,2sleep00-21swapper/00
281168290,3sleep10-21swapper/11
256002291,4sleep0256015-21ssh0
212922291,4sleep2212936-21ssh2
205121291,3sleep20-21swapper/22
203438290,4sleep1181976-21diskmemload1
144973290,1sleep10-21swapper/11
116793291,4sleep1116853-21ld1
116742291,4sleep0116791-21ntpq0
102018291,4sleep0102170-21cat0
289827280,3sleep1181976-21diskmemload1
271775281,3sleep0271783-21ssh0
241076280,2sleep10-21swapper/11
144131281,3sleep0802-21snmpd0
138160281,3sleep0138206-21/usr/sbin/munin0
128688281,2sleep20-21swapper/22
127676280,1sleep00-21swapper/00
125886281,3sleep1127239-21memory1
112242280,1sleep10-21swapper/11
106733280,2sleep10-21swapper/11
102285280,5sleep2102448-21gltestperf2
91145270,2sleep20-21swapper/22
78077270,4sleep00-21swapper/00
74490270,1sleep10-21swapper/11
68140271,3sleep00-21swapper/00
63864270,1sleep10-21swapper/11
57189271,3sleep20-21swapper/22
181744270,1sleep10-21swapper/11
177469271,2sleep20-21swapper/22
175637270,3sleep2175686-21ntpq2
168433271,2sleep20-21swapper/22
166850271,3sleep1168431-21cut1
159387270,3sleep20-21swapper/22
153938270,2sleep20-21swapper/22
128262270,1sleep20-21swapper/22
123072270,3sleep00-21swapper/00
92831261,2sleep10-21swapper/11
91063260,1sleep20-21swapper/22
91063260,1sleep20-21swapper/22
89943261,2sleep00-21swapper/00
89943261,2sleep00-21swapper/00
80020260,1sleep10-21swapper/11
70148261,2sleep20-21swapper/22
69539260,1sleep00-21swapper/00
56511260,1sleep10-21swapper/11
180600260,2sleep0180913-21strings0
165907260,4sleep00-21swapper/00
151429261,2sleep00-21swapper/00
134016260,2sleep20-21swapper/22
122737260,4sleep10-21swapper/11
155299250,1sleep10-21swapper/11
144238250,1sleep20-21swapper/22
143414250,1sleep10-21swapper/11
138492250,2sleep10-21swapper/11
122827250,1sleep20-21swapper/22
100807250,1sleep10-21swapper/11
107571240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional