You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 08:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Fri Jul 10, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27344721270,2sleep20-21swapper/22
4100915211756,56sleep30-21swapper/33
410087928455,12sleep10-21swapper/11
410059828454,12sleep20-21swapper/22
410082827656,14sleep00-21swapper/00
3510932460,2sleep00-21swapper/00
410110899286,18cyclictest66950irq/132-enp1s0-TxRx-33
1814482261,20sleep1181458-21ssh1
410110899248,11cyclictest0-21swapper/33
857192231,3sleep00-21swapper/00
410110899237,13cyclictest0-21swapper/33
4101108992320,1cyclictest0-21swapper/33
4101108992320,1cyclictest0-21swapper/33
4101108992320,1cyclictest0-21swapper/33
4101108992319,2cyclictest0-21swapper/33
4101108992219,1cyclictest0-21swapper/33
4101108992218,2cyclictest0-21swapper/33
4101108992218,2cyclictest0-21swapper/33
4101108992218,2cyclictest0-21swapper/33
410110899218,10cyclictest0-21swapper/33
410110899216,2cyclictest0-21swapper/33
410110899216,13cyclictest0-21swapper/33
4101108992118,2cyclictest0-21swapper/33
4101108992118,1cyclictest0-21swapper/33
4101108992117,2cyclictest0-21swapper/33
4101108992117,2cyclictest0-21swapper/33
4101108992117,2cyclictest0-21swapper/33
4101108992117,2cyclictest0-21swapper/33
4101108992117,2cyclictest0-21swapper/33
4101108992117,2cyclictest0-21swapper/33
4101108992117,2cyclictest0-21swapper/33
4101108992116,2cyclictest0-21swapper/33
4101108992116,2cyclictest0-21swapper/33
410110899211,16cyclictest0-21swapper/33
410110899211,16cyclictest0-21swapper/33
410110899211,15cyclictest0-21swapper/33
410110899211,15cyclictest0-21swapper/33
1261622211,16sleep2126176-21ssh2
410110899202,14cyclictest4119004-21taskset3
4101108992017,1cyclictest0-21swapper/33
4101108992017,1cyclictest0-21swapper/33
4101108992017,1cyclictest0-21swapper/33
4101108992017,1cyclictest0-21swapper/33
4101108992016,2cyclictest0-21swapper/33
4101108992016,2cyclictest0-21swapper/33
4101108992016,2cyclictest0-21swapper/33
4101108992016,2cyclictest0-21swapper/33
4101108992016,2cyclictest0-21swapper/33
4101108992016,2cyclictest0-21swapper/33
4101108992015,2cyclictest541ktimers/33
4101108992014,3cyclictest0-21swapper/33
410110899201,16cyclictest0-21swapper/33
410110899201,14cyclictest0-21swapper/33
1801702201,2sleep00-21swapper/00
410110899199,6cyclictest4139681-21sleep3
4101108991917,1cyclictest0-21swapper/33
4101108991916,2cyclictest0-21swapper/33
4101108991916,2cyclictest0-21swapper/33
4101108991916,1cyclictest0-21swapper/33
4101108991916,1cyclictest0-21swapper/33
4101108991916,1cyclictest0-21swapper/33
4101108991915,2cyclictest541ktimers/33
4101108991915,2cyclictest0-21swapper/33
4101108991915,2cyclictest0-21swapper/33
4101108991915,2cyclictest0-21swapper/33
410110899191,14cyclictest0-21swapper/33
3713422191,4sleep134726-21diskmemload1
1952452191,4sleep2195259-21sh2
505782180,1sleep20-21swapper/22
410110899183,12cyclictest0-21swapper/33
4101108991815,1cyclictest0-21swapper/33
2742242180,2sleep0237240-21kworker/u16:1+rpciod0
2328712181,4sleep0232917-21idleruntime-cro0
4101108991714,1cyclictest0-21swapper/33
410110899171,13cyclictest0-21swapper/33
410110899170,13cyclictest0-21swapper/33
410110899170,13cyclictest0-21swapper/33
3155372171,12sleep1315549-21ssh1
3101202171,11sleep034726-21diskmemload0
2899942171,2sleep20-21swapper/22
1191722171,3sleep00-21swapper/00
1093582172,4sleep134726-21diskmemload1
498632161,2sleep10-21swapper/11
4101108991613,1cyclictest0-21swapper/33
410110899160,13cyclictest0-21swapper/33
3980952161,3sleep20-21swapper/22
2246752167,5sleep2224776-21cut2
1988252160,3sleep20-21swapper/22
1377962161,3sleep10-21swapper/11
770872151,9sleep077102-21ssh0
410110899158,4cyclictest4191820-21chrt3
4101108991512,1cyclictest0-21swapper/33
4101108991511,2cyclictest0-21swapper/33
2386512151,8sleep034726-21diskmemload0
2386512151,8sleep034726-21diskmemload0
2220532151,9sleep1222064-21ssh1
2171572150,6sleep2218365-21ssh2
1721762151,4sleep134726-21diskmemload1
436992141,9sleep043713-21ssh0
35642141,9sleep10-21swapper/11
2934752141,2sleep1171rcu_preempt1
2819552141,7sleep134726-21diskmemload1
2556532141,9sleep00-21swapper/00
2241432141,8sleep0224157-21ssh0
1706542141,9sleep0170668-21ssh0
1039942141,7sleep2104018-21apt-get2
922892131,7sleep034726-21diskmemload0
673632131,7sleep134726-21diskmemload1
41818112130,2sleep00-21swapper/00
41556312130,2sleep10-21swapper/11
41086112131,6sleep04108805-21diskstats0
4101108991311,1cyclictest0-21swapper/33
3436632131,8sleep2343673-21ssh2
2976222131,4sleep2297721-21ssh2
250782131,8sleep226035-21cut2
1964032130,2sleep10-21swapper/11
1873182130,4sleep2187328-21ssh2
1538292131,7sleep1153843-21ssh1
1520222131,2sleep20-21swapper/22
130252131,8sleep213238-21/usr/sbin/munin2
1182332130,3sleep2118246-21ssh2
1098962131,5sleep00-21swapper/00
1032702131,8sleep134726-21diskmemload1
952812121,7sleep234726-21diskmemload2
41815422121,7sleep24181726-21grep2
41658272121,7sleep14165964-21cat1
41062432121,5sleep04106299-21munin-run0
3959172120,1sleep00-21swapper/00
3611902121,6sleep1361201-21ssh1
3349502121,7sleep0334960-21ssh0
3256712121,7sleep034726-21diskmemload0
3177852121,6sleep2317800-21ssh2
315992121,7sleep033351-21cut0
2920232120,8sleep10-21swapper/11
2761062120,5sleep00-21swapper/00
2668662121,7sleep2266880-21ssh2
2655492121,5sleep00-21swapper/00
2552742121,7sleep1255275-21vmstat1
2469622121,5sleep234726-21diskmemload2
2398632120,2sleep10-21swapper/11
2398632120,2sleep10-21swapper/11
2130212121,7sleep1213034-21rm1
1637272121,3sleep20-21swapper/22
1553262121,7sleep0155333-21cron0
1548182121,7sleep0154829-21ssh0
1318422121,3sleep0133534-21grep0
1195892121,4sleep10-21swapper/11
922092110,4sleep10-21swapper/11
832522111,6sleep183266-21ssh1
742752111,4sleep234726-21diskmemload2
659842110,2sleep00-21swapper/00
615512111,6sleep261834-21kthreadcore2
41870732111,5sleep14187331-21df_inode1
41387072111,5sleep24138759-21ntp_states2
41242452112,4sleep04124241-21apt-get0
3790992112,3sleep234726-21diskmemload2
3779292111,5sleep134726-21diskmemload1
3683422111,5sleep2368546-21memory2
3627732110,1sleep00-21swapper/00
3531852111,4sleep234726-21diskmemload2
3526742111,5sleep134726-21diskmemload1
350772111,5sleep134726-21diskmemload1
350772111,5sleep134726-21diskmemload1
3451142110,1sleep10-21swapper/11
3358372111,5sleep234726-21diskmemload2
3256822111,5sleep134726-21diskmemload1
3097252110,1sleep10-21swapper/11
3015162110,1sleep00-21swapper/00
288522111,5sleep129088-21users1
2555332111,3sleep20-21swapper/22
1904642111,5sleep0190522-21/usr/sbin/munin0
1780562110,1sleep10-21swapper/11
1729822111,6sleep2173223-21expr2
1447282111,6sleep0144740-21ssh0
1385192111,5sleep1138786-21df1
1286942111,3sleep00-21swapper/00
1028492111,3sleep00-21swapper/00
836882100,1sleep20-21swapper/22
568802100,2sleep20-21swapper/22
41609492101,4sleep24161152-21diskstats2
41576732102,3sleep04159431-21tr0
41294562101,4sleep24129475-21apt-get2
41211212101,5sleep04122809-21wc0
407382101,4sleep034726-21diskmemload0
407382101,4sleep034726-21diskmemload0
399102101,2sleep20-21swapper/22
399102101,2sleep20-21swapper/22
3952612101,4sleep134726-21diskmemload1
3870562101,4sleep034726-21diskmemload0
3697052101,4sleep034726-21diskmemload0
3438922101,4sleep034726-21diskmemload0
3350052101,4sleep1327599-40find1
3178132101,3sleep00-21swapper/00
3073722101,3sleep20-21swapper/22
2727762101,3sleep10-21swapper/11
2482882100,1sleep00-21swapper/00
2481562102,4sleep1331ktimers/11
2297432101,3sleep134726-21diskmemload1
2137202101,3sleep00-21swapper/00
2076762101,5sleep2207898-21head2
1886662101,2sleep00-21swapper/00
1691222101,2sleep20-21swapper/22
1448072101,3sleep20-21swapper/22
1349872101,3sleep234726-21diskmemload2
1234872101,5sleep1125188-21tr1
91725291,2sleep20-21swapper/22
74484291,3sleep10-21swapper/11
58460291,2sleep00-21swapper/00
57264290,2sleep10-21swapper/11
4182359291,4sleep14182771-21grep1
4176032290,1sleep00-21swapper/00
4170308291,4sleep24170519-21timerwakeupswit2
4146768291,4sleep14148467-21cut1
4145727291,4sleep04146765-21kthreadcore0
4134682291,3sleep039542-21systemd-journal0
4132209291,4sleep14133313-21/usr/sbin/munin1
4124505291,4sleep24124608-21perl2
4106300291,3sleep14106746-21memory1
387474290,5sleep20-21swapper/22
381787291,4sleep134726-21diskmemload1
378127291,4sleep0378137-21ssh0
361491291,2sleep20-21swapper/22
292011291,2sleep00-21swapper/00
276858291,4sleep2278102-21cut2
259977291,4sleep1261520-21tr1
2468294,2sleep0221rcuc/00
239323290,2sleep20-21swapper/22
239323290,2sleep20-21swapper/22
204442291,4sleep0204456-21ssh0
204053291,3sleep134726-21diskmemload1
19761291,4sleep219891-21grep2
9352281,3sleep010381-21kthreadcore0
4191963280,1sleep20-21swapper/22
4180569280,1sleep10-21swapper/11
4171321280,2sleep20-21swapper/22
4167100281,4sleep04168737-21kthreadcore0
4150896281,3sleep14150993-21ps1
4144657280,1sleep10-21swapper/11
4123951281,3sleep11-21systemd1
4108516280,2sleep20-21swapper/22
4101252281,2sleep14101271-21apt-get1
324294280,2sleep20-21swapper/22
24800283,2sleep00-21swapper/00
156382282,2sleep10-21swapper/11
14754281,3sleep016346-21kthreadcore0
10383281,3sleep112093-21cut1
4192011271,2sleep04192030-21latency_hist0
4182772271,2sleep24184409-21kthreadcore2
4176561271,2sleep00-21swapper/00
4175979270,1sleep10-21swapper/11
4160864271,1sleep04160883-21apt-get0
4155632271,1sleep20-21swapper/22
4155179270,1sleep00-21swapper/00
4144656270,1sleep00-21swapper/00
4134683271,2sleep10-21swapper/11
4134254270,1sleep00-21swapper/00
4125406271,2sleep10-21swapper/11
4119013270,2sleep20-21swapper/22
4101808271,2sleep20-21swapper/22
3542270,1sleep00-21swapper/00
29743270,2sleep20-21swapper/22
2888270,1sleep20-21swapper/22
24106270,1sleep00-21swapper/00
18636270,1sleep20-21swapper/22
4166375261,2sleep14166430-21grep1
4150409261,2sleep20-21swapper/22
4139946260,2sleep20-21swapper/22
4118402260,1sleep20-21swapper/22
4117984261,2sleep00-21swapper/00
4113772261,1sleep10-21swapper/11
3539262,1sleep20-21swapper/22
2983260,4sleep10-21swapper/11
4148468250,1sleep20-21swapper/22
33352250,1sleep10-21swapper/11
19251250,1sleep10-21swapper/11
14294250,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional