You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 15:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Tue Aug 11, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
152401212459,60sleep30-21swapper/33
15715628956,12sleep20-21swapper/22
15728928354,13sleep10-21swapper/11
15727827957,16sleep00-21swapper/00
15749599238,12cyclictest0-21swapper/33
157495992317,4cyclictest0-21swapper/33
15749599222,14cyclictest487972-21taskset3
5664082211,3sleep20-21swapper/22
3628242201,15sleep2362836-21sh2
15749599202,12cyclictest426196-21taskset3
157495992016,2cyclictest0-21swapper/33
15749599201,16cyclictest0-21swapper/33
15749599201,16cyclictest0-21swapper/33
15749599201,15cyclictest0-21swapper/33
15749599201,15cyclictest0-21swapper/33
15749599191,7cyclictest344182-21taskset3
15749599191,15cyclictest0-21swapper/33
15749599191,14cyclictest0-21swapper/33
15749599191,14cyclictest0-21swapper/33
15749599191,14cyclictest0-21swapper/33
15749599191,14cyclictest0-21swapper/33
15749599191,14cyclictest0-21swapper/33
15749599191,14cyclictest0-21swapper/33
15749599190,2cyclictest0-21swapper/33
15749599190,17cyclictest0-21swapper/33
15749599190,16cyclictest0-21swapper/33
5575732181,3sleep20-21swapper/22
2929002181,3sleep10-21swapper/11
157495991815,1cyclictest0-21swapper/33
157495991811,4cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,14cyclictest0-21swapper/33
15749599181,13cyclictest190628-21taskset3
15749599181,13cyclictest0-21swapper/33
15749599181,13cyclictest0-21swapper/33
15749599181,13cyclictest0-21swapper/33
15749599181,13cyclictest0-21swapper/33
15749599181,13cyclictest0-21swapper/33
15749599181,13cyclictest0-21swapper/33
15749599181,13cyclictest0-21swapper/33
15749599180,14cyclictest0-21swapper/33
15749599180,14cyclictest0-21swapper/33
6000562171,2sleep00-21swapper/00
1967842171,11sleep0197030-21grep0
15749599177,7cyclictest185514-21turbostat3
157495991711,4cyclictest0-21swapper/33
15749599171,13cyclictest0-21swapper/33
15749599171,13cyclictest0-21swapper/33
15749599171,13cyclictest0-21swapper/33
6093742161,2sleep20-21swapper/22
6093742161,2sleep20-21swapper/22
4158672161,10sleep1415882-21modprobe1
2751192161,3sleep00-21swapper/00
15749599161,13cyclictest0-21swapper/33
15749599161,12cyclictest0-21swapper/33
15749599161,12cyclictest0-21swapper/33
15749599161,11cyclictest0-21swapper/33
15749599160,13cyclictest0-21swapper/33
15749599160,13cyclictest0-21swapper/33
15749599160,13cyclictest0-21swapper/33
15749599160,13cyclictest0-21swapper/33
15749599160,13cyclictest0-21swapper/33
15749599160,13cyclictest0-21swapper/33
5567532150,2sleep10-21swapper/11
5184732151,9sleep2513992-21sendmail-msp2
5107362151,2sleep00-21swapper/00
4452412151,5sleep1445902-21kthreadcore1
4407082151,10sleep0440722-21ssh0
2292112151,10sleep2230922-21tr2
1754492151,9sleep1175652-21diskstats1
15749599151,12cyclictest0-21swapper/33
15749599150,12cyclictest0-21swapper/33
6114202141,6sleep1611452-21ssh1
5982602142,8sleep20-21swapper/22
5873782141,2sleep00-21swapper/00
5464792141,8sleep2284726-21diskmemload2
5340012141,9sleep10-21swapper/11
5265742141,6sleep1527232-21sed1
5051452141,3sleep0505258-21latency_hist0
4764392141,9sleep2476450-21ssh2
4406792141,8sleep2284726-21diskmemload2
4261632141,7sleep2284726-21diskmemload2
3878302141,3sleep0284726-21diskmemload0
3868882140,2sleep10-21swapper/11
3206542141,9sleep20-21swapper/22
3095792141,8sleep1284726-21diskmemload1
2640702142,7sleep1264082-21apt-get1
15749599148,4cyclictest247791-21chrt3
15749599141,12cyclictest0-21swapper/33
15749599141,10cyclictest222051-21turbostat3
15749599140,2cyclictest0-21swapper/33
15749599140,12cyclictest0-21swapper/33
15749599140,11cyclictest0-21swapper/33
15749599140,11cyclictest0-21swapper/33
6478712131,8sleep1649456-21grep1
6290942131,8sleep0629307-21cstates0
6259102131,3sleep1625921-21ssh1
6167132131,3sleep0616724-21ssh0
5193512131,8sleep0519363-21ssh0
4947782131,8sleep0494793-21ssh0
4062222131,6sleep00-21swapper/00
3951422131,8sleep0395284-21ssh0
3604782131,5sleep0360485-21ssh0
3365582131,8sleep0336572-21ssh0
2921422131,8sleep0284726-21diskmemload0
1750932131,6sleep0175150-21munin-run0
5747162120,5sleep10-21swapper/11
5048602121,6sleep2284726-21diskmemload2
4933562121,5sleep1284726-21diskmemload1
4879402121,5sleep2284726-21diskmemload2
4770562121,7sleep0477067-21ssh0
4349422122,5sleep1434961-21apt-get1
4341942121,6sleep1284726-21diskmemload1
4237632121,7sleep1284726-21diskmemload1
4141252121,6sleep2284726-21diskmemload2
4090682125,3sleep0284726-21diskmemload0
3785672120,1sleep00-21swapper/00
3519512121,7sleep2351962-21ssh2
3512402121,3sleep1351470-21ssh1
3196202121,6sleep1284726-21diskmemload1
2915942121,7sleep2291605-21ssh2
2594202121,6sleep2260281-21grep2
2471632121,5sleep1247233-21ntp_states1
6433842111,3sleep20-21swapper/22
6391682111,6sleep0640789-21grep0
6286022111,6sleep0628616-21ssh0
6231362110,2sleep2624220-21ssh2
6086722111,6sleep1608685-21ssh1
6086722111,6sleep1608685-21ssh1
5994222111,5sleep1284726-21diskmemload1
5841502111,3sleep10-21swapper/11
5767502113,4sleep00-21swapper/00
5752022111,3sleep20-21swapper/22
5566382111,6sleep0556652-21ssh0
5131502110,5sleep20-21swapper/22
4576822110,1sleep00-21swapper/00
4521642111,6sleep2452178-21ssh2
4485122110,5sleep0443370-21sendmail-msp0
4247882111,3sleep20-21swapper/22
3891132111,6sleep2389127-21ssh2
3516602111,6sleep0106713-21snmpd0
3353542111,4sleep1284726-21diskmemload1
3346812111,3sleep20-21swapper/22
3277552111,6sleep0327768-21rm0
3180652111,6sleep0318078-21rm0
2748302111,6sleep2274310-21sendmail-msp2
1806112111,5sleep0180869-21df_inode0
1619942111,5sleep1162211-21users1
6524992101,3sleep00-21swapper/00
6517212101,2sleep20-21swapper/22
6455002101,4sleep1284726-21diskmemload1
6087142100,2sleep00-21swapper/00
6087142100,2sleep00-21swapper/00
5929062101,4sleep2284726-21diskmemload2
5826722101,2sleep20-21swapper/22
5745842100,1sleep00-21swapper/00
5630462101,4sleep0563304-21users0
5584492101,4sleep10-21swapper/11
5489752101,4sleep1284726-21diskmemload1
5401302101,3sleep20-21swapper/22
5291922101,4sleep0284726-21diskmemload0
5290092101,3sleep20-21swapper/22
5187702101,3sleep10-21swapper/11
4973892101,5sleep1498390-21kthreadcore1
4864402101,3sleep00-21swapper/00
4853882100,1sleep10-21swapper/11
4789692101,4sleep2478673-21sendmail-msp2
4687502101,3sleep00-21swapper/00
4663342101,2sleep10-21swapper/11
4529212101,2sleep20-21swapper/22
4335532101,4sleep0284726-21diskmemload0
4227932101,6sleep0284726-21diskmemload0
4069942100,1sleep20-21swapper/22
4047962101,4sleep1284726-21diskmemload1
3956772101,3sleep20-21swapper/22
3786392100,6sleep2378651-21sh2
3711372101,4sleep1284726-21diskmemload1
3710662100,3sleep00-21swapper/00
3456592101,4sleep2284726-21diskmemload2
3438992101,4sleep1284726-21diskmemload1
3379202101,4sleep0337918-21apt-get0
3096632100,3sleep00-21swapper/00
3081072101,3sleep20-21swapper/22
3010442100,5sleep20-21swapper/22
2989012101,3sleep1284726-21diskmemload1
2693142101,4sleep1269326-21apt-get1
2591432101,5sleep0259343-21tail0
2071842101,4sleep2207255-21irqstats2
1913432100,6sleep1191017-21/usr/sbin/munin1
1857802101,4sleep2186039-21df_inode2
1759362101,2sleep00-21swapper/00
1625932101,4sleep2162650-21munin-run2
633809290,1sleep10-21swapper/11
618178291,4sleep2618189-21ssh2
540905291,4sleep0541123-21diskstats0
538502291,4sleep0538520-21ssh0
470782291,4sleep1471167-21cut1
467612291,2sleep20-21swapper/22
458918291,2sleep10-21swapper/11
390852290,1sleep10-21swapper/11
375485291,5sleep1377023-21kthreadcore1
367814291,4sleep2368450-21missed_timers2
360658291,4sleep1360660-21ssh1
320251291,3sleep1284726-21diskmemload1
315700291,4sleep2315824-21ps2
302253290,1sleep00-21swapper/00
264906291,4sleep2266521-21cut2
241973291,3sleep2242026-21ntp_states2
238026290,4sleep1238045-21apt-get1
232453291,4sleep0232480-21latency_hist0
219126291,4sleep0220879-21grep0
186319291,4sleep1187113-21grep1
185757290,4sleep0185776-21apt-get0
169904291,3sleep2169890-21sendmail-msp2
165149291,4sleep0165362-21hddtemp_smartct0
635888280,2sleep20-21swapper/22
589496281,3sleep1589716-21unixbench-2d1
506776281,4sleep1507904-21tr1
284337280,1sleep20-21swapper/22
279793280,3sleep1279812-21apt-get1
270010281,3sleep2271631-21kthreadcore2
267916280,1sleep00-21swapper/00
257433282,2sleep2257606-21strings2
253889281,3sleep14192558-21rs:main1
237523280,1sleep00-21swapper/00
222849281,3sleep2222906-21kthreadcore2
211915281,3sleep14192558-21rs:main1
211910281,2sleep0211912-21apt-get0
211880280,4sleep287-1kworker/2:1H+kblockd2
211403281,4sleep1106713-21snmpd1
201475281,2sleep2201494-21apt-get2
191551281,3sleep2191919-21cut2
164858280,2sleep10-21swapper/11
279401270,1sleep10-21swapper/11
274154270,1sleep00-21swapper/00
258853270,2sleep10-21swapper/11
258232270,1sleep00-21swapper/00
248401270,2sleep00-21swapper/00
243211271,3sleep2243411-21cstates2
217137270,2sleep20-21swapper/22
210727271,2sleep00-21swapper/00
203133271,3sleep00-21swapper/00
201765271,3sleep10-21swapper/11
197032271,2sleep10-21swapper/11
170114271,2sleep10-21swapper/11
169744270,1sleep20-21swapper/22
162421270,1sleep00-21swapper/00
284498260,1sleep00-21swapper/00
235666261,2sleep2236646-21latency2
232744260,2sleep10-21swapper/11
226886260,1sleep10-21swapper/11
226473261,2sleep00-21swapper/00
217136260,1sleep10-21swapper/11
196259260,4sleep20-21swapper/22
193407261,2sleep00-21swapper/00
185487261,2sleep10-21swapper/11
180586260,1sleep20-21swapper/22
180202260,1sleep20-21swapper/22
253033250,1sleep20-21swapper/22
252943250,1sleep10-21swapper/11
243485250,1sleep00-21swapper/00
237975240,1sleep00-21swapper/00
227532240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional