You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 19:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Wed May 13, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
41510032143123,11sleep10-21swapper/11
4151050212955,69sleep30-21swapper/33
5735721140,3sleep1321rcuc/11
414661928558,14sleep20-21swapper/22
415100227956,18sleep00-21swapper/00
415146099286,14cyclictest0-21swapper/33
415146099271,23cyclictest0-21swapper/33
415146099271,14cyclictest0-21swapper/33
415146099241,3cyclictest0-21swapper/33
415146099241,3cyclictest0-21swapper/33
415146099241,2cyclictest0-21swapper/33
415146099241,2cyclictest0-21swapper/33
415146099240,2cyclictest0-21swapper/33
415146099240,2cyclictest0-21swapper/33
415146099231,3cyclictest0-21swapper/33
415146099231,15cyclictest0-21swapper/33
415146099230,2cyclictest0-21swapper/33
415146099230,2cyclictest0-21swapper/33
415146099224,2cyclictest0-21swapper/33
415146099224,2cyclictest0-21swapper/33
415146099221,2cyclictest0-21swapper/33
415146099221,2cyclictest0-21swapper/33
415146099220,2cyclictest0-21swapper/33
415146099220,20cyclictest0-21swapper/33
4255172211,10sleep1425524-21ssh1
415146099211,15cyclictest88804-21turbostat3
415146099210,2cyclictest0-21swapper/33
415146099210,2cyclictest0-21swapper/33
415146099210,2cyclictest0-21swapper/33
415146099201,16cyclictest0-21swapper/33
415146099201,15cyclictest66950irq/132-enp1s0-TxRx-33
415146099201,15cyclictest0-21swapper/33
415146099201,15cyclictest0-21swapper/33
415146099201,15cyclictest0-21swapper/33
415146099201,15cyclictest0-21swapper/33
415146099201,15cyclictest0-21swapper/33
415146099201,15cyclictest0-21swapper/33
415146099201,15cyclictest0-21swapper/33
415146099201,10cyclictest0-21swapper/33
415146099200,18cyclictest0-21swapper/33
415146099193,10cyclictest66950irq/132-enp1s0-TxRx-33
415146099191,7cyclictest72413-21turbostat3
4151460991913,4cyclictest0-21swapper/33
415146099191,15cyclictest0-21swapper/33
415146099191,15cyclictest0-21swapper/33
415146099191,14cyclictest0-21swapper/33
415146099191,14cyclictest0-21swapper/33
415146099191,13cyclictest0-21swapper/33
415146099190,11cyclictest0-21swapper/33
2133382191,7sleep088785-21diskmemload0
1972722191,13sleep088785-21diskmemload0
1048632191,9sleep2104874-21ssh2
415146099187,8cyclictest34808-21turbostat3
415146099183,12cyclictest66950irq/132-enp1s0-TxRx-33
415146099181,7cyclictest77770-21turbostat3
4151460991812,4cyclictest164331-21taskset3
415146099181,14cyclictest0-21swapper/33
415146099181,14cyclictest0-21swapper/33
415146099181,14cyclictest0-21swapper/33
415146099181,14cyclictest0-21swapper/33
415146099181,13cyclictest83337-21taskset3
960292171,10sleep20-21swapper/22
4196872171,3sleep00-21swapper/00
415146099178,2cyclictest0-21swapper/33
415146099171,13cyclictest0-21swapper/33
415146099170,9cyclictest0-21swapper/33
415146099170,14cyclictest0-21swapper/33
3421042170,1sleep10-21swapper/11
2241612172,11sleep1224201-21perf1
1209592171,6sleep088785-21diskmemload0
1209592171,6sleep088785-21diskmemload0
1201032171,3sleep20-21swapper/22
1201032171,3sleep20-21swapper/22
64952160,2sleep20-21swapper/22
4151460991611,3cyclictest0-21swapper/33
415146099161,13cyclictest0-21swapper/33
415146099161,12cyclictest50860-21sleep3
415146099161,12cyclictest0-21swapper/33
415146099161,12cyclictest0-21swapper/33
415146099160,13cyclictest0-21swapper/33
415146099160,13cyclictest0-21swapper/33
415146099160,13cyclictest0-21swapper/33
415146099160,12cyclictest0-21swapper/33
415146099160,12cyclictest0-21swapper/33
3747442165,2sleep2171rcu_preempt2
3667922161,4sleep288785-21diskmemload2
1551892161,2sleep20-21swapper/22
1403392161,7sleep2140558-21hddtemp_smartct2
1144622161,2sleep10-21swapper/11
1144622161,2sleep10-21swapper/11
1060982161,3sleep20-21swapper/22
4151460991512,2cyclictest0-21swapper/33
415146099150,13cyclictest0-21swapper/33
3935012151,2sleep10-21swapper/11
3679792151,5sleep10-21swapper/11
3597122151,2sleep00-21swapper/00
3311842151,9sleep0331244-21ntpq0
3236542150,1sleep20-21swapper/22
3176472151,8sleep072-21kcompactd00
3070072151,4sleep288785-21diskmemload2
2816192151,9sleep088785-21diskmemload0
2662562151,4sleep288785-21diskmemload2
2415572151,9sleep2241809-21df2
463402141,9sleep20-21swapper/22
4346062141,8sleep10-21swapper/11
41683702141,2sleep00-21swapper/00
4116672140,2sleep10-21swapper/11
3516272140,1sleep10-21swapper/11
3002262141,9sleep2300237-21ssh2
2744142141,8sleep188785-21diskmemload1
4452312131,8sleep0445250-21cron0
4413932132,6sleep288785-21diskmemload2
41858012132,6sleep04185819-21apt-get0
41567212131,6sleep14156769-21munin-run1
3658862130,4sleep00-21swapper/00
3586622131,8sleep1358676-21ssh1
3260892131,7sleep188785-21diskmemload1
3169402131,7sleep10-21swapper/11
3088972131,8sleep1308907-21ssh1
3000202131,2sleep10-21swapper/11
2747692131,8sleep2274815-21perf2
2325942131,8sleep1232608-21ssh1
2216292131,7sleep088785-21diskmemload0
1881942131,3sleep1188208-21ssh1
1293452131,8sleep088785-21diskmemload0
554672121,7sleep055521-21ntpq0
41684112121,5sleep14168460-21ntpq1
3909222121,6sleep2391123-21users2
3849362120,1sleep00-21swapper/00
3438472121,4sleep20-21swapper/22
3191562121,3sleep0319802-21tr0
301262121,7sleep029726-21/usr/sbin/munin0
2909632120,1sleep10-21swapper/11
2314222121,8sleep288785-21diskmemload2
2150562120,1sleep10-21swapper/11
2049882121,7sleep0205000-21ssh0
1981162121,7sleep2198127-21ssh2
1297862123,4sleep134-21ksoftirqd/11
1130332121,7sleep188785-21diskmemload1
783142112,4sleep078333-21apt-get0
643122111,6sleep062031-21/usr/sbin/munin0
506402110,4sleep10-21swapper/11
4450952110,3sleep10-21swapper/11
4347252111,3sleep00-21swapper/00
4279652111,5sleep288785-21diskmemload2
41851682111,4sleep24185068-31sort2
41752712111,5sleep14175484-21hddtemp_smartct1
41752712111,5sleep14175484-21hddtemp_smartct1
41557892111,7sleep04151628-21/usr/sbin/munin0
4086532111,5sleep288785-21diskmemload2
4040142111,7sleep1405608-21kthreadcore1
3924872111,5sleep088785-21diskmemload0
3504212111,4sleep088785-21diskmemload0
3353722111,3sleep20-21swapper/22
3070752111,4sleep088785-21diskmemload0
2929682111,5sleep0293179-21hddtemp_smartct0
2829272111,4sleep188785-21diskmemload1
2738732111,3sleep00-21swapper/00
2657752111,3sleep00-21swapper/00
2564852111,5sleep088785-21diskmemload0
1992492111,4sleep2199275-21apt-get2
1965702110,1sleep10-21swapper/11
1893732110,1sleep00-21swapper/00
1712662110,7sleep088785-21diskmemload0
1635642110,1sleep20-21swapper/22
1550752110,1sleep10-21swapper/11
1530092112,6sleep0153129-21timerandwakeup0
1402052111,6sleep1140331-21sh1
1315302112,5sleep2131660-21cut2
1287942111,3sleep20-21swapper/22
956152101,4sleep088785-21diskmemload0
836962101,4sleep183948-21df1
794842101,5sleep181209-21tr1
660262101,4sleep266083-21ntpq2
518582101,4sleep251929-21irqstats2
4352202101,3sleep20-21swapper/22
4265622101,4sleep088785-21diskmemload0
41915742101,4sleep04191690-21irqstats0
41747182101,5sleep24174724-21cron2
4160372101,4sleep2416100-21ntpq2
41516272101,4sleep24151646-21apt-get2
404282101,4sleep240447-21apt-get2
3834732100,2sleep10-21swapper/11
3736692101,4sleep088785-21diskmemload0
3581902101,3sleep288785-21diskmemload2
350822101,3sleep035101-21apt-get0
3416722101,2sleep00-21swapper/00
3312462101,4sleep10-21swapper/11
297272101,4sleep129746-21apt-get1
2908842101,3sleep20-21swapper/22
283672100,2sleep10-21swapper/11
2804722101,4sleep288785-21diskmemload2
2636672101,3sleep10-21swapper/11
2500002101,3sleep10-21swapper/11
2483412101,4sleep188785-21diskmemload1
2481372101,4sleep088785-21diskmemload0
2398042100,4sleep10-21swapper/11
2394702102,3sleep088785-21diskmemload0
238492101,4sleep223924-21mktemp2
2329442101,3sleep20-21swapper/22
2245412101,3sleep0224572-21ssh0
2206422101,3sleep20-21swapper/22
2059012101,5sleep1205913-21ssh1
1728452101,4sleep188785-21diskmemload1
1380902101,3sleep10-21swapper/11
1102762101,4sleep0110334-21ntpq0
1051322100,1sleep00-21swapper/00
1038602101,4sleep188785-21diskmemload1
94463291,3sleep188785-21diskmemload1
8406291,4sleep110051-21cut1
83247290,4sleep20-21swapper/22
76765291,4sleep176810-21ntpq1
67886291,4sleep268450-21cut2
45243290,1sleep10-21swapper/11
4192878291,4sleep2701-21kthreadcore2
4191691291,4sleep14192877-21grep1
4187296291,4sleep14189063-21cut1
4185046291,4sleep14185108-31find1
4174981291,3sleep24175000-21apt-get2
4174981291,3sleep24175000-21apt-get2
4169827291,4sleep04169940-21entropy0
403200291,2sleep00-21swapper/00
401911290,3sleep10-21swapper/11
400515291,5sleep2400530-21rm2
396431291,4sleep0398121-21cut0
374770290,1sleep10-21swapper/11
35708291,4sleep137365-21kthreadcore1
33833291,3sleep233878-21ntpq2
332532291,4sleep2332545-21ssh2
314700291,3sleep288785-21diskmemload2
288085291,4sleep0288133-21ntpq0
28462291,4sleep228512-21ntpq2
250028291,4sleep288785-21diskmemload2
23161291,5sleep023404-21tail0
213019291,4sleep2213034-21ssh2
186432291,4sleep2186498-21modprobe2
180335291,3sleep20-21swapper/22
179747291,2sleep10-21swapper/11
175075291,4sleep0176704-21grep0
171107291,2sleep20-21swapper/22
162796291,3sleep088785-21diskmemload0
162234291,2sleep10-21swapper/11
144942290,4sleep00-21swapper/00
131442291,2sleep00-21swapper/00
11852291,4sleep011887-21ld0
73171281,4sleep073365-21cat0
4582281,3sleep06310-21latency0
44524281,2sleep00-21swapper/00
4174367280,3sleep10-21swapper/11
4156770281,3sleep24157914-21grep2
23735280,4sleep10-21swapper/11
72928271,3sleep272957-21forks2
6658271,3sleep10-21swapper/11
56706270,2sleep00-21swapper/00
55528270,4sleep10-21swapper/11
4190722270,1sleep20-21swapper/22
4185141271,2sleep00-21swapper/00
4159586271,2sleep10-21swapper/11
4158983270,2sleep00-21swapper/00
384313270,3sleep2384327-21ssh2
24670271,3sleep00-21swapper/00
18308270,1sleep10-21swapper/11
13109270,2sleep20-21swapper/22
12725270,4sleep20-21swapper/22
83674260,1sleep00-21swapper/00
72239260,1sleep10-21swapper/11
72053260,4sleep00-21swapper/00
56983261,2sleep20-21swapper/22
4179821260,1sleep00-21swapper/00
4179821260,1sleep00-21swapper/00
13443260,1sleep00-21swapper/00
83949250,2sleep20-21swapper/22
62325250,2sleep10-21swapper/11
46049250,1sleep00-21swapper/00
37370250,1sleep20-21swapper/22
4164604240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional