You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 04:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Sun Mar 15, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3217327212568,53sleep30-21swapper/33
3217327212568,53sleep30-21swapper/33
321737428466,12sleep20-21swapper/22
321737428466,12sleep20-21swapper/22
321670628360,13sleep00-21swapper/00
321670628360,13sleep00-21swapper/00
321719828152,10sleep10-21swapper/11
321719828152,10sleep10-21swapper/11
321754999341,19cyclictest0-21swapper/33
321754999281,12cyclictest0-21swapper/33
321754999243,8cyclictest0-21swapper/33
35490642231,12sleep03349161-21diskmemload0
321754999230,12cyclictest0-21swapper/33
321754999221,18cyclictest3506645-21taskset3
321754999221,16cyclictest0-21swapper/33
321754999221,15cyclictest0-21swapper/33
321754999216,11cyclictest3357921-21turbostat3
321754999212,14cyclictest33377902chrt3
3217549992118,1cyclictest0-21swapper/33
3217549992118,1cyclictest0-21swapper/33
321754999211,2cyclictest36059512sleep33
321754999211,16cyclictest0-21swapper/33
321754999211,15cyclictest0-21swapper/33
321754999203,12cyclictest3614205-21taskset3
321754999201,6cyclictest0-21swapper/33
321754999201,15cyclictest34362712chrt3
321754999201,15cyclictest3374245-21taskset3
321754999199,7cyclictest3283760-21chrt3
321754999191,15cyclictest0-21swapper/33
321754999191,15cyclictest0-21swapper/33
321754999191,14cyclictest0-21swapper/33
321754999191,14cyclictest0-21swapper/33
321754999191,14cyclictest0-21swapper/33
321754999191,14cyclictest0-21swapper/33
321754999191,14cyclictest0-21swapper/33
321754999191,14cyclictest0-21swapper/33
321754999191,14cyclictest0-21swapper/33
321754999190,15cyclictest0-21swapper/33
321754999190,15cyclictest0-21swapper/33
321754999190,14cyclictest37170512sleep33
36310522181,3sleep13349161-21diskmemload1
34013492181,5sleep23349161-21diskmemload2
3217549991812,4cyclictest0-21swapper/33
321754999181,14cyclictest0-21swapper/33
321754999181,14cyclictest0-21swapper/33
321754999181,14cyclictest0-21swapper/33
321754999181,14cyclictest0-21swapper/33
321754999181,14cyclictest0-21swapper/33
321754999181,14cyclictest0-21swapper/33
321754999181,14cyclictest0-21swapper/33
321754999181,14cyclictest0-21swapper/33
321754999180,14cyclictest0-21swapper/33
321754999180,14cyclictest0-21swapper/33
35411472171,5sleep13349161-21diskmemload1
34177422171,2sleep20-21swapper/22
33916972171,3sleep03349161-21diskmemload0
33572832171,3sleep20-21swapper/22
321754999171,3cyclictest0-21swapper/33
321754999171,14cyclictest0-21swapper/33
321754999171,13cyclictest0-21swapper/33
321754999171,13cyclictest0-21swapper/33
321754999171,13cyclictest0-21swapper/33
321754999171,13cyclictest0-21swapper/33
321754999171,13cyclictest0-21swapper/33
321754999171,12cyclictest0-21swapper/33
37156102161,2sleep20-21swapper/22
36717892161,10sleep03666850-21/usr/sbin/munin0
35605552161,2sleep00-21swapper/00
35128462161,4sleep13512907-21ntpq1
34988042161,4sleep13349161-21diskmemload1
33114522161,4sleep13311471-21apt-get1
3217549991613,1cyclictest0-21swapper/33
321754999161,12cyclictest0-21swapper/33
321754999161,12cyclictest0-21swapper/33
321754999160,14cyclictest0-21swapper/33
321754999160,14cyclictest0-21swapper/33
321754999160,13cyclictest0-21swapper/33
321754999160,13cyclictest0-21swapper/33
36810672151,4sleep03681081-21ssh0
36807772151,4sleep13349161-21diskmemload1
36196462151,3sleep10-21swapper/11
35777772151,3sleep13349161-21diskmemload1
35682672150,1sleep10-21swapper/11
35575562150,3sleep10-21swapper/11
35167212150,2sleep20-21swapper/22
34702162151,2sleep20-21swapper/22
34551642151,4sleep13349161-21diskmemload1
34421722151,9sleep03349161-21diskmemload0
34080622151,10sleep03408073-21sh0
33760662151,8sleep20-21swapper/22
321754999157,3cyclictest32250652sleep33
321754999151,4cyclictest32739582sleep33
321754999150,12cyclictest0-21swapper/33
321754999150,12cyclictest0-21swapper/33
321754999150,12cyclictest0-21swapper/33
36055142141,9sleep13605733-21ssh1
34895162140,3sleep23349161-21diskmemload2
34376432141,4sleep23437654-21ssh2
32588762146,4sleep13260625-21tr1
321754999141,11cyclictest0-21swapper/33
321754999140,11cyclictest0-21swapper/33
321754999140,11cyclictest0-21swapper/33
35882122131,5sleep03588427-21hddtemp_smartct0
35864312131,6sleep23549289-21kworker/2:0+events2
35759252131,8sleep03575936-21ssh0
35589942131,7sleep23349161-21diskmemload2
35331912131,8sleep03533204-21rm0
35143582131,8sleep03514370-21ssh0
34986922131,1sleep00-21swapper/00
34875632131,8sleep13487574-21ssh1
34604512131,8sleep03460465-21ssh0
34464672131,6sleep23446491-21apt-get2
34269072131,7sleep03349161-21diskmemload0
34164952131,9sleep00-21swapper/00
33823852130,2sleep00-21swapper/00
33748882133,2sleep2171rcu_preempt2
33646862131,4sleep23364697-21ssh2
32522712131,6sleep13252526-21df1
321754999137,3cyclictest33053232sleep33
321754999131,1cyclictest0-21swapper/33
37182942121,5sleep13349161-21diskmemload1
36931822121,7sleep2802-21snmpd2
36724702121,6sleep13349161-21diskmemload1
36711642121,7sleep23671239-21cut2
36664032121,7sleep03666419-21ssh0
36590822123,5sleep23660743-21cut2
36549252121,5sleep20-21swapper/22
36370692121,7sleep03637079-21ssh0
36370692121,7sleep03637079-21ssh0
36225272120,5sleep00-21swapper/00
36141722121,4sleep20-21swapper/22
35846392121,7sleep13349161-21diskmemload1
35666902121,6sleep03349161-21diskmemload0
35352482121,3sleep03535469-21ssh0
35243792121,5sleep00-21swapper/00
34363542121,6sleep13349161-21diskmemload1
34338842121,7sleep23433894-21ssh2
34186902121,6sleep13349161-21diskmemload1
33901682121,6sleep13349161-21diskmemload1
33488242120,1sleep10-21swapper/11
33221732121,5sleep13322221-21munin-run1
33000562120,1sleep10-21swapper/11
37095112111,4sleep13349161-21diskmemload1
37060202111,5sleep03706081-21ntpq0
36900612111,5sleep13349161-21diskmemload1
36462872110,2sleep10-21swapper/11
36368862111,4sleep20-21swapper/22
36368862111,4sleep20-21swapper/22
36275262111,7sleep03622949-21/usr/sbin/munin0
36108572110,4sleep20-21swapper/22
36043052110,1sleep00-21swapper/00
35966942111,7sleep10-21swapper/11
35955792111,4sleep23349161-21diskmemload2
35320452111,3sleep10-21swapper/11
35244072111,3sleep20-21swapper/22
35231142111,4sleep13349161-21diskmemload1
35076532111,3sleep20-21swapper/22
35045992111,6sleep03504779-21ssh0
34691832112,4sleep1331ktimers/11
34453452111,4sleep13349161-21diskmemload1
33934242111,5sleep13393456-21ssh1
33713282111,5sleep03371398-21ntp_states0
33495122111,4sleep13349161-21diskmemload1
33387562111,7sleep23339017-21/usr/sbin/munin2
33252602111,5sleep13326336-21memory1
32679842111,5sleep23268032-21munin-run2
32357272111,5sleep03235746-21apt-get0
32301182111,5sleep13230142-21munin-run1
32221362111,6sleep13222408-21unixbench_singl1
32181952111,5sleep23218266-21irqstats2
37073032101,4sleep23349161-21diskmemload2
37004562101,3sleep00-21swapper/00
37002252101,3sleep13349161-21diskmemload1
36922852101,4sleep03349161-21diskmemload0
36909922101,3sleep20-21swapper/22
36807342100,1sleep20-21swapper/22
36633802101,4sleep13349161-21diskmemload1
36575912101,2sleep00-21swapper/00
36553172101,2sleep10-21swapper/11
36480862101,3sleep23349161-21diskmemload2
36326212101,5sleep13349161-21diskmemload1
36326212101,5sleep13349161-21diskmemload1
36305582101,5sleep23630571-21ssh2
36095822101,4sleep03609643-21ntpq0
35967622101,4sleep23596986-21diskstats2
35834132101,5sleep03583477-21rm0
35613322101,4sleep23561356-21apt-get2
35494832101,4sleep13349161-21diskmemload1
35486702101,3sleep23349161-21diskmemload2
35324132100,1sleep20-21swapper/22
34998312101,4sleep13499838-21apt-get1
34952772101,5sleep23495332-21ntpq2
34791892100,1sleep00-21swapper/00
34781742101,3sleep13349161-21diskmemload1
34531412101,3sleep13349161-21diskmemload1
34517822101,5sleep03338049-21kworker/u16:3+rpciod0
34262162101,3sleep20-21swapper/22
34092982101,3sleep13349161-21diskmemload1
33813302100,3sleep10-21swapper/11
33717012101,4sleep13371957-21users1
33650182101,2sleep00-21swapper/00
33622932101,4sleep13362441-21/usr/sbin/munin1
33555982100,1sleep00-21swapper/00
33380102100,3sleep13338002-21fwupd1
33235372101,5sleep03325259-21cut0
32939552101,4sleep03294012-21ntpq0
32848652101,5sleep13285069-21if_enp1s01
32629182101,4sleep13262937-21apt-get1
32578722101,4sleep23257975-21perl2
3715160291,4sleep03715319-21ls0
3646330292,3sleep00-21swapper/00
3596071290,3sleep13349161-21diskmemload1
3575080291,4sleep23575094-21ssh2
3540843291,2sleep20-21swapper/22
3482028291,4sleep03482245-21diskstats0
3472838291,2sleep20-21swapper/22
3470449291,4sleep03470460-21ssh0
3455289291,4sleep23455550-21cat2
3435075291,2sleep00-21swapper/00
3426150290,1sleep10-21swapper/11
3402558291,4sleep23402782-21/usr/sbin/munin2
3398409290,5sleep03398421-21ssh0
3384587291,2sleep20-21swapper/22
3344925291,4sleep03346609-21grep0
3312033291,4sleep23313498-21tr2
3309966291,4sleep13310025-21ld1
3304569291,4sleep03304617-21ntpq0
3273874291,4sleep23273956-21fschecks_time2
3257288290,2sleep20-21swapper/22
3251697291,4sleep23252156-21find2
3246612290,4sleep13246631-21apt-get1
3245388291,4sleep13245605-21/usr/sbin/munin1
3229835290,1sleep00-21swapper/00
3337786280,2sleep272-21kcompactd02
3333360281,4sleep00-21swapper/00
3327758280,3sleep23327777-21apt-get2
3319065281,3sleep23320905-21kthreadcore2
3305849281,3sleep23305854-21sort2
3301873280,5sleep13303628-21cut1
3289842280,3sleep23289861-21apt-get2
3278919281,2sleep20-21swapper/22
3270159281,3sleep20-21swapper/22
3252249281,2sleep00-21swapper/00
3246070280,1sleep20-21swapper/22
3245108281,3sleep03245162-21ntpq0
3240082281,4sleep23240342-21sed2
3219565281,4sleep03221377-21tr0
3349001270,1sleep20-21swapper/22
3343581270,1sleep00-21swapper/00
3339299271,2sleep13340235-21cut1
3331575271,2sleep13331748-21strings1
3317089271,2sleep00-21swapper/00
3307397271,3sleep00-21swapper/00
3294598270,1sleep10-21swapper/11
3288855271,3sleep20-21swapper/22
3283919270,1sleep00-21swapper/00
3274825271,3sleep1331ktimers/11
3274148270,3sleep03274197-21kthreadcore0
3272999270,1sleep00-21swapper/00
3268814271,2sleep10-21swapper/11
3251476270,1sleep00-21swapper/00
3239806271,3sleep13239861-21ntpq1
3232118271,3sleep10-21swapper/11
3230968271,2sleep03232116-21kthreadcore0
3327373260,1sleep20-21swapper/22
3315307260,2sleep03315480-21strings0
3300017260,1sleep20-21swapper/22
3299911260,1sleep00-21swapper/00
3284801261,2sleep00-21swapper/00
3283114261,1sleep10-21swapper/11
3257592260,1sleep00-21swapper/00
3235284260,4sleep20-21swapper/22
3327936250,1sleep00-21swapper/00
3305240250,1sleep20-21swapper/22
3262770250,1sleep00-21swapper/00
3225546250,2sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional