You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-06 - 05:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Sat Jun 06, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
33275262147107,36sleep30-21swapper/33
349232321240,2sleep00-21swapper/00
377125821220,2sleep00-21swapper/00
339292621050,4sleep0221rcuc/00
332742528466,12sleep20-21swapper/22
332760328356,7sleep00-21swapper/00
332768827963,10sleep10-21swapper/11
332786399251,18cyclictest37467372sleep33
332786399241,13cyclictest0-21swapper/33
3327863992218,2cyclictest0-21swapper/33
332786399221,18cyclictest0-21swapper/33
332786399211,16cyclictest0-21swapper/33
3327863992017,1cyclictest0-21swapper/33
3327863992012,4cyclictest3808997-21turbostat3
332786399201,15cyclictest66950irq/132-enp1s0-TxRx-33
332786399201,15cyclictest0-21swapper/33
3327863992010,6cyclictest3774061-21turbostat3
37222872192,11sleep23722398-21irqstats2
35898592191,6sleep13589899-21munin-run1
35898592191,6sleep13589899-21munin-run1
33366332191,8sleep13338368-21cut1
3327863991916,1cyclictest0-21swapper/33
332786399191,15cyclictest0-21swapper/33
332786399191,15cyclictest0-21swapper/33
332786399191,14cyclictest0-21swapper/33
332786399191,14cyclictest0-21swapper/33
332786399191,14cyclictest0-21swapper/33
332786399191,14cyclictest0-21swapper/33
332786399190,17cyclictest0-21swapper/33
37195682181,3sleep00-21swapper/00
35184102180,2sleep20-21swapper/22
332786399182,10cyclictest3448043-21taskset3
3327863991816,1cyclictest0-21swapper/33
3327863991816,1cyclictest0-21swapper/33
332786399181,5cyclictest0-21swapper/33
3327863991812,3cyclictest0-21swapper/33
332786399181,14cyclictest0-21swapper/33
332786399181,14cyclictest0-21swapper/33
332786399181,14cyclictest0-21swapper/33
332786399181,14cyclictest0-21swapper/33
332786399181,14cyclictest0-21swapper/33
332786399181,14cyclictest0-21swapper/33
332786399181,14cyclictest0-21swapper/33
332786399181,14cyclictest0-21swapper/33
332786399181,13cyclictest3809326-21chrt3
332786399181,13cyclictest0-21swapper/33
332786399181,13cyclictest0-21swapper/33
332786399181,13cyclictest0-21swapper/33
332786399180,15cyclictest0-21swapper/33
332786399180,15cyclictest0-21swapper/33
332786399180,14cyclictest0-21swapper/33
332786399180,14cyclictest0-21swapper/33
36981902171,5sleep13699061-21memory1
35944132171,11sleep03594463-21ntpq0
35707192176,7sleep13327857-21cyclictest1
34813422171,3sleep10-21swapper/11
34675652171,3sleep00-21swapper/00
34588232172,4sleep13458837-21apt-get1
332786399171,3cyclictest0-21swapper/33
3327863991713,2cyclictest0-21swapper/33
332786399171,14cyclictest0-21swapper/33
332786399171,13cyclictest0-21swapper/33
332786399171,13cyclictest0-21swapper/33
332786399171,13cyclictest0-21swapper/33
332786399171,13cyclictest0-21swapper/33
332786399171,13cyclictest0-21swapper/33
332786399171,12cyclictest0-21swapper/33
332786399170,14cyclictest0-21swapper/33
332786399170,14cyclictest0-21swapper/33
332786399170,14cyclictest0-21swapper/33
332786399170,14cyclictest0-21swapper/33
36904542161,4sleep13690512-21ntpq1
36679462161,5sleep03458448-21diskmemload0
34587722163,2sleep2171rcu_preempt2
33670832160,11sleep20-21swapper/22
332786399166,6cyclictest3384161-21taskset3
332786399160,13cyclictest0-21swapper/33
332786399160,13cyclictest0-21swapper/33
332786399160,13cyclictest0-21swapper/33
38263232151,3sleep03458448-21diskmemload0
37622472151,2sleep00-21swapper/00
36134662151,2sleep20-21swapper/22
36131402151,4sleep13458448-21diskmemload1
35533362150,1sleep10-21swapper/11
35530482151,4sleep23553062-21ssh2
35172742151,3sleep03458448-21diskmemload0
34199332151,8sleep03419981-21ntpq0
332786399150,13cyclictest0-21swapper/33
332786399150,12cyclictest0-21swapper/33
332786399150,12cyclictest0-21swapper/33
332786399150,12cyclictest0-21swapper/33
38142792140,2sleep10-21swapper/11
37192432141,8sleep23458448-21diskmemload2
36501342141,10sleep23458448-21diskmemload2
35624212141,2sleep10-21swapper/11
35182482141,2sleep10-21swapper/11
332786399148,3cyclictest34321772sleep33
332786399143,7cyclictest0-21swapper/33
332786399141,1cyclictest0-21swapper/33
3327863991411,1cyclictest0-21swapper/33
332786399140,12cyclictest0-21swapper/33
332786399140,12cyclictest0-21swapper/33
37720082131,2sleep10-21swapper/11
37533642131,6sleep00-21swapper/00
37011772130,7sleep00-21swapper/00
36591142131,1sleep10-21swapper/11
35803162132,2sleep1171rcu_preempt1
35690192130,2sleep20-21swapper/22
35433112131,3sleep13543322-21ssh1
34815852131,8sleep03458448-21diskmemload0
34746942131,8sleep13458448-21diskmemload1
34574242130,4sleep23457477-21ld2
332786399131,9cyclictest3383571-21turbostat3
37873632121,6sleep13787418-21ntpq1
37736302121,6sleep23458448-21diskmemload2
37575192121,7sleep23757682-21cut2
37446112121,8sleep13744625-21ssh1
37308242121,5sleep03730853-21fschecks_count0
37121412121,6sleep03458448-21diskmemload0
36753662121,7sleep23675377-21ssh2
36682122124,4sleep23668225-21sh2
36557262121,7sleep23655862-21ssh2
36317422121,6sleep23458448-21diskmemload2
36067752120,8sleep00-21swapper/00
36035482120,2sleep23458448-21diskmemload2
35888352121,7sleep03588849-21ssh0
35888352121,7sleep03588849-21ssh0
35632702121,2sleep03563277-21ssh0
35626972121,6sleep23458448-21diskmemload2
35440702121,7sleep23544083-21ssh2
35256582121,2sleep03525672-21ssh0
35008272121,5sleep00-21swapper/00
34856922121,7sleep23486493-21cat2
38136192111,5sleep23813684-21ntpq2
37370032111,5sleep23458448-21diskmemload2
37301502111,6sleep03730271-21latency_hist0
37040412111,3sleep20-21swapper/22
36859012111,5sleep13685982-21munin-run1
36766232111,5sleep03458448-21diskmemload0
36654822110,2sleep10-21swapper/11
36498212111,6sleep13458448-21diskmemload1
36406872111,2sleep10-21swapper/11
36381402111,5sleep03638196-21ntpq0
36307072111,3sleep00-21swapper/00
35519032111,7sleep03551917-21ssh0
35374392111,5sleep03537453-21ssh0
34209822111,6sleep13421061-21latency_hist1
34108472111,5sleep03411057-21hddtemp_smartct0
34093982111,6sleep13409483-21ps1
33950362111,6sleep13395165-21cut1
33931562111,5sleep13393257-21ntpq1
33725482110,1sleep00-21swapper/00
38238502100,1sleep10-21swapper/11
38066122101,4sleep23458448-21diskmemload2
38057372101,3sleep10-21swapper/11
37970512101,2sleep20-21swapper/22
37960142101,4sleep13796076-21ntpq1
37837182101,5sleep03784216-21ssh0
37830952100,1sleep20-21swapper/22
37814342101,5sleep23781447-21sh2
37812732101,2sleep10-21swapper/11
37538682100,1sleep10-21swapper/11
37503292101,6sleep23752045-21kthreadcore2
37425182101,4sleep03743584-21memory0
37203922100,1sleep10-21swapper/11
37094362100,4sleep10-21swapper/11
37006512101,4sleep23458448-21diskmemload2
36940912100,5sleep03694102-21ssh0
36764232101,3sleep10-21swapper/11
36556012101,3sleep00-21swapper/00
36480012101,2sleep00-21swapper/00
36338122101,4sleep23458448-21diskmemload2
36216692100,1sleep10-21swapper/11
36210322101,5sleep03621291-21unixbench_multi0
36139352101,4sleep03458448-21diskmemload0
36061362101,4sleep13458448-21diskmemload1
35902802101,4sleep13590545-21df_inode1
35872492101,2sleep20-21swapper/22
35872492101,2sleep20-21swapper/22
35800482101,3sleep00-21swapper/00
35780222101,3sleep20-21swapper/22
35695422101,4sleep03458448-21diskmemload0
35373112101,4sleep23458448-21diskmemload2
35203932100,2sleep10-21swapper/11
35102472101,2sleep00-21swapper/00
35084882101,3sleep172-21kcompactd01
34922912101,4sleep13458448-21diskmemload1
34809282100,6sleep23476374-21/usr/sbin/munin2
34725872100,1sleep20-21swapper/22
34649342101,2sleep00-21swapper/00
34519422101,4sleep13452082-21ntp_192.168.1141
34507232101,4sleep03451941-21memory0
34440402103,3sleep23445784-21grep2
34305892100,2sleep20-21swapper/22
34212422101,4sleep03421261-21apt-get0
34057102101,4sleep23405781-21irqstats2
33838242101,4sleep03383843-21apt-get0
33784802101,5sleep03378498-21apt-get0
33718442101,5sleep23371892-21ntpq2
33408102101,4sleep23340829-21apt-get2
3823161291,3sleep23458448-21diskmemload2
3809299290,3sleep00-21swapper/00
3805202291,4sleep03805471-21sed0
3798581291,3sleep00-21swapper/00
3781664290,2sleep00-21swapper/00
3744729291,4sleep23458448-21diskmemload2
3730506291,3sleep10-21swapper/11
3727846291,3sleep13458448-21diskmemload1
3690514291,4sleep23690569-21/usr/sbin/munin2
3683325291,4sleep23683339-21ssh2
3630569291,3sleep10-21swapper/11
3622338291,4sleep23622349-21ssh2
3594811291,4sleep23590247-21/usr/sbin/munin2
3544000291,3sleep03458448-21diskmemload0
3535031291,2sleep10-21swapper/11
3524233291,5sleep23524282-21ntpq2
3507834291,3sleep23458448-21diskmemload2
3498919291,4sleep23498932-21sh2
3498754291,4sleep13498768-21ssh1
3448021291,2sleep20-21swapper/22
3442658291,2sleep10-21swapper/11
3437055291,4sleep23437081-21munin-run2
3394550291,4sleep23394746-21expr2
3378501291,4sleep13378478-21/usr/sbin/munin1
3344982291,4sleep13345106-21grep1
3342814291,4sleep03344639-21cut0
3329585291,4sleep23331369-21cut2
3762260281,2sleep10-21swapper/11
3677909281,3sleep03458448-21diskmemload0
3453647280,1sleep00-21swapper/00
3436006281,3sleep13436063-21ntpq1
3430716280,3sleep03430764-21ntpq0
3414880280,1sleep10-21swapper/11
3404931280,1sleep00-21swapper/00
3404834280,2sleep00-21swapper/00
3394826280,1sleep00-21swapper/00
3387922281,3sleep13387961-21/usr/sbin/munin1
3377199281,3sleep13377232-21ntpq1
3367777281,3sleep13367274-21apt-get1
3352359280,4sleep23353964-21kthreadcore2
3350457280,1sleep20-21swapper/22
3453352270,2sleep10-21swapper/11
3438203271,3sleep23439898-21kthreadcore2
3437864271,3sleep13438202-21cat1
3417873271,3sleep20-21swapper/22
3404598270,1sleep20-21swapper/22
3384101271,3sleep20-21swapper/22
3362176271,2sleep13362195-21apt-get1
3361801271,3sleep1802-21snmpd1
3357556271,3sleep00-21swapper/00
3351472270,2sleep10-21swapper/11
3348362270,4sleep00-21swapper/00
3346423270,3sleep13346635-21smartctl1
3335423270,2sleep00-21swapper/00
3328608271,3sleep13329584-21kthreadcore1
3328032270,1sleep00-21swapper/00
3446580260,2sleep00-21swapper/00
3432247260,3sleep00-21swapper/00
3426051260,1sleep10-21swapper/11
3425629261,2sleep20-21swapper/22
3415660261,2sleep20-21swapper/22
3404870260,1sleep10-21swapper/11
3389734261,2sleep20-21swapper/22
3377962260,1sleep00-21swapper/00
3374063261,2sleep20-21swapper/22
3357110260,2sleep20-21swapper/22
3356486260,1sleep00-21swapper/00
3378762250,1sleep20-21swapper/22
3362466250,1sleep00-21swapper/00
3338370250,1sleep20-21swapper/22
3437314240,1sleep00-21swapper/00
3426398240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional