You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-18 - 20:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Wed Mar 18, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1191233212751,70sleep30-21swapper/33
119135428658,10sleep10-21swapper/11
119132128656,11sleep20-21swapper/22
119081928268,6sleep00-21swapper/00
14690772320,3sleep10-21swapper/11
15556892251,19sleep01321873-21diskmemload0
119163599221,7cyclictest1557761-21taskset3
119163599211,16cyclictest1426781-21taskset3
1191635992111,6cyclictest1339764-21turbostat3
13275112201,3sleep00-21swapper/00
1191635992017,1cyclictest0-21swapper/33
1191635992013,4cyclictest1664365-21taskset3
119163599201,17cyclictest0-21swapper/33
119163599201,16cyclictest0-21swapper/33
119163599201,15cyclictest66950irq/132-enp1s0-TxRx-33
119163599201,15cyclictest66950irq/132-enp1s0-TxRx-33
119163599201,15cyclictest1637719-21taskset3
119163599201,15cyclictest0-21swapper/33
1191635992010,6cyclictest16459562sleep33
119163599191,14cyclictest0-21swapper/33
119163599191,14cyclictest0-21swapper/33
119163599191,14cyclictest0-21swapper/33
119163599191,14cyclictest0-21swapper/33
119163599191,14cyclictest0-21swapper/33
119163599191,14cyclictest0-21swapper/33
1191635991910,6cyclictest66950irq/132-enp1s0-TxRx-33
119163599190,2cyclictest0-21swapper/33
119163599190,2cyclictest0-21swapper/33
119163599190,15cyclictest66950irq/132-enp1s0-TxRx-33
119163599190,15cyclictest0-21swapper/33
15484922181,3sleep21548507-21ssh2
13244482181,12sleep21326205-21tr2
119163599181,15cyclictest0-21swapper/33
119163599181,14cyclictest0-21swapper/33
119163599181,14cyclictest0-21swapper/33
119163599181,14cyclictest0-21swapper/33
119163599181,14cyclictest0-21swapper/33
119163599181,14cyclictest0-21swapper/33
119163599181,13cyclictest0-21swapper/33
119163599181,13cyclictest0-21swapper/33
119163599181,13cyclictest0-21swapper/33
119163599180,14cyclictest0-21swapper/33
119163599180,14cyclictest0-21swapper/33
16205402171,2sleep00-21swapper/00
14088392171,2sleep00-21swapper/00
13562402171,1sleep10-21swapper/11
13156432171,4sleep11315696-21ntp_states1
119163599178,6cyclictest1241475-21taskset3
1191635991715,1cyclictest0-21swapper/33
119163599171,4cyclictest0-21swapper/33
1191635991713,2cyclictest0-21swapper/33
1191635991712,3cyclictest0-21swapper/33
1191635991711,3cyclictest0-21swapper/33
119163599171,13cyclictest0-21swapper/33
119163599170,14cyclictest0-21swapper/33
119163599170,14cyclictest0-21swapper/33
119163599170,13cyclictest0-21swapper/33
16530382161,3sleep00-21swapper/00
16526652161,3sleep20-21swapper/22
15753362161,2sleep00-21swapper/00
15636162161,10sleep21563696-21memory2
15314802167,4sleep2431ktimers/22
15221162161,4sleep01321873-21diskmemload0
14513202161,4sleep11321873-21diskmemload1
14269672160,1sleep20-21swapper/22
13810702161,2sleep20-21swapper/22
119163599161,3cyclictest0-21swapper/33
1191635991613,1cyclictest0-21swapper/33
119163599161,12cyclictest0-21swapper/33
119163599160,14cyclictest0-21swapper/33
119163599160,13cyclictest0-21swapper/33
119163599160,13cyclictest0-21swapper/33
119163599160,12cyclictest0-21swapper/33
16459852151,3sleep11321873-21diskmemload1
16263612158,1sleep20-21swapper/22
16033102150,2sleep10-21swapper/11
15236152150,6sleep21523629-21ssh2
15120212151,3sleep20-21swapper/22
14755392151,9sleep11475616-21memory1
12569332151,3sleep21256984-21ntpq2
12569332151,3sleep21256984-21ntpq2
119163599159,4cyclictest1252596-21turbostat3
119163599157,4cyclictest12942412sleep33
1191635991512,2cyclictest0-21swapper/33
119163599150,12cyclictest0-21swapper/33
119163599150,12cyclictest0-21swapper/33
119163599150,12cyclictest0-21swapper/33
119163599150,12cyclictest0-21swapper/33
119163599150,12cyclictest0-21swapper/33
119163599150,12cyclictest0-21swapper/33
15127692141,9sleep11512780-21ssh1
14973002140,3sleep20-21swapper/22
14701062141,6sleep21470118-21rm2
14285922141,9sleep21430183-21cut2
13922262141,2sleep2171rcu_preempt2
13483182141,2sleep10-21swapper/11
119163599149,2cyclictest0-21swapper/33
119163599140,2cyclictest0-21swapper/33
119163599140,12cyclictest0-21swapper/33
119163599140,11cyclictest0-21swapper/33
119163599140,11cyclictest0-21swapper/33
16803022131,8sleep11680313-21ssh1
16714562131,4sleep11671470-21ssh1
16384012130,6sleep10-21swapper/11
16382092131,2sleep01321873-21diskmemload0
15758192131,3sleep21575830-21ssh2
15497632130,3sleep11321873-21diskmemload1
15044102131,8sleep21504424-21ssh2
14592732131,3sleep01459287-21ssh0
14433332131,8sleep11443346-21sh1
14184772130,2sleep20-21swapper/22
13805302130,2sleep0171rcu_preempt0
13746022131,7sleep11321873-21diskmemload1
13644532131,7sleep11321873-21diskmemload1
13627492131,5sleep00-21swapper/00
13396152131,2sleep11321873-21diskmemload1
13224582131,4sleep11322468-21ssh1
13208332131,7sleep21320879-21ntpq2
13013662131,7sleep21301957-21cut2
12629332130,1sleep10-21swapper/11
12253572131,6sleep11225305-31find1
11981542130,2sleep00-21swapper/00
119163599138,3cyclictest1279491-21chrt3
119163599130,2cyclictest0-21swapper/33
16458662121,7sleep01321873-21diskmemload0
16002692120,2sleep20-21swapper/22
15935922121,7sleep11593604-21sh1
15285462120,7sleep11528614-21ntpq1
15284832122,5sleep01528544-21ntpq0
14885032121,7sleep21488510-21ssh2
14876772121,6sleep11321873-21diskmemload1
14628882121,7sleep01463132-21cstates0
14612702121,5sleep20-21swapper/22
14592882121,7sleep11459300-21sh1
14354372125,2sleep10-21swapper/11
14263932121,7sleep11426403-21ssh1
14172902121,7sleep11417300-21ssh1
13736682121,3sleep00-21swapper/00
13552702121,7sleep21355281-21ssh2
13402252121,5sleep21340455-21diskstats2
13363982121,3sleep20-21swapper/22
13062252121,5sleep21306428-21diskstats2
12528432122,5sleep11252862-21apt-get1
12528432122,5sleep11252862-21apt-get1
12371792121,7sleep21238701-21cut2
12360962121,4sleep272-21kcompactd02
12156902121,7sleep21216503-21cut2
16884272111,3sleep00-21swapper/00
16869342111,5sleep11686990-21ntpq1
16802472111,3sleep00-21swapper/00
16719752111,2sleep20-21swapper/22
16564262111,5sleep01656453-21apt-get0
16388412111,5sleep21638872-21apt-get2
16281572111,4sleep01321873-21diskmemload0
16183042111,5sleep11321873-21diskmemload1
16125952111,5sleep21612857-21df2
16106652111,5sleep01321873-21diskmemload0
15833952111,4sleep00-21swapper/00
15550742111,5sleep11555130-21ntpq1
15396872111,3sleep10-21swapper/11
15329672110,7sleep01321873-21diskmemload0
15161062111,6sleep11516325-21grep1
15063192111,6sleep01506523-21cut0
14947642111,7sleep11321873-21diskmemload1
14798942110,5sleep20-21swapper/22
14781842115,1sleep00-21swapper/00
14535882110,7sleep21453599-21ssh2
14515562111,5sleep01321873-21diskmemload0
14009082111,2sleep20-21swapper/22
13999702111,5sleep01321873-21diskmemload0
13840852111,4sleep01384348-21df0
13661072111,6sleep21366276-21cut2
13567252111,4sleep00-21swapper/00
13217072111,7sleep01321709-21sshd0
12675352111,5sleep11267582-21ntpq1
12475492110,2sleep10-21swapper/11
12218552111,6sleep01223621-21cut0
16895122101,2sleep20-21swapper/22
16781152101,4sleep21678182-21ntpq2
16713892101,2sleep00-21swapper/00
16636412101,6sleep21663653-21rm2
16633702101,3sleep11321873-21diskmemload1
16354652101,3sleep20-21swapper/22
16255512101,4sleep11625601-21ntpq1
15943412101,6sleep01321873-21diskmemload0
15859212101,2sleep20-21swapper/22
15836662101,3sleep10-21swapper/11
15834762100,5sleep21321873-21diskmemload2
15653532102,3sleep01321873-21diskmemload0
15598882101,6sleep11559983-21ssh1
15569492101,5sleep20-21swapper/22
15478032100,1sleep00-21swapper/00
15129632101,4sleep01512973-21ssh0
15051622100,1sleep10-21swapper/11
14932682101,5sleep01493359-21ntpq0
14332212101,2sleep00-21swapper/00
14193482101,5sleep01419574-21idleruntime0
14147902101,2sleep00-21swapper/00
14084942101,4sleep11321873-21diskmemload1
13839262101,4sleep11321873-21diskmemload1
13811102101,5sleep11321873-21diskmemload1
13721142101,3sleep20-21swapper/22
13366322101,3sleep00-21swapper/00
12903452101,5sleep01290452-21/usr/sbin/munin0
12800642101,5sleep11280097-21kthreadcore1
12746632101,4sleep21274734-21irqstats2
12515802101,4sleep01251623-21ntpq0
12264462101,4sleep21226517-21irqstats2
12139252101,4sleep01213978-21ntp_states0
1652227293,3sleep1331ktimers/11
1608202291,3sleep11321873-21diskmemload1
1607980291,4sleep21608041-21ntpq2
1602689291,3sleep00-21swapper/00
1572983291,3sleep11321873-21diskmemload1
1532939291,3sleep21532953-21ssh2
1486998291,3sleep01321873-21diskmemload0
1441678291,2sleep20-21swapper/22
1408100291,4sleep21408112-21rm2
1397859291,3sleep11321873-21diskmemload1
1340678291,3sleep01321873-21diskmemload0
1312097291,4sleep01313726-21grep0
1290667291,4sleep21290766-21grep2
1289769290,2sleep10-21swapper/11
1268837291,2sleep20-21swapper/22
1263520291,4sleep21263778-21df_inode2
1258173291,2sleep20-21swapper/22
1236597290,4sleep10-21swapper/11
1235643291,4sleep01235704-21sed0
1214274290,4sleep11214491-21users1
1213817291,4sleep21213869-21collect22
1204969291,4sleep21205085-21irqstats2
1440838281,3sleep00-21swapper/00
1321630280,1sleep10-21swapper/11
1306141281,3sleep01306159-21/usr/sbin/munin0
1289150281,4sleep01289178-21grep0
1252262280,1sleep20-21swapper/22
1241897281,3sleep21241916-21apt-get2
1227856281,3sleep01229692-21kthreadcore0
1219065281,3sleep11219186-21ntpq1
1204021280,4sleep10-21swapper/11
1306709271,3sleep11307737-21kthreadcore1
1300532270,1sleep00-21swapper/00
1281312271,3sleep21283120-21cut2
1273701270,1sleep00-21swapper/00
1273651270,1sleep10-21swapper/11
1268291271,3sleep01268292-21pool-udisksd0
1258451271,3sleep01258504-21fschecks_count0
1257702270,1sleep00-21swapper/00
1257702270,1sleep00-21swapper/00
1246662270,1sleep10-21swapper/11
1246603270,1sleep00-21swapper/00
1225382271,2sleep21225268-31sort2
1219859270,1sleep00-21swapper/00
1199168270,2sleep20-21swapper/22
1192377271,2sleep20-21swapper/22
1191808270,2sleep10-21swapper/11
1305587260,4sleep10-21swapper/11
1300364260,4sleep10-21swapper/11
1295013260,1sleep10-21swapper/11
1283451260,2sleep00-21swapper/00
1276013261,2sleep00-21swapper/00
1241389260,1sleep00-21swapper/00
1236115260,4sleep10-21swapper/11
1230829260,1sleep10-21swapper/11
1207960260,2sleep11159720-21kworker/u16:3+kvfree_rcu_reclaim1
1206166261,2sleep00-21swapper/00
1196871260,1sleep00-21swapper/00
1315395250,1sleep20-21swapper/22
1303698250,1sleep00-21swapper/00
1300056250,1sleep20-21swapper/22
1278936250,3sleep10-21swapper/11
1284807240,2sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional