You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 08:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Tue Jul 21, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
164988221440,2sleep00-21swapper/00
1268649211853,56sleep30-21swapper/33
126859828654,12sleep20-21swapper/22
126875727860,12sleep10-21swapper/11
126861727656,14sleep00-21swapper/00
1269099992511,10cyclictest1440320-21turbostat3
17384282231,15sleep11738424-21apt-get1
126909999212,5cyclictest17289952chrt3
126909999211,17cyclictest0-21swapper/33
126909999211,14cyclictest0-21swapper/33
1269099992017,1cyclictest0-21swapper/33
1269099992012,5cyclictest0-21swapper/33
126909999201,15cyclictest0-21swapper/33
126909999201,15cyclictest0-21swapper/33
126909999200,16cyclictest0-21swapper/33
126909999191,15cyclictest0-21swapper/33
126909999191,15cyclictest0-21swapper/33
126909999191,15cyclictest0-21swapper/33
126909999191,14cyclictest0-21swapper/33
126909999191,14cyclictest0-21swapper/33
126909999191,14cyclictest0-21swapper/33
126909999191,14cyclictest0-21swapper/33
126909999191,13cyclictest0-21swapper/33
1269099991910,6cyclictest0-21swapper/33
1269099991910,5cyclictest0-21swapper/33
126909999190,13cyclictest0-21swapper/33
16122282181,3sleep10-21swapper/11
15845562181,13sleep01584626-21ssh0
15699412181,6sleep21569956-21ssh2
126909999184,2cyclictest66950irq/132-enp1s0-TxRx-33
126909999181,4cyclictest0-21swapper/33
1269099991814,3cyclictest66950irq/132-enp1s0-TxRx-33
1269099991814,3cyclictest66950irq/132-enp1s0-TxRx-33
1269099991812,3cyclictest0-21swapper/33
1269099991811,5cyclictest0-21swapper/33
126909999181,14cyclictest0-21swapper/33
126909999181,14cyclictest0-21swapper/33
126909999181,14cyclictest0-21swapper/33
126909999181,14cyclictest0-21swapper/33
126909999181,14cyclictest0-21swapper/33
126909999181,14cyclictest0-21swapper/33
126909999181,14cyclictest0-21swapper/33
126909999181,14cyclictest0-21swapper/33
126909999181,14cyclictest0-21swapper/33
126909999181,13cyclictest0-21swapper/33
126909999181,13cyclictest0-21swapper/33
126909999181,13cyclictest0-21swapper/33
126909999181,13cyclictest0-21swapper/33
126909999181,13cyclictest0-21swapper/33
15840832178,5sleep11584199-21grep1
15525982171,3sleep10-21swapper/11
14645942171,3sleep10-21swapper/11
14564452171,3sleep20-21swapper/22
14124782170,1sleep20-21swapper/22
1269099991712,4cyclictest66950irq/132-enp1s0-TxRx-33
1269099991712,3cyclictest0-21swapper/33
1269099991712,3cyclictest0-21swapper/33
126909999171,14cyclictest0-21swapper/33
126909999171,13cyclictest0-21swapper/33
126909999171,13cyclictest0-21swapper/33
126909999171,12cyclictest0-21swapper/33
126909999171,12cyclictest0-21swapper/33
126909999170,15cyclictest0-21swapper/33
126909999170,14cyclictest0-21swapper/33
17287222161,3sleep20-21swapper/22
17262702161,7sleep11726281-21ssh1
16574272160,3sleep10-21swapper/11
15079402161,4sleep01507955-21ssh0
14908542168,4sleep11490868-21ssh1
14457182161,2sleep00-21swapper/00
13330672161,11sleep21333134-21cut2
13138872161,4sleep11314084-21cut1
126909999161,3cyclictest0-21swapper/33
1269099991613,1cyclictest0-21swapper/33
126909999161,12cyclictest0-21swapper/33
126909999160,13cyclictest0-21swapper/33
126909999160,13cyclictest0-21swapper/33
126909999160,12cyclictest0-21swapper/33
17624012150,2sleep00-21swapper/00
17180792151,2sleep20-21swapper/22
16483502151,4sleep11396802-21diskmemload1
14563892151,7sleep10-21swapper/11
14402932151,7sleep11440359-21munin-run1
14302562150,1sleep10-21swapper/11
14236082151,9sleep21423719-21/usr/sbin/munin2
126909999158,4cyclictest12976202sleep33
126909999150,13cyclictest0-21swapper/33
126909999150,13cyclictest0-21swapper/33
126909999150,12cyclictest0-21swapper/33
126909999150,12cyclictest0-21swapper/33
16401132141,2sleep20-21swapper/22
16232832141,3sleep01623311-21apt-get0
15334482141,3sleep01396802-21diskmemload0
15057612141,8sleep21505835-21/usr/sbin/munin2
14829072141,2sleep20-21swapper/22
14026222141,8sleep01402637-21ssh0
13557562141,4sleep21356837-21tr2
13237822141,8sleep11323564-21sendmail-msp1
1269099991412,1cyclictest0-21swapper/33
126909999140,2cyclictest0-21swapper/33
126909999140,12cyclictest0-21swapper/33
126909999140,12cyclictest0-21swapper/33
126909999140,11cyclictest0-21swapper/33
126909999140,11cyclictest0-21swapper/33
17523342131,6sleep11396802-21diskmemload1
17172932131,8sleep01717308-21ssh0
17027882131,4sleep01702801-21rm0
16651472131,7sleep01665154-21ssh0
16651472131,7sleep01665154-21ssh0
16482522132,5sleep21396802-21diskmemload2
15729842130,8sleep11574634-21cut1
15255982130,3sleep11396802-21diskmemload1
14644962130,8sleep21396802-21diskmemload2
14530982132,6sleep01396802-21diskmemload0
14208422131,2sleep11396802-21diskmemload1
12890302131,9sleep00-21swapper/00
12743282131,4sleep01275016-21kthreadcore0
17549932121,6sleep21396802-21diskmemload2
17124242121,7sleep10-21swapper/11
16998072120,6sleep11692566-40find1
16949072121,5sleep20-21swapper/22
16836772121,7sleep01683769-21cat0
16743562120,3sleep21674370-21ssh2
16647952121,3sleep10-21swapper/11
16647952121,3sleep10-21swapper/11
16549372121,6sleep21396802-21diskmemload2
16481272121,2sleep01648140-21ssh0
16275122121,7sleep21396802-21diskmemload2
16200982121,7sleep01620109-21ssh0
16144922121,7sleep21614506-21ssh2
16139392121,5sleep20-21swapper/22
15987552121,7sleep21600363-21cut2
15516442121,6sleep21396802-21diskmemload2
15423892120,3sleep21542402-21ssh2
15006642121,7sleep11500674-21ssh1
14817662121,7sleep01481772-21sh0
14373242121,3sleep21437335-21ssh2
126909999129,2cyclictest0-21swapper/33
17617602110,1sleep10-21swapper/11
17356102110,7sleep21396802-21diskmemload2
17300202111,6sleep11730096-21fschecks_count1
17112352111,3sleep10-21swapper/11
17078422111,6sleep01707905-21open_files0
16884222111,2sleep11688608-21timerandwakeup1
16840682111,4sleep21684095-21apt-get2
16378952110,3sleep01637908-21ssh0
16371392111,6sleep11396802-21diskmemload1
16274292111,5sleep11627511-21ntp_states1
16049042111,3sleep00-21swapper/00
15888832111,3sleep11589021-21ssh1
15674852111,3sleep10-21swapper/11
15600482110,1sleep10-21swapper/11
15267642110,1sleep00-21swapper/00
15064182111,4sleep10-21swapper/11
14914382113,3sleep21491449-21ssh2
14891542111,2sleep00-21swapper/00
14831472111,6sleep11396802-21diskmemload1
14727892111,3sleep10-21swapper/11
14709892111,6sleep01471108-21/usr/sbin/munin0
14413112111,6sleep21441374-21kthreadcore2
14148982111,6sleep01415080-21tail0
14061672111,5sleep01406391-21hddtemp_smartct0
14031942111,3sleep10-21swapper/11
13345532111,5sleep21334770-21hddtemp_smartct2
13081132111,4sleep01308132-21apt-get0
13081132111,4sleep01308132-21apt-get0
13078432110,1sleep10-21swapper/11
13025772111,5sleep21302643-21sendmail-msp2
12806962110,1sleep20-21swapper/22
17562752101,4sleep21756624-21kthreadcore2
17460822100,6sleep21746098-21sshd-session2
17378382101,4sleep01396802-21diskmemload0
17286952101,3sleep00-21swapper/00
16910322100,1sleep00-21swapper/00
16812052101,4sleep11396802-21diskmemload1
16709732101,5sleep11671132-21/usr/sbin/munin1
16665902101,4sleep01396802-21diskmemload0
16632882101,3sleep21396802-21diskmemload2
16632882101,3sleep21396802-21diskmemload2
16228142101,4sleep11396802-21diskmemload1
16110842101,5sleep01396802-21diskmemload0
16054822100,2sleep10-21swapper/11
15934312101,3sleep00-21swapper/00
15778182101,3sleep00-21swapper/00
15761902100,1sleep20-21swapper/22
15684352100,1sleep00-21swapper/00
15606512101,4sleep01396802-21diskmemload0
15591942101,4sleep21396802-21diskmemload2
15438542101,4sleep01396802-21diskmemload0
15405822101,5sleep11540673-21wc1
15347092100,1sleep20-21swapper/22
15172302100,5sleep11396802-21diskmemload1
15153812101,3sleep00-21swapper/00
14737312100,6sleep21396802-21diskmemload2
14377582101,2sleep00-21swapper/00
14193142101,2sleep20-21swapper/22
14133872101,4sleep11396802-21diskmemload1
13971672101,4sleep21397196-21apt-get2
13958842102,4sleep11395993-21/usr/sbin/munin1
13695882100,1sleep00-21swapper/00
13673032101,6sleep21368999-21kthreadcore2
13653832101,5sleep21365449-21sendmail-msp2
13551812101,4sleep11355200-21apt-get1
13447342101,4sleep11344753-21apt-get1
13028722101,3sleep20-21swapper/22
12861492101,4sleep11286393-21users1
1747073291,3sleep01747094-21apt-get0
1743549291,3sleep01396802-21diskmemload0
1709147291,4sleep21709160-21ssh2
1675484291,2sleep20-21swapper/22
1592549291,4sleep21592699-21ntp_192.168.1142
1586954290,1sleep20-21swapper/22
1547089291,4sleep01548757-21ssh0
1532956290,1sleep10-21swapper/11
1525135291,2sleep20-21swapper/22
1517700290,4sleep21396802-21diskmemload2
1498212291,5sleep20-21swapper/22
1462948290,2sleep00-21swapper/00
1447614291,2sleep10-21swapper/11
1428410291,2sleep00-21swapper/00
1386605290,1sleep00-21swapper/00
1377278290,5sleep11378934-21tr1
1344727291,4sleep0205-21jbd2/sda2-80
1340891291,4sleep11342588-21cut1
1340046291,4sleep01340890-21cut0
1329518290,4sleep11329602-21irqstats1
1324785293,3sleep21326416-21kthreadcore2
1297607291,2sleep00-21swapper/00
1287132291,3sleep21287151-21apt-get2
1281804291,4sleep21281577-21sendmail-msp2
1269300291,4sleep11269554-21df1
1498659281,3sleep00-21swapper/00
1374976280,5sleep11375084-21sendmail_mailqu1
1371461281,3sleep01371494-21kthreadcore0
1365671280,4sleep10-21swapper/11
1349947281,2sleep00-21swapper/00
1344729281,2sleep21344506-21sendmail-msp2
1344440281,3sleep21344566-21wc2
1336247281,3sleep11338015-21kthreadcore1
1333796280,1sleep00-21swapper/00
1318533281,2sleep20-21swapper/22
1317921280,2sleep00-21swapper/00
1307701280,2sleep00-21swapper/00
1292041280,2sleep10-21swapper/11
1269817281,3sleep01270098-21kthreadcore0
1391812270,2sleep00-21swapper/00
1391468270,1sleep20-21swapper/22
1391177270,1sleep10-21swapper/11
1382263271,3sleep00-21swapper/00
1380773270,1sleep20-21swapper/22
1376681271,2sleep00-21swapper/00
1375781270,1sleep20-21swapper/22
1365210270,1sleep10-21swapper/11
1354572270,1sleep10-21swapper/11
1351955271,2sleep21353716-21tr2
1334840271,2sleep00-21swapper/00
1323319270,1sleep00-21swapper/00
1318532270,1sleep10-21swapper/11
1314085270,3sleep21315693-21cut2
1308135270,2sleep10-21swapper/11
1308135270,2sleep10-21swapper/11
1302379270,1sleep10-21swapper/11
1297260270,1sleep10-21swapper/11
1281453270,4sleep10-21swapper/11
1386207260,4sleep10-21swapper/11
1360982261,2sleep00-21swapper/00
1355471261,2sleep00-21swapper/00
1309499261,2sleep20-21swapper/22
1309499261,2sleep20-21swapper/22
1297026260,4sleep20-21swapper/22
1292900260,2sleep01293421-21cat0
1282095260,2sleep00-21swapper/00
1395994250,1sleep20-21swapper/22
1381353250,1sleep20-21swapper/22
1323786250,1sleep00-21swapper/00
1269272250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional