You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 06:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Tue Jul 07, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
145160321530,4sleep0221rcuc/00
148915521490,3sleep11489040-21sendmail-msp1
1083535212363,55sleep30-21swapper/33
108369628669,11sleep10-21swapper/11
108101728657,7sleep00-21swapper/00
108378728062,12sleep20-21swapper/22
13622332204,5sleep21362246-21sh2
108405199201,15cyclictest66950irq/132-enp1s0-TxRx-33
108405199201,15cyclictest66950irq/132-enp1s0-TxRx-33
108405199201,15cyclictest0-21swapper/33
108405199201,15cyclictest0-21swapper/33
108405199201,15cyclictest0-21swapper/33
108405199201,15cyclictest0-21swapper/33
108405199201,15cyclictest0-21swapper/33
14139472191,13sleep01207119-21diskmemload0
11696272190,2sleep20-21swapper/22
108405199191,6cyclictest0-21swapper/33
108405199191,15cyclictest66950irq/132-enp1s0-TxRx-33
108405199191,15cyclictest0-21swapper/33
108405199191,15cyclictest0-21swapper/33
108405199191,14cyclictest0-21swapper/33
108405199191,14cyclictest0-21swapper/33
108405199191,14cyclictest0-21swapper/33
108405199191,14cyclictest0-21swapper/33
108405199191,14cyclictest0-21swapper/33
108405199191,14cyclictest0-21swapper/33
108405199191,14cyclictest0-21swapper/33
1084051991910,5cyclictest66950irq/132-enp1s0-TxRx-33
15402612185,8sleep0151ktimers/00
14710872181,3sleep20-21swapper/22
12333452181,4sleep01234803-21ps0
108405199189,6cyclictest1159964-21turbostat3
108405199181,14cyclictest0-21swapper/33
108405199181,14cyclictest0-21swapper/33
108405199181,14cyclictest0-21swapper/33
108405199181,14cyclictest0-21swapper/33
108405199181,14cyclictest0-21swapper/33
108405199181,14cyclictest0-21swapper/33
108405199181,14cyclictest0-21swapper/33
108405199181,14cyclictest0-21swapper/33
108405199181,14cyclictest0-21swapper/33
108405199181,14cyclictest0-21swapper/33
108405199181,13cyclictest0-21swapper/33
108405199181,13cyclictest0-21swapper/33
108405199181,13cyclictest0-21swapper/33
108405199181,13cyclictest0-21swapper/33
108405199181,13cyclictest0-21swapper/33
108405199181,13cyclictest0-21swapper/33
108405199181,13cyclictest0-21swapper/33
108405199181,13cyclictest0-21swapper/33
108405199180,15cyclictest0-21swapper/33
108405199180,14cyclictest0-21swapper/33
15321802171,3sleep00-21swapper/00
12948062171,7sleep11290615-21/usr/sbin/munin1
10870272171,5sleep01087150-21diskstats0
108405199176,8cyclictest1144309-21turbostat3
1084051991713,2cyclictest0-21swapper/33
1084051991711,3cyclictest0-21swapper/33
108405199171,13cyclictest0-21swapper/33
108405199171,13cyclictest0-21swapper/33
108405199171,13cyclictest0-21swapper/33
108405199171,13cyclictest0-21swapper/33
108405199170,14cyclictest0-21swapper/33
108405199170,14cyclictest0-21swapper/33
108405199170,14cyclictest0-21swapper/33
108405199170,13cyclictest0-21swapper/33
14485792161,11sleep21449920-21cut2
14063892161,10sleep01406442-21basename0
13624362161,3sleep00-21swapper/00
13226862161,4sleep21207119-21diskmemload2
12310512161,2sleep20-21swapper/22
108405199169,5cyclictest0-21swapper/33
108405199161,12cyclictest0-21swapper/33
108405199161,11cyclictest0-21swapper/33
108405199160,13cyclictest0-21swapper/33
108405199160,13cyclictest0-21swapper/33
15389982158,2sleep20-21swapper/22
14544442151,4sleep11454458-21ssh1
14384672151,7sleep20-21swapper/22
13892252151,4sleep21207119-21diskmemload2
13651822151,2sleep20-21swapper/22
13565352151,2sleep00-21swapper/00
13563662151,3sleep11207119-21diskmemload1
13375472151,3sleep01207119-21diskmemload0
13278432151,2sleep20-21swapper/22
12818602150,1sleep10-21swapper/11
12818602150,1sleep10-21swapper/11
11497642151,2sleep20-21swapper/22
108405199157,6cyclictest1170411-21turbostat3
108405199153,8cyclictest0-21swapper/33
108405199151,9cyclictest11602262sleep33
108405199151,2cyclictest0-21swapper/33
108405199150,12cyclictest0-21swapper/33
108405199150,12cyclictest0-21swapper/33
15324632140,2sleep10-21swapper/11
15088052141,9sleep11508912-21ps1
14638132141,2sleep20-21swapper/22
13301602141,3sleep01207119-21diskmemload0
13126802141,3sleep10-21swapper/11
12712262141,8sleep139542-21systemd-journal1
12157082141,2sleep10-21swapper/11
12118912141,10sleep00-21swapper/00
1084051991412,1cyclictest0-21swapper/33
108405199141,10cyclictest0-21swapper/33
108405199141,10cyclictest0-21swapper/33
108405199140,2cyclictest0-21swapper/33
108405199140,11cyclictest0-21swapper/33
108405199140,10cyclictest11545092sleep33
15177862131,3sleep11518663-21kthreadcore1
15047702131,3sleep11504780-21ssh1
14456242131,8sleep21445635-21ssh2
13741312131,8sleep00-21swapper/00
13652302132,5sleep11365253-21apt-get1
13462462131,7sleep21207119-21diskmemload2
12652542133,4sleep0151ktimers/00
12640522131,8sleep21264062-21ssh2
12550752131,7sleep01207119-21diskmemload0
12253632131,8sleep01226913-21tr0
12145432131,8sleep21207119-21diskmemload2
11869612131,7sleep11187451-21cut1
11709562131,8sleep21171110-21grep2
15241932121,7sleep01207119-21diskmemload0
15225342121,7sleep21522548-21ssh2
14983552121,2sleep00-21swapper/00
14965312121,7sleep11489342-40find1
14858972121,7sleep21485908-21ssh2
14797272121,6sleep21207119-21diskmemload2
14698092121,6sleep10-21swapper/11
14634792121,4sleep10-21swapper/11
14473162121,7sleep11447326-21ssh1
14358002121,3sleep11436028-21ssh1
14289782121,6sleep01207119-21diskmemload0
14236142121,7sleep21424029-21grep2
14218492121,7sleep21207119-21diskmemload2
14148882122,6sleep1106713-21snmpd1
14115502121,5sleep10-21swapper/11
13952572121,6sleep01207119-21diskmemload0
13865992121,7sleep01386609-21ssh0
13788932120,5sleep10-21swapper/11
13560232120,8sleep21356036-21sh2
12876212121,7sleep01207119-21diskmemload0
12472562121,5sleep11207119-21diskmemload1
12365362120,2sleep21207119-21diskmemload2
12160812121,7sleep01216137-21ssh0
12021442121,5sleep10-21swapper/11
11874532121,7sleep21189111-21tr2
15490602110,2sleep11549072-21ssh1
15404622111,5sleep11207119-21diskmemload1
15144542111,5sleep01207119-21diskmemload0
14973882111,4sleep21207119-21diskmemload2
14698882111,5sleep01207119-21diskmemload0
14456482112,4sleep01207119-21diskmemload0
14046932111,5sleep21207119-21diskmemload2
13954132111,3sleep10-21swapper/11
13634702111,4sleep11207119-21diskmemload1
13451562111,6sleep11207119-21diskmemload1
13213342110,7sleep01207119-21diskmemload0
12906452115,1sleep00-21swapper/00
12866272111,5sleep11286851-21sed1
12731192111,3sleep00-21swapper/00
12640892111,3sleep10-21swapper/11
12550862111,3sleep10-21swapper/11
12484472111,6sleep01207119-21diskmemload0
12318662111,6sleep11231878-21ssh1
12080742111,6sleep11208421-21tr1
11780392110,2sleep00-21swapper/00
11078282111,5sleep11108078-21df_inode1
11003482111,5sleep11101229-21sed1
10973462111,5sleep01097605-21df_inode0
15500912101,3sleep21207119-21diskmemload2
15486482101,3sleep20-21swapper/22
15482622101,5sleep01207119-21diskmemload0
15248962101,5sleep21524984-21tr2
15157062100,3sleep20-21swapper/22
15046912101,5sleep21504702-21ssh2
15006502101,5sleep01502074-21kthreadcore0
14889152101,4sleep01207119-21diskmemload0
14792912101,4sleep11207119-21diskmemload1
14781812100,4sleep00-21swapper/00
14631652101,3sleep00-21swapper/00
14306342100,1sleep10-21swapper/11
14225062101,3sleep00-21swapper/00
14067472101,3sleep20-21swapper/22
13959882101,4sleep21207119-21diskmemload2
13821302101,5sleep11382192-21cut1
13783142101,2sleep20-21swapper/22
13713022100,1sleep00-21swapper/00
13382682101,4sleep11207119-21diskmemload1
13373862101,3sleep20-21swapper/22
13237352101,2sleep10-21swapper/11
13140752101,3sleep20-21swapper/22
13117112101,2sleep00-21swapper/00
13066502101,3sleep10-21swapper/11
13065612101,3sleep20-21swapper/22
13018082100,1sleep00-21swapper/00
12822672101,4sleep21282291-21apt-get2
12740062101,4sleep21207119-21diskmemload2
12740062101,4sleep21207119-21diskmemload2
12661102101,2sleep20-21swapper/22
12376962101,4sleep11207119-21diskmemload1
12059842101,5sleep01206138-21cut0
11916692100,2sleep10-21swapper/11
11759122101,3sleep20-21swapper/22
11660172101,5sleep01166060-21kthreadcore0
11555782100,6sleep01156971-21cut0
11399022101,5sleep21141169-21cat2
11288132101,4sleep11128815-21apt-get1
11080792100,6sleep21108207-21tr2
10970472101,4sleep21097083-21munin-run2
1552721291,4sleep11552733-21ssh1
1550774291,4sleep01550786-21rm0
1403714291,2sleep10-21swapper/11
1323096290,1sleep10-21swapper/11
1294932291,2sleep20-21swapper/22
1274370291,4sleep01274077-21/usr/sbin/munin0
1274370291,4sleep01274077-21/usr/sbin/munin0
1256769291,4sleep21256783-21ssh2
1240820291,4sleep21240830-21ssh2
1222222290,2sleep20-21swapper/22
1202823291,4sleep21204408-21cut2
1181181290,3sleep20-21swapper/22
1165183291,3sleep11165221-21grep1
1160862291,4sleep01162429-21tr0
1144845291,3sleep21144969-21perl2
1144265291,3sleep072-21kcompactd00
1139055291,4sleep21139111-21munin-run2
1117688292,3sleep11117691-21sshd-auth1
1437284281,3sleep01437295-21ssh0
1345285281,3sleep01207119-21diskmemload0
1197415281,3sleep21197491-21uname2
1191648281,2sleep00-21swapper/00
1186143281,3sleep11186167-21latency_hist1
1171237280,2sleep00-21swapper/00
1159139281,4sleep11159262-21sensors_temp1
1155294281,3sleep21154996-21/usr/sbin/munin2
1154374281,3sleep11154377-21sshd-auth1
1144584280,1sleep10-21swapper/11
1139329280,2sleep10-21swapper/11
1128781281,2sleep01128498-21apt-get0
1120131281,3sleep11121873-21kthreadcore1
1120131281,3sleep11121873-21kthreadcore1
1118854281,4sleep01120130-21cat0
1118854281,4sleep01120130-21cat0
1107815281,3sleep01107817-21cpu0
1102626280,4sleep21102844-21df_inode2
1102559280,2sleep00-21swapper/00
1201801270,1sleep00-21swapper/00
1191031270,1sleep00-21swapper/00
1180180270,3sleep1321rcuc/11
1166828271,2sleep10-21swapper/11
1145127271,2sleep00-21swapper/00
1138941270,1sleep00-21swapper/00
1138302271,2sleep10-21swapper/11
1123525270,2sleep00-21swapper/00
1107429270,1sleep10-21swapper/11
1097322270,2sleep20-21swapper/22
1092653271,2sleep10-21swapper/11
1092091270,2sleep00-21swapper/00
1087151271,3sleep10-21swapper/11
1084935270,3sleep21086277-21grep2
1084424270,1sleep10-21swapper/11
1193909261,2sleep20-21swapper/22
1181145260,2sleep00-21swapper/00
1172253261,2sleep10-21swapper/11
1154687260,4sleep00-21swapper/00
1129895261,2sleep20-21swapper/22
1125498261,2sleep20-21swapper/22
1123001260,1sleep20-21swapper/22
1123001260,1sleep20-21swapper/22
1117797260,1sleep20-21swapper/22
1114335260,3sleep0221rcuc/00
1196899250,1sleep10-21swapper/11
1164038250,1sleep20-21swapper/22
1123547250,1sleep10-21swapper/11
1084607250,1sleep00-21swapper/00
1084341250,3sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional