You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 21:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Thu Jul 23, 2026 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137301221650,2sleep20-21swapper/22
1018063212552,68sleep30-21swapper/33
101825928862,17sleep00-21swapper/00
101809628653,12sleep10-21swapper/11
101803728466,11sleep20-21swapper/22
1018571992511,10cyclictest1189934-21turbostat3
101857199211,16cyclictest0-21swapper/33
101857199211,16cyclictest0-21swapper/33
101857199210,3cyclictest0-21swapper/33
13803532201,3sleep00-21swapper/00
101857199203,14cyclictest66950irq/132-enp1s0-TxRx-33
101857199203,11cyclictest1442475-21taskset3
101857199201,3cyclictest0-21swapper/33
101857199201,15cyclictest1247393-21chrt3
101857199201,15cyclictest0-21swapper/33
101857199201,14cyclictest0-21swapper/33
101857199201,14cyclictest0-21swapper/33
13703152191,2sleep20-21swapper/22
11812102197,8sleep21181285-21systemd-run2
101857199192,12cyclictest1320846-21taskset3
101857199191,15cyclictest0-21swapper/33
1018571991911,4cyclictest0-21swapper/33
101857199191,14cyclictest0-21swapper/33
14872162181,12sleep01488787-21cut0
101857199182,11cyclictest1233897-21taskset3
1018571991815,1cyclictest0-21swapper/33
1018571991814,2cyclictest0-21swapper/33
1018571991814,2cyclictest0-21swapper/33
1018571991813,4cyclictest0-21swapper/33
1018571991811,4cyclictest0-21swapper/33
101857199181,14cyclictest0-21swapper/33
101857199181,14cyclictest0-21swapper/33
101857199181,14cyclictest0-21swapper/33
101857199181,13cyclictest0-21swapper/33
101857199181,13cyclictest0-21swapper/33
101857199181,13cyclictest0-21swapper/33
101857199181,13cyclictest0-21swapper/33
101857199181,12cyclictest0-21swapper/33
101857199180,15cyclictest0-21swapper/33
101857199180,14cyclictest0-21swapper/33
101857199180,14cyclictest0-21swapper/33
101857199180,14cyclictest0-21swapper/33
15104182171,4sleep01146349-21diskmemload0
15104182171,4sleep01146349-21diskmemload0
14379692176,2sleep0171rcu_preempt0
14305332171,2sleep00-21swapper/00
13369342171,4sleep21146349-21diskmemload2
11903222171,4sleep01190346-21apt-get0
11724132171,2sleep10-21swapper/11
11708252171,2sleep00-21swapper/00
101857199171,4cyclictest0-21swapper/33
1018571991714,1cyclictest0-21swapper/33
1018571991714,1cyclictest0-21swapper/33
101857199171,13cyclictest0-21swapper/33
101857199171,13cyclictest0-21swapper/33
101857199171,13cyclictest0-21swapper/33
101857199171,13cyclictest0-21swapper/33
101857199171,13cyclictest0-21swapper/33
101857199170,14cyclictest0-21swapper/33
101857199170,14cyclictest0-21swapper/33
101857199170,14cyclictest0-21swapper/33
101857199170,13cyclictest0-21swapper/33
13689192160,4sleep01369054-21/usr/sbin/munin0
10993982161,2sleep00-21swapper/00
101857199161,3cyclictest0-21swapper/33
101857199161,14cyclictest0-21swapper/33
101857199161,13cyclictest0-21swapper/33
101857199161,12cyclictest0-21swapper/33
1018571991610,4cyclictest0-21swapper/33
101857199160,3cyclictest0-21swapper/33
101857199160,3cyclictest0-21swapper/33
101857199160,14cyclictest0-21swapper/33
101857199160,13cyclictest0-21swapper/33
101857199160,13cyclictest0-21swapper/33
101857199160,12cyclictest0-21swapper/33
14563602151,3sleep10-21swapper/11
13208142152,6sleep21320838-21apt-get2
12910882151,10sleep21291101-21ssh2
12841552150,2sleep20-21swapper/22
12728032151,9sleep11272887-21ntp_states1
11612122151,8sleep10-21swapper/11
1018571991512,2cyclictest0-21swapper/33
1018571991510,3cyclictest0-21swapper/33
101857199150,13cyclictest0-21swapper/33
101857199150,13cyclictest0-21swapper/33
101857199150,12cyclictest0-21swapper/33
101857199150,12cyclictest0-21swapper/33
15087462140,2sleep20-21swapper/22
15087462140,2sleep20-21swapper/22
14564742141,8sleep21146349-21diskmemload2
14408872141,8sleep21440902-21ssh2
14153742141,2sleep20-21swapper/22
13377812142,8sleep14192556-21in:imuxsock1
13006992140,2sleep10-21swapper/11
12575992140,2sleep20-21swapper/22
11735632141,9sleep00-21swapper/00
11685462141,5sleep21168716-21timerandwakeup2
101857199148,4cyclictest1089742-21chrt3
1018571991412,1cyclictest0-21swapper/33
1018571991412,1cyclictest0-21swapper/33
101857199140,12cyclictest0-21swapper/33
101857199140,12cyclictest0-21swapper/33
101857199140,12cyclictest0-21swapper/33
101857199140,11cyclictest0-21swapper/33
14945162131,7sleep21494784-21df2
14839982130,1sleep00-21swapper/00
14737072131,3sleep01473718-21ssh0
14656822131,3sleep01146349-21diskmemload0
14321682131,2sleep10-21swapper/11
13971802130,3sleep01397193-21ssh0
13817032131,6sleep21146349-21diskmemload2
13556012131,6sleep10-21swapper/11
13531832131,8sleep01146349-21diskmemload0
13356342131,6sleep01146349-21diskmemload0
13093712130,9sleep21309384-21rm2
12406592131,3sleep11240667-21ssh1
12307522130,3sleep01230766-21ssh0
12235162131,2sleep0171rcu_preempt0
12165782131,6sleep21216798-21df_inode2
11883062131,8sleep01188318-21ssh0
11157382131,3sleep11117233-21ps1
10848552131,8sleep01086510-21cut0
14917812120,8sleep11491789-21ssh1
14724602121,7sleep11468444-21/usr/sbin/munin1
14655002121,5sleep10-21swapper/11
14413962121,7sleep11441406-21ssh1
14045702121,4sleep00-21swapper/00
13537982121,7sleep11353809-21ssh1
13163392121,7sleep11316444-21ps1
13163392121,7sleep11316444-21ps1
13005332121,7sleep01300547-21ssh0
12926162121,5sleep11146349-21diskmemload1
12686572121,3sleep00-21swapper/00
12675932120,8sleep11267602-21sh1
12398582121,3sleep21239869-21ssh2
12312332121,2sleep11231244-21ssh1
12064632121,7sleep11206465-21ssh1
11470572121,7sleep11147255-21sed1
10497582121,5sleep11050870-21memory1
10423842121,7sleep01043967-21cut0
15074192111,4sleep10-21swapper/11
15074192111,4sleep10-21swapper/11
15012972111,5sleep11146349-21diskmemload1
14835052111,2sleep1171rcu_preempt1
14738892111,4sleep21146349-21diskmemload2
14493942110,2sleep10-21swapper/11
14475212110,7sleep21447535-21ssh2
14305602111,3sleep20-21swapper/22
14159682111,6sleep11415981-21rm1
13922892111,6sleep21393963-21kthreadcore2
13865862111,6sleep01386596-21ssh0
13691642112,2sleep11369177-21rm1
13455092111,5sleep01146349-21diskmemload0
13270182111,3sleep00-21swapper/00
13211442111,5sleep11321368-21hddtemp_smartct1
13106592110,2sleep01310673-21ssh0
12835862111,6sleep11146349-21diskmemload1
12687552111,4sleep21268290-21sendmail-msp2
12577282111,5sleep01146349-21diskmemload0
12492302111,5sleep21249241-21ssh2
12487452111,6sleep11248755-21ssh1
12339112111,4sleep01234171-21df0
12242932111,7sleep11224303-21ssh1
12041452112,4sleep21146349-21diskmemload2
11963482111,3sleep20-21swapper/22
11885962110,2sleep20-21swapper/22
11257422111,5sleep01125883-21diskstats0
11257422111,5sleep01125883-21diskstats0
10889042111,4sleep21088903-21apt-get2
10577982111,5sleep01058008-21hddtemp_smartct0
14933752101,4sleep21493386-21ssh2
14830052101,4sleep21146349-21diskmemload2
14559412101,4sleep01146349-21diskmemload0
14492832101,3sleep00-21swapper/00
14240342101,4sleep01146349-21diskmemload0
14170542101,5sleep21417748-21cut2
14154022101,4sleep01146349-21diskmemload0
14068922100,6sleep21406903-21ssh2
14065762101,2sleep10-21swapper/11
13870622100,2sleep10-21swapper/11
13794432101,3sleep10-21swapper/11
13604122100,1sleep20-21swapper/22
13555652101,4sleep01355597-21apt-get0
13551502101,4sleep21355193-21munin-run2
13179462102,4sleep21317960-21ssh2
13179462102,4sleep21317960-21ssh2
13174152101,4sleep01146349-21diskmemload0
13174152101,4sleep01146349-21diskmemload0
13039852101,4sleep11304103-21irqstats1
13017302100,4sleep20-21swapper/22
12822932101,4sleep01146349-21diskmemload0
12650902100,1sleep20-21swapper/22
12599532102,4sleep01259945-21apt-get0
12512792101,4sleep11251548-21df_inode1
12424472101,4sleep01146349-21diskmemload0
12314982101,2sleep20-21swapper/22
12126282101,4sleep21146349-21diskmemload2
12078072101,5sleep01207972-21cstates0
11990742101,4sleep01198689-21sendmail-msp0
11972972101,4sleep11146349-21diskmemload1
11871552101,4sleep11146349-21diskmemload1
11793432100,1sleep10-21swapper/11
11627262101,4sleep21162737-21ssh2
11517542102,3sleep21146349-21diskmemload2
11516312101,4sleep01146349-21diskmemload0
11203552101,4sleep21120374-21apt-get2
11200872101,5sleep01120209-21cut0
11148392101,6sleep11114938-21latency_hist1
11102562100,1sleep20-21swapper/22
10865112101,4sleep11087529-21memory1
10801482102,4sleep21081874-21cut2
10731952101,4sleep21072977-21sendmail-msp2
10682732101,5sleep21068489-21idleruntime2
10529642101,5sleep21054565-21kthreadcore2
1499889291,2sleep00-21swapper/00
1464042291,4sleep21464198-21grep2
1424406291,3sleep11146349-21diskmemload1
1395814291,4sleep11395829-21ssh1
1343840291,2sleep20-21swapper/22
1334533290,3sleep10-21swapper/11
1285974291,3sleep01146349-21diskmemload0
1213496291,2sleep10-21swapper/11
1160635291,2sleep00-21swapper/00
1141952291,4sleep01143418-21cat0
1135800291,4sleep11135966-21wc1
1131156291,4sleep21131356-21grep2
1115124291,3sleep21114905-21sendmail-msp2
1074210291,3sleep11075854-21cut1
1052283291,3sleep10-21swapper/11
1046976291,4sleep21046995-21apt-get2
1037043291,4sleep11038191-21cut1
1031256291,2sleep20-21swapper/22
1023413290,2sleep20-21swapper/22
1134873281,4sleep00-21swapper/00
1125592281,2sleep20-21swapper/22
1125592281,2sleep20-21swapper/22
1111063281,3sleep00-21swapper/00
1109312280,2sleep00-21swapper/00
1100435281,3sleep11102086-21kthreadcore1
1099058280,1sleep20-21swapper/22
1094136280,4sleep00-21swapper/00
1093853281,3sleep11093932-21munin-run1
1083218280,1sleep00-21swapper/00
1078435280,2sleep10-21swapper/11
1077875280,2sleep072-21kcompactd00
1062781281,3sleep01062827-21/usr/sbin/munin0
1048106281,3sleep01049757-21tr0
1045926280,1sleep10-21swapper/11
1038192281,3sleep21039919-21grep2
1031550281,3sleep11031762-21ls1
1030955281,3sleep21031024-21systemd-run2
1020509281,3sleep11022258-21cut1
1146216270,1sleep20-21swapper/22
1143420271,2sleep10-21swapper/11
1140768270,1sleep00-21swapper/00
1136116271,2sleep10-21swapper/11
1135967271,2sleep21135866-21sendmail-msp2
1130499270,1sleep10-21swapper/11
1130499270,1sleep10-21swapper/11
1125210270,1sleep10-21swapper/11
1104654270,2sleep20-21swapper/22
1094796271,3sleep11096384-21kthreadcore1
1093561270,1sleep00-21swapper/00
1083672270,2sleep20-21swapper/22
1067547270,1sleep20-21swapper/22
1062181270,1sleep20-21swapper/22
1057502270,2sleep10-21swapper/11
1052265271,2sleep00-21swapper/00
1031247271,2sleep00-21swapper/00
1030591270,1sleep10-21swapper/11
1026027270,2sleep00-21swapper/00
1023526270,1sleep00-21swapper/00
1122353260,2sleep01124110-21cut0
1109485260,1sleep10-21swapper/11
1104068260,1sleep20-21swapper/22
1072589262,2sleep01018565-21cyclictest0
1072570260,1sleep10-21swapper/11
1064338261,2sleep10-21swapper/11
1036471260,2sleep00-21swapper/00
1046674250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional