You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 20:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sat Mar 28, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40819012125105,12sleep20-21swapper/22
4081828211954,60sleep30-21swapper/33
408164928653,12sleep10-21swapper/11
408180527455,14sleep00-21swapper/00
1110432301,4sleep0111224-21ssh0
408212299231,19cyclictest0-21swapper/33
408212299220,19cyclictest0-21swapper/33
1333152221,8sleep0133325-21sh0
4082122992119,1cyclictest0-21swapper/33
4082122992115,4cyclictest0-21swapper/33
632092201,7sleep163239-21ssh1
408212299209,8cyclictest368920-21chrt3
408212299203,13cyclictest0-21swapper/33
408212299201,7cyclictest4105005-21chrt3
408212299201,15cyclictest0-21swapper/33
408212299201,15cyclictest0-21swapper/33
408212299201,15cyclictest0-21swapper/33
1551702200,15sleep119034-21diskmemload1
41802922191,6sleep24180352-21sendmail-msp2
408212299199,6cyclictest4137507-21taskset3
4082122991915,2cyclictest0-21swapper/33
4082122991912,5cyclictest0-21swapper/33
4082122991912,4cyclictest0-21swapper/33
408212299191,15cyclictest0-21swapper/33
408212299191,15cyclictest0-21swapper/33
408212299191,14cyclictest0-21swapper/33
408212299191,14cyclictest0-21swapper/33
408212299191,14cyclictest0-21swapper/33
408212299191,14cyclictest0-21swapper/33
408212299191,14cyclictest0-21swapper/33
408212299190,15cyclictest0-21swapper/33
519772182,8sleep119034-21diskmemload1
4082122991812,3cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,14cyclictest0-21swapper/33
408212299181,13cyclictest4153341-21taskset3
408212299181,13cyclictest0-21swapper/33
408212299181,13cyclictest0-21swapper/33
4082122991810,5cyclictest281570-21taskset3
408212299180,15cyclictest0-21swapper/33
408212299180,14cyclictest0-21swapper/33
408212299180,14cyclictest0-21swapper/33
408212299171,4cyclictest0-21swapper/33
4082122991713,2cyclictest0-21swapper/33
408212299171,15cyclictest0-21swapper/33
4082122991711,4cyclictest0-21swapper/33
4082122991711,4cyclictest0-21swapper/33
408212299171,14cyclictest0-21swapper/33
408212299171,14cyclictest0-21swapper/33
4082122991711,3cyclictest0-21swapper/33
408212299171,13cyclictest0-21swapper/33
408212299171,13cyclictest0-21swapper/33
408212299171,13cyclictest0-21swapper/33
408212299171,12cyclictest0-21swapper/33
408212299170,14cyclictest0-21swapper/33
408212299170,14cyclictest0-21swapper/33
408212299170,13cyclictest0-21swapper/33
2646912171,3sleep20-21swapper/22
2203362171,3sleep00-21swapper/00
2002952170,7sleep00-21swapper/00
601472161,10sleep119034-21diskmemload1
41205072161,4sleep24120562-21ntpq2
408212299161,12cyclictest0-21swapper/33
408212299161,12cyclictest0-21swapper/33
408212299160,13cyclictest0-21swapper/33
408212299160,13cyclictest0-21swapper/33
3591912161,3sleep00-21swapper/00
2696042161,10sleep2269659-21ntpq2
2475602160,4sleep1321rcuc/11
2460792161,2sleep20-21swapper/22
2080132161,5sleep2208073-21ntpq2
1855382161,10sleep119034-21diskmemload1
1059372161,11sleep2105948-21ssh2
41271152151,3sleep04126908-21fwupd0
41244422151,9sleep04125691-21memory0
41244422151,9sleep04125691-21memory0
41160822151,7sleep24115973-31sort2
408212299151,11cyclictest4121784-21chrt3
408212299151,11cyclictest4121784-21chrt3
4082122991510,3cyclictest0-21swapper/33
408212299150,13cyclictest0-21swapper/33
408212299150,13cyclictest0-21swapper/33
408212299150,12cyclictest0-21swapper/33
408212299150,12cyclictest0-21swapper/33
3838332151,9sleep1383896-21ntpq1
3601572151,4sleep1360169-21ssh1
3174312151,4sleep119034-21diskmemload1
2801652150,3sleep10-21swapper/11
2352612151,2sleep00-21swapper/00
2278712151,3sleep219034-21diskmemload2
1485142151,5sleep1148526-21ssh1
139682151,9sleep214224-21df2
1225772150,1sleep10-21swapper/11
1146862151,4sleep219034-21diskmemload2
982422142,7sleep098264-21apt-get0
791992141,3sleep00-21swapper/00
691412141,7sleep219034-21diskmemload2
4082122991411,2cyclictest0-21swapper/33
408212299141,11cyclictest0-21swapper/33
3781562141,9sleep2378167-21ssh2
3090402141,5sleep20-21swapper/22
2897312141,2sleep00-21swapper/00
1919132141,8sleep219034-21diskmemload2
1508522140,1sleep20-21swapper/22
538212131,8sleep053833-21ssh0
41741052131,8sleep04174228-21mailstats0
41706542131,8sleep24172318-21kthreadcore2
41632832131,8sleep24163321-21ntpq2
40828712131,8sleep04084250-21grep0
373092131,8sleep037485-21head0
3686722131,7sleep00-21swapper/00
3441532131,8sleep0344410-21ssh0
3066882131,8sleep119034-21diskmemload1
2997242131,2sleep00-21swapper/00
2796202131,8sleep2279633-21ssh2
2461552131,8sleep0246169-21ssh0
2382522131,7sleep119034-21diskmemload1
2089762131,6sleep119034-21diskmemload1
2031052131,8sleep2203483-21dpkg2
1912432135,4sleep172-21kcompactd01
1468582131,7sleep2147114-21users2
1394632131,2sleep10-21swapper/11
613502121,5sleep00-21swapper/00
41312392122,5sleep14131289-21ntpq1
40896522121,6sleep24089671-21apt-get2
3841822121,7sleep2384233-21grep2
3766142121,6sleep119034-21diskmemload1
3672842121,7sleep2367295-21ssh2
3590122121,7sleep2359020-21ssh2
3440142121,3sleep10-21swapper/11
3425432121,6sleep00-21swapper/00
3236672121,6sleep119034-21diskmemload1
3228892121,7sleep2322974-21find2
2964802120,2sleep2171rcu_preempt2
2819782121,2sleep00-21swapper/00
2727452121,7sleep1272748-21ssh1
2715072121,7sleep0271519-21ssh0
2521092121,5sleep1252165-21ntpq1
2197422121,7sleep1219753-21ssh1
1814332121,8sleep2181488-21ntpq2
1734622121,7sleep0173473-21ssh0
1734622121,7sleep0173473-21ssh0
1322822121,7sleep1132293-21ssh1
1307072123,1sleep2171rcu_preempt2
1240602121,6sleep219034-21diskmemload2
1200222121,7sleep019034-21diskmemload0
1049062121,8sleep1104918-21sh1
952792111,6sleep195290-21sh1
803342111,6sleep2802-21snmpd2
610172110,7sleep261031-21ssh2
423122110,2sleep10-21swapper/11
41418912111,5sleep24141939-21ntpq2
41398632111,6sleep04137845-21/usr/sbin/munin0
41373432112,6sleep1802-21snmpd1
41160712111,5sleep14116006-31find1
41061762111,3sleep14106825-21kthreadcore1
3695112110,6sleep10-21swapper/11
3504302110,8sleep1350443-21rm1
3327302111,1sleep00-21swapper/00
3183082111,5sleep0318336-21apt-get0
3153602110,7sleep219034-21diskmemload2
2893382111,4sleep119034-21diskmemload1
2620392111,6sleep019034-21diskmemload0
2616492111,3sleep10-21swapper/11
260782111,3sleep20-21swapper/22
2540082111,6sleep2254022-21ssh2
245992111,6sleep024609-21ssh0
2354922111,3sleep20-21swapper/22
2231562111,6sleep1224860-21kthreadcore1
2041902111,4sleep00-21swapper/00
1733802111,3sleep20-21swapper/22
1733802111,3sleep20-21swapper/22
1684272111,2sleep10-21swapper/11
1684272111,2sleep10-21swapper/11
1676082111,5sleep119034-21diskmemload1
144962115,2sleep10-21swapper/11
121512111,6sleep112635-21grep1
966312100,2sleep00-21swapper/00
945272101,3sleep20-21swapper/22
883982101,4sleep119034-21diskmemload1
859752101,3sleep20-21swapper/22
760912102,3sleep176191-21ntpq1
694752101,2sleep00-21swapper/00
498572101,4sleep249916-21ntpq2
41902512101,4sleep04190293-21ntpq0
41472512100,5sleep14147289-21ntpq1
41327802101,4sleep24132993-21hddtemp_smartct2
41204572101,4sleep14120505-21ntpq1
41159512100,5sleep04116006-31find0
41068282103,3sleep20-21swapper/22
40975652101,4sleep04098868-21memory0
40823832101,5sleep14082587-21/usr/sbin/munin1
3854472101,3sleep00-21swapper/00
3708092102,3sleep0370842-21apt-get0
356532100,2sleep20-21swapper/22
3448352101,6sleep2344884-21/usr/sbin/munin2
340462101,2sleep10-21swapper/11
3347302101,2sleep10-21swapper/11
331772101,3sleep00-21swapper/00
32622101,4sleep13520-21df_inode1
3013752101,2sleep00-21swapper/00
2978122101,2sleep10-21swapper/11
2892542101,4sleep219034-21diskmemload2
248662101,4sleep119034-21diskmemload1
2183322101,4sleep219034-21diskmemload2
1999572101,3sleep10-21swapper/11
1663352101,5sleep0166349-21ssh0
1654942101,3sleep20-21swapper/22
1569502101,6sleep0156963-21sh0
1498682101,3sleep00-21swapper/00
1375922101,4sleep2137651-21ntpq2
1305722100,1sleep00-21swapper/00
1131932101,2sleep10-21swapper/11
103452101,5sleep012149-21cut0
4184821291,4sleep14184942-21ntpq1
4153880291,3sleep24153899-21apt-get2
4132512291,5sleep00-21swapper/00
4129276291,4sleep24131027-21sed2
4109794291,4sleep04109938-21mailstats0
4087393291,4sleep04087557-21munin-node0
3800293,3sleep04624-21kthreadcore0
36895290,1sleep20-21swapper/22
336142291,4sleep2336364-21hddtemp_smartct2
327753291,4sleep2329256-21ps2
314770291,4sleep0314784-21ssh0
2888290,1sleep10-21swapper/11
254784291,3sleep019034-21diskmemload0
227285291,2sleep00-21swapper/00
185877290,3sleep00-21swapper/00
177176291,4sleep0177193-21ssh0
86158281,3sleep086168-21ssh0
4191828281,2sleep04191850-21apt-get0
4186229281,3sleep14186444-21cat1
4169894280,2sleep10-21swapper/11
4153606280,1sleep10-21swapper/11
4149051281,3sleep20-21swapper/22
4125983281,3sleep14126125-21mailstats1
4125983281,3sleep14126125-21mailstats1
4125832280,3sleep24125870-21ntpq2
4125832280,3sleep24125870-21ntpq2
4094996280,2sleep20-21swapper/22
8302270,1sleep20-21swapper/22
4191199270,1sleep20-21swapper/22
4185936271,3sleep00-21swapper/00
4169435270,1sleep10-21swapper/11
4164563270,2sleep20-21swapper/22
4155886271,2sleep00-21swapper/00
4148221270,1sleep00-21swapper/00
4143219271,2sleep20-21swapper/22
4138135271,3sleep10-21swapper/11
4121170270,1sleep00-21swapper/00
4094380270,4sleep10-21swapper/11
13684271,3sleep213918-21latency_hist2
4191796260,2sleep20-21swapper/22
4181421261,2sleep20-21swapper/22
4179949260,1sleep10-21swapper/11
4179943260,4sleep00-21swapper/00
4164036260,1sleep10-21swapper/11
4159504261,2sleep00-21swapper/00
4158316260,3sleep10-21swapper/11
4153620261,2sleep04110469-21kworker/u16:1+kvfree_rcu_reclaim0
4105057260,1sleep10-21swapper/11
4100861260,2sleep20-21swapper/22
4099843260,4sleep10-21swapper/11
4086086260,2sleep24086341-21strings2
18904260,1sleep00-21swapper/00
4169618250,2sleep00-21swapper/00
4105354251,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional