You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-06 - 15:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Wed May 06, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
197422212057,59sleep30-21swapper/33
1976512111103,5sleep10-21swapper/11
19756228264,12sleep20-21swapper/22
19749828262,14sleep00-21swapper/00
19778199260,13cyclictest0-21swapper/33
19778199221,17cyclictest512983-21chrt3
19778199221,17cyclictest462878-21taskset3
19778199211,16cyclictest0-21swapper/33
4971192201,14sleep1329496-21diskmemload1
197781992017,1cyclictest0-21swapper/33
19778199201,6cyclictest215831-21taskset3
19778199201,16cyclictest673017-21chrt3
19778199201,16cyclictest673017-21chrt3
19778199201,16cyclictest0-21swapper/33
19778199201,15cyclictest622318-21taskset3
19778199201,15cyclictest5873882sleep33
19778199201,15cyclictest264795-21turbostat3
19778199201,15cyclictest0-21swapper/33
19778199201,15cyclictest0-21swapper/33
19778199201,15cyclictest0-21swapper/33
19778199194,13cyclictest0-21swapper/33
19778199191,15cyclictest0-21swapper/33
19778199191,15cyclictest0-21swapper/33
19778199191,15cyclictest0-21swapper/33
19778199191,15cyclictest0-21swapper/33
19778199191,15cyclictest0-21swapper/33
19778199191,14cyclictest0-21swapper/33
19778199191,14cyclictest0-21swapper/33
19778199191,14cyclictest0-21swapper/33
19778199191,14cyclictest0-21swapper/33
19778199191,14cyclictest0-21swapper/33
19778199184,11cyclictest0-21swapper/33
197781991812,4cyclictest0-21swapper/33
197781991812,3cyclictest0-21swapper/33
197781991812,3cyclictest0-21swapper/33
19778199181,14cyclictest529650-21taskset3
19778199181,14cyclictest0-21swapper/33
19778199181,14cyclictest0-21swapper/33
19778199181,14cyclictest0-21swapper/33
19778199181,14cyclictest0-21swapper/33
19778199181,14cyclictest0-21swapper/33
19778199181,14cyclictest0-21swapper/33
19778199181,14cyclictest0-21swapper/33
19778199181,14cyclictest0-21swapper/33
19778199181,14cyclictest0-21swapper/33
19778199181,13cyclictest0-21swapper/33
19778199181,13cyclictest0-21swapper/33
197781991810,6cyclictest547661-21taskset3
5714612171,2sleep00-21swapper/00
5245702170,2sleep10-21swapper/11
2221212171,6sleep1222359-21grep1
19778199177,6cyclictest253811-21turbostat3
19778199172,10cyclictest465119-21taskset3
197781991711,4cyclictest0-21swapper/33
19778199171,13cyclictest0-21swapper/33
19778199171,13cyclictest0-21swapper/33
19778199171,13cyclictest0-21swapper/33
19778199171,12cyclictest286153-21turbostat3
197781991710,4cyclictest0-21swapper/33
19778199170,13cyclictest0-21swapper/33
19778199170,11cyclictest0-21swapper/33
5395232161,2sleep20-21swapper/22
5203032161,3sleep20-21swapper/22
3786482161,2sleep20-21swapper/22
3683622161,12sleep0364019-21/usr/sbin/munin0
19778199169,2cyclictest0-21swapper/33
197781991614,1cyclictest0-21swapper/33
19778199161,13cyclictest0-21swapper/33
19778199160,14cyclictest0-21swapper/33
19778199160,13cyclictest0-21swapper/33
6048182151,2sleep00-21swapper/00
5643512151,3sleep20-21swapper/22
5390242151,4sleep0329496-21diskmemload0
5114622151,3sleep20-21swapper/22
4467242151,6sleep1446739-21ssh1
3543792151,2sleep00-21swapper/00
3437832151,4sleep0343797-21ssh0
19778199153,9cyclictest0-21swapper/33
197781991512,2cyclictest0-21swapper/33
19778199150,13cyclictest0-21swapper/33
19778199150,12cyclictest0-21swapper/33
19778199150,10cyclictest0-21swapper/33
6812402141,3sleep1329496-21diskmemload1
6807182141,3sleep0329496-21diskmemload0
6734622141,2sleep20-21swapper/22
6734622141,2sleep20-21swapper/22
6082702141,7sleep172-21kcompactd01
5542092141,5sleep2554223-21ssh2
5046412141,3sleep2504655-21ssh2
5046412141,3sleep2504655-21ssh2
5037362141,4sleep172-21kcompactd01
5037362141,4sleep172-21kcompactd01
4568672141,4sleep0457084-21diskstats0
4384152142,2sleep1171rcu_preempt1
3524642142,2sleep2329496-21diskmemload2
3433652141,2sleep10-21swapper/11
2493092140,6sleep0250533-21cut0
2319712141,4sleep1231967-21gdbus1
19778199148,3cyclictest291122-21chrt3
19778199148,3cyclictest2422172sleep33
19778199148,3cyclictest2029602sleep33
19778199141,9cyclictest205330-21chrt3
19778199141,2cyclictest0-21swapper/33
19778199141,2cyclictest0-21swapper/33
197781991411,2cyclictest0-21swapper/33
6659952131,8sleep2666006-21ssh2
6482962131,8sleep2648307-21sh2
6316672131,8sleep0631679-21ssh0
6296072131,7sleep1629663-21ntp_states1
6220822130,3sleep0329496-21diskmemload0
5890242131,6sleep2329496-21diskmemload2
5568512131,3sleep1556867-21ssh1
5478892131,6sleep20-21swapper/22
5158832131,9sleep10-21swapper/11
5138922131,8sleep1513905-21ssh1
4641572131,2sleep20-21swapper/22
4398862131,4sleep2439899-21sh2
3623922134,1sleep0171rcu_preempt0
2743382131,7sleep0274444-21ntpq0
2490272131,8sleep2249203-21grep2
2103782131,3sleep0584-21gdbus0
6753172120,6sleep10-21swapper/11
6753172120,6sleep10-21swapper/11
6566452121,6sleep0329496-21diskmemload0
4863902124,4sleep10-21swapper/11
4475802121,5sleep00-21swapper/00
4196862121,7sleep2419697-21ssh2
4129472121,7sleep0412959-21rm0
4129472121,7sleep0412959-21rm0
3976532121,7sleep1397667-21ssh1
3816382121,7sleep2382901-21cut2
3712972121,7sleep1329496-21diskmemload1
3702982121,7sleep2370309-21ssh2
3628612121,7sleep20-21swapper/22
3075552121,7sleep2307702-21apt-config2
2851272121,5sleep2285176-21ntpq2
2717952121,3sleep20-21swapper/22
6835822111,5sleep2329496-21diskmemload2
6555532111,4sleep2329496-21diskmemload2
6476172111,3sleep00-21swapper/00
6342592111,6sleep2635000-21cat2
6325332111,4sleep2329496-21diskmemload2
6245712111,5sleep1329496-21diskmemload1
6057272111,4sleep2329496-21diskmemload2
5906692111,4sleep0329496-21diskmemload0
5817872111,5sleep2329496-21diskmemload2
5815802111,5sleep1331ktimers/11
5802032111,6sleep0580213-21ssh0
5731882112,5sleep2802-21snmpd2
5649742111,4sleep1329496-21diskmemload1
5565512110,2sleep00-21swapper/00
5362692111,5sleep1536335-21ntp_states1
4947562111,5sleep0494969-21users0
4797762111,4sleep2329496-21diskmemload2
4556242111,4sleep1329496-21diskmemload1
4544542111,3sleep00-21swapper/00
4389442111,3sleep20-21swapper/22
4274272111,5sleep1427482-21ntpq1
4216932111,5sleep0329496-21diskmemload0
4049402110,2sleep0404951-21ssh0
4048142111,4sleep20-21swapper/22
3780142111,5sleep0329496-21diskmemload0
3350552110,5sleep0329496-21diskmemload0
3340572111,4sleep2334114-21ntpq2
3228602111,5sleep0322906-21ntpq0
2701132110,2sleep1270144-21perf1
2701132110,2sleep1270144-21perf1
2474732111,5sleep1247520-21ntpq1
2425442111,6sleep0242776-21unixbench_singl0
2159192110,1sleep10-21swapper/11
6648072101,4sleep1329496-21diskmemload1
6643672101,3sleep00-21swapper/00
6555402101,3sleep1329496-21diskmemload1
6420882101,4sleep1642109-21apt-get1
6379462101,4sleep0638009-21ntp_states0
6229442101,4sleep2329496-21diskmemload2
6159652101,4sleep0329496-21diskmemload0
6142972101,4sleep2329496-21diskmemload2
6073612101,3sleep10-21swapper/11
5989952101,3sleep0329496-21diskmemload0
5913302101,4sleep1591592-21ssh1
5895322100,6sleep1329496-21diskmemload1
5614662101,4sleep0561526-21ntpq0
5472702101,4sleep0329496-21diskmemload0
5457982101,3sleep10-21swapper/11
5315072101,3sleep20-21swapper/22
5288412101,3sleep0329496-21diskmemload0
5034492100,5sleep0503451-21ssh0
5034492100,5sleep0503451-21ssh0
4873632101,3sleep20-21swapper/22
4818782101,4sleep0329496-21diskmemload0
4706312101,4sleep2329496-21diskmemload2
4702172100,2sleep10-21swapper/11
4540872101,3sleep20-21swapper/22
4316652101,4sleep0431691-21apt-get0
4291392101,4sleep0329496-21diskmemload0
4219872101,3sleep10-21swapper/11
4141622101,3sleep20-21swapper/22
4141622101,3sleep20-21swapper/22
4057422101,3sleep172-21kcompactd01
4057422101,3sleep172-21kcompactd01
3960862100,1sleep10-21swapper/11
3893912101,2sleep20-21swapper/22
3623532101,3sleep10-21swapper/11
3463192101,3sleep1329496-21diskmemload1
3460902100,1sleep20-21swapper/22
3246742101,5sleep1324883-21hddtemp_smartct1
3232772101,4sleep2323474-21users2
2917912101,4sleep2292029-21df_inode2
2763212101,6sleep0277824-21/usr/sbin/munin0
2528292101,4sleep1252883-21ntpq1
2431642101,5sleep2243389-21latency_hist2
2202952101,4sleep2220343-21ntpq2
2109102101,4sleep2210389-21apt-get2
639863291,3sleep1329496-21diskmemload1
596933291,3sleep2329496-21diskmemload2
565957290,4sleep1566180-21hddtemp_smartct1
506641290,2sleep00-21swapper/00
495741291,3sleep2329496-21diskmemload2
479204291,4sleep1479218-21ssh1
473564291,3sleep0329496-21diskmemload0
470063291,4sleep0470074-21ssh0
461300291,3sleep10-21swapper/11
428643291,2sleep20-21swapper/22
392162291,3sleep0329496-21diskmemload0
387730291,3sleep10-21swapper/11
381337291,4sleep0381554-21hddtemp_smartct0
377785291,4sleep1377797-21ssh1
336257291,3sleep1329496-21diskmemload1
323887291,4sleep00-21swapper/00
298409291,4sleep0300147-21tr0
296829291,4sleep1297088-21cat1
264784291,4sleep2555-21dbus-daemon2
259251291,3sleep0259171-21apt-get0
258003291,4sleep2258143-21ntp_192.168.1142
227495291,4sleep2227708-21hddtemp_smartct2
216275291,3sleep1216294-21apt-get1
673032281,3sleep0673036-21ssh0
673032281,3sleep0673036-21ssh0
520461281,3sleep0329496-21diskmemload0
324384280,3sleep2324403-21apt-get2
313448281,2sleep00-21swapper/00
306008281,4sleep0306499-21needreboot0
290462280,1sleep20-21swapper/22
286402280,2sleep10-21swapper/11
281339281,4sleep0281443-21fschecks_time0
275459280,1sleep10-21swapper/11
265042281,2sleep00-21swapper/00
265042281,2sleep00-21swapper/00
260201281,3sleep1260281-21kernelversion1
259135283,3sleep1331ktimers/11
244754281,3sleep20-21swapper/22
227196280,2sleep00-21swapper/00
225683280,4sleep2225734-21ld2
221081280,1sleep00-21swapper/00
323521270,1sleep10-21swapper/11
318480270,1sleep20-21swapper/22
318244270,2sleep10-21swapper/11
308661271,2sleep20-21swapper/22
308077270,2sleep10-21swapper/11
307528270,1sleep10-21swapper/11
301291270,1sleep10-21swapper/11
286043270,1sleep10-21swapper/11
255089271,3sleep00-21swapper/00
242920270,1sleep10-21swapper/11
237588270,1sleep10-21swapper/11
237584270,1sleep20-21swapper/22
205307270,2sleep20-21swapper/22
202931270,1sleep20-21swapper/22
202735270,4sleep10-21swapper/11
313113260,1sleep00-21swapper/00
275738260,1sleep20-21swapper/22
269426261,2sleep2265043-21/usr/sbin/munin2
269426261,2sleep2265043-21/usr/sbin/munin2
248186260,4sleep00-21swapper/00
233503261,2sleep00-21swapper/00
226631260,1sleep00-21swapper/00
215660260,1sleep00-21swapper/00
209216260,3sleep10-21swapper/11
302139250,1sleep20-21swapper/22
291745250,1sleep00-21swapper/00
291477250,3sleep00-21swapper/00
270933250,2sleep10-21swapper/11
201624250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional