You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 03:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Tue Mar 24, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14902722260,2sleep2149018-21ntpq2
3899901213678,54sleep30-21swapper/33
390002429260,12sleep20-21swapper/22
389988128151,12sleep10-21swapper/11
390002227556,13sleep00-21swapper/00
1521302291,2sleep00-21swapper/00
41018912281,7sleep04102371-21cut0
3900325992623,2cyclictest0-21swapper/33
39617762240,4sleep03961780-21fschecks_time0
39258962240,4sleep1321rcuc/11
390032599243,17cyclictest66950irq/132-enp1s0-TxRx-33
3900325992421,1cyclictest0-21swapper/33
3900325992321,1cyclictest0-21swapper/33
3900325992220,1cyclictest0-21swapper/33
3900325992218,2cyclictest0-21swapper/33
3900325992212,7cyclictest0-21swapper/33
257812221,5sleep025841-21ntpq0
3900325992119,1cyclictest0-21swapper/33
390032599211,3cyclictest0-21swapper/33
3900325992111,7cyclictest0-21swapper/33
390032599211,16cyclictest66950irq/132-enp1s0-TxRx-33
390032599210,18cyclictest0-21swapper/33
1147852211,4sleep14030695-21diskmemload1
3900325992018,1cyclictest0-21swapper/33
3900325992017,1cyclictest0-21swapper/33
3900325992016,2cyclictest0-21swapper/33
3900325992016,2cyclictest0-21swapper/33
3900325992015,2cyclictest0-21swapper/33
3900325992015,2cyclictest0-21swapper/33
390032599200,15cyclictest41337612chrt3
384962201,7sleep14030695-21diskmemload1
390032599195,12cyclictest19784-21taskset3
3900325991917,1cyclictest0-21swapper/33
3900325991917,1cyclictest0-21swapper/33
3900325991917,1cyclictest0-21swapper/33
3900325991917,1cyclictest0-21swapper/33
3900325991917,1cyclictest0-21swapper/33
3900325991917,1cyclictest0-21swapper/33
3900325991916,2cyclictest0-21swapper/33
3900325991916,2cyclictest0-21swapper/33
3900325991915,2cyclictest0-21swapper/33
3900325991915,2cyclictest0-21swapper/33
3900325991915,2cyclictest0-21swapper/33
3900325991915,2cyclictest0-21swapper/33
3900325991912,4cyclictest0-21swapper/33
390032599191,15cyclictest90568-21taskset3
390032599191,14cyclictest0-21swapper/33
390032599191,14cyclictest0-21swapper/33
14382190,2sleep10-21swapper/11
108402191,7sleep04030695-21diskmemload0
41601882182,4sleep03900320-21cyclictest0
41066912181,3sleep20-21swapper/22
3900325991816,1cyclictest0-21swapper/33
3900325991816,1cyclictest0-21swapper/33
3900325991816,1cyclictest0-21swapper/33
3900325991816,1cyclictest0-21swapper/33
3900325991815,1cyclictest0-21swapper/33
3900325991815,1cyclictest0-21swapper/33
3900325991814,2cyclictest0-21swapper/33
3900325991812,3cyclictest0-21swapper/33
390032599181,14cyclictest0-21swapper/33
390032599181,14cyclictest0-21swapper/33
390032599181,13cyclictest0-21swapper/33
390032599181,13cyclictest0-21swapper/33
390032599180,13cyclictest0-21swapper/33
1753562181,6sleep1175420-21ntpq1
1002812181,5sleep24030695-21diskmemload2
917782171,4sleep14030695-21diskmemload1
899732171,3sleep20-21swapper/22
732262171,3sleep00-21swapper/00
41709322171,8sleep14030695-21diskmemload1
41709322171,8sleep14030695-21diskmemload1
40895302171,9sleep00-21swapper/00
3900325991715,1cyclictest0-21swapper/33
3900325991715,1cyclictest0-21swapper/33
3900325991714,1cyclictest0-21swapper/33
390032599171,13cyclictest0-21swapper/33
390032599171,13cyclictest0-21swapper/33
390032599171,12cyclictest0-21swapper/33
390032599171,12cyclictest0-21swapper/33
1960042171,3sleep20-21swapper/22
12862170,1sleep20-21swapper/22
1271162171,11sleep2127284-21ls2
1163032171,8sleep0116317-21ssh0
808032161,4sleep24030695-21diskmemload2
41448682161,10sleep24030695-21diskmemload2
41349622160,1sleep20-21swapper/22
40663662161,4sleep14066397-21ssh1
390032599164,9cyclictest3998701-21turbostat3
390032599162,12cyclictest0-21swapper/33
3900325991614,1cyclictest0-21swapper/33
3900325991614,1cyclictest0-21swapper/33
2048302161,5sleep14030695-21diskmemload1
1860272161,4sleep24030695-21diskmemload2
795252151,8sleep14030695-21diskmemload1
611862151,4sleep14030695-21diskmemload1
41680692151,10sleep04168083-21ssh0
41680692151,10sleep04168083-21ssh0
41237622158,4sleep24123783-21ssh2
41007132151,9sleep14030695-21diskmemload1
40879002151,9sleep24088014-21ntpq2
40400362151,9sleep04040068-21ssh0
3900325991512,1cyclictest0-21swapper/33
390032599150,13cyclictest0-21swapper/33
390032599150,13cyclictest0-21swapper/33
390032599150,12cyclictest0-21swapper/33
390032599150,12cyclictest0-21swapper/33
390032599150,11cyclictest0-21swapper/33
1685422151,2sleep20-21swapper/22
1685422151,2sleep20-21swapper/22
1611642150,3sleep20-21swapper/22
1239402151,8sleep14030695-21diskmemload1
1233662151,9sleep00-21swapper/00
550642141,4sleep055074-21ssh0
41693742141,2sleep20-21swapper/22
41693742141,2sleep20-21swapper/22
40713972141,8sleep04030695-21diskmemload0
390032599146,6cyclictest0-21swapper/33
3900325991412,1cyclictest0-21swapper/33
1713742141,6sleep2171596-21hddtemp_smartct2
890592130,4sleep10-21swapper/11
564542131,8sleep14030695-21diskmemload1
41442432131,8sleep14144256-21sh1
40533662131,3sleep04030695-21diskmemload0
295052131,3sleep129515-21ssh1
1840582130,2sleep10-21swapper/11
1627162131,8sleep1162881-21interrupts1
1627162131,8sleep1162881-21interrupts1
1312582131,7sleep1131316-21ntpq1
823072121,7sleep082324-21ssh0
629192121,7sleep24030695-21diskmemload2
448812121,4sleep10-21swapper/11
41765632121,6sleep04176822-21users0
41544792121,7sleep14154705-21ssh1
41361562121,7sleep04136172-21ssh0
41356712121,7sleep14030695-21diskmemload1
41097322121,7sleep14109942-21ssh1
40967412121,5sleep24096793-21ntpq2
40764662121,3sleep04078155-21cut0
40659682122,5sleep14065996-21munin-run1
40625612121,7sleep24030695-21diskmemload2
40555642121,7sleep14030695-21diskmemload1
40477792120,4sleep14047793-21ssh1
40327122121,7sleep04034409-21kthreadcore0
39924242121,6sleep03992477-21ntp_states0
39619622121,7sleep13962602-21cat1
39559642121,5sleep13955519-21apt-get1
1508722120,7sleep14030695-21diskmemload1
1421722121,7sleep0142184-21ssh0
728972110,7sleep272910-21ssh2
641822111,5sleep00-21swapper/00
41881402111,3sleep00-21swapper/00
41537252111,3sleep10-21swapper/11
41520112111,2sleep20-21swapper/22
41453452111,3sleep00-21swapper/00
41256932112,4sleep14030695-21diskmemload1
41163792111,3sleep20-21swapper/22
40975722110,1sleep00-21swapper/00
40889282111,6sleep14030695-21diskmemload1
40753772111,5sleep14075512-21perl1
40723832111,3sleep20-21swapper/22
40643222111,7sleep00-21swapper/00
40565952111,3sleep20-21swapper/22
40464362110,3sleep20-21swapper/22
40358132110,3sleep24035957-21ssh2
40203252111,5sleep14020583-21df_inode1
39976872111,5sleep23997736-21ntpq2
397472111,6sleep239842-21fschecks_time2
39185452111,5sleep23918798-21df2
39182642111,5sleep23918312-21munin-run2
373232111,3sleep00-21swapper/00
305682111,3sleep20-21swapper/22
1961812111,3sleep00-21swapper/00
194012111,4sleep14030695-21diskmemload1
181522111,4sleep24030695-21diskmemload2
1798722111,5sleep0179904-21ssh0
171022111,4sleep00-21swapper/00
1685162111,5sleep0150520-21kworker/0:2+events0
1685162111,5sleep0150520-21kworker/0:2+events0
1577162111,5sleep0157776-21ntpq0
1086992111,4sleep14030695-21diskmemload1
977462101,2sleep00-21swapper/00
974162101,2sleep20-21swapper/22
893282101,3sleep00-21swapper/00
661652101,4sleep166246-21irqstats1
41863982101,3sleep20-21swapper/22
41840762101,3sleep14030695-21diskmemload1
41785692101,4sleep14030695-21diskmemload1
41778672101,2sleep20-21swapper/22
41601042101,3sleep20-21swapper/22
41142922101,4sleep04114355-21ntpq0
41012152101,3sleep14030695-21diskmemload1
40807892100,2sleep20-21swapper/22
40310102101,3sleep14030695-21diskmemload1
40259062101,4sleep14026115-21hddtemp_smartct1
40258102101,4sleep04025905-21diskstats0
40200192101,4sleep04020091-21munin-run0
40166762101,5sleep24018481-21tr2
40093692101,5sleep24009386-21latency_hist2
39822992101,5sleep0255-21systemd-journal0
39760492101,5sleep03976089-21ntpq0
39510302101,6sleep03951975-21cat0
219172101,6sleep222111-21gltestperf2
1973162101,5sleep2197350-21ssh2
1969122101,4sleep1196970-21munin-run1
1760712100,2sleep00-21swapper/00
1603282101,3sleep10-21swapper/11
1405372101,4sleep14030695-21diskmemload1
1364202101,3sleep20-21swapper/22
1328722101,2sleep00-21swapper/00
1143932101,4sleep24030695-21diskmemload2
1060002101,3sleep00-21swapper/00
9536291,2sleep10-21swapper/11
804290,1sleep00-21swapper/00
53555291,3sleep2431ktimers/22
4629291,2sleep20-21swapper/22
4140643291,5sleep04140767-21ssh0
4121119291,3sleep00-21swapper/00
4030685291,4sleep24030722-21ssh2
4020584291,4sleep24020689-21grep2
4015521291,4sleep14016675-21cat1
4009909291,4sleep14009982-21fschecks_time1
4008353291,4sleep04008406-21ntpq0
4005676291,5sleep10-21swapper/11
4000522291,4sleep24002316-21cut2
3998576290,2sleep10-21swapper/11
3987390291,4sleep13987647-21grep1
3972736291,5sleep13974415-21kthreadcore1
3966987291,4sleep03967111-21perl0
3956641292,3sleep03957763-21cat0
3939878291,4sleep03940096-21df_inode0
3938563291,4sleep23938617-21ntpq2
3929685291,4sleep03929952-21grep0
3923442290,2sleep10-21swapper/11
3910602291,4sleep23911726-21memory2
3907844291,3sleep00-21swapper/00
3905573291,4sleep03905782-21wc0
203816291,2sleep00-21swapper/00
124654291,4sleep2124665-21ssh2
44420281,3sleep044431-21ssh0
4020304281,2sleep00-21swapper/00
3982425281,3sleep23982435-21apt-get2
3966702280,1sleep10-21swapper/11
3956360281,4sleep23956550-21grep2
3945139280,3sleep03945157-21apt-get0
3943970281,4sleep13944040-21head1
3918087280,1sleep00-21swapper/00
3908894281,3sleep13910598-21kthreadcore1
3901054281,3sleep23902564-21grep2
4010797271,3sleep00-21swapper/00
3999538271,3sleep14000521-21cat1
3993278270,1sleep10-21swapper/11
3988308270,2sleep20-21swapper/22
3987911270,1sleep20-21swapper/22
3982265270,1sleep10-21swapper/11
3967263271,2sleep20-21swapper/22
3962603270,3sleep23963477-21kthreadcore2
3955161270,1sleep20-21swapper/22
3951976271,2sleep13953776-21kthreadcore1
3950003270,1sleep20-21swapper/22
3949850270,1sleep10-21swapper/11
3934484270,2sleep10-21swapper/11
3929953271,3sleep13931659-21kthreadcore1
3900473271,2sleep13900492-21apt-get1
4014355260,1sleep20-21swapper/22
3984825261,2sleep00-21swapper/00
3944548260,1sleep20-21swapper/22
3933266261,2sleep20-21swapper/22
3928010261,2sleep20-21swapper/22
3924425261,2sleep00-21swapper/00
3998358250,1sleep00-21swapper/00
3976146250,1sleep20-21swapper/22
3938324250,1sleep00-21swapper/00
3918849250,2sleep00-21swapper/00
3913524250,1sleep10-21swapper/11
3998691240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional