You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 01:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Tue Aug 25, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
126654121700,2sleep190-21kswapd01
1102760212453,66sleep30-21swapper/33
110259629073,10sleep20-21swapper/22
110263928062,12sleep10-21swapper/11
110278127353,14sleep00-21swapper/00
13009802380,3sleep10-21swapper/11
110306099271,22cyclictest15106802sleep33
1103060992511,2cyclictest0-21swapper/33
110306099241,2cyclictest0-21swapper/33
110306099240,12cyclictest0-21swapper/33
14472342230,2sleep30-21swapper/33
1103060992320,1cyclictest0-21swapper/33
110306099231,10cyclictest66950irq/132-enp1s0-TxRx-33
110306099223,14cyclictest1406956-21taskset3
110306099220,1cyclictest0-21swapper/33
14991982211,8sleep11499442-21grep1
13924772211,15sleep01393407-21memory0
110306099213,7cyclictest0-21swapper/33
110306099211,14cyclictest0-21swapper/33
110306099210,17cyclictest0-21swapper/33
14498612201,10sleep11449868-21ssh1
110306099202,14cyclictest1194242-21turbostat3
110306099202,14cyclictest1194242-21turbostat3
110306099201,15cyclictest0-21swapper/33
110306099201,15cyclictest0-21swapper/33
13892432191,8sleep21388982-21sendmail-msp2
1103060991916,1cyclictest0-21swapper/33
1103060991913,4cyclictest0-21swapper/33
1103060991911,5cyclictest0-21swapper/33
110306099191,15cyclictest0-21swapper/33
110306099191,15cyclictest0-21swapper/33
110306099191,14cyclictest0-21swapper/33
110306099191,14cyclictest0-21swapper/33
110306099191,14cyclictest0-21swapper/33
110306099191,14cyclictest0-21swapper/33
110306099191,14cyclictest0-21swapper/33
110306099191,14cyclictest0-21swapper/33
110306099191,13cyclictest1561860-21chrt3
110306099190,16cyclictest0-21swapper/33
110306099190,15cyclictest0-21swapper/33
11032622181,6sleep11103515-21df1
1103060991814,2cyclictest0-21swapper/33
1103060991812,4cyclictest0-21swapper/33
1103060991812,3cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,14cyclictest0-21swapper/33
110306099181,13cyclictest0-21swapper/33
110306099181,13cyclictest0-21swapper/33
110306099181,13cyclictest0-21swapper/33
110306099181,13cyclictest0-21swapper/33
110306099180,14cyclictest0-21swapper/33
110306099180,14cyclictest0-21swapper/33
110306099180,14cyclictest0-21swapper/33
15793312172,4sleep11579328-21apt-get1
15618482171,10sleep21231082-21diskmemload2
14931162171,6sleep01493131-21ssh0
14683442171,4sleep21468372-21ssh2
14513602171,11sleep21451562-21grep2
13692182171,3sleep10-21swapper/11
12208372171,7sleep21220584-21sendmail-msp2
11113252171,11sleep11112923-21cut1
110306099171,4cyclictest0-21swapper/33
1103060991712,4cyclictest66950irq/132-enp1s0-TxRx-33
110306099171,13cyclictest0-21swapper/33
15847662161,2sleep00-21swapper/00
15767082160,3sleep00-21swapper/00
14922522161,2sleep10-21swapper/11
14298402161,10sleep21231082-21diskmemload2
14111492161,3sleep00-21swapper/00
14055752161,4sleep21231082-21diskmemload2
14047782161,4sleep11231082-21diskmemload1
13876012161,4sleep11231082-21diskmemload1
13436332161,2sleep00-21swapper/00
13022522160,8sleep134-21ksoftirqd/11
1103060991611,3cyclictest0-21swapper/33
110306099161,11cyclictest0-21swapper/33
110306099160,14cyclictest0-21swapper/33
110306099160,13cyclictest0-21swapper/33
110306099160,13cyclictest0-21swapper/33
15390052150,3sleep20-21swapper/22
14604462151,5sleep11461971-21cut1
13357602151,3sleep01231082-21diskmemload0
12834842150,3sleep11283497-21ssh1
12824002151,2sleep20-21swapper/22
12377182151,5sleep21237721-21ssh2
12237772151,4sleep01224754-21memory0
12044522151,4sleep0205-21jbd2/sda2-80
1103060991512,1cyclictest0-21swapper/33
110306099151,1cyclictest0-21swapper/33
110306099150,13cyclictest0-21swapper/33
110306099150,12cyclictest0-21swapper/33
15579882141,3sleep20-21swapper/22
14586712141,3sleep11231082-21diskmemload1
14150852141,4sleep21415098-21sh2
12932112141,9sleep01293385-21/usr/sbin/munin0
12372302140,2sleep10-21swapper/11
11688972141,3sleep11170514-21cut1
110306099148,3cyclictest1136362-21chrt3
1103060991410,2cyclictest0-21swapper/33
110306099140,12cyclictest0-21swapper/33
15753432131,8sleep21575358-21ssh2
15306052131,7sleep21231082-21diskmemload2
15301192131,8sleep01530133-21ssh0
14383362131,6sleep00-21swapper/00
14302892131,4sleep11430303-21ssh1
13875602131,8sleep21387573-21ssh2
13797772131,7sleep01231082-21diskmemload0
13797772131,7sleep01231082-21diskmemload0
13329622131,5sleep10-21swapper/11
12748222131,8sleep01274836-21ssh0
12461072131,7sleep21231082-21diskmemload2
12155802131,8sleep20-21swapper/22
11472882131,2sleep00-21swapper/00
110306099139,2cyclictest0-21swapper/33
110306099130,11cyclictest0-21swapper/33
110306099130,10cyclictest1230788-21chrt3
15850972121,7sleep21585108-21ssh2
15463302121,7sleep11546344-21ssh1
15285702121,3sleep10-21swapper/11
15155862121,6sleep01231082-21diskmemload0
15109722121,7sleep11231082-21diskmemload1
15068262121,7sleep11231082-21diskmemload1
15016832121,3sleep20-21swapper/22
14652872121,7sleep21465298-21ssh2
13866472125,2sleep01231082-21diskmemload0
13681062121,6sleep01231082-21diskmemload0
13619252121,6sleep01361939-21ssh0
13453952121,5sleep01345429-21ssh0
13434912121,5sleep10-21swapper/11
13266942121,5sleep00-21swapper/00
13092512120,6sleep01231082-21diskmemload0
12980862120,7sleep21231082-21diskmemload2
12840602122,4sleep01231082-21diskmemload0
12659272121,6sleep01231082-21diskmemload0
12577322121,4sleep10-21swapper/11
12410052121,6sleep11242535-21grep1
11944992121,5sleep21194518-21apt-get2
11260052121,5sleep11126064-21munin-run1
11196862121,1sleep21119799-21strings2
11105102121,5sleep00-21swapper/00
15669712111,7sleep11231082-21diskmemload1
15576172110,3sleep10-21swapper/11
15510802111,3sleep20-21swapper/22
15488752111,3sleep00-21swapper/00
15461832111,4sleep21231082-21diskmemload2
15176802111,6sleep21231082-21diskmemload2
15168422111,4sleep11231082-21diskmemload1
15009842111,4sleep01231082-21diskmemload0
14839812111,5sleep01231082-21diskmemload0
14685292111,5sleep01468665-21diskstats0
14658302110,7sleep01465844-21ssh0
14156082111,6sleep21415625-21sshd-auth2
13898762111,5sleep11389959-21irqstats1
13771792111,2sleep1171rcu_preempt1
13771792111,2sleep1171rcu_preempt1
13103902111,6sleep21310658-21df2
13081232110,5sleep20-21swapper/22
12812322111,1sleep00-21swapper/00
12722582111,3sleep20-21swapper/22
12650702111,3sleep20-21swapper/22
12364942111,4sleep01231082-21diskmemload0
12052592111,5sleep21205382-21diskstats2
11998212111,5sleep21199535-21sendmail-msp2
11938982110,2sleep00-21swapper/00
11938982110,2sleep00-21swapper/00
11734722111,3sleep00-21swapper/00
11396172111,5sleep11140678-21memory1
15714022101,5sleep11572947-21ssh1
15582312101,2sleep00-21swapper/00
15487352100,2sleep10-21swapper/11
15400692100,2sleep0171rcu_preempt0
15397582101,5sleep01539799-21/usr/sbin/munin0
15390442101,4sleep11539058-21ssh1
15095062101,3sleep00-21swapper/00
15044422100,1sleep20-21swapper/22
14935982101,3sleep21231082-21diskmemload2
14843472101,5sleep11484362-21ssh1
14682212101,2sleep10-21swapper/11
14571722101,4sleep01231082-21diskmemload0
14476702100,2sleep00-21swapper/00
14472222100,3sleep20-21swapper/22
14386252100,2sleep20-21swapper/22
14384552100,1sleep10-21swapper/11
14172332101,4sleep01418884-21cut0
14151282101,3sleep10-21swapper/11
13681332101,3sleep21231082-21diskmemload2
13625712101,5sleep11362614-21awk1
13519642101,3sleep21231082-21diskmemload2
13508672101,3sleep10-21swapper/11
13415752101,5sleep21341586-21ssh2
13352192100,1sleep20-21swapper/22
13264122101,3sleep20-21swapper/22
13177062101,4sleep11231082-21diskmemload1
12922682101,2sleep10-21swapper/11
12920822102,4sleep21292085-21ssh2
12561562101,2sleep10-21swapper/11
12540222100,1sleep20-21swapper/22
12521292101,5sleep01252999-21memory0
12258362100,1sleep10-21swapper/11
12050932101,3sleep00-21swapper/00
11998062101,4sleep11199818-21apt-get1
11947852100,1sleep10-21swapper/11
11887172101,3sleep172-21kcompactd01
11776342101,5sleep21177783-21awk2
11679422101,5sleep01168126-21grep0
11630272101,5sleep21163090-21cpuspeed_turbos2
11535852100,2sleep10-21swapper/11
11469222101,5sleep11146925-21sshd-auth1
11157522101,4sleep11115520-21sendmail-msp1
11032472101,4sleep01103259-21apt-get0
1566859291,2sleep00-21swapper/00
1509060291,2sleep20-21swapper/22
1504041291,4sleep01504055-21ssh0
1477137291,4sleep21477296-21wc2
1426999291,4sleep01428444-21latency0
1421969291,4sleep11421984-21sshd-auth1
1398141290,3sleep00-21swapper/00
1378763290,1sleep20-21swapper/22
1378763290,1sleep20-21swapper/22
1360295291,4sleep21360296-21sh2
1318963291,4sleep10-21swapper/11
1315492291,4sleep01315506-21ssh0
1240253290,3sleep00-21swapper/00
1220806291,3sleep139542-21systemd-journal1
1188977291,4sleep01189226-21cat0
1178723291,3sleep139542-21systemd-journal1
1168726291,4sleep01168896-21cat0
1162730291,4sleep11162890-21sed1
1147552291,4sleep21147695-21grep2
1141539290,1sleep20-21swapper/22
1136185290,1sleep00-21swapper/00
1115778291,4sleep01115754-21/usr/sbin/munin0
1210890281,3sleep01211701-21kthreadcore0
1210352281,3sleep10-21swapper/11
1189311281,3sleep11189485-21cstates1
1189311281,3sleep11189485-21cstates1
1184662281,3sleep21186261-21kthreadcore2
1153028281,3sleep01153583-21kthreadcore0
1146991281,3sleep00-21swapper/00
1131070280,1sleep10-21swapper/11
1121050280,2sleep20-21swapper/22
1230981270,1sleep20-21swapper/22
1230741270,1sleep00-21swapper/00
1220484270,1sleep00-21swapper/00
1215558270,2sleep10-21swapper/11
1205667271,2sleep11206999-21kthreadcore1
1195940271,2sleep00-21swapper/00
1193867270,2sleep20-21swapper/22
1193867270,2sleep20-21swapper/22
1183547270,4sleep00-21swapper/00
1178768270,2sleep20-21swapper/22
1178341270,1sleep10-21swapper/11
1163005271,1sleep10-21swapper/11
1148503271,3sleep11150168-21kthreadcore1
1146660270,1sleep20-21swapper/22
1138106270,3sleep01139616-21cut0
1136264270,1sleep10-21swapper/11
1136126270,1sleep20-21swapper/22
1131181270,1sleep20-21swapper/22
1127088271,3sleep00-21swapper/00
1115288270,1sleep20-21swapper/22
1172973260,4sleep20-21swapper/22
1162528260,4sleep00-21swapper/00
1161710260,2sleep21157499-21sendmail-msp2
1157433261,2sleep20-21swapper/22
1108208260,1sleep20-21swapper/22
1210613240,1sleep20-21swapper/22
1121340240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional