You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 07:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Wed May 13, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
866272212452,67sleep30-21swapper/33
94176421130,4sleep1321rcuc/11
86143029256,12sleep10-21swapper/11
86616227961,12sleep20-21swapper/22
86143427654,16sleep00-21swapper/00
86669599231,18cyclictest932352-21taskset3
86669599218,1cyclictest0-21swapper/33
86669599211,17cyclictest0-21swapper/33
86669599211,16cyclictest1080534-21taskset3
86669599203,13cyclictest0-21swapper/33
86669599201,3cyclictest0-21swapper/33
86669599201,16cyclictest1272203-21taskset3
86669599201,15cyclictest66950irq/132-enp1s0-TxRx-33
86669599201,13cyclictest0-21swapper/33
866695992010,6cyclictest66950irq/132-enp1s0-TxRx-33
86669599194,3cyclictest66950irq/132-enp1s0-TxRx-33
866695991915,2cyclictest0-21swapper/33
86669599191,15cyclictest900745-21turbostat3
86669599191,15cyclictest0-21swapper/33
86669599191,15cyclictest0-21swapper/33
86669599191,14cyclictest66950irq/132-enp1s0-TxRx-33
86669599191,14cyclictest0-21swapper/33
86669599191,14cyclictest0-21swapper/33
86669599191,14cyclictest0-21swapper/33
86669599190,16cyclictest0-21swapper/33
86669599190,16cyclictest0-21swapper/33
86669599190,16cyclictest0-21swapper/33
86669599190,15cyclictest11955342sleep33
12148972190,5sleep01214941-21munin-run0
86669599183,13cyclictest0-21swapper/33
86669599181,4cyclictest0-21swapper/33
866695991812,4cyclictest0-21swapper/33
866695991812,4cyclictest0-21swapper/33
866695991811,4cyclictest0-21swapper/33
86669599181,14cyclictest0-21swapper/33
86669599181,14cyclictest0-21swapper/33
86669599181,14cyclictest0-21swapper/33
86669599181,14cyclictest0-21swapper/33
86669599181,14cyclictest0-21swapper/33
86669599181,14cyclictest0-21swapper/33
86669599181,14cyclictest0-21swapper/33
86669599181,14cyclictest0-21swapper/33
86669599181,14cyclictest0-21swapper/33
86669599181,13cyclictest962735-21taskset3
86669599181,13cyclictest0-21swapper/33
86669599181,13cyclictest0-21swapper/33
86669599181,13cyclictest0-21swapper/33
86669599181,13cyclictest0-21swapper/33
86669599181,13cyclictest0-21swapper/33
86669599181,13cyclictest0-21swapper/33
86669599180,14cyclictest0-21swapper/33
86669599180,14cyclictest0-21swapper/33
86669599180,12cyclictest0-21swapper/33
10396062182,8sleep21039637-21ssh2
866695991711,3cyclictest0-21swapper/33
86669599171,13cyclictest0-21swapper/33
86669599171,13cyclictest0-21swapper/33
86669599171,13cyclictest0-21swapper/33
86669599170,14cyclictest0-21swapper/33
86669599170,14cyclictest0-21swapper/33
86669599170,13cyclictest0-21swapper/33
86669599170,13cyclictest0-21swapper/33
13294572171,3sleep20-21swapper/22
11810192170,4sleep00-21swapper/00
11129112171,3sleep20-21swapper/22
11129112171,3sleep20-21swapper/22
86669599160,3cyclictest959511-21chrt3
86669599160,13cyclictest0-21swapper/33
86669599160,12cyclictest0-21swapper/33
13209672161,2sleep10-21swapper/11
11224522161,4sleep0996918-21diskmemload0
10441632161,3sleep10-21swapper/11
9993862151,2sleep10-21swapper/11
86669599151,9cyclictest879720-21taskset3
86669599151,9cyclictest879720-21taskset3
86669599151,3cyclictest0-21swapper/33
86669599151,11cyclictest0-21swapper/33
86669599150,13cyclictest0-21swapper/33
86669599150,13cyclictest0-21swapper/33
86669599150,12cyclictest0-21swapper/33
86669599150,12cyclictest0-21swapper/33
86669599150,12cyclictest0-21swapper/33
12985022151,2sleep10-21swapper/11
11642012151,4sleep0996918-21diskmemload0
11559932151,4sleep21156005-21ssh2
866695991412,1cyclictest0-21swapper/33
13223882141,9sleep0996918-21diskmemload0
11048872141,3sleep1996918-21diskmemload1
10700552141,8sleep21070056-21ssh2
10278442141,8sleep0996918-21diskmemload0
9931842131,7sleep1994920-21cut1
86669599130,11cyclictest0-21swapper/33
86669599130,11cyclictest0-21swapper/33
13401842131,4sleep21340196-21ssh2
13245732131,8sleep11324784-21smartctl1
13049632131,7sleep00-21swapper/00
12808012131,7sleep2996918-21diskmemload2
12808012131,7sleep2996918-21diskmemload2
12649852131,7sleep2996918-21diskmemload2
12483892131,8sleep01248410-21sendmail0
12425782131,6sleep20-21swapper/22
12291602131,8sleep11229169-21sh1
12136052131,3sleep21213619-21ssh2
11951302131,8sleep01195141-21ssh0
11563072131,4sleep11156317-21ssh1
11293102131,8sleep01129322-21ssh0
10975382131,8sleep11097546-21ssh1
10640912131,8sleep0996918-21diskmemload0
10136032131,3sleep01013606-21ssh0
9489692121,7sleep1949002-21awk1
9370142121,6sleep1937061-21ntpq1
86669599125,3cyclictest9652132sleep33
13079872121,2sleep00-21swapper/00
12719292121,7sleep2996918-21diskmemload2
12556742121,7sleep21255686-21rm2
12286772121,7sleep01228687-21ssh0
12052942121,8sleep11205301-21ssh1
11977672121,6sleep1996918-21diskmemload1
11784742120,7sleep2996918-21diskmemload2
11564442121,7sleep11156661-21ssh1
11471472121,5sleep00-21swapper/00
11279472121,7sleep11128143-21ssh1
10821272121,4sleep21082668-21rm2
10040082120,1sleep20-21swapper/22
9811912111,5sleep2981209-21apt-get2
9758882111,5sleep1976146-21df_inode1
9377622110,1sleep00-21swapper/00
9219172110,4sleep00-21swapper/00
9012872111,7sleep1901412-21cut1
13218572110,6sleep20-21swapper/22
13081302110,7sleep11309707-21cut1
12698742111,3sleep10-21swapper/11
12633162111,3sleep10-21swapper/11
12409532111,6sleep11242576-21cut1
12068702111,5sleep11206894-21apt-get1
11984502111,4sleep2996918-21diskmemload2
11976882111,3sleep20-21swapper/22
11891942111,7sleep2996918-21diskmemload2
11717842111,4sleep0996918-21diskmemload0
11535092111,3sleep00-21swapper/00
11476812111,6sleep1996918-21diskmemload1
10862492111,5sleep11086440-21users1
10711992111,3sleep10-21swapper/11
10709922111,5sleep0996918-21diskmemload0
10578572111,6sleep21059539-21cut2
10537532111,3sleep10-21swapper/11
10395792111,3sleep00-21swapper/00
10291742111,7sleep11029187-21sh1
10223952111,5sleep11022428-21munin-run1
10223952111,5sleep11022428-21munin-run1
10129532111,5sleep1996918-21diskmemload1
10024982111,7sleep01002505-21ssh0
9868092101,4sleep1987018-21hddtemp_smartct1
9799222101,4sleep0979971-21ntpq0
9655382100,6sleep2965755-21awk2
9276782101,5sleep0927727-21/usr/sbin/munin0
9217572101,6sleep2802-21snmpd2
8944412101,4sleep1894498-21ntpq1
8909002100,6sleep0892180-21grep0
13470362101,4sleep00-21swapper/00
13409582101,6sleep21341144-21gltestperf2
13308032101,3sleep00-21swapper/00
13142032101,4sleep2996918-21diskmemload2
13037342100,6sleep11299210-21/usr/sbin/munin1
12979712101,3sleep00-21swapper/00
12899802101,4sleep2996918-21diskmemload2
12899802101,4sleep2996918-21diskmemload2
12882232101,3sleep10-21swapper/11
12882232101,3sleep10-21swapper/11
12880722101,4sleep0996918-21diskmemload0
12880722101,4sleep0996918-21diskmemload0
12782072101,4sleep01278265-21ntpq0
12782072101,4sleep01278265-21ntpq0
12733392101,4sleep0996918-21diskmemload0
12563602100,6sleep11256370-21ssh1
12368232100,3sleep10-21swapper/11
12219382101,2sleep00-21swapper/00
12206382101,3sleep10-21swapper/11
12156012101,4sleep21215714-21perl2
11868682101,3sleep10-21swapper/11
11736512101,4sleep1996918-21diskmemload1
11365592101,3sleep20-21swapper/22
11364032101,4sleep01136538-21users0
11232942101,3sleep20-21swapper/22
11223732101,4sleep1996918-21diskmemload1
11204292101,5sleep21120440-21ssh2
11040352100,4sleep20-21swapper/22
10976502101,3sleep20-21swapper/22
10858312101,3sleep00-21swapper/00
10808952101,4sleep1996918-21diskmemload1
10731412101,4sleep21073166-21apt-get2
10622592101,3sleep10-21swapper/11
10356192101,3sleep10-21swapper/11
10354052101,5sleep21031162-21/usr/sbin/munin2
10277382100,2sleep21027752-21ssh2
10134382101,3sleep20-21swapper/22
956252291,4sleep0958043-21grep0
949004290,1sleep20-21swapper/22
937328291,4sleep2937335-21sensors_temp2
890051290,2sleep00-21swapper/00
883748291,4sleep2883803-21ntpq2
883748291,4sleep2883803-21ntpq2
874369291,3sleep0874627-21df_inode0
1345669290,2sleep10-21swapper/11
1332508291,3sleep10-21swapper/11
1332497290,1sleep00-21swapper/00
1304427291,2sleep20-21swapper/22
1298463291,3sleep2996918-21diskmemload2
1280523290,1sleep10-21swapper/11
1280523290,1sleep10-21swapper/11
1262247291,3sleep0996918-21diskmemload0
1256618291,2sleep00-21swapper/00
1238506291,3sleep20-21swapper/22
1237450290,1sleep00-21swapper/00
1228327291,2sleep20-21swapper/22
1205041290,1sleep00-21swapper/00
1187196291,3sleep00-21swapper/00
1170666290,3sleep2996918-21diskmemload2
1170470291,2sleep10-21swapper/11
1161767291,3sleep2996918-21diskmemload2
1139998291,2sleep20-21swapper/22
1131539290,1sleep10-21swapper/11
1110899290,1sleep00-21swapper/00
1110899290,1sleep00-21swapper/00
1103344291,4sleep01103355-21ssh0
1090873291,4sleep01092476-21tr0
1081071290,4sleep0996918-21diskmemload0
1054143291,3sleep2996918-21diskmemload2
1052314291,3sleep01052372-21ntpq0
1036891291,4sleep01036902-21ssh0
1020861291,2sleep20-21swapper/22
1020861291,2sleep20-21swapper/22
1019462291,3sleep0996918-21diskmemload0
1019462291,3sleep0996918-21diskmemload0
996839280,1sleep00-21swapper/00
996821281,2sleep2802-21snmpd2
970654281,4sleep2970821-21df_inode2
970537281,3sleep0970556-21apt-get0
954556280,2sleep20-21swapper/22
943352281,3sleep20-21swapper/22
939969281,3sleep0941761-21kthreadcore0
921781280,1sleep10-21swapper/11
911945280,1sleep20-21swapper/22
910510280,1sleep20-21swapper/22
902408281,3sleep0904158-21kthreadcore0
895951281,3sleep1896163-21smartctl1
890034280,1sleep20-21swapper/22
885023280,2sleep10-21swapper/11
882702280,5sleep00-21swapper/00
882702280,5sleep00-21swapper/00
872689280,4sleep1874109-21grep1
1106640281,2sleep10-21swapper/11
1106640281,2sleep10-21swapper/11
991216270,1sleep20-21swapper/22
986076270,1sleep10-21swapper/11
975607271,2sleep1975816-21latency_hist1
970238270,1sleep10-21swapper/11
969987270,1sleep00-21swapper/00
959312270,1sleep10-21swapper/11
954038270,1sleep00-21swapper/00
953248271,2sleep1866691-21cyclictest1
943887271,1sleep00-21swapper/00
932572270,1sleep10-21swapper/11
932563270,1sleep20-21swapper/22
927089270,1sleep20-21swapper/22
927062272,3sleep0151ktimers/00
927040270,1sleep10-21swapper/11
915718271,2sleep00-21swapper/00
913116271,2sleep10-21swapper/11
911163270,1sleep00-21swapper/00
895656270,2sleep20-21swapper/22
884474270,1sleep10-21swapper/11
884474270,1sleep10-21swapper/11
879144270,1sleep20-21swapper/22
984448260,1sleep00-21swapper/00
976147260,3sleep20-21swapper/22
964610260,1sleep00-21swapper/00
959882260,1sleep10-21swapper/11
944492260,2sleep20-21swapper/22
911116260,1sleep10-21swapper/11
895864261,2sleep00-21swapper/00
871749260,1sleep10-21swapper/11
867016260,3sleep0867233-21cat0
986604250,2sleep00-21swapper/00
961008250,1sleep20-21swapper/22
904965250,1sleep20-21swapper/22
871228250,2sleep20-21swapper/22
890618240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional