You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 19:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sun May 24, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
112274121270,3sleep1321rcuc/11
903452211853,60sleep30-21swapper/33
90338829073,11sleep20-21swapper/22
90337629070,14sleep00-21swapper/00
90329827959,16sleep10-21swapper/11
903607992411,10cyclictest1375654-21turbostat3
903607992211,8cyclictest0-21swapper/33
12956212221,16sleep11034284-21diskmemload1
90360799211,7cyclictest1304422-21taskset3
90360799211,16cyclictest66950irq/132-enp1s0-TxRx-33
90360799210,19cyclictest0-21swapper/33
13232732211,14sleep11323304-21ssh1
11359042211,6sleep01034284-21diskmemload0
90360799206,12cyclictest0-21swapper/33
90360799202,4cyclictest1173190-21taskset3
903607992016,2cyclictest0-21swapper/33
903607992016,2cyclictest0-21swapper/33
90360799201,15cyclictest66950irq/132-enp1s0-TxRx-33
903607992010,6cyclictest1191513-21taskset3
90360799191,15cyclictest0-21swapper/33
90360799191,15cyclictest0-21swapper/33
90360799191,14cyclictest66950irq/132-enp1s0-TxRx-33
90360799191,14cyclictest1267821-21taskset3
90360799191,14cyclictest0-21swapper/33
90360799191,14cyclictest0-21swapper/33
90360799191,14cyclictest0-21swapper/33
90360799191,14cyclictest0-21swapper/33
90360799190,17cyclictest0-21swapper/33
90360799190,15cyclictest0-21swapper/33
90360799189,6cyclictest1401858-21turbostat3
90360799187,8cyclictest959648-21turbostat3
90360799184,11cyclictest0-21swapper/33
903607991812,4cyclictest66950irq/132-enp1s0-TxRx-33
90360799181,15cyclictest0-21swapper/33
90360799181,14cyclictest970254-21taskset3
90360799181,14cyclictest0-21swapper/33
90360799181,14cyclictest0-21swapper/33
90360799181,14cyclictest0-21swapper/33
90360799181,13cyclictest0-21swapper/33
90360799181,13cyclictest0-21swapper/33
90360799181,13cyclictest0-21swapper/33
90360799180,15cyclictest66950irq/132-enp1s0-TxRx-33
90360799180,14cyclictest0-21swapper/33
90360799180,14cyclictest0-21swapper/33
13458462181,6sleep21034284-21diskmemload2
90360799174,9cyclictest0-21swapper/33
90360799171,4cyclictest0-21swapper/33
90360799171,4cyclictest0-21swapper/33
903607991714,2cyclictest0-21swapper/33
90360799171,14cyclictest0-21swapper/33
90360799171,13cyclictest0-21swapper/33
90360799171,13cyclictest0-21swapper/33
90360799171,13cyclictest0-21swapper/33
90360799171,13cyclictest0-21swapper/33
90360799171,13cyclictest0-21swapper/33
90360799170,15cyclictest0-21swapper/33
90360799170,14cyclictest0-21swapper/33
12504662171,4sleep21034284-21diskmemload2
12155082170,1sleep00-21swapper/00
11366332170,1sleep10-21swapper/11
90360799161,12cyclictest0-21swapper/33
90360799161,12cyclictest0-21swapper/33
90360799160,13cyclictest0-21swapper/33
90360799160,13cyclictest0-21swapper/33
90360799160,13cyclictest0-21swapper/33
90360799160,13cyclictest0-21swapper/33
90360799160,13cyclictest0-21swapper/33
90360799160,13cyclictest0-21swapper/33
13462812161,2sleep00-21swapper/00
13112232161,4sleep01034284-21diskmemload0
13054252161,11sleep01-21systemd0
11623462161,3sleep00-21swapper/00
9652452151,4sleep2965264-21apt-get2
9348432151,6sleep2936642-21cut2
90360799156,6cyclictest1002334-21sleep3
90360799150,13cyclictest0-21swapper/33
90360799150,12cyclictest0-21swapper/33
90360799150,12cyclictest0-21swapper/33
90360799150,12cyclictest0-21swapper/33
14010152151,8sleep11034284-21diskmemload1
13564582151,7sleep00-21swapper/00
13462472151,2sleep10-21swapper/11
13364742150,3sleep2421rcuc/22
12312922151,9sleep11231349-21ntpq1
11809102151,4sleep01034284-21diskmemload0
11545382151,3sleep01154551-21ssh0
11101742151,9sleep00-21swapper/00
11001722151,10sleep11100222-21ntpq1
10835552151,3sleep00-21swapper/00
10238812151,3sleep10-21swapper/11
903607991411,2cyclictest0-21swapper/33
90360799141,11cyclictest0-21swapper/33
90360799141,10cyclictest980952-21taskset3
90360799140,12cyclictest0-21swapper/33
90360799140,12cyclictest0-21swapper/33
90360799140,11cyclictest0-21swapper/33
13869032141,9sleep21388659-21tr2
13029262140,10sleep11034284-21diskmemload1
12786482141,8sleep00-21swapper/00
12705572141,2sleep10-21swapper/11
12609162141,8sleep11034284-21diskmemload1
11999192141,9sleep01199929-21ssh0
11916382141,2sleep01034284-21diskmemload0
11899842141,4sleep11189994-21ssh1
11443432148,2sleep20-21swapper/22
9759022131,6sleep1975921-21apt-get1
13319342130,2sleep01034284-21diskmemload0
13127532131,2sleep10-21swapper/11
12264232130,2sleep01034284-21diskmemload0
12261502130,3sleep21034284-21diskmemload2
12168012130,1sleep20-21swapper/22
11894442131,4sleep21189455-21sh2
11613262131,8sleep21161381-21ntp_kernel_pll_2
11476952131,6sleep10-21swapper/11
11449502136,3sleep01144964-21ssh0
11057962131,8sleep10-21swapper/11
11014852131,7sleep21034284-21diskmemload2
10643662131,8sleep11064818-21sed1
10606262130,2sleep00-21swapper/00
10393312131,8sleep01039559-21users0
10336152131,8sleep11033860-21awk1
90360799120,1cyclictest0-21swapper/33
13759582121,5sleep10-21swapper/11
12771122121,6sleep21034284-21diskmemload2
12422022121,5sleep21034284-21diskmemload2
12079942121,3sleep10-21swapper/11
12069312121,6sleep01034284-21diskmemload0
11637182120,7sleep11034284-21diskmemload1
11098422121,6sleep21034284-21diskmemload2
10765892121,6sleep21034284-21diskmemload2
10439882121,3sleep11044020-21kthreadcore1
10332242121,6sleep21033272-21ntpq2
10065382121,5sleep21006627-21ntpq2
9854042111,6sleep2985487-21ps2
9472762111,6sleep0947882-21needreboot0
13654602111,5sleep11034284-21diskmemload1
13220262112,4sleep11034284-21diskmemload1
13209122111,7sleep21320926-21sh2
13061302111,5sleep21306354-21hddtemp_smartct2
12883152111,5sleep01288343-21apt-get0
12836082111,2sleep01283673-21ssh0
12697452111,4sleep00-21swapper/00
12686002111,5sleep21034284-21diskmemload2
12229012111,5sleep11223156-21users1
11792022111,6sleep21179452-21grep2
11729462111,5sleep01034284-21diskmemload0
11724142111,2sleep20-21swapper/22
11664942111,5sleep11167312-21tr1
10863522111,3sleep10-21swapper/11
10814442111,5sleep21082495-21memory2
10738152111,5sleep01073915-21ntpq0
10600002111,4sleep21034284-21diskmemload2
10521662111,5sleep11052383-21diskstats1
10504032111,3sleep00-21swapper/00
10346452111,5sleep11034671-21apt-get1
10172692112,4sleep21017317-21ntpq2
10078942111,4sleep21007913-21apt-get2
9818872101,4sleep0983531-21cut0
9655272101,4sleep0965630-21perl0
9276942101,4sleep10-21swapper/11
9276942101,4sleep10-21swapper/11
9208382101,4sleep0920893-21ntpq0
9090522101,5sleep2910164-21kthreadcore2
13996532101,4sleep21034284-21diskmemload2
13989732101,4sleep00-21swapper/00
13897792101,4sleep11034284-21diskmemload1
13809802101,4sleep21034284-21diskmemload2
13736172100,3sleep21373628-21ssh2
13641672101,3sleep20-21swapper/22
13626552101,4sleep01034284-21diskmemload0
13373752101,3sleep10-21swapper/11
13299952101,2sleep00-21swapper/00
13229362101,4sleep21034284-21diskmemload2
13200882101,3sleep01034284-21diskmemload0
12945292101,2sleep20-21swapper/22
12851612101,3sleep20-21swapper/22
12676452101,4sleep11034284-21diskmemload1
12600992100,1sleep00-21swapper/00
12596262101,4sleep21259640-21ssh2
12505452101,3sleep01034284-21diskmemload0
12499412100,2sleep10-21swapper/11
12319442101,2sleep00-21swapper/00
12078992101,3sleep20-21swapper/22
11985752100,1sleep10-21swapper/11
11798082101,3sleep10-21swapper/11
11551782101,2sleep20-21swapper/22
11530392100,3sleep10-21swapper/11
11264232101,5sleep21126484-21ld2
11205942101,4sleep21034284-21diskmemload2
11176732101,5sleep11117737-21ssh1
10938302101,2sleep20-21swapper/22
10906072101,2sleep10-21swapper/11
10697642102,5sleep11069826-21/usr/sbin/munin1
10686912101,3sleep00-21swapper/00
10669912101,2sleep20-21swapper/22
10495652101,4sleep21034284-21diskmemload2
10247562101,5sleep21026428-21kthreadcore2
10040622101,6sleep01005821-21cut0
10012762101,6sleep00-21swapper/00
992479291,4sleep0994017-21cut0
992479291,4sleep0994017-21cut0
971122291,4sleep1971590-21grep1
961297291,4sleep1963039-21cut1
953174291,4sleep2953299-21ld2
952386291,4sleep1953173-21sed1
950646291,4sleep0952382-21kthreadcore0
937865291,4sleep0937497-31find0
933056291,4sleep1933121-21grep1
1390925291,3sleep01034284-21diskmemload0
1382025290,1sleep00-21swapper/00
1374035291,3sleep01034284-21diskmemload0
1367591291,4sleep11367704-21perl1
1355068291,4sleep21355082-21ssh2
1354017291,4sleep11354074-21ssh1
1302141290,1sleep20-21swapper/22
1284387291,4sleep11284401-21ssh1
1243946291,2sleep00-21swapper/00
1236341291,4sleep11237145-21ssh1
1234839290,5sleep21034284-21diskmemload2
1192100291,5sleep21192315-21cstates2
1130870291,2sleep20-21swapper/22
1117881291,4sleep01118014-21ssh0
1101570290,1sleep00-21swapper/00
1095018291,3sleep00-21swapper/00
1039172291,3sleep20-21swapper/22
1017383291,4sleep01017470-21/usr/sbin/munin0
1008170291,4sleep01007895-21/usr/sbin/munin0
996251281,1sleep10-21swapper/11
996251281,1sleep10-21swapper/11
987117281,3sleep0988111-21kthreadcore0
986029280,1sleep10-21swapper/11
976673281,3sleep20-21swapper/22
960482281,3sleep0961296-21kthreadcore0
944479281,3sleep2945513-21kthreadcore2
943910281,3sleep1943927-21apt1
943514280,1sleep00-21swapper/00
938734280,1sleep10-21swapper/11
927179280,1sleep00-21swapper/00
916342280,3sleep1916184-21apt-get1
915854281,2sleep10-21swapper/11
915190281,3sleep0915244-21ntpq0
1214023281,2sleep11034284-21diskmemload1
1121826281,3sleep01122109-21ssh0
1013234280,3sleep10-21swapper/11
998010271,3sleep1999691-21tr1
992168271,2sleep2992382-21smartctl2
992168271,2sleep2992382-21smartctl2
980972271,3sleep00-21swapper/00
971591271,3sleep20-21swapper/22
970307270,1sleep10-21swapper/11
954569270,1sleep10-21swapper/11
943624270,1sleep20-21swapper/22
932599270,1sleep00-21swapper/00
932599270,1sleep00-21swapper/00
932468270,1sleep20-21swapper/22
932468270,1sleep20-21swapper/22
922333270,2sleep20-21swapper/22
908868271,3sleep1909049-21latency_hist1
908549270,1sleep00-21swapper/00
1006725270,3sleep11006770-21pmu-power1
997481261,2sleep20-21swapper/22
991603260,1sleep10-21swapper/11
990718261,2sleep20-21swapper/22
958397260,2sleep0958586-21strings0
955145261,2sleep20-21swapper/22
920647260,2sleep20-21swapper/22
908001261,2sleep20-21swapper/22
1034004260,4sleep00-21swapper/00
1023556260,4sleep10-21swapper/11
1018518260,1sleep20-21swapper/22
975610250,1sleep00-21swapper/00
964704250,1sleep20-21swapper/22
926207250,2sleep1926380-21strings1
1028838250,1sleep00-21swapper/00
1021951250,1sleep00-21swapper/00
1012378250,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional