You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 09:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Mon May 25, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19603221860,3sleep0195855-21ssh0
618212368,51sleep30-21swapper/33
44403321210,3sleep10-21swapper/11
33428359,6sleep00-21swapper/00
66728161,12sleep20-21swapper/22
31827961,12sleep10-21swapper/11
830992219,1cyclictest0-21swapper/33
830992219,1cyclictest0-21swapper/33
830992117,2cyclictest0-21swapper/33
83099211,16cyclictest0-21swapper/33
83099203,13cyclictest0-21swapper/33
83099203,13cyclictest0-21swapper/33
83099202,15cyclictest66950irq/132-enp1s0-TxRx-33
830992016,2cyclictest0-21swapper/33
4096502200,1sleep10-21swapper/11
2343982201,2sleep10-21swapper/11
830991915,3cyclictest66950irq/132-enp1s0-TxRx-33
83099191,15cyclictest0-21swapper/33
83099191,15cyclictest0-21swapper/33
83099191,14cyclictest321262-21taskset3
83099191,14cyclictest0-21swapper/33
83099191,14cyclictest0-21swapper/33
83099191,14cyclictest0-21swapper/33
83099191,14cyclictest0-21swapper/33
83099191,14cyclictest0-21swapper/33
83099191,14cyclictest0-21swapper/33
83099191,14cyclictest0-21swapper/33
830991910,5cyclictest131244-21turbostat3
83099190,15cyclictest4125492sleep33
83099182,12cyclictest55976-21turbostat3
830991814,2cyclictest0-21swapper/33
830991812,3cyclictest0-21swapper/33
83099181,15cyclictest0-21swapper/33
83099181,14cyclictest0-21swapper/33
83099181,14cyclictest0-21swapper/33
83099181,14cyclictest0-21swapper/33
83099181,14cyclictest0-21swapper/33
83099181,14cyclictest0-21swapper/33
83099181,13cyclictest0-21swapper/33
83099181,13cyclictest0-21swapper/33
83099181,13cyclictest0-21swapper/33
83099180,14cyclictest0-21swapper/33
4467602181,4sleep1131228-21diskmemload1
3724282181,5sleep0372477-21ntpq0
2857952180,2sleep10-21swapper/11
2169542180,9sleep0131228-21diskmemload0
1724042181,2sleep00-21swapper/00
1724042181,2sleep00-21swapper/00
83099171,4cyclictest0-21swapper/33
83099171,3cyclictest0-21swapper/33
830991713,2cyclictest0-21swapper/33
83099171,14cyclictest0-21swapper/33
83099171,13cyclictest0-21swapper/33
83099171,13cyclictest0-21swapper/33
83099171,13cyclictest0-21swapper/33
83099171,13cyclictest0-21swapper/33
83099171,13cyclictest0-21swapper/33
83099171,13cyclictest0-21swapper/33
83099171,13cyclictest0-21swapper/33
83099171,13cyclictest0-21swapper/33
83099170,14cyclictest0-21swapper/33
83099170,14cyclictest0-21swapper/33
83099170,14cyclictest0-21swapper/33
83099170,14cyclictest0-21swapper/33
83099170,14cyclictest0-21swapper/33
3933322170,2sleep00-21swapper/00
2599452171,12sleep0259960-21ssh0
2330222171,4sleep00-21swapper/00
83099163,11cyclictest0-21swapper/33
83099161,13cyclictest0-21swapper/33
83099161,13cyclictest0-21swapper/33
83099160,14cyclictest0-21swapper/33
83099160,13cyclictest0-21swapper/33
83099160,13cyclictest0-21swapper/33
83099160,12cyclictest0-21swapper/33
4880602161,3sleep00-21swapper/00
3221932161,2sleep00-21swapper/00
2182872160,1sleep20-21swapper/22
1101422161,5sleep2110352-21hddtemp_smartct2
83099151,2cyclictest0-21swapper/33
830991512,2cyclictest0-21swapper/33
83099151,12cyclictest0-21swapper/33
83099150,13cyclictest0-21swapper/33
83099150,13cyclictest0-21swapper/33
83099150,12cyclictest0-21swapper/33
4946102151,8sleep0494670-21ntpq0
3220692151,1sleep20-21swapper/22
3139182151,2sleep10-21swapper/11
2861972151,3sleep0131228-21diskmemload0
2711282151,9sleep2271188-21munin-run2
2711282151,9sleep2271188-21munin-run2
2699342150,3sleep00-21swapper/00
2699342150,3sleep00-21swapper/00
2509402151,4sleep2131228-21diskmemload2
1841062151,4sleep1131228-21diskmemload1
1378532151,8sleep2131228-21diskmemload2
1154332150,9sleep1114925-21apt-get1
830991411,2cyclictest0-21swapper/33
83099141,10cyclictest0-21swapper/33
830991410,2cyclictest0-21swapper/33
83099140,11cyclictest39876-21chrt3
4510272141,9sleep0451084-21ntpq0
3390372141,3sleep2131228-21diskmemload2
2962692141,9sleep1296279-21ssh1
2621972141,9sleep2131228-21diskmemload2
2160532141,2sleep10-21swapper/11
2043132142,6sleep1205550-21memory1
986112131,3sleep298677-21https2
830991310,1cyclictest0-21swapper/33
83099130,11cyclictest0-21swapper/33
4859472131,8sleep1486013-21ssh1
4704022131,7sleep2131228-21diskmemload2
4597392131,8sleep0459798-21ntpq0
4597392131,8sleep0459798-21ntpq0
4437512131,4sleep2443762-21diskmemload2
4149992130,2sleep1321rcuc/11
3766452131,7sleep1376882-21cat1
3661142131,8sleep0366126-21rm0
3661142131,8sleep0366126-21rm0
3305642131,9sleep10-21swapper/11
2933342131,8sleep0293422-21awk0
2497302131,8sleep0249817-21ssh0
1811362131,8sleep1131228-21diskmemload1
1566202131,5sleep1156634-21ssh1
1407142131,8sleep2131228-21diskmemload2
1243672131,8sleep0124792-21/usr/sbin/munin0
929822120,1sleep20-21swapper/22
567922121,7sleep057289-21grep0
4803662121,7sleep0480380-21ssh0
4798712121,7sleep1479887-21modprobe1
390452121,6sleep139092-21ntpq1
3648232121,6sleep2131228-21diskmemload2
3648232121,6sleep2131228-21diskmemload2
3243682121,7sleep0324381-21apt-get0
2896352121,3sleep2290591-21kthreadcore2
2855612121,3sleep2285572-21ssh2
2453202121,7sleep1245440-21grep1
2240742126,2sleep1131228-21diskmemload1
1885292122,2sleep0171rcu_preempt0
1483842120,3sleep0148398-21ssh0
991692110,2sleep10-21swapper/11
816152111,5sleep081663-21ntpq0
656132111,5sleep165659-21ntpq1
4963372111,3sleep20-21swapper/22
4902222111,4sleep10-21swapper/11
4816462111,5sleep2481658-21ssh2
467972111,6sleep148522-21kthreadcore1
4249172111,5sleep0424980-21ntp_states0
4166122111,5sleep2416843-21users2
4001142111,4sleep2131228-21diskmemload2
3919332110,1sleep20-21swapper/22
3741332111,5sleep2131228-21diskmemload2
3595722111,5sleep1359604-21ssh1
3595722111,5sleep1359604-21ssh1
3582052111,5sleep2131228-21diskmemload2
3561742111,4sleep1131228-21diskmemload1
349552111,5sleep034974-21apt-get0
3480172112,4sleep0131228-21diskmemload0
3423732111,5sleep290-21kswapd02
3372852111,5sleep1337344-21ntpq1
3065582112,4sleep2306588-21apt-get2
3058422111,5sleep00-21swapper/00
3045412111,5sleep2131228-21diskmemload2
3041302111,5sleep10-21swapper/11
2773802111,6sleep0277391-21ssh0
2426602110,5sleep00-21swapper/00
2323662111,5sleep20-21swapper/22
1830082111,3sleep20-21swapper/22
1714612111,7sleep1171471-21ssh1
1714612111,7sleep1171471-21ssh1
1665232111,4sleep20-21swapper/22
1665232111,4sleep20-21swapper/22
1580782112,5sleep2131228-21diskmemload2
1454822111,3sleep10-21swapper/11
1371692110,7sleep0131228-21diskmemload0
1212992110,7sleep1121389-21irqstats1
1050912111,6sleep1106385-21tr1
976332101,4sleep097682-21ntpq0
869872101,4sleep087036-21ntpq0
509872101,4sleep051200-21df_inode0
4806882101,5sleep2480699-21ssh2
4641942100,1sleep00-21swapper/00
4626482101,2sleep20-21swapper/22
4626482101,2sleep20-21swapper/22
4522032101,3sleep20-21swapper/22
4422822101,4sleep0442336-21ntpq0
4355512101,4sleep2131228-21diskmemload2
4258522101,4sleep1131228-21diskmemload1
4248112100,1sleep20-21swapper/22
4197762101,3sleep00-21swapper/00
4115372101,6sleep2131228-21diskmemload2
4026032101,3sleep0131228-21diskmemload0
3924352101,2sleep10-21swapper/11
3851952100,1sleep00-21swapper/00
3569232100,1sleep00-21swapper/00
3463352100,3sleep10-21swapper/11
3412402100,2sleep0171rcu_preempt0
336342101,5sleep233685-21ntpq2
3289372101,5sleep2328988-21switchtime2
3227662101,4sleep1131228-21diskmemload1
3144622101,2sleep00-21swapper/00
2755482101,4sleep1275691-21ntp_192.168.1141
2714682101,5sleep21418-21sendmail-mta2
2596132101,4sleep1131228-21diskmemload1
2441022101,2sleep10-21swapper/11
2365512101,5sleep2236580-21grep2
2189562100,1sleep20-21swapper/22
2087572101,3sleep2131228-21diskmemload2
2083322102,4sleep0208343-21sh0
1987642100,2sleep10-21swapper/11
1982272101,3sleep20-21swapper/22
1803612101,4sleep0131228-21diskmemload0
1577792101,4sleep00-21swapper/00
1542512101,3sleep20-21swapper/22
1357522101,5sleep1135811-21ntpq1
1261682101,5sleep2126424-21df2
99461291,4sleep099673-21hddtemp_smartct0
88532291,4sleep188635-21/usr/sbin/munin1
88273291,4sleep088531-21/usr/sbin/munin0
71612291,2sleep00-21swapper/00
66887290,3sleep266906-21apt-get2
470090290,1sleep10-21swapper/11
435680291,2sleep00-21swapper/00
434341291,4sleep1434356-21ssh1
408730290,5sleep0408742-21ssh0
399836293,3sleep134-21ksoftirqd/11
384246291,3sleep1131228-21diskmemload1
384117291,4sleep2384128-21ssh2
29906291,4sleep130120-21hddtemp_smartct1
269439291,2sleep10-21swapper/11
269439291,2sleep10-21swapper/11
225484290,3sleep00-21swapper/00
183943291,4sleep2183954-21ssh2
164888291,2sleep10-21swapper/11
153092291,4sleep0153195-21ssh0
14179291,4sleep014261-21sed0
130193291,3sleep1130234-21ntpq1
110442291,4sleep0111945-21kthreadcore0
99742281,3sleep2100948-21kthreadcore2
458037281,3sleep10-21swapper/11
458037281,3sleep10-21swapper/11
45329281,3sleep045356-21/usr/sbin/munin0
29327280,2sleep20-21swapper/22
25647281,3sleep027395-21kthreadcore0
23928280,2sleep10-21swapper/11
18980281,2sleep00-21swapper/00
131070280,2sleep00-21swapper/00
109238280,4sleep20-21swapper/22
79469271,2sleep20-21swapper/22
77125270,1sleep20-21swapper/22
72505271,3sleep172707-21expr1
72288271,2sleep072504-21diskstats0
62950271,3sleep20-21swapper/22
61163270,1sleep20-21swapper/22
61001270,1sleep10-21swapper/11
45590270,2sleep00-21swapper/00
45220270,1sleep20-21swapper/22
34437270,1sleep00-21swapper/00
29027270,1sleep10-21swapper/11
19259271,3sleep20-21swapper/22
13671270,1sleep20-21swapper/22
113940270,2sleep10-21swapper/11
95700261,2sleep10-21swapper/11
8875261,2sleep20-21swapper/22
87801260,4sleep20-21swapper/22
87698260,1sleep10-21swapper/11
70731260,2sleep170904-21strings1
6518261,2sleep17236-21cat1
55338260,3sleep10-21swapper/11
50904260,4sleep20-21swapper/22
14263261,2sleep10-21swapper/11
124874260,2sleep20-21swapper/22
116017261,2sleep20-21swapper/22
114975261,2sleep00-21swapper/00
8318250,3sleep00-21swapper/00
81665250,1sleep10-21swapper/11
61981250,1sleep00-21swapper/00
4843250,1sleep20-21swapper/22
45871250,1sleep20-21swapper/22
40002250,3sleep20-21swapper/22
1168250,1sleep10-21swapper/11
104519250,1sleep00-21swapper/00
1024250,1sleep00-21swapper/00
40556240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional