You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 03:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Wed Aug 26, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
392443921370,2sleep20-21swapper/22
3506074212251,66sleep30-21swapper/33
350631828452,10sleep20-21swapper/22
350619228062,12sleep10-21swapper/11
350607127451,6sleep00-21swapper/00
398487522211,7sleep03984888-21sh0
350652499222,15cyclictest0-21swapper/33
350652499221,18cyclictest0-21swapper/33
350652499213,14cyclictest39858872sleep33
3506524992118,1cyclictest0-21swapper/33
350652499211,7cyclictest3956486-21chrt3
3506524992116,3cyclictest0-21swapper/33
350652499211,16cyclictest0-21swapper/33
350652499211,16cyclictest0-21swapper/33
350652499211,16cyclictest0-21swapper/33
350652499211,16cyclictest0-21swapper/33
350652499201,6cyclictest3555575-21taskset3
350652499201,16cyclictest3813502-21turbostat3
350652499201,16cyclictest3762272-21chrt3
350652499201,16cyclictest3519212-21chrt3
350652499201,16cyclictest0-21swapper/33
350652499201,15cyclictest0-21swapper/33
350652499201,15cyclictest0-21swapper/33
350652499201,15cyclictest0-21swapper/33
37444522191,2sleep20-21swapper/22
37444522191,2sleep20-21swapper/22
350652499198,7cyclictest3865246-21taskset3
350652499194,12cyclictest3897275-21chrt3
3506524991912,4cyclictest0-21swapper/33
350652499191,15cyclictest0-21swapper/33
350652499191,15cyclictest0-21swapper/33
350652499191,15cyclictest0-21swapper/33
350652499191,15cyclictest0-21swapper/33
350652499191,15cyclictest0-21swapper/33
350652499191,14cyclictest0-21swapper/33
350652499191,14cyclictest0-21swapper/33
350652499191,14cyclictest0-21swapper/33
350652499191,14cyclictest0-21swapper/33
350652499191,14cyclictest0-21swapper/33
350652499191,14cyclictest0-21swapper/33
350652499191,14cyclictest0-21swapper/33
350652499191,14cyclictest0-21swapper/33
350652499191,14cyclictest0-21swapper/33
39145592181,3sleep23634487-21diskmemload2
38886282181,11sleep23634487-21diskmemload2
38053572181,4sleep13805378-21apt-get1
36833302181,3sleep10-21swapper/11
36038472181,9sleep23603041-21sendmail-msp2
350652499182,12cyclictest3728913-21taskset3
3506524991812,3cyclictest0-21swapper/33
3506524991812,3cyclictest0-21swapper/33
3506524991812,3cyclictest0-21swapper/33
350652499181,14cyclictest0-21swapper/33
350652499181,13cyclictest0-21swapper/33
350652499181,12cyclictest3788809-21chrt3
350652499180,14cyclictest0-21swapper/33
38614492171,3sleep10-21swapper/11
38022022171,13sleep00-21swapper/00
36752422171,2sleep00-21swapper/00
350652499172,12cyclictest0-21swapper/33
3506524991712,3cyclictest0-21swapper/33
350652499171,13cyclictest0-21swapper/33
350652499171,13cyclictest0-21swapper/33
350652499171,13cyclictest0-21swapper/33
350652499171,12cyclictest0-21swapper/33
350652499171,12cyclictest0-21swapper/33
350652499170,13cyclictest0-21swapper/33
39344462161,10sleep13634487-21diskmemload1
38620262161,3sleep03634487-21diskmemload0
37604612160,1sleep10-21swapper/11
37601982161,4sleep23634487-21diskmemload2
37108042161,9sleep20-21swapper/22
36649322161,5sleep13665125-21ssh1
3506524991611,3cyclictest0-21swapper/33
350652499161,12cyclictest0-21swapper/33
3506524991610,4cyclictest0-21swapper/33
350652499160,13cyclictest0-21swapper/33
350652499160,13cyclictest0-21swapper/33
350652499160,13cyclictest0-21swapper/33
39932552151,2sleep00-21swapper/00
39236842151,2sleep00-21swapper/00
38029142151,9sleep23634487-21diskmemload2
37957322150,2sleep20-21swapper/22
37701712151,4sleep13770181-21ssh1
37105332150,1sleep10-21swapper/11
35714062150,2sleep10-21swapper/11
350652499151,2cyclictest0-21swapper/33
3506524991512,2cyclictest0-21swapper/33
3506524991512,1cyclictest0-21swapper/33
3506524991510,3cyclictest0-21swapper/33
350652499150,12cyclictest0-21swapper/33
350652499150,12cyclictest0-21swapper/33
350652499150,12cyclictest0-21swapper/33
39928332141,9sleep23992846-21sh2
39212422141,9sleep13921260-21ssh1
39128782141,8sleep03912888-21ssh0
38948152141,3sleep00-21swapper/00
38714522141,8sleep13871462-21ssh1
38706572141,4sleep03870671-21ssh0
38649042141,3sleep23634487-21diskmemload2
38549452141,4sleep13854957-21ssh1
38116862141,9sleep23634487-21diskmemload2
37953162141,3sleep03795326-21ssh0
37878382141,4sleep03787932-21ssh0
37765552141,9sleep03776569-21ssh0
36706662141,8sleep23672329-21grep2
36406682141,10sleep23640680-21rm2
36253452141,9sleep03626815-21grep0
350652499141,8cyclictest3624253-21taskset3
38946182130,1sleep20-21swapper/22
38879152131,5sleep00-21swapper/00
38790402130,2sleep20-21swapper/22
38535862131,7sleep03634487-21diskmemload0
350652499131,10cyclictest35922482sleep33
350652499130,2cyclictest0-21swapper/33
39859282121,6sleep23985941-21ssh2
39491912121,7sleep13634487-21diskmemload1
39430432120,3sleep00-21swapper/00
38475362121,7sleep00-21swapper/00
38390872121,7sleep23839297-21ssh2
38287972121,3sleep23828808-21ssh2
38223562121,2sleep00-21swapper/00
38204662121,7sleep13820477-21sh1
38192862121,7sleep03819300-21ssh0
37925422120,2sleep1171rcu_preempt1
37797762121,6sleep13634487-21diskmemload1
37702502121,7sleep03770264-21ssh0
37689232120,1sleep20-21swapper/22
37430502121,7sleep03743064-21ssh0
37430502121,7sleep03743064-21ssh0
37424832120,8sleep13742502-21ssh1
37424832120,8sleep13742502-21ssh1
37176162121,7sleep13717627-21ssh1
36958992121,7sleep23697493-21grep2
36824802121,5sleep20-21swapper/22
36403502121,6sleep03634487-21diskmemload0
36245202121,7sleep13624590-21fschecks_time1
35665022121,5sleep03566521-21apt-get0
39782722111,5sleep13978385-21perl1
39677132111,3sleep10-21swapper/11
39660892111,3sleep00-21swapper/00
39543162111,6sleep03956020-21kthreadcore0
39438172110,5sleep20-21swapper/22
39406782110,5sleep244-21ksoftirqd/22
39068612111,6sleep13906873-21ssh1
39066672111,6sleep03906678-21ssh0
39041012111,4sleep23634487-21diskmemload2
38310042111,5sleep13831270-21df1
38196482111,4sleep23634487-21diskmemload2
37973332111,5sleep13797427-21irqstats1
37768272110,5sleep20-21swapper/22
37535932111,3sleep20-21swapper/22
37534322110,2sleep10-21swapper/11
37344652111,2sleep23734476-21ssh2
37254462111,4sleep20-21swapper/22
37086652110,6sleep03634487-21diskmemload0
36985672111,6sleep03698714-21/usr/sbin/munin0
36941282111,4sleep20-21swapper/22
36930372111,3sleep00-21swapper/00
36507742110,2sleep00-21swapper/00
36086822111,5sleep13608848-21diskstats1
35538832112,5sleep03554629-21wc0
35470432112,6sleep10-21swapper/11
35326842111,5sleep13533634-21memory1
35251232111,7sleep03526637-21cut0
35238682110,7sleep23518957-21sendmail-msp2
39870852101,5sleep13987509-21grep1
39694542101,4sleep03969721-21df0
39666402101,4sleep23634487-21diskmemload2
39571082101,3sleep23634487-21diskmemload2
39526292101,4sleep13952851-21hddtemp_smartct1
39511352101,3sleep00-21swapper/00
39407702101,4sleep13634487-21diskmemload1
39323832100,1sleep20-21swapper/22
39311292101,5sleep03931188-21sed0
39130862101,3sleep10-21swapper/11
38949762101,5sleep13895207-21grep1
38639222100,3sleep20-21swapper/22
38361302100,1sleep20-21swapper/22
38309882101,4sleep03634487-21diskmemload0
38280132101,3sleep10-21swapper/11
37862492101,4sleep23634487-21diskmemload2
37597772101,3sleep00-21swapper/00
37347882101,6sleep13734799-21ssh1
37334082100,1sleep00-21swapper/00
37252252101,2sleep00-21swapper/00
37203792101,5sleep13720593-21hddtemp_smartct1
36993732100,1sleep10-21swapper/11
36823052101,5sleep03682318-21ssh0
36665442101,3sleep03634487-21diskmemload0
36661642100,1sleep20-21swapper/22
36585292101,4sleep03634487-21diskmemload0
36584972101,3sleep20-21swapper/22
36354052101,4sleep13635486-21irqstats1
36137912101,4sleep23613810-21apt-get2
35875602101,4sleep23587579-21apt-get2
35823702101,4sleep13582493-21diskstats1
35769822101,4sleep13576984-21apt-get1
35461092101,5sleep03546145-21kthreadcore0
35188892101,4sleep13518979-21munin-run1
3976249291,3sleep13634487-21diskmemload1
3969366291,2sleep20-21swapper/22
3889773290,1sleep10-21swapper/11
3879768291,3sleep13634487-21diskmemload1
3844403291,3sleep13634487-21diskmemload1
3809703290,4sleep03809856-21find0
3776081290,1sleep10-21swapper/11
3751996290,2sleep00-21swapper/00
3716288291,3sleep03634487-21diskmemload0
3669569291,4sleep13670664-21cut1
3659716291,2sleep10-21swapper/11
3648626291,3sleep13634487-21diskmemload1
3643253291,3sleep23634487-21diskmemload2
3618375291,5sleep0106713-21snmpd0
3602840290,1sleep10-21swapper/11
3593054291,5sleep03593078-21fschecks_count0
3576987291,4sleep23577243-21df2
3571462291,4sleep23571658-21cat2
3545202291,3sleep2106713-21snmpd2
3540236290,1sleep10-21swapper/11
3539824290,2sleep00-21swapper/00
3524249291,4sleep01-21systemd0
3519307291,3sleep13519504-21diskstats1
3878469281,3sleep03878479-21ssh0
3847908281,3sleep23847938-21apt-get2
3717483281,2sleep20-21swapper/22
3686326281,3sleep14192558-21rs:main1
3634381281,3sleep04192558-21rs:main0
3633777281,3sleep13629528-21/usr/sbin/munin1
3624231281,2sleep20-21swapper/22
3620864281,3sleep13622616-21kthreadcore1
3609131281,3sleep03610342-21kthreadcore0
3596816280,4sleep20-21swapper/22
3592776281,3sleep13592791-21acpi1
3592496281,3sleep03592499-21sshd-auth0
3586480281,3sleep23581976-21sendmail-msp2
3582771281,3sleep03582809-21kthreadcore0
3581910281,3sleep03582108-21latency_hist0
3571733280,2sleep20-21swapper/22
3565984281,3sleep24192556-21in:imuxsock2
3560911281,3sleep00-21swapper/00
3555970280,3sleep13555967-21apt-get1
3540801281,3sleep039542-21systemd-journal0
3539886280,2sleep10-21swapper/11
3529305280,2sleep10-21swapper/11
3619006270,2sleep00-21swapper/00
3614127271,3sleep13511898-21munin-node1
3607263271,2sleep10-21swapper/11
3603259271,2sleep00-21swapper/00
3592436270,1sleep10-21swapper/11
3576694271,2sleep10-21swapper/11
3576468270,1sleep00-21swapper/00
3561739271,2sleep00-21swapper/00
3560032271,2sleep20-21swapper/22
3550737270,2sleep10-21swapper/11
3531008271,3sleep00-21swapper/00
3530142271,3sleep20-21swapper/22
3514492271,3sleep23515567-21kthreadcore2
3511335270,1sleep10-21swapper/11
3508366271,2sleep00-21swapper/00
3506700270,2sleep20-21swapper/22
3623796260,1sleep20-21swapper/22
3612081261,2sleep20-21swapper/22
3599499260,2sleep20-21swapper/22
3598028260,1sleep00-21swapper/00
3561050260,2sleep10-21swapper/11
3551304261,2sleep20-21swapper/22
3550282261,3sleep2431ktimers/22
3535011261,2sleep20-21swapper/22
3524819261,1sleep20-21swapper/22
3512096261,2sleep00-21swapper/00
3630036250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional