You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 05:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sun Mar 22, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3285670213160,67sleep30-21swapper/33
328559528467,11sleep10-21swapper/11
328562228064,10sleep20-21swapper/22
328532228057,17sleep00-21swapper/00
3285832992724,1cyclictest0-21swapper/33
3285832992720,5cyclictest0-21swapper/33
3285832992321,1cyclictest0-21swapper/33
3285832992319,2cyclictest0-21swapper/33
3285832992219,1cyclictest0-21swapper/33
33629602211,8sleep13363182-21df1
328583299211,8cyclictest3703731-21chrt3
3285832992118,1cyclictest0-21swapper/33
3285832992117,2cyclictest0-21swapper/33
328583299211,16cyclictest0-21swapper/33
328583299211,14cyclictest0-21swapper/33
328583299207,2cyclictest0-21swapper/33
3285832992018,1cyclictest0-21swapper/33
3285832992017,1cyclictest0-21swapper/33
3285832992017,1cyclictest0-21swapper/33
3285832992016,2cyclictest0-21swapper/33
328583299201,15cyclictest0-21swapper/33
328583299200,3cyclictest0-21swapper/33
37049312191,9sleep23704945-21ssh2
36950702191,3sleep00-21swapper/00
328583299194,12cyclictest0-21swapper/33
3285832991916,2cyclictest0-21swapper/33
3285832991916,1cyclictest0-21swapper/33
3285832991916,1cyclictest0-21swapper/33
3285832991915,2cyclictest0-21swapper/33
3285832991915,2cyclictest0-21swapper/33
3285832991915,2cyclictest0-21swapper/33
3285832991915,2cyclictest0-21swapper/33
3285832991915,2cyclictest0-21swapper/33
3285832991914,2cyclictest0-21swapper/33
328583299191,15cyclictest66950irq/132-enp1s0-TxRx-33
328583299191,15cyclictest0-21swapper/33
328583299191,14cyclictest66950irq/132-enp1s0-TxRx-33
328583299191,14cyclictest0-21swapper/33
328583299191,14cyclictest0-21swapper/33
328583299191,14cyclictest0-21swapper/33
328583299190,16cyclictest0-21swapper/33
35271792181,3sleep00-21swapper/00
34760992181,7sleep13415812-21diskmemload1
3285832991816,1cyclictest0-21swapper/33
3285832991816,1cyclictest0-21swapper/33
3285832991815,1cyclictest0-21swapper/33
3285832991815,1cyclictest0-21swapper/33
3285832991814,2cyclictest0-21swapper/33
3285832991814,2cyclictest0-21swapper/33
3285832991813,2cyclictest0-21swapper/33
328583299181,14cyclictest36234062chrt3
328583299181,14cyclictest0-21swapper/33
328583299181,14cyclictest0-21swapper/33
328583299181,14cyclictest0-21swapper/33
328583299180,14cyclictest0-21swapper/33
37727772171,3sleep00-21swapper/00
37639592171,3sleep00-21swapper/00
37479932171,3sleep00-21swapper/00
34928112171,12sleep13492822-21ssh1
34387512171,3sleep00-21swapper/00
34317912170,2sleep10-21swapper/11
328583299174,1cyclictest0-21swapper/33
328583299171,13cyclictest0-21swapper/33
328583299171,13cyclictest0-21swapper/33
328583299171,13cyclictest0-21swapper/33
328583299171,13cyclictest0-21swapper/33
3285832991710,4cyclictest3378462-21taskset3
37297562161,2sleep20-21swapper/22
36701812160,1sleep10-21swapper/11
36437452161,2sleep00-21swapper/00
361469521611,1sleep20-21swapper/22
35800722161,9sleep00-21swapper/00
35646032161,2sleep20-21swapper/22
35552142169,4sleep23285825-21cyclictest2
35114132161,4sleep23511426-21ssh2
34596352161,4sleep13415812-21diskmemload1
34431232161,10sleep03443659-21cut0
34301402161,1sleep00-21swapper/00
32974052161,10sleep23297459-21ntpq2
328583299163,11cyclictest0-21swapper/33
3285832991613,1cyclictest0-21swapper/33
328583299160,13cyclictest0-21swapper/33
36593352150,1sleep10-21swapper/11
36294882151,7sleep13630709-21memory1
36241852151,6sleep03624195-21ssh0
35900392151,4sleep23415812-21diskmemload2
35353182159,2sleep23485090-21kworker/2:2+events_freezable2
34854462150,2sleep10-21swapper/11
34738552152,3sleep23415812-21diskmemload2
3285832991513,1cyclictest0-21swapper/33
3285832991513,1cyclictest0-21swapper/33
3285832991513,1cyclictest0-21swapper/33
3285832991513,1cyclictest0-21swapper/33
328583299151,2cyclictest0-21swapper/33
3285832991512,1cyclictest0-21swapper/33
328583299150,12cyclictest0-21swapper/33
328583299150,12cyclictest0-21swapper/33
328583299150,12cyclictest0-21swapper/33
37827712141,9sleep13782782-21sh1
37316972140,4sleep03731732-21ssh0
37035402141,2sleep00-21swapper/00
36618002143,4sleep0151ktimers/00
36422822143,3sleep1331ktimers/11
36252652140,3sleep13415812-21diskmemload1
35912262140,3sleep03415812-21diskmemload0
34162312142,6sleep03416499-21df_inode0
33950902142,8sleep03395253-21/usr/sbin/munin0
3285832991412,1cyclictest0-21swapper/33
328583299140,2cyclictest0-21swapper/33
328583299140,12cyclictest0-21swapper/33
37211612131,7sleep23415812-21diskmemload2
37113832130,2sleep10-21swapper/11
36434582131,8sleep23643471-21sh2
36085852131,3sleep23415812-21diskmemload2
35745342131,5sleep13574747-21hddtemp_smartct1
35608332131,7sleep13415812-21diskmemload1
35466042131,7sleep13415812-21diskmemload1
35304082132,5sleep13530433-21apt-get1
35027202131,6sleep23415812-21diskmemload2
34837282131,7sleep23415812-21diskmemload2
33468742131,8sleep03346890-21awk0
37774592121,7sleep03779146-21cut0
37647822120,7sleep13415812-21diskmemload1
37578442121,6sleep13415812-21diskmemload1
37406022121,5sleep13415812-21diskmemload1
36711102121,6sleep03671762-21grep0
36595572121,5sleep20-21swapper/22
36495572121,6sleep13415812-21diskmemload1
36490172121,7sleep23649217-21vmstat2
36325652120,7sleep20-21swapper/22
36134822121,5sleep03613548-21ntpq0
35920512121,6sleep03415812-21diskmemload0
35722072120,5sleep23572220-21sh2
35179232121,4sleep072-21kcompactd00
35090602121,4sleep1331ktimers/11
35020532121,6sleep13415812-21diskmemload1
34676502120,6sleep20-21swapper/22
34675812120,2sleep10-21swapper/11
34221862121,6sleep13415812-21diskmemload1
33933132120,2sleep13393486-21strings1
33899762121,8sleep23390045-21/usr/sbin/munin2
32923942121,5sleep03293196-21ntpq0
37818112110,7sleep23781822-21ssh2
37640362111,3sleep20-21swapper/22
37495602112,4sleep23749582-21apt-get2
37478692111,6sleep23747884-21ssh2
37279162115,3sleep0151ktimers/00
37123442111,4sleep23415812-21diskmemload2
37103592111,6sleep03710373-21ssh0
37020182111,4sleep10-21swapper/11
36866502111,3sleep00-21swapper/00
36769222111,7sleep23676936-21ssh2
36610452111,3sleep00-21swapper/00
36454422111,6sleep03647080-21cut0
35727842110,5sleep00-21swapper/00
35613342110,1sleep00-21swapper/00
35552922111,5sleep03415812-21diskmemload0
35461232111,4sleep00-21swapper/00
35442152111,4sleep23415812-21diskmemload2
35207352110,4sleep2431ktimers/22
34948192111,5sleep23494849-21munin-run2
34665462111,3sleep00-21swapper/00
34576472111,5sleep23415812-21diskmemload2
34570582111,7sleep00-21swapper/00
34467622111,5sleep13446824-21ssh1
34394412111,3sleep20-21swapper/22
34228742111,3sleep20-21swapper/22
34107602111,5sleep13410779-21apt-get1
33988402110,2sleep20-21swapper/22
33936002111,5sleep03393653-21ntp_states0
33559742110,1sleep10-21swapper/11
33402922111,6sleep23335933-21/usr/sbin/munin2
33241262111,6sleep23319979-21/usr/sbin/munin2
33240702111,6sleep13324119-21ntpq1
33199782111,5sleep03319997-21apt-get0
33143722111,6sleep23314391-21ls2
32973572111,5sleep13297403-21ntpq1
32904512111,5sleep13290695-21users1
37743242101,2sleep20-21swapper/22
37723862101,4sleep13415812-21diskmemload1
37576812100,1sleep00-21swapper/00
37376042101,6sleep23415812-21diskmemload2
37306622101,4sleep13415812-21diskmemload1
37198202101,4sleep13415812-21diskmemload1
36962572101,6sleep13415812-21diskmemload1
36880112101,4sleep23688277-21df2
36864002101,5sleep13686411-21ssh1
36851572100,1sleep20-21swapper/22
36673862101,3sleep10-21swapper/11
36660392101,4sleep23666090-21ntpq2
36312202101,4sleep03415812-21diskmemload0
36228792101,4sleep23415812-21diskmemload2
36145282100,5sleep13415812-21diskmemload1
36072362100,1sleep00-21swapper/00
36050462101,5sleep13605063-21/usr/sbin/munin1
35968992101,3sleep23415812-21diskmemload2
35831292101,4sleep13583266-21diskstats1
35798342101,3sleep20-21swapper/22
35658222101,3sleep13415812-21diskmemload1
35387562101,6sleep03538832-21/usr/sbin/munin0
35286112101,3sleep13415812-21diskmemload1
35274032101,4sleep23415812-21diskmemload2
35127542101,3sleep10-21swapper/11
34860872101,4sleep03415812-21diskmemload0
34830872101,3sleep00-21swapper/00
34490582101,3sleep20-21swapper/22
34416792101,2sleep10-21swapper/11
34300562101,2sleep20-21swapper/22
33541492101,2sleep00-21swapper/00
33354422100,3sleep172-21kcompactd01
33293662101,5sleep03329413-21ntpq0
33093142101,5sleep13309316-21/usr/sbin/munin1
33001602101,6sleep13301946-21cut1
32989422100,6sleep23299053-21grep2
32910692101,5sleep23291196-21sed2
3735860291,4sleep13735908-21ntpq1
3721135291,3sleep03415812-21diskmemload0
3596503290,3sleep13415812-21diskmemload1
3553063291,2sleep10-21swapper/11
3503952291,3sleep03415812-21diskmemload0
3501680291,2sleep00-21swapper/00
3468859291,3sleep03468878-21apt-get0
3409773291,4sleep03409778-21/usr/sbin/munin0
3388053291,4sleep13388193-21ntp_192.168.1141
3383879290,4sleep10-21swapper/11
3381637291,4sleep03382727-21memory0
3372561290,1sleep20-21swapper/22
3330607291,2sleep20-21swapper/22
3314632291,4sleep23314651-21apt-get2
3298661290,3sleep00-21swapper/00
3384168280,2sleep20-21swapper/22
3378821281,2sleep20-21swapper/22
3378347280,1sleep00-21swapper/00
3373516281,2sleep23373535-21apt-get2
3369587281,3sleep1321rcuc/11
3356909280,1sleep20-21swapper/22
3347442281,3sleep13348500-21tr1
3341841281,3sleep23342965-21kthreadcore2
3319502280,2sleep10-21swapper/11
3308831280,4sleep23308833-21sshd2
3308831280,4sleep23308833-21sshd2
3411401271,3sleep20-21swapper/22
3400106270,2sleep20-21swapper/22
3400105270,1sleep10-21swapper/11
3373068270,1sleep00-21swapper/00
3367785270,1sleep00-21swapper/00
3362614271,2sleep10-21swapper/11
3348501271,3sleep20-21swapper/22
3346019270,5sleep10-21swapper/11
3345451271,2sleep03345564-21ps0
3335932270,2sleep10-21swapper/11
3329652271,3sleep13329815-21timerwakeupswit1
3319427270,1sleep00-21swapper/00
3308045270,2sleep00-21swapper/00
3308045270,2sleep00-21swapper/00
3405455261,2sleep13405696-21cstates1
3405048260,1sleep00-21swapper/00
3378523260,1sleep10-21swapper/11
3367866260,1sleep20-21swapper/22
3357867261,2sleep00-21swapper/00
3357533260,1sleep20-21swapper/22
3325601261,2sleep20-21swapper/22
3305764261,2sleep10-21swapper/11
3305764261,2sleep10-21swapper/11
3410216250,3sleep20-21swapper/22
3395254250,1sleep10-21swapper/11
3389155250,1sleep00-21swapper/00
3339828250,2sleep00-21swapper/00
3309597250,2sleep00-21swapper/00
3286139250,1sleep00-21swapper/00
3411055240,1sleep00-21swapper/00
3330350240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional