You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 10:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Mon May 11, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
332936213343,85sleep30-21swapper/33
35719921210,2sleep2357198-21perl2
33285328161,12sleep20-21swapper/22
33283628163,12sleep10-21swapper/11
33312027959,14sleep00-21swapper/00
7523362230,2sleep00-21swapper/00
33330999231,19cyclictest0-21swapper/33
6542742221,10sleep1654336-21awk1
33330999213,14cyclictest529498-21taskset3
33330999211,16cyclictest733794-21taskset3
33330999211,16cyclictest66950irq/132-enp1s0-TxRx-33
5908202202,6sleep014-21ksoftirqd/00
33330999208,9cyclictest666642-21turbostat3
333309992016,2cyclictest0-21swapper/33
333309992013,3cyclictest0-21swapper/33
33330999201,16cyclictest0-21swapper/33
33330999201,15cyclictest0-21swapper/33
333309992011,1cyclictest0-21swapper/33
33330999199,1cyclictest0-21swapper/33
33330999194,13cyclictest0-21swapper/33
333309991914,4cyclictest66950irq/132-enp1s0-TxRx-33
33330999191,2cyclictest0-21swapper/33
33330999191,15cyclictest0-21swapper/33
33330999191,15cyclictest0-21swapper/33
33330999191,14cyclictest0-21swapper/33
33330999191,14cyclictest0-21swapper/33
33330999191,14cyclictest0-21swapper/33
33330999191,14cyclictest0-21swapper/33
33330999191,14cyclictest0-21swapper/33
33330999191,14cyclictest0-21swapper/33
33330999191,14cyclictest0-21swapper/33
33330999191,14cyclictest0-21swapper/33
33330999190,16cyclictest0-21swapper/33
79787521810,4sleep1797974-21ssh1
7091492182,5sleep1709172-21apt-get1
33330999184,2cyclictest0-21swapper/33
333309991812,3cyclictest0-21swapper/33
333309991812,3cyclictest0-21swapper/33
333309991811,4cyclictest66950irq/132-enp1s0-TxRx-33
33330999181,14cyclictest0-21swapper/33
33330999181,14cyclictest0-21swapper/33
33330999181,14cyclictest0-21swapper/33
33330999181,14cyclictest0-21swapper/33
33330999181,14cyclictest0-21swapper/33
33330999181,14cyclictest0-21swapper/33
33330999181,14cyclictest0-21swapper/33
33330999181,14cyclictest0-21swapper/33
33330999181,13cyclictest351162-21taskset3
33330999181,13cyclictest0-21swapper/33
33330999181,13cyclictest0-21swapper/33
33330999181,13cyclictest0-21swapper/33
5613812171,12sleep1561439-21ntpq1
4960912171,4sleep1464099-21diskmemload1
33330999171,3cyclictest0-21swapper/33
33330999171,13cyclictest0-21swapper/33
33330999171,12cyclictest0-21swapper/33
33330999170,14cyclictest0-21swapper/33
7647702161,4sleep2464099-21diskmemload2
7229222161,3sleep10-21swapper/11
7086772160,3sleep20-21swapper/22
5896112161,5sleep1464099-21diskmemload1
4577132161,5sleep1457770-21ntpq1
4148862161,4sleep1414933-21ntpq1
33330999169,3cyclictest0-21swapper/33
333309991612,2cyclictest394344-21taskset3
333309991612,2cyclictest0-21swapper/33
33330999160,13cyclictest0-21swapper/33
33330999160,13cyclictest0-21swapper/33
33330999160,13cyclictest0-21swapper/33
33330999160,13cyclictest0-21swapper/33
33330999160,13cyclictest0-21swapper/33
7494762155,2sleep2171rcu_preempt2
6396552151,4sleep0464099-21diskmemload0
6312112151,9sleep1464099-21diskmemload1
6226322151,4sleep1464099-21diskmemload1
33330999158,4cyclictest0-21swapper/33
333309991512,1cyclictest0-21swapper/33
333309991512,1cyclictest0-21swapper/33
333309991512,1cyclictest0-21swapper/33
333309991511,2cyclictest0-21swapper/33
33330999151,12cyclictest0-21swapper/33
33330999151,10cyclictest362148-21taskset3
33330999150,12cyclictest0-21swapper/33
33330999150,12cyclictest0-21swapper/33
33330999150,12cyclictest0-21swapper/33
8081792141,3sleep0464099-21diskmemload0
7722412141,7sleep1772293-21ntpq1
7562772141,3sleep1321rcuc/11
7054112141,7sleep0705606-21users0
6484502141,8sleep1464099-21diskmemload1
5796412141,3sleep1464099-21diskmemload1
5566182141,8sleep1464099-21diskmemload1
5408062141,7sleep1540883-21/usr/sbin/munin1
5127202141,8sleep2464099-21diskmemload2
4893322141,4sleep0489345-21diskmemload0
33330999140,12cyclictest0-21swapper/33
7917822131,3sleep0791793-21ssh0
7490922135,4sleep1464099-21diskmemload1
7175582131,7sleep0594576-21kworker/u16:0+xprtiod0
6971512131,3sleep0697161-21ssh0
6821512131,6sleep00-21swapper/00
6820262131,7sleep2464099-21diskmemload2
6669602131,5sleep20-21swapper/22
6406102131,7sleep1464099-21diskmemload1
6064862131,8sleep2606497-21ssh2
5376442131,3sleep0537652-21ssh0
5306022131,8sleep0464099-21diskmemload0
5045232131,2sleep10-21swapper/11
3988392131,6sleep0398885-21ntpq0
333309991311,1cyclictest0-21swapper/33
33330999130,1cyclictest0-21swapper/33
8183162121,5sleep20-21swapper/22
8015132121,7sleep2801523-21ssh2
7831592120,7sleep2464099-21diskmemload2
7680212121,5sleep0768052-21ssh0
7428462121,8sleep0743116-21ssh0
7386332121,5sleep2738692-21ntpq2
7324392120,3sleep1732450-21ssh1
6635982121,2sleep00-21swapper/00
6150912121,7sleep2615105-21ssh2
5985132120,4sleep1598524-21ssh1
5555222121,7sleep2555533-21sh2
5201312121,8sleep2520145-21ssh2
5118272121,7sleep00-21swapper/00
5044122121,6sleep0464099-21diskmemload0
4148432121,5sleep0414884-21ntpq0
3509092120,1sleep00-21swapper/00
8162462111,5sleep00-21swapper/00
8087582111,7sleep1808772-21ssh1
7745082111,7sleep2774522-21ssh2
7665492111,5sleep00-21swapper/00
7405012111,3sleep10-21swapper/11
7333082110,7sleep2733315-21ssh2
6902922111,4sleep1464099-21diskmemload1
6831922110,2sleep10-21swapper/11
6746902110,1sleep00-21swapper/00
6630952111,4sleep10-21swapper/11
6460982111,5sleep2646197-21cat2
6457032111,5sleep0645742-21ntpq0
6405282111,4sleep2464099-21diskmemload2
6320812111,5sleep20-21swapper/22
6213722111,6sleep2621387-21ssh2
6075572111,5sleep0464099-21diskmemload0
5781372112,4sleep2578182-21ntpq2
5721512111,6sleep0572154-21ssh0
5696982111,2sleep20-21swapper/22
5377572111,6sleep1537768-21ssh1
5257212111,3sleep10-21swapper/11
5238872111,5sleep2524106-21hddtemp_smartct2
5210282111,6sleep1521037-21ssh1
5126602111,3sleep10-21swapper/11
4956912111,2sleep2495702-21ssh2
4863342111,3sleep20-21swapper/22
4802492111,6sleep0464099-21diskmemload0
4711212110,1sleep20-21swapper/22
4590232112,4sleep2459042-21apt-get2
4329662111,6sleep1433738-21kthreadcore1
4258642111,5sleep0425918-21ntpq0
3676902111,5sleep2367813-21diskstats2
3464592111,5sleep2346672-21hddtemp_smartct2
3373912111,6sleep1337531-21ntp_192.168.1141
8161642101,3sleep10-21swapper/11
7994002101,3sleep0464099-21diskmemload0
7919352101,4sleep1464099-21diskmemload1
7851652101,4sleep2464099-21diskmemload2
7840032101,4sleep0464099-21diskmemload0
7648082101,6sleep1464099-21diskmemload1
7586152100,3sleep20-21swapper/22
7380252101,5sleep0738562-21expr0
7143242101,2sleep00-21swapper/00
6925022101,4sleep2692726-21hddtemp_smartct2
6905412101,3sleep20-21swapper/22
6732922101,4sleep1464099-21diskmemload1
6640002101,3sleep20-21swapper/22
6569182101,3sleep2464099-21diskmemload2
6500662101,4sleep0650090-21apt-get0
6248342101,2sleep00-21swapper/00
6145012100,1sleep00-21swapper/00
6079552101,3sleep1464099-21diskmemload1
5958062101,2sleep20-21swapper/22
5863862101,4sleep0464099-21diskmemload0
5726902101,6sleep1572698-21ssh1
5471042101,3sleep0464099-21diskmemload0
5460422100,2sleep20-21swapper/22
5322712101,4sleep2532493-21hddtemp_smartct2
5191912100,5sleep0519295-21collect20
4958122101,3sleep0464099-21diskmemload0
4885992101,4sleep1464099-21diskmemload1
4783252101,3sleep20-21swapper/22
4729452101,2sleep10-21swapper/11
4685112101,5sleep1468614-21ssh1
4321212101,4sleep2432193-21munin-run2
4257162101,5sleep2425807-21gcc2
3935392101,5sleep2393589-21ntpq2
3897402101,4sleep0389949-21hddtemp_smartct0
3838652101,4sleep0384107-21df_inode0
3716142101,4sleep1371664-21ntpq1
3559362101,4sleep0356180-21users0
3521692101,5sleep2353852-21kthreadcore2
808901291,2sleep20-21swapper/22
730186292,3sleep0730250-21/usr/sbin/munin0
722598291,2sleep20-21swapper/22
716314290,3sleep20-21swapper/22
706076291,3sleep1464099-21diskmemload1
699554291,4sleep1699565-21ssh1
689392291,3sleep00-21swapper/00
623054291,2sleep00-21swapper/00
604493291,4sleep1604507-21ssh1
582281291,3sleep2464099-21diskmemload2
573863291,3sleep00-21swapper/00
564881291,2sleep00-21swapper/00
562366291,2sleep20-21swapper/22
556094291,3sleep0464099-21diskmemload0
502627291,2sleep20-21swapper/22
468745291,5sleep0468835-21proc_pri0
448366291,4sleep2448547-21ls2
443099290,4sleep0443325-21df_inode0
437714291,3sleep1437733-21apt-get1
431233291,5sleep1431335-21processes1
416402291,3sleep0416421-21apt-get0
416311290,4sleep2415878-21apt-get2
400117290,3sleep2400136-21apt-get2
377928294,2sleep0251irq_work/00
367524291,3sleep0367543-21apt-get0
362679291,4sleep2362963-21cut2
344998290,1sleep00-21swapper/00
781995280,3sleep1464099-21diskmemload1
453297280,1sleep00-21swapper/00
433740281,3sleep2435487-21grep2
422033281,2sleep1422157-21perl1
409285281,2sleep10-21swapper/11
400411281,3sleep1400625-21smartctl1
382967281,3sleep0383184-21users0
382795280,4sleep10-21swapper/11
373141280,1sleep10-21swapper/11
340824281,3sleep2341069-21cstates2
333974281,3sleep2334418-21kthreadcore2
458748270,1sleep00-21swapper/00
454888271,3sleep20-21swapper/22
448344271,2sleep1448363-21apt-get1
441632270,4sleep00-21swapper/00
438958271,3sleep20-21swapper/22
437235270,1sleep00-21swapper/00
431847270,1sleep00-21swapper/00
421208270,1sleep10-21swapper/11
405467270,2sleep20-21swapper/22
404850270,1sleep00-21swapper/00
398887270,2sleep10-21swapper/11
389449270,2sleep10-21swapper/11
362458271,3sleep10-21swapper/11
358122271,2sleep00-21swapper/00
351469270,2sleep10-21swapper/11
350942270,1sleep10-21swapper/11
340802270,2sleep10-21swapper/11
338550270,1sleep00-21swapper/00
463889260,4sleep10-21swapper/11
463845260,1sleep00-21swapper/00
447901260,1sleep20-21swapper/22
443326260,3sleep1443461-21expr1
415856260,4sleep20-21swapper/22
409819261,2sleep0409925-21timerandwakeup0
395067261,2sleep20-21swapper/22
388616260,1sleep10-21swapper/11
388584260,1sleep20-21swapper/22
377838260,4sleep20-21swapper/22
366882260,1sleep00-21swapper/00
378482250,2sleep20-21swapper/22
357095250,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional