You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 00:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Fri May 22, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
323994212056,59sleep30-21swapper/33
32399728062,9sleep10-21swapper/11
32375728061,12sleep20-21swapper/22
32389927855,7sleep00-21swapper/00
32413999242,17cyclictest0-21swapper/33
324139992320,1cyclictest0-21swapper/33
324139992318,2cyclictest0-21swapper/33
324139992218,2cyclictest0-21swapper/33
324139992218,2cyclictest0-21swapper/33
32413999221,18cyclictest541ktimers/33
7006752212,8sleep0455679-21diskmemload0
32413999214,14cyclictest66950irq/132-enp1s0-TxRx-33
32413999212,15cyclictest66950irq/132-enp1s0-TxRx-33
324139992117,2cyclictest0-21swapper/33
32413999211,16cyclictest551538-21taskset3
324139992110,8cyclictest481371-21chrt3
324139992110,7cyclictest795158-21taskset3
324139992110,7cyclictest401804-21chrt3
6200522202,8sleep1455679-21diskmemload1
5688942201,3sleep00-21swapper/00
4797722201,3sleep10-21swapper/11
32413999206,10cyclictest0-21swapper/33
324139992018,1cyclictest0-21swapper/33
32413999201,15cyclictest66950irq/132-enp1s0-TxRx-33
32413999201,15cyclictest647637-21taskset3
32413999201,15cyclictest0-21swapper/33
324139992011,2cyclictest0-21swapper/33
32413999196,10cyclictest0-21swapper/33
32413999195,12cyclictest0-21swapper/33
32413999194,12cyclictest0-21swapper/33
32413999193,13cyclictest0-21swapper/33
324139991916,1cyclictest0-21swapper/33
324139991915,2cyclictest0-21swapper/33
324139991915,2cyclictest0-21swapper/33
324139991914,2cyclictest0-21swapper/33
324139991914,2cyclictest0-21swapper/33
32413999191,15cyclictest699042-21taskset3
32413999191,15cyclictest0-21swapper/33
32413999190,15cyclictest727846-21taskset3
5081712181,3sleep10-21swapper/11
4654282181,3sleep2466124-21kthreadcore2
32413999189,5cyclictest3772092sleep33
32413999182,13cyclictest541ktimers/33
32413999182,11cyclictest455707-21turbostat3
324139991816,1cyclictest0-21swapper/33
324139991815,1cyclictest0-21swapper/33
324139991814,2cyclictest0-21swapper/33
324139991814,2cyclictest0-21swapper/33
324139991814,2cyclictest0-21swapper/33
324139991814,2cyclictest0-21swapper/33
324139991814,2cyclictest0-21swapper/33
324139991812,4cyclictest0-21swapper/33
32413999181,14cyclictest0-21swapper/33
32413999181,14cyclictest0-21swapper/33
32413999181,14cyclictest0-21swapper/33
32413999181,14cyclictest0-21swapper/33
32413999181,13cyclictest5352062sleep33
32413999181,13cyclictest0-21swapper/33
32413999181,13cyclictest0-21swapper/33
32413999180,15cyclictest0-21swapper/33
6637972171,3sleep00-21swapper/00
5679622171,3sleep20-21swapper/22
32413999174,1cyclictest0-21swapper/33
32413999173,10cyclictest0-21swapper/33
324139991715,1cyclictest0-21swapper/33
324139991715,1cyclictest0-21swapper/33
324139991713,2cyclictest0-21swapper/33
32413999170,14cyclictest0-21swapper/33
32413999170,14cyclictest0-21swapper/33
8042912161,2sleep20-21swapper/22
7422612161,10sleep1455679-21diskmemload1
73438121610,3sleep1455679-21diskmemload1
7015052161,9sleep10-21swapper/11
6885762161,4sleep1455679-21diskmemload1
6833502161,3sleep10-21swapper/11
6388722160,12sleep20-21swapper/22
6126102161,2sleep00-21swapper/00
5219772160,8sleep1522022-21cat1
5139492160,2sleep00-21swapper/00
324139991614,1cyclictest0-21swapper/33
324139991614,1cyclictest0-21swapper/33
324139991614,1cyclictest0-21swapper/33
324139991614,1cyclictest0-21swapper/33
324139991612,2cyclictest0-21swapper/33
32413999161,12cyclictest0-21swapper/33
32413999160,13cyclictest0-21swapper/33
7613302151,10sleep0455679-21diskmemload0
7228682152,4sleep1723550-21cut1
5786882151,4sleep1455679-21diskmemload1
32413999150,13cyclictest0-21swapper/33
32413999150,13cyclictest0-21swapper/33
7347232141,8sleep0455679-21diskmemload0
7062752141,9sleep2706330-21ssh2
6732942141,3sleep1673305-21ssh1
6478182140,1sleep10-21swapper/11
6311072141,8sleep1455679-21diskmemload1
6154492140,3sleep00-21swapper/00
5953972141,4sleep1455679-21diskmemload1
5880152141,8sleep00-21swapper/00
5772122141,8sleep0455679-21diskmemload0
5511112145,2sleep1171rcu_preempt1
5162582141,9sleep2516271-21ssh2
4009982141,9sleep1401046-21ntpq1
32413999140,11cyclictest0-21swapper/33
8133272131,3sleep1813341-21ssh1
7957532131,8sleep2795767-21ssh2
7778772131,3sleep0777880-21ssh0
7777302131,2sleep10-21swapper/11
7527062130,5sleep20-21swapper/22
6929182130,2sleep20-21swapper/22
6307592131,2sleep20-21swapper/22
5590502131,4sleep1559064-21ssh1
5545902131,7sleep0556282-21tr0
5053102131,8sleep0505324-21ssh0
4064902131,8sleep0406555-21cut0
3905362130,1sleep00-21swapper/00
8127832121,5sleep2455679-21diskmemload2
8038162121,2sleep10-21swapper/11
7712002121,5sleep00-21swapper/00
7641692121,7sleep10-21swapper/11
7435932121,6sleep2455679-21diskmemload2
7271652121,7sleep2455679-21diskmemload2
6545952121,6sleep1455679-21diskmemload1
6481862120,1sleep20-21swapper/22
6215482121,7sleep2621563-21ssh2
5941452120,1sleep20-21swapper/22
5759142121,5sleep2455679-21diskmemload2
5677462121,3sleep1567757-21ssh1
5579192121,5sleep20-21swapper/22
5407222121,7sleep0455679-21diskmemload0
5322222121,7sleep2532231-21sh2
4818642121,3sleep2481878-21ssh2
4808832121,3sleep00-21swapper/00
4439522121,6sleep2443999-21ntpq2
4439522121,6sleep2443999-21ntpq2
4292622121,7sleep1429440-21expr1
3898192121,7sleep2389856-21collect22
3587942120,7sleep1358734-31find1
3417402121,7sleep2341777-21switchtime2
3293432120,1sleep20-21swapper/22
7783512111,5sleep20-21swapper/22
7677322111,5sleep2767784-21ntpq2
7621052111,6sleep2455679-21diskmemload2
7509182110,4sleep00-21swapper/00
7107142112,4sleep1455679-21diskmemload1
7020832111,5sleep190-21kswapd01
6926372110,2sleep00-21swapper/00
6818032111,6sleep0681814-21ssh0
6670362112,4sleep0667058-21apt-get0
5843932111,3sleep20-21swapper/22
5503322111,2sleep00-21swapper/00
5393492112,4sleep1539445-21ntpq1
5332282111,3sleep10-21swapper/11
5321002111,6sleep0455679-21diskmemload0
5241662111,5sleep0455679-21diskmemload0
4973342111,6sleep0455679-21diskmemload0
4899982111,6sleep00-21swapper/00
4878812111,3sleep20-21swapper/22
4825272111,5sleep1482712-21diskstats1
4618392110,3sleep20-21swapper/22
4547322111,5sleep2454785-21ntp_states2
4343072111,5sleep0434355-21munin-run0
4289782111,6sleep1429083-21latency_hist1
4173802111,5sleep1417437-21ntp_states1
3954532111,5sleep1395503-21ntpq1
3913782111,4sleep0391397-21apt-get0
3588142111,4sleep072-21kcompactd00
3522492111,5sleep2352390-21ntpq2
3373872111,4sleep0337406-21apt-get0
3294352111,5sleep0330088-21diskstats0
3265212111,6sleep1328234-21/usr/sbin/munin1
8225392101,2sleep20-21swapper/22
8224182101,4sleep1455679-21diskmemload1
8211932101,4sleep0455679-21diskmemload0
8127152100,2sleep00-21swapper/00
8044152102,3sleep0455679-21diskmemload0
7938932101,5sleep0793953-21ntpq0
7863282105,2sleep00-21swapper/00
7524472101,3sleep10-21swapper/11
7425632101,3sleep0455679-21diskmemload0
7194962101,3sleep00-21swapper/00
7169132101,4sleep2455679-21diskmemload2
7161762101,4sleep0455679-21diskmemload0
7097882101,2sleep0171rcu_preempt0
6815832101,4sleep2455679-21diskmemload2
6725002101,3sleep20-21swapper/22
6662922101,2sleep10-21swapper/11
6555072101,4sleep0455679-21diskmemload0
6388062101,4sleep1455679-21diskmemload1
6042322101,4sleep0455679-21diskmemload0
6034782101,2sleep10-21swapper/11
5849302100,1sleep00-21swapper/00
5522972101,4sleep2552309-21ssh2
5434782100,2sleep20-21swapper/22
5243962101,4sleep2455679-21diskmemload2
5078202101,3sleep20-21swapper/22
4993002100,6sleep2499312-21ssh2
4976042101,4sleep1455679-21diskmemload1
4647532101,4sleep0455679-21diskmemload0
4556712101,5sleep0455679-21diskmemload0
4503542101,4sleep1450367-21munin-run1
4493712101,5sleep2449422-21ld2
4239392101,4sleep2423957-21apt-get2
4117492101,4sleep1411803-21ntpq1
3972542101,4sleep0397463-21hddtemp_smartct0
3753852101,4sleep0375598-21hddtemp_smartct0
3751172101,6sleep2375266-21cstates2
3470472101,4sleep2347093-21ntpq2
796244291,4sleep1796255-21ssh1
787720291,4sleep2455679-21diskmemload2
787044290,1sleep10-21swapper/11
757506291,2sleep10-21swapper/11
733233291,2sleep20-21swapper/22
691547291,3sleep2455679-21diskmemload2
658297291,2sleep20-21swapper/22
655325291,4sleep2655340-21ssh2
640748292,3sleep00-21swapper/00
638740291,4sleep0455679-21diskmemload0
628446291,2sleep00-21swapper/00
610995290,4sleep10-21swapper/11
605517291,3sleep20-21swapper/22
605120291,2sleep20-21swapper/22
583583291,2sleep10-21swapper/11
513614291,3sleep1455679-21diskmemload1
461911290,2sleep00-21swapper/00
455973290,1sleep10-21swapper/11
435435291,4sleep0437100-21cut0
428903290,1sleep00-21swapper/00
418326291,4sleep0418571-21date0
402356290,2sleep20-21swapper/22
402335291,3sleep1402354-21apt-get1
390834290,3sleep172-21kcompactd01
380469291,4sleep2380727-21df_inode2
375095291,3sleep1375114-21apt-get1
368401291,4sleep0368514-21ntpq0
361036291,4sleep2362860-21cut2
358795290,1sleep20-21swapper/22
349510291,4sleep0351243-21cut0
324925291,4sleep0326519-21cut0
469183281,3sleep1455679-21diskmemload1
439466280,2sleep10-21swapper/11
423525280,1sleep10-21swapper/11
422759281,3sleep2422847-21ps2
411805281,3sleep2407662-21/usr/sbin/munin2
407661281,2sleep00-21swapper/00
397536281,3sleep2421rcuc/22
391343280,3sleep2390846-21apt-get2
385503280,1sleep00-21swapper/00
337973280,4sleep10-21swapper/11
330089281,3sleep1330782-21grep1
444637270,1sleep10-21swapper/11
444637270,1sleep10-21swapper/11
434083270,1sleep20-21swapper/22
418299270,1sleep00-21swapper/00
380448270,2sleep10-21swapper/11
374825270,1sleep10-21swapper/11
370819271,3sleep0372524-21kthreadcore0
368521270,2sleep10-21swapper/11
359636270,4sleep00-21swapper/00
451220261,2sleep10-21swapper/11
434868261,2sleep20-21swapper/22
413232260,1sleep20-21swapper/22
373675261,2sleep20-21swapper/22
359709261,2sleep10-21swapper/11
347762260,4sleep10-21swapper/11
336749260,1sleep20-21swapper/22
450234250,3sleep00-21swapper/00
444611250,3sleep00-21swapper/00
444611250,3sleep00-21swapper/00
351245250,1sleep10-21swapper/11
345335250,1sleep00-21swapper/00
364368240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional