You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 15:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Tue Jul 28, 2026 12:43:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1633328995542,10cyclictest1708738-21munin-run10:15:010
1633329995242,5cyclictest0-21swapper/110:30:521
1633329995242,5cyclictest0-21swapper/110:30:521
1633329995137,9cyclictest0-21swapper/110:35:511
1633329995126,21cyclictest1648841-21fschecks_count08:00:401
1633329995126,21cyclictest1648841-21fschecks_count08:00:401
1633329995124,23cyclictest0-21swapper/111:40:011
1633329994940,6cyclictest421-21dbus-daemon12:35:011
1633328994937,8cyclictest1772318-21systemd-run12:00:010
163332899492,41cyclictest1697786-21ssh09:55:480
1633329994710,17cyclictest1788646-21ssh12:25:521
163332899473,28cyclictest1761430-21rm11:40:520
1633329994628,15cyclictest1726964-21kworker/u8:8-rpciod11:00:011
1633330994537,5cyclictest1696650-21ssh09:55:012
1633329994531,10cyclictest1727643-21entropy10:45:391
163332999452,7cyclictest1679546-21ssh09:25:511
1633329994527,15cyclictest1636318-21munin-run07:20:011
1633330994435,5cyclictest1634836-21systemd-run07:15:012
1633329994418,7cyclictest171rcu_preempt10:20:461
1633329994418,7cyclictest171rcu_preempt10:20:461
1633330994335,5cyclictest1763230-21munin-plugin-st11:45:012
1633330994334,6cyclictest265-21systemd-journal11:15:012
163332999434,14cyclictest1785577-21ntpq12:20:501
163332999433,7cyclictest1634168-21ntpq07:10:451
163332999432,16cyclictest1761380-21ntpq11:40:501
163332999432,16cyclictest1749345-21ntpq11:20:531
163332999431,11cyclictest1755377-21ntpq11:30:511
163332999431,11cyclictest1659736-21ntpq08:35:501
1633329994228,10cyclictest1672773-21acpi09:15:361
1633329994228,10cyclictest1672773-21acpi09:15:361
163332999422,5cyclictest1740234-21ntpq11:05:521
163332999422,5cyclictest1682539-21ntpq09:30:461
163332999421,16cyclictest1737125-21ntpq11:00:481
163332999421,12cyclictest1658196-21ntpq08:30:511
1633328994234,5cyclictest1676373-21missed_timers09:20:440
1633328994234,5cyclictest1676373-21missed_timers09:20:440
163332899423,30cyclictest1703901-21ssh10:05:560
1633330994132,6cyclictest1-21systemd07:20:012
163332999412,7cyclictest1652237-21ntpq08:10:511
163332999412,6cyclictest1697866-21ntpq09:55:511
163332999412,6cyclictest1644690-21ntpq07:45:461
163332999411,16cyclictest1770354-21ntpq11:55:471
163332999411,13cyclictest1732996-21/usr/sbin/munin10:55:021
163332999411,10cyclictest1691676-21ntpq09:45:461
163332999411,10cyclictest1653790-21ntpq08:15:541
163332999411,10cyclictest1646198-21ntpq07:50:481
1633328994130,8cyclictest1-21systemd12:38:310
1633330994035,2cyclictest0-21swapper/212:10:012
1633330994035,2cyclictest0-21swapper/212:10:012
1633330994034,3cyclictest0-21swapper/209:10:012
1633330994033,3cyclictest0-21swapper/210:05:572
163332999407,30cyclictest1768583-21sh11:52:521
163332999404,13cyclictest1703887-21ntpq10:05:561
163332999403,6cyclictest1716007-21ntpq10:25:511
163332999403,6cyclictest1716007-21ntpq10:25:511
1633329994032,5cyclictest1000-21mta-sts-daemon07:55:491
1633329994029,7cyclictest843869-21cups-browsed11:18:271
163332999401,5cyclictest1661176-21ntpq08:40:471
163332999401,15cyclictest1794689-21ntpq12:35:541
163332999401,11cyclictest1667778-21users09:01:011
163332999401,10cyclictest1688630-21ntpq09:40:451
1633328994025,12cyclictest1654851-21fschecks_count08:20:390
1633330993930,6cyclictest1-21systemd09:15:012
1633330993929,5cyclictest0-21swapper/210:20:482
1633330993929,5cyclictest0-21swapper/210:20:482
163332999398,27cyclictest1784363-21cron12:20:011
163332999394,30cyclictest843869-21cups-browsed08:50:521
163332999392,6cyclictest1776581-21ntpq12:05:541
163332999392,6cyclictest1776581-21ntpq12:05:541
163332999391,9cyclictest1640260-21ntpq07:30:501
163332999391,8cyclictest1752403-21ntpq11:25:541
163332999391,8cyclictest1752403-21ntpq11:25:541
163332999391,5cyclictest1668740-21ntpq09:05:531
163332999391,11cyclictest1641692-21ntpq07:35:481
1633328993933,2cyclictest0-21swapper/009:15:480
1633328993933,2cyclictest0-21swapper/009:15:480
1633328993931,5cyclictest1784764-21gpgconf12:20:280
1633330993831,3cyclictest0-21swapper/210:45:012
1633330993828,5cyclictest1002-21mta-sts-daemon09:45:512
163332999383,5cyclictest1650702-21ntpq08:05:511
163332999382,5cyclictest1779565-21ntpq12:10:571
163332999382,11cyclictest1709794-21/usr/sbin/munin10:15:461
163332999382,11cyclictest1709794-21/usr/sbin/munin10:15:461
163332999381,9cyclictest1774119-21ssh12:01:101
163332999381,9cyclictest1725053-21ntpq10:40:491
163332999381,9cyclictest1685207-21df_inode09:35:411
163332999381,7cyclictest1654871-21/usr/sbin/munin08:20:391
163332999381,6cyclictest1662677-21ntpq08:45:501
163332999381,5cyclictest1638744-21ntpq07:25:491
163332999381,11cyclictest1765494-21ssh11:47:381
163332999381,11cyclictest1694784-21awk09:50:511
163332999380,9cyclictest1643166-21ntp_kernel_err07:40:451
1633330993730,3cyclictest0-21swapper/211:20:012
1633330993729,5cyclictest1646691-21unixbench_multi07:50:592
1633330993729,5cyclictest1637851-21systemd-run07:25:022
1633330993728,6cyclictest1716516-21unixbench_singl10:25:592
1633330993728,6cyclictest1716516-21unixbench_singl10:25:592
163332999373,5cyclictest1637250-21ntpq07:20:501
163332999371,4cyclictest1665712-21ntpq08:55:491
163332999371,4cyclictest1665712-21ntpq08:55:491
1633329993713,21cyclictest0-21swapper/111:10:521
163332999371,13cyclictest1670498-21ntpq09:10:511
163332999371,11cyclictest1708454-21ssh10:14:091
1633328993730,4cyclictest1648511-21latency_hist08:00:020
1633330993631,4cyclictest1633327-21cyclictest10:45:552
163332999363,6cyclictest1700866-21ntpq10:00:501
163332999361,5cyclictest1676122-21/usr/sbin/munin09:20:401
163332999361,5cyclictest1676122-21/usr/sbin/munin09:20:401
1633328993630,4cyclictest1748829-21cut11:20:400
1633328993630,4cyclictest1726930-21cron10:45:010
1633328993630,4cyclictest1003-21mta-sts-daemon10:00:530
1633328993629,4cyclictest1782809-21switchtime12:15:590
1633330993528,4cyclictest1002-21mta-sts-daemon09:33:382
1633330993528,4cyclictest1001-21mta-sts-daemon09:59:342
1633330993528,3cyclictest0-21swapper/210:15:022
1633330993527,5cyclictest1734689-21unixbench_singl10:55:592
1633330993527,4cyclictest0-21swapper/212:15:472
1633330993524,7cyclictest420-21avahi-daemon09:29:582
163332999351,9cyclictest1656740-21ntpq08:25:491
1633328993529,4cyclictest1001-21mta-sts-daemon09:35:010
1633328993522,9cyclictest1670431-21ssh09:10:490
1633328993516,12cyclictest151ktimers/009:50:490
1633330993427,3cyclictest0-21swapper/210:00:482
1633330993426,5cyclictest1648305-21munin-run08:00:012
1633330993426,3cyclictest0-21swapper/211:35:482
1633330993426,3cyclictest0-21swapper/211:09:482
1633330993425,6cyclictest843869-21cups-browsed10:39:092
1633330993424,7cyclictest1719637-21time-dir10:31:042
1633330993424,7cyclictest1719637-21time-dir10:31:042
1633330993423,8cyclictest485-21in:imuxsock10:55:012
1633328993425,3cyclictest0-21swapper/010:55:390
1633328993424,6cyclictest420-21avahi-daemon09:48:090
1633328993423,8cyclictest1752258-21sh11:25:480
1633328993423,8cyclictest1752258-21sh11:25:480
1633328993420,11cyclictest0-21swapper/010:35:400
1633330993327,3cyclictest0-21swapper/211:05:012
1633330993326,4cyclictest1781344-21munin-plugin-st12:15:002
1633330993326,3cyclictest0-21swapper/211:25:592
1633330993326,3cyclictest0-21swapper/211:25:592
1633330993325,5cyclictest1764857-21unixbench-2d11:45:592
1633330993325,5cyclictest1642306-21sendmail07:40:012
1633330993324,7cyclictest1665128-21aten_r2power_fr08:55:382
1633330993324,7cyclictest1665128-21aten_r2power_fr08:55:382
1633330993319,3cyclictest171rcu_preempt12:30:022
1633328993326,4cyclictest1718556-21fschecks_count10:30:410
1633328993326,4cyclictest1718556-21fschecks_count10:30:410
1633328993321,9cyclictest1654317-21cut08:20:010
1633328993317,10cyclictest151ktimers/007:20:500
1633330993226,3cyclictest0-21swapper/212:00:012
1633330993226,3cyclictest0-21swapper/211:35:002
1633330993224,5cyclictest1711743-21ssh10:20:012
1633330993224,5cyclictest1711743-21ssh10:20:012
1633330993224,3cyclictest0-21swapper/209:35:552
1633330993217,10cyclictest843869-21cups-browsed09:18:232
1633330993217,10cyclictest843869-21cups-browsed09:18:232
163332899323,5cyclictest171rcu_preempt08:40:510
1633328993226,3cyclictest1003-21mta-sts-daemon10:50:500
1633328993225,5cyclictest0-21swapper/008:40:010
163332899322,24cyclictest1685683-21ssh09:35:520
1633328993221,8cyclictest265-21systemd-journal07:50:010
1633328993221,7cyclictest1788413-21missed_timers12:25:450
1633328993220,9cyclictest1791415-21missed_timers12:30:480
1633328993220,9cyclictest1690587-21cron09:45:010
1633328993214,13cyclictest151ktimers/008:50:480
163332899320,28cyclictest1737350-21mailstats11:00:550
163333099313,23cyclictest1676571-21ssh09:20:502
163333099313,23cyclictest1676571-21ssh09:20:502
1633330993124,4cyclictest1769250-21ssh11:55:012
1633330993124,3cyclictest0-21swapper/208:20:012
1633330993123,5cyclictest1690593-21latency_hist09:45:012
1633330993123,4cyclictest1656568-21missed_timers08:25:442
1633330993122,7cyclictest1792144-21unixbench_multi12:31:042
163332899313,6cyclictest171rcu_preempt11:35:520
1633328993126,3cyclictest487-21rs:main0
1633328993125,4cyclictest1745080-21systemd-run11:15:020
1633328993124,4cyclictest1776511-21ntp_offset12:05:510
1633328993124,4cyclictest1776511-21ntp_offset12:05:510
163332899311,23cyclictest1773525-21ssh12:00:520
1633328993110,13cyclictest151ktimers/011:30:490
1633330993023,4cyclictest0-21swapper/212:20:592
1633330993022,5cyclictest1658417-21systemctl08:31:002
163333099302,20cyclictest1794722-21ssh12:35:552
1633330993021,6cyclictest1663235-21head08:46:052
1633329993022,4cyclictest1633506-21latency_hist07:10:031
1633328993024,3cyclictest0-21swapper/010:20:010
1633328993024,3cyclictest0-21swapper/010:20:010
1633328993020,5cyclictest0-21swapper/009:00:010
1633328993020,5cyclictest0-21swapper/009:00:010
163332899301,15cyclictest0-21swapper/011:45:530
1633330992921,5cyclictest1660355-21latency_hist08:40:022
1633330992921,4cyclictest0-21swapper/211:20:502
1633328992924,3cyclictest1003-21mta-sts-daemon09:28:460
1633328992923,3cyclictest1766806-21aten2.4-expect11:50:380
1633328992917,10cyclictest0-21swapper/010:45:580
1633328992821,5cyclictest439-21systemd-logind10:26:270
1633328992821,5cyclictest439-21systemd-logind10:26:270
1633328992812,3cyclictest0-21swapper/007:35:010
163332899281,22cyclictest665-21snmpd12:10:580
1633328992811,14cyclictest1666839-21fschecks_count09:00:420
163333099274,18cyclictest1773447-21ssh12:00:492
163333099273,3cyclictest1644728-21ntpq07:45:482
1633330992721,3cyclictest0-21swapper/208:25:012
1633330992719,5cyclictest843869-21cups-browsed07:31:122
1633330992713,11cyclictest351ktimers/207:25:512
1633328992720,4cyclictest1681304-21kworker/u8:1-rpciod10:21:320
1633328992720,4cyclictest1681304-21kworker/u8:1-rpciod10:21:320
1633328992720,4cyclictest1635327-21fschecks_count07:15:420
1633330992616,3cyclictest1633327-21cyclictest08:05:522
163332899262,4cyclictest0-21swapper/007:35:480
1633328992621,3cyclictest0-21swapper/007:25:580
1633328992621,2cyclictest0-21swapper/009:10:020
1633328992620,4cyclictest1652412-21systemctl08:10:560
1633328992620,4cyclictest1633327-21cyclictest07:45:010
1633328992619,5cyclictest0-21swapper/011:20:010
163332899261,18cyclictest1740177-21ssh11:05:510
1633330992519,3cyclictest0-21swapper/208:50:582
1633330992516,6cyclictest1660835-21fschecks_count08:40:422
1633330992516,5cyclictest1634838-21kworker/u8:5-xprtiod09:01:192
1633328992518,4cyclictest1634805-21cron07:15:000
1633328992517,6cyclictest843869-21cups-browsed08:31:030
1633330992418,3cyclictest1000-21mta-sts-daemon08:11:002
1633330992412,8cyclictest265-21systemd-journal08:05:012
1633330992412,8cyclictest265-21systemd-journal08:05:012
1633328992418,4cyclictest1649419-21sendmail_mailtr08:00:550
1633328992418,4cyclictest1649419-21sendmail_mailtr08:00:550
1633328992417,5cyclictest421-21dbus-daemon07:10:020
1633328992416,6cyclictest1003-21mta-sts-daemon08:29:350
163333099230,4cyclictest171rcu_preempt07:40:482
1633328992316,5cyclictest485-21in:imuxsock08:08:220
1633328992217,2cyclictest0-21swapper/007:50:590
163333199192,9cyclictest0-21swapper/310:30:533
163333199192,9cyclictest0-21swapper/310:30:533
1633330991914,2cyclictest0-21swapper/207:10:022
163333199182,9cyclictest0-21swapper/309:25:533
163333199172,9cyclictest0-21swapper/312:25:503
163333199172,9cyclictest0-21swapper/312:11:003
163333199172,8cyclictest0-21swapper/312:30:563
163333199171,9cyclictest0-21swapper/312:15:553
163333199171,8cyclictest0-21swapper/310:10:513
163333199163,7cyclictest0-21swapper/311:50:473
163333199162,9cyclictest0-21swapper/311:10:553
163333199162,8cyclictest0-21swapper/309:35:513
163333199162,8cyclictest0-21swapper/309:30:493
163333199161,9cyclictest0-21swapper/311:40:473
163333199161,9cyclictest0-21swapper/310:05:553
163333199161,8cyclictest0-21swapper/310:35:503
163333199161,8cyclictest0-21swapper/308:40:533
163333199161,7cyclictest0-21swapper/308:00:493
163333199161,7cyclictest0-21swapper/308:00:493
163333199161,10cyclictest0-21swapper/311:00:493
163333199152,8cyclictest0-21swapper/311:45:523
163333199152,8cyclictest0-21swapper/310:20:423
163333199152,8cyclictest0-21swapper/310:20:423
163333199152,8cyclictest0-21swapper/309:15:493
163333199152,8cyclictest0-21swapper/309:15:493
163333199152,7cyclictest0-21swapper/309:20:443
163333199152,7cyclictest0-21swapper/309:20:443
163333199152,6cyclictest0-21swapper/309:05:523
163333199152,5cyclictest0-21swapper/308:45:503
163333199151,8cyclictest0-21swapper/311:20:473
163333199151,8cyclictest0-21swapper/310:55:483
163333199151,8cyclictest0-21swapper/310:50:383
163333199151,8cyclictest0-21swapper/310:45:463
163333199143,6cyclictest0-21swapper/311:35:553
163333199143,5cyclictest0-21swapper/307:10:493
163333199142,8cyclictest0-21swapper/311:55:383
163333199142,8cyclictest0-21swapper/311:25:553
163333199142,8cyclictest0-21swapper/311:25:553
163333199142,8cyclictest0-21swapper/310:25:383
163333199142,8cyclictest0-21swapper/310:25:383
163333199142,7cyclictest0-21swapper/309:10:513
163333199141,8cyclictest0-21swapper/312:00:543
163333199141,8cyclictest0-21swapper/311:30:193
163333199141,8cyclictest0-21swapper/311:15:513
163333199141,8cyclictest0-21swapper/309:45:473
163333199141,7cyclictest0-21swapper/307:25:473
163333199141,6cyclictest0-21swapper/308:25:453
163333199141,6cyclictest0-21swapper/308:15:543
163333199141,6cyclictest0-21swapper/307:30:493
163333199132,7cyclictest0-21swapper/312:20:473
163333199132,7cyclictest0-21swapper/309:40:373
163333199132,5cyclictest0-21swapper/308:50:503
163333199131,8cyclictest0-21swapper/311:05:383
163333199131,8cyclictest0-21swapper/310:40:433
163333199131,7cyclictest0-21swapper/312:35:433
163333199131,7cyclictest0-21swapper/312:05:493
163333199131,7cyclictest0-21swapper/312:05:493
163333199131,7cyclictest0-21swapper/310:15:483
163333199131,7cyclictest0-21swapper/310:15:483
163333199131,6cyclictest0-21swapper/308:40:013
163333199122,7cyclictest0-21swapper/309:50:533
163333199122,7cyclictest0-21swapper/307:55:483
163333199121,7cyclictest0-21swapper/309:55:383
163333199121,7cyclictest0-21swapper/308:31:033
163333199121,7cyclictest0-21swapper/307:35:473
163333199121,6cyclictest0-21swapper/309:00:513
163333199121,6cyclictest0-21swapper/308:05:503
163333199121,6cyclictest0-21swapper/307:40:533
163333199120,6cyclictest0-21swapper/308:20:473
163333199112,5cyclictest0-21swapper/308:10:473
163333199112,5cyclictest0-21swapper/307:45:413
163333199112,5cyclictest0-21swapper/307:20:493
163333199112,5cyclictest0-21swapper/307:15:463
163333199111,6cyclictest0-21swapper/310:00:373
163333199102,5cyclictest0-21swapper/307:50:413
16333319992,5cyclictest0-21swapper/308:55:413
16333319992,5cyclictest0-21swapper/308:55:413
16333319991,5cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional