You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-18 - 11:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Mon May 18, 2026 00:43:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1710593996523,14cyclictest1866512-21kworker/u8:2+rpciod00:00:560
1710593996523,14cyclictest1866512-21kworker/u8:2+rpciod00:00:560
1710595996220,7cyclictest171rcu_preempt23:35:582
1710593996125,11cyclictest1817357-21grep22:36:040
1710595995343,5cyclictest0-21swapper/220:15:582
1710594995346,3cyclictest0-21swapper/122:16:021
1710595995232,17cyclictest1889079-21sh00:05:502
1710595995146,2cyclictest0-21swapper/223:20:562
1710595995146,2cyclictest0-21swapper/223:20:562
1710594994940,5cyclictest421-21dbus-daemon22:35:021
1710593994840,5cyclictest1904146-21systemd-run00:25:010
1710595994639,3cyclictest0-21swapper/200:40:012
1710595994637,6cyclictest1871895-21latency_hist23:45:022
1710595994636,5cyclictest0-21swapper/200:20:532
1710595994635,7cyclictest1829003-21ssh22:51:012
1710595994635,7cyclictest1829003-21ssh22:51:012
1710594994637,6cyclictest1867782-21true23:40:011
1710594994626,4cyclictest171rcu_preempt00:00:001
171059599452,36cyclictest1776423-21ssh21:45:592
1710594994539,3cyclictest0-21swapper/122:51:051
1710594994539,3cyclictest0-21swapper/122:51:051
1710595994436,5cyclictest1803260-21readlink22:20:002
1710594994433,7cyclictest1837068-21cron23:01:001
171059599435,27cyclictest1905463-21ssh00:25:542
1710595994334,5cyclictest1900120-21ssh00:20:012
1710595994332,8cyclictest1851639-21sendmail23:20:012
1710595994329,6cyclictest420-21avahi-daemon21:27:312
1710595994329,6cyclictest420-21avahi-daemon21:27:312
1710594994335,4cyclictest0-21swapper/122:35:561
1710594994333,7cyclictest1797162-21unixbench_singl22:11:071
1710594994333,7cyclictest1797162-21unixbench_singl22:11:071
1710595994231,7cyclictest1745092-21systemd-run21:05:012
1710593994231,6cyclictest151ktimers/021:10:570
1710595994132,6cyclictest1752301-21mailstats21:16:082
1710595994130,6cyclictest1793056-21grep22:06:062
1710595994127,9cyclictest421-21dbus-daemon21:55:582
171059599412,31cyclictest1764119-21ssh21:30:542
171059599412,31cyclictest1764119-21ssh21:30:542
1710594994133,5cyclictest1892904-21fschecks_count00:10:481
1710595994034,3cyclictest0-21swapper/223:50:482
1710595994028,9cyclictest1885182-21needreboot00:00:532
1710595994028,9cyclictest1885182-21needreboot00:00:532
171059599402,28cyclictest1856951-21ssh23:25:532
171059599402,28cyclictest1856951-21ssh23:25:532
1710594994032,5cyclictest421-21dbus-daemon20:05:011
1710594994031,6cyclictest421-21dbus-daemon22:00:011
1710594994031,6cyclictest4037003-21cups-browsed21:28:361
1710594994031,6cyclictest4037003-21cups-browsed21:28:361
1710594994029,7cyclictest1780008-21idleruntime21:50:501
171059499402,29cyclictest1824883-21ssh22:45:571
1710595993931,5cyclictest1883907-21cron00:00:012
1710595993929,6cyclictest1840981-21ntp_kernel_pll_23:05:562
1710594993931,5cyclictest1736064-21munin-plugin-st20:35:001
1710593993928,9cyclictest1739067-21cron20:45:010
1710595993830,5cyclictest1783085-21grep21:55:012
1710594993832,3cyclictest0-21swapper/123:15:011
1710594993831,3cyclictest0-21swapper/122:00:541
1710594993830,5cyclictest1779177-21latency_hist21:50:021
1710594993829,5cyclictest1055-21mta-sts-daemon22:40:571
1710594993829,4cyclictest0-21swapper/123:25:561
1710594993829,4cyclictest0-21swapper/123:25:561
171059399384,22cyclictest12350irq/32-eth023:55:560
1710593993831,4cyclictest1861515-21timerwakeupswit23:31:050
1710593993828,7cyclictest420-21avahi-daemon20:00:010
1710593993827,4cyclictest1852570-21fschecks_count23:20:490
1710593993827,4cyclictest1852570-21fschecks_count23:20:490
171059399382,28cyclictest1837082-21ssh23:01:010
171059599378,22cyclictest1812668-21ssh22:30:552
1710595993729,5cyclictest1896080-21latency_hist00:15:022
1710595993729,5cyclictest1737561-21munin-plugin-st20:40:002
1710595993727,8cyclictest695-21cron19:15:002
1710595993724,9cyclictest1726449-21ntp_kernel_pll_20:00:532
1710594993730,3cyclictest0-21swapper/121:40:011
1710594993729,5cyclictest420-21avahi-daemon19:11:001
1710594993729,5cyclictest1754719-21systemd-run21:20:011
1710594993729,5cyclictest1728586-21systemd-run20:10:011
1710594993729,5cyclictest1056-21mta-sts-daemon23:50:511
1710594993728,6cyclictest4037003-21cups-browsed22:05:521
171059399373,28cyclictest1824888-21ssh22:45:570
1710593993730,4cyclictest0-21swapper/022:55:480
1710593993729,5cyclictest1784820-21awk21:56:080
1710593993729,5cyclictest1756359-21mailstats21:21:050
1710595993630,4cyclictest1696446-21kworker/u8:14+xprtiod19:45:012
1710595993628,5cyclictest1819434-21cron22:40:012
1710595993627,5cyclictest420-21avahi-daemon22:40:432
1710595993625,8cyclictest1729437-21ntp_kernel_pll_20:10:532
171059499362,27cyclictest1772322-21ssh21:40:561
1710593993629,4cyclictest1781059-21unixbench_singl21:51:080
1710593993628,5cyclictest1788504-21ntp_offset22:00:580
171059599355,6cyclictest171rcu_preempt20:55:522
1710595993528,4cyclictest1719159-21mailstats19:36:012
1710595993527,5cyclictest1909305-21latency00:30:512
1710595993526,5cyclictest1735589-21proc_pri20:31:022
1710595993526,4cyclictest0-21swapper/222:12:192
1710595993526,4cyclictest0-21swapper/222:12:192
1710595993524,7cyclictest1758876-21latency_hist21:25:032
1710594993529,2cyclictest0-21swapper/122:26:061
1710594993527,4cyclictest1750624-21idleruntime-cro21:15:011
1710594993527,4cyclictest0-21swapper/123:15:541
1710594993525,7cyclictest439-21systemd-logind00:10:011
1710593993525,8cyclictest1768742-21timerwakeupswit21:36:070
1710593993525,7cyclictest1713629-21latency_hist19:20:010
1710593993525,7cyclictest1713629-21latency_hist19:20:010
171059599344,23cyclictest1837037-21ssh23:00:592
171059599344,21cyclictest1845066-21ssh23:11:012
171059599342,23cyclictest1748010-21ssh21:10:592
171059499345,24cyclictest1913506-21ssh00:35:531
1710594993429,2cyclictest0-21swapper/120:20:011
1710594993426,5cyclictest1734592-21systemd-run20:30:011
1710594993426,5cyclictest1734592-21systemd-run20:30:011
1710594993426,4cyclictest0-21swapper/100:30:501
1710594993425,6cyclictest261-21systemd-journal20:00:011
1710593993428,3cyclictest0-21swapper/023:45:010
1710593993427,3cyclictest0-21swapper/023:45:500
1710593993424,6cyclictest151ktimers/020:30:570
1710593993420,9cyclictest0-21swapper/022:10:010
1710595993325,5cyclictest1768774-21unixbench-2d21:36:072
1710595993322,8cyclictest1791140-21cut22:05:012
1710594993326,4cyclictest1736395-21aten_r2power_fr20:35:451
1710594993325,5cyclictest1741093-21fschecks_time20:50:471
1710594993320,9cyclictest1-21systemd00:15:451
171059399332,25cyclictest1804561-21ssh22:20:530
171059599323,21cyclictest1772338-21ssh21:40:572
171059599322,5cyclictest0-21swapper/220:20:592
1710595993224,5cyclictest4037003-21cups-browsed22:27:432
1710595993224,5cyclictest1860320-21aten2.4-expect23:30:432
1710595993224,5cyclictest1705256-21kworker/u8:5+xprtiod20:45:012
1710595993221,8cyclictest1832999-21ntpq22:55:592
1710594993224,5cyclictest1766864-21munin-plugin-st21:35:001
1710594993224,5cyclictest1766864-21munin-plugin-st21:35:001
171059499322,23cyclictest1885430-21ssh00:01:021
171059499322,23cyclictest1885430-21ssh00:01:021
171059399323,23cyclictest1820849-21ssh22:40:570
1710593993226,4cyclictest1051-21mta-sts-daemon23:51:020
1710593993224,6cyclictest1847610-21sh23:15:010
1710593993224,5cyclictest420-21avahi-daemon22:33:350
1710593993220,9cyclictest421-21dbus-daemon00:25:530
1710595993126,2cyclictest0-21swapper/223:50:012
1710595993125,3cyclictest0-21swapper/219:50:012
1710595993125,3cyclictest0-21swapper/219:35:012
1710595993125,2cyclictest0-21swapper/222:45:482
1710595993123,5cyclictest420-21avahi-daemon20:50:492
1710595993123,5cyclictest1745883-21needreboot21:05:532
1710595993123,5cyclictest1-21systemd20:30:012
1710595993123,5cyclictest1-21systemd20:30:012
1710595993121,8cyclictest1727920-21ntp_kernel_err20:05:542
171059499314,22cyclictest1853113-21ssh23:21:011
171059499314,22cyclictest1853113-21ssh23:21:011
1710594993123,4cyclictest0-21swapper/123:05:581
1710594993122,7cyclictest1901211-21latency00:20:501
1710594993122,6cyclictest1722567-21cron19:50:011
1710593993124,4cyclictest1771859-21sh21:40:470
1710593993124,4cyclictest1752359-21switchtime21:16:090
1710593993123,6cyclictest4037003-21cups-browsed21:28:480
1710593993123,6cyclictest4037003-21cups-browsed21:28:480
1710593993123,6cyclictest1808214-21fschecks_count22:25:470
171059399312,21cyclictest1893387-21ssh00:10:570
1710593993121,7cyclictest1848550-21gltestperf23:15:480
171059399311,24cyclictest1865205-21ssh23:36:030
1710595993023,4cyclictest695-21cron20:50:012
1710595993023,4cyclictest1056-21mta-sts-daemon19:20:482
1710595993023,4cyclictest1054-21mta-sts-daemon19:50:582
1710594993024,3cyclictest0-21swapper/122:20:441
1710594993023,4cyclictest1863747-21basename23:35:001
1710594993023,4cyclictest1729108-21gltestperf20:10:471
1710594993021,6cyclictest1-21systemd20:50:011
1710594993021,5cyclictest1743586-21systemd-run21:00:011
1710593993024,3cyclictest0-21swapper/022:15:000
1710593993024,3cyclictest0-21swapper/022:15:000
1710593993023,4cyclictest1803266-21tr22:20:000
171059399302,22cyclictest1776403-21ssh21:45:580
1710593993020,7cyclictest1910115-21unixbench_multi00:31:080
1710593993016,8cyclictest1731473-21unixbench_singl20:16:060
171059399301,23cyclictest1856968-21ssh23:25:530
171059399301,23cyclictest1856968-21ssh23:25:530
1710594992922,4cyclictest1056-21mta-sts-daemon19:25:571
1710594992919,7cyclictest1055-21mta-sts-daemon22:55:581
1710593992923,4cyclictest1727598-21fschecks_count20:05:480
1710593992922,5cyclictest1737592-21systemd-run20:40:010
1710593992913,13cyclictest151ktimers/019:10:520
1710595992821,3cyclictest0-21swapper/220:00:012
1710594992822,3cyclictest0-21swapper/119:35:551
1710594992822,3cyclictest0-21swapper/100:25:561
171059499282,21cyclictest1873062-21ssh23:45:541
1710594992820,5cyclictest1755654-21gltestperf21:20:491
1710594992819,5cyclictest0-21swapper/119:25:001
1710593992822,4cyclictest1841023-21ntp_kernel_pll_23:05:570
1710593992822,4cyclictest1742070-21systemd-run20:55:010
171059399282,16cyclictest428-21mta-sts-daemon21:00:550
1710593992821,5cyclictest1-21systemd21:06:050
1710593992820,6cyclictest1722145-21mailstats19:46:030
1710593992818,8cyclictest1883891-21cups-browsed00:16:310
171059599272,19cyclictest1804732-21ssh22:21:002
1710594992721,3cyclictest0-21swapper/123:41:011
1710594992718,6cyclictest548-21gdbus21:06:031
1710594992717,7cyclictest1745072-21/usr/sbin/munin21:05:021
1710593992719,5cyclictest0-21swapper/022:50:470
1710593992719,5cyclictest0-21swapper/022:50:470
1710593992719,4cyclictest0-21swapper/021:30:560
1710593992719,4cyclictest0-21swapper/021:30:560
1710593992715,9cyclictest0-21swapper/019:35:020
171059399271,21cyclictest1913535-21rm00:35:540
1710595992618,5cyclictest1716597-21systemd-run19:30:012
1710595992618,5cyclictest1713574-21idleruntime-cro19:20:002
1710595992618,5cyclictest1713574-21idleruntime-cro19:20:002
1710594992620,3cyclictest0-21swapper/120:45:011
1710594992619,4cyclictest1720045-21cpuspeed19:40:481
1710594992618,5cyclictest1733102-21systemd-run20:25:011
1710593992620,4cyclictest1727072-21cron20:05:010
1710593992519,4cyclictest1730064-21munin-run20:15:010
1710593992518,5cyclictest1743602-21latency_hist21:00:020
1710593992518,4cyclictest427-21polkitd19:38:420
1710593992516,6cyclictest1888555-21/usr/sbin/munin00:05:480
1710594992417,4cyclictest1712587-21fschecks_count19:15:491
1710594992417,4cyclictest1712587-21fschecks_count19:15:491
171059499232,16cyclictest1723495-21ntpq19:50:571
171059399231,9cyclictest1732935-21unixbench-2d20:21:070
1710593992317,3cyclictest0-21swapper/020:45:530
1710593992316,4cyclictest1721082-21munin-plugin-st19:45:010
1710593992315,4cyclictest12450irq/33-eth019:25:570
1710593992314,7cyclictest1733948-21ntp_kernel_pll_20:25:540
1710593992314,7cyclictest1733948-21ntp_kernel_pll_20:25:540
1710593992312,8cyclictest548-21gdbus19:51:000
171059499223,13cyclictest428-21mta-sts-daemon19:30:551
1710594992115,3cyclictest0-21swapper/119:10:011
1710594992115,3cyclictest0-21swapper/119:10:011
1710593992112,7cyclictest1714405-21needreboot19:20:520
171059699202,12cyclictest0-21swapper/323:51:023
1710595992011,6cyclictest1710653-21latency_hist19:10:012
1710595992011,6cyclictest1710653-21latency_hist19:10:012
171059699192,10cyclictest0-21swapper/300:35:533
171059699184,8cyclictest0-21swapper/300:10:533
171059699183,9cyclictest0-21swapper/323:40:563
171059699182,8cyclictest0-21swapper/322:00:553
171059699182,11cyclictest0-21swapper/321:25:593
171059699182,11cyclictest0-21swapper/321:25:593
171059699182,10cyclictest0-21swapper/321:55:583
171059699182,10cyclictest0-21swapper/321:20:553
171059699181,10cyclictest0-21swapper/322:25:553
171059699172,9cyclictest0-21swapper/323:35:593
171059699172,9cyclictest0-21swapper/319:20:583
171059699172,8cyclictest0-21swapper/323:25:553
171059699172,8cyclictest0-21swapper/323:25:553
171059699172,8cyclictest0-21swapper/321:10:563
171059699172,10cyclictest0-21swapper/300:25:533
171059699171,9cyclictest0-21swapper/300:20:463
171059699171,10cyclictest0-21swapper/322:10:583
171059699171,10cyclictest0-21swapper/322:10:583
171059699163,9cyclictest0-21swapper/322:40:543
171059699163,8cyclictest0-21swapper/322:20:483
171059699163,8cyclictest0-21swapper/322:15:543
171059699163,8cyclictest0-21swapper/321:00:013
171059699163,8cyclictest0-21swapper/300:15:583
171059699163,7cyclictest0-21swapper/322:05:453
171059699163,6cyclictest0-21swapper/320:55:013
171059699162,9cyclictest0-21swapper/323:05:543
171059699162,9cyclictest0-21swapper/322:56:003
171059699162,9cyclictest0-21swapper/300:05:533
171059699162,9cyclictest0-21swapper/300:00:583
171059699162,9cyclictest0-21swapper/300:00:583
171059699162,8cyclictest0-21swapper/321:45:583
171059699162,8cyclictest0-21swapper/321:16:003
171059699162,7cyclictest0-21swapper/321:00:583
171059699161,9cyclictest0-21swapper/323:55:583
171059699161,9cyclictest0-21swapper/323:20:513
171059699161,9cyclictest0-21swapper/323:20:513
171059699161,9cyclictest0-21swapper/300:30:493
171059699161,8cyclictest0-21swapper/323:15:553
171059699161,7cyclictest0-21swapper/319:25:593
171059399161,12cyclictest1710600-21sh19:10:010
171059399161,12cyclictest1710600-21sh19:10:010
171059699154,5cyclictest0-21swapper/320:46:003
171059699153,7cyclictest0-21swapper/322:35:513
171059699152,8cyclictest0-21swapper/323:30:533
171059699152,8cyclictest0-21swapper/323:01:033
171059699152,8cyclictest0-21swapper/321:35:563
171059699152,8cyclictest0-21swapper/321:30:553
171059699152,8cyclictest0-21swapper/321:30:553
171059699152,7cyclictest0-21swapper/321:40:453
171059699151,9cyclictest0-21swapper/323:10:543
171059699151,7cyclictest0-21swapper/320:35:553
171059699151,7cyclictest0-21swapper/319:50:573
171059699151,6cyclictest0-21swapper/319:30:523
171059699142,8cyclictest0-21swapper/322:45:493
171059699142,8cyclictest0-21swapper/320:30:573
171059699142,7cyclictest0-21swapper/321:50:503
171059699142,7cyclictest0-21swapper/320:15:023
171059699142,5cyclictest0-21swapper/320:15:533
171059699141,8cyclictest0-21swapper/323:45:573
171059699141,7cyclictest0-21swapper/320:05:583
171059699141,7cyclictest0-21swapper/320:00:523
171059699141,6cyclictest0-21swapper/319:15:553
171059699141,6cyclictest0-21swapper/319:15:553
171059699132,7cyclictest0-21swapper/322:50:523
171059699132,7cyclictest0-21swapper/322:50:523
171059699132,7cyclictest0-21swapper/319:40:013
171059699132,6cyclictest0-21swapper/320:20:463
171059699131,6cyclictest0-21swapper/320:25:513
171059699131,6cyclictest0-21swapper/320:25:513
171059699123,4cyclictest0-21swapper/319:45:553
171059699122,6cyclictest0-21swapper/322:30:443
171059699122,6cyclictest0-21swapper/319:55:543
171059699112,5cyclictest0-21swapper/320:40:543
171059699111,7cyclictest0-21swapper/321:10:023
171059699111,7cyclictest0-21swapper/319:40:563
171059699111,6cyclictest0-21swapper/319:10:533
171059699103,5cyclictest0-21swapper/319:10:013
171059699103,5cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional