You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-31 - 09:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun May 31, 2026 00:42:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
180761299642,9cyclictest1857696-21ssh21:31:032
180761299642,9cyclictest1857696-21ssh21:31:032
1807610995140,8cyclictest1853396-21latency_hist21:25:020
1807610994932,13cyclictest12350irq/32-eth022:16:050
1807610994932,13cyclictest12350irq/32-eth022:16:050
1807612994839,5cyclictest1901167-21sendmail22:40:012
1807612994831,14cyclictest1952990-21aten_r2power_po00:00:502
1807610994733,11cyclictest1828584-21latency_hist20:20:010
1807612994640,3cyclictest0-21swapper/221:06:042
1807611994537,5cyclictest1810647-21latency_hist19:20:021
1807610994418,16cyclictest1844962-21ssh21:11:060
1807610994418,16cyclictest1844962-21ssh21:11:060
180761299435,34cyclictest1911920-21ssh22:56:012
1807612994333,6cyclictest1831617-21systemd-run20:30:012
180761299431,11cyclictest1810001-21ntpq19:16:042
1807610994329,8cyclictest1843588-21diskmemload00:11:050
180761299428,3cyclictest0-21swapper/220:10:592
180761299423,5cyclictest1851266-21ntpq21:21:032
180761299423,5cyclictest1818961-21ntpq19:46:002
180761299422,6cyclictest1937533-21ntpq23:36:022
180761299422,6cyclictest1820556-21ntpq19:51:072
180761299422,6cyclictest1816020-21ntpq19:36:022
1807612994217,21cyclictest86535-21cups-browsed23:01:152
1807612994216,23cyclictest0-21swapper/223:51:042
180761299421,11cyclictest1940795-21ntpq23:41:022
180761299421,11cyclictest1905531-21ntpq22:46:002
180761299421,11cyclictest186608350cat21:45:022
1807611994234,5cyclictest1850027-21ssh21:20:001
1807611994230,9cyclictest1975904-21sh00:36:021
180761299419,2cyclictest0-21swapper/221:01:002
180761299419,2cyclictest0-21swapper/221:01:002
180761299414,27cyclictest1886381-21ssh22:16:012
180761299414,27cyclictest1886381-21ssh22:16:012
180761299412,8cyclictest1944066-21ssh23:46:072
180761299412,6cyclictest1883265-21ntpq22:11:052
180761299412,5cyclictest1896107-21ntpq22:31:062
180761299412,5cyclictest1838486-21ntpq20:51:002
180761299412,5cyclictest1838486-21ntpq20:51:002
180761299412,17cyclictest1860882-21ssh21:36:032
1807612994120,19cyclictest1807609-21cyclictest22:00:572
180761299411,11cyclictest1934459-21ntpq23:31:052
180761299411,11cyclictest1817523-21ntpq19:41:022
180761299411,11cyclictest1817523-21ntpq19:41:022
180761299411,10cyclictest1918382-21ntpq23:06:022
180761299411,10cyclictest1840028-21ntpq20:56:042
180761299411,10cyclictest1840028-21ntpq20:56:042
180761299411,10cyclictest1833949-21ntpq20:36:012
180761299411,10cyclictest1808496-21ntpq19:11:002
1807611994133,5cyclictest1818123-21systemd-run19:45:011
1807611994133,5cyclictest1818123-21systemd-run19:45:011
180761099413,28cyclictest1915100-21ssh23:01:000
1807610994130,9cyclictest1875595-21sendmail22:00:010
180761299407,10cyclictest421-21dbus-daemon20:05:012
180761299403,10cyclictest1963156-21ntpq00:16:032
180761299402,8cyclictest1844908-21ssh21:11:042
180761299402,8cyclictest1844908-21ssh21:11:042
180761299402,5cyclictest1828001-21ntpq20:16:022
180761299402,5cyclictest1821996-21ntpq19:56:052
180761299401,8cyclictest1909340-21/usr/sbin/munin22:51:142
1807612994018,20cyclictest1807609-21cyclictest23:23:472
180761299401,14cyclictest1892961-21ps22:26:072
180761299401,11cyclictest1814556-21ntpq19:31:042
180761299401,10cyclictest1829506-21ntpq20:21:022
180761299401,10cyclictest1811556-21ntpq19:21:032
1807611994032,5cyclictest1960483-21grep00:11:121
1807611994030,8cyclictest1939503-21munin-plugin-st23:40:021
1807611994029,4cyclictest0-21swapper/100:06:001
1807610994027,10cyclictest0-21swapper/020:40:010
1807610994024,11cyclictest151ktimers/021:41:000
180761299393,5cyclictest1837013-21ntpq20:46:032
1807612993915,21cyclictest0-21swapper/200:00:022
180761299391,16cyclictest1968412-21date00:25:022
180761299391,13cyclictest1813050-21ntpq19:26:032
180761299391,12cyclictest1902395-21ntpq22:41:012
180761299391,10cyclictest1870237-21memory21:50:562
1807611993931,5cyclictest1828631-21systemd-run20:20:021
1807611993927,9cyclictest624-21lldpd23:16:001
1807610993924,11cyclictest421-21dbus-daemon22:30:010
1807612993831,3cyclictest0-21swapper/200:35:012
180761299382,7cyclictest1854484-21ssh21:26:052
180761299382,7cyclictest1854484-21ssh21:26:052
180761299382,5cyclictest1924777-21ntpq23:16:032
180761299381,12cyclictest1868647-21ssh21:48:272
180761299381,10cyclictest1835567-21ntpq20:41:042
1807611993831,3cyclictest0-21swapper/122:10:011
1807611993830,5cyclictest1885182-21systemd-run22:15:011
1807610993828,6cyclictest1824125-21systemd20:05:010
180761299372,13cyclictest1880041-21ssh22:06:042
180761299372,13cyclictest1873714-21ntp_states21:56:042
180761299371,8cyclictest1930981-21memory23:25:572
1807612993711,23cyclictest1847798-21latency21:15:592
1807611993731,3cyclictest0-21swapper/122:41:041
1807611993727,7cyclictest1827079-21cron20:15:011
180761199370,5cyclictest1873691-21ssh21:56:021
1807610993731,3cyclictest0-21swapper/023:31:110
1807610993729,5cyclictest1-21systemd19:40:010
1807610993729,4cyclictest0-21swapper/000:26:010
180761099372,30cyclictest1870401-21ssh21:51:000
180761299368,3cyclictest1968676-21ssh00:25:312
180761299362,12cyclictest1975872-21ssh00:36:002
180761299361,9cyclictest1832464-21ntpq20:30:592
180761299361,7cyclictest1824735-21/usr/sbin/munin20:05:592
180761299361,11cyclictest1921632-21ntpq23:11:042
180761199368,9cyclictest28-21ksoftirqd/122:51:061
1807611993628,5cyclictest1929946-21systemd-run23:25:011
1807611993627,6cyclictest421-21dbus-daemon22:40:011
180761099364,25cyclictest1876881-21sshd22:01:030
180761099363,25cyclictest1821972-21sshd19:56:040
1807610993626,5cyclictest0-21swapper/000:36:060
1807610993625,8cyclictest1821101-21true19:55:010
180761099362,28cyclictest1963083-21ssh00:16:000
180761299352,6cyclictest1956709-21ntpq00:06:002
1807612993512,20cyclictest1889185-21fschecks_count22:20:542
180761299351,10cyclictest1959999-21ssh00:11:032
1807611993528,4cyclictest0-21swapper/121:00:001
1807611993528,4cyclictest0-21swapper/121:00:001
1807611993519,9cyclictest1863976-21ssh21:40:591
180761199350,16cyclictest28-21ksoftirqd/120:46:061
180761099353,27cyclictest1895933-21ssh22:30:590
1807610993529,4cyclictest1946426-21/usr/sbin/munin23:51:010
1807610993522,11cyclictest1836096-21idleruntime-cro20:45:010
1807611993426,5cyclictest1942738-21systemd-run23:45:011
1807611993425,6cyclictest1-21systemd23:55:021
180761199341,27cyclictest271ktimers/122:31:061
180761099343,24cyclictest1972812-21ssh00:31:050
180761099343,21cyclictest1940767-21ssh23:41:010
1807610993428,4cyclictest1945948-21latency_hist23:50:020
1807610993417,9cyclictest12350irq/32-eth023:06:000
1807611993326,4cyclictest1914546-21aten_r2power_fr23:00:511
1807611993325,5cyclictest86535-21cups-browsed19:51:141
1807611993324,6cyclictest624-21lldpd23:47:011
1807611993323,7cyclictest1968282-21/usr/sbin/munin00:25:011
180761199332,12cyclictest28-21ksoftirqd/100:31:041
1807611993315,4cyclictest171rcu_preempt20:36:011
1807610993327,3cyclictest0-21swapper/023:20:550
1807610993325,4cyclictest1833160-21latency_hist20:35:020
1807610993323,7cyclictest1840665-21latency_hist21:00:010
1807610993323,7cyclictest1840665-21latency_hist21:00:010
1807610993319,11cyclictest0-21swapper/022:55:010
1807611993225,3cyclictest0-21swapper/100:20:011
1807611993224,5cyclictest1933252-21latency_hist23:30:021
1807611993224,5cyclictest1913937-21ssh23:00:011
1807611993217,3cyclictest171rcu_preempt19:46:041
1807611993215,6cyclictest1054-21mta-sts-daemon22:00:301
1807610993225,4cyclictest1955526-21systemd-run00:05:010
1807610993225,4cyclictest1854452-21ssh21:26:030
1807610993225,4cyclictest1854452-21ssh21:26:030
1807610993224,4cyclictest0-21swapper/022:41:020
1807610993224,4cyclictest0-21swapper/021:36:020
180761099322,26cyclictest1848197-21/usr/sbin/munin21:16:100
180761099322,25cyclictest1950327-21sh23:56:030
180761099322,24cyclictest1912020-21ssh22:56:040
1807610993213,12cyclictest1937553-21ssh23:36:030
180761199317,16cyclictest428-21mta-sts-daemon23:06:061
180761199315,8cyclictest28-21ksoftirqd/119:26:001
180761199312,7cyclictest28-21ksoftirqd/121:46:031
1807611993125,3cyclictest0-21swapper/100:26:021
1807611993124,3cyclictest0-21swapper/121:11:021
1807611993124,3cyclictest0-21swapper/121:11:021
1807611993123,5cyclictest1886702-21grep22:16:101
1807611993123,5cyclictest1886702-21grep22:16:101
1807611993123,5cyclictest1859575-21cron21:35:001
1807611993123,5cyclictest1859575-21cron21:35:001
1807611993123,5cyclictest1843590-21munin-run21:10:001
1807611993123,5cyclictest1841119-21fschecks_time21:00:541
1807611993123,5cyclictest1841119-21fschecks_time21:00:541
1807611993123,3cyclictest0-21swapper/123:11:021
1807611993123,3cyclictest0-21swapper/122:46:021
180761199312,24cyclictest1860868-21sh21:36:031
1807611993121,7cyclictest0-21swapper/123:55:561
1807611993116,4cyclictest171rcu_preempt20:25:591
1807610993122,6cyclictest1354370-21kworker/u8:13+xprtiod19:15:010
1807610993120,8cyclictest1966437-21sh00:21:050
1807610993117,10cyclictest12350irq/32-eth022:36:030
1807611993023,4cyclictest1892967-21processes22:26:071
1807611993023,4cyclictest1055-21mta-sts-daemon20:07:551
1807611993022,5cyclictest1953428-21missed_timers00:00:571
1807611993022,5cyclictest0-21swapper/121:55:001
1807611993021,7cyclictest1832290-21latency20:30:551
1807611993020,7cyclictest1851743-21timerwakeupswit21:21:131
1807610993023,4cyclictest1830118-21systemd-run20:25:010
1807610993019,6cyclictest1843588-21diskmemload21:46:020
180761099301,25cyclictest1924760-21diskmemload23:16:020
1807612992914,12cyclictest0-21swapper/219:10:022
180761199297,8cyclictest28-21ksoftirqd/119:10:021
180761199296,16cyclictest261-21systemd-journal20:01:031
1807611992922,5cyclictest271ktimers/123:31:031
1807611992922,3cyclictest0-21swapper/119:31:011
1807611992917,3cyclictest171rcu_preempt19:35:551
1807611992915,8cyclictest171rcu_preempt19:56:021
1807611992914,4cyclictest1888384-21kworker/u8:3-xprtiod22:22:391
1807611992914,3cyclictest171rcu_preempt19:10:591
1807611992913,2cyclictest171rcu_preempt20:21:021
1807610992924,3cyclictest1879423-21aten_r2power_po22:05:510
1807610992922,4cyclictest1882789-21fschecks_count22:10:550
1807610992922,3cyclictest1807609-21cyclictest20:10:510
180761099292,22cyclictest1905626-21sshd22:46:040
180761099292,14cyclictest1830600-21diskstats20:25:540
1807610992920,7cyclictest1841402-21needreboot21:00:580
1807610992920,7cyclictest1841402-21needreboot21:00:580
1807611992816,3cyclictest171rcu_preempt21:26:141
1807611992816,3cyclictest171rcu_preempt21:26:141
1807611992812,3cyclictest171rcu_preempt19:20:591
1807610992822,4cyclictest1056-21mta-sts-daemon19:45:000
1807610992822,4cyclictest1056-21mta-sts-daemon19:45:000
1807610992822,3cyclictest1859710-21latency_hist21:35:020
1807610992822,3cyclictest1859710-21latency_hist21:35:020
1807610992822,3cyclictest0-21swapper/000:06:000
1807610992821,3cyclictest0-21swapper/023:11:040
1807611992712,3cyclictest171rcu_preempt20:51:011
1807611992712,3cyclictest171rcu_preempt20:51:011
1807610992722,3cyclictest1055-21mta-sts-daemon22:21:120
1807610992715,9cyclictest1809742-21kernelversion19:15:570
1807610992712,13cyclictest151ktimers/019:31:010
1807611992613,3cyclictest171rcu_preempt20:40:501
1807610992620,4cyclictest1933339-21ssh23:30:030
1807610992620,3cyclictest0-21swapper/020:55:000
1807610992620,3cyclictest0-21swapper/020:55:000
1807610992616,7cyclictest1842609-21fschecks_count21:05:530
1807610992518,4cyclictest1813709-21kworker/u8:0-xprtiod20:10:020
1807610992516,6cyclictest1836767-21cat20:45:570
1807610992417,5cyclictest620-21sshd19:49:030
180761099232,16cyclictest1811637-21ps19:21:060
180761099231,19cyclictest1813229-21mailstats19:26:090
180761399192,8cyclictest0-21swapper/322:36:003
180761399182,6cyclictest0-21swapper/321:11:023
180761399182,6cyclictest0-21swapper/321:11:023
180761099181,9cyclictest1-21systemd19:10:010
180761399171,7cyclictest0-21swapper/323:15:593
180761399171,7cyclictest0-21swapper/322:11:063
180761399162,7cyclictest0-21swapper/323:06:013
180761399162,6cyclictest0-21swapper/323:36:053
180761399161,9cyclictest0-21swapper/322:51:043
180761399161,8cyclictest0-21swapper/300:06:043
180761399160,8cyclictest0-21swapper/323:11:023
180761399152,8cyclictest0-21swapper/321:06:043
180761399152,6cyclictest0-21swapper/322:46:013
180761399151,9cyclictest0-21swapper/300:21:013
180761399151,7cyclictest0-21swapper/323:56:043
180761399151,7cyclictest0-21swapper/323:21:043
180761399151,7cyclictest0-21swapper/322:25:593
180761399151,7cyclictest0-21swapper/322:16:023
180761399151,7cyclictest0-21swapper/322:16:023
180761399151,7cyclictest0-21swapper/300:36:003
180761399143,4cyclictest0-21swapper/320:36:023
180761399142,7cyclictest0-21swapper/300:16:043
180761399142,7cyclictest0-21swapper/300:00:593
180761399142,6cyclictest0-21swapper/323:46:043
180761399142,6cyclictest0-21swapper/321:21:013
180761399142,6cyclictest0-21swapper/320:11:013
180761399142,6cyclictest0-21swapper/300:26:043
180761399141,7cyclictest0-21swapper/322:06:043
180761399141,7cyclictest0-21swapper/321:36:003
180761399141,7cyclictest0-21swapper/321:26:023
180761399141,7cyclictest0-21swapper/321:26:023
180761399141,6cyclictest0-21swapper/322:31:003
180761399141,6cyclictest0-21swapper/320:51:013
180761399141,6cyclictest0-21swapper/320:51:013
180761399134,5cyclictest0-21swapper/320:01:033
180761399132,7cyclictest0-21swapper/319:31:023
180761399132,6cyclictest0-21swapper/323:51:013
180761399132,6cyclictest0-21swapper/323:31:043
180761399132,6cyclictest0-21swapper/322:25:013
180761399132,5cyclictest0-21swapper/319:51:033
180761399132,5cyclictest0-21swapper/319:21:013
180761399131,7cyclictest0-21swapper/323:01:043
180761399131,7cyclictest0-21swapper/320:41:043
180761399131,6cyclictest0-21swapper/321:40:593
180761399131,6cyclictest0-21swapper/319:10:583
180761399131,6cyclictest0-21swapper/300:31:023
180761399122,6cyclictest0-21swapper/323:26:013
180761399122,6cyclictest0-21swapper/322:01:033
180761399122,5cyclictest0-21swapper/321:46:023
180761399122,5cyclictest0-21swapper/319:46:043
180761399121,6cyclictest0-21swapper/322:55:543
180761399121,6cyclictest0-21swapper/321:50:593
180761399121,6cyclictest0-21swapper/321:16:033
180761399121,6cyclictest0-21swapper/319:56:043
180761399121,6cyclictest0-21swapper/319:36:003
180761399121,6cyclictest0-21swapper/319:26:063
180761399121,5cyclictest0-21swapper/320:16:003
180761399121,5cyclictest0-21swapper/320:06:043
180761399112,6cyclictest0-21swapper/323:41:043
180761399112,5cyclictest0-21swapper/321:56:003
180761399112,5cyclictest0-21swapper/320:25:513
180761399111,6cyclictest0-21swapper/321:30:593
180761399111,6cyclictest0-21swapper/321:30:593
180761399111,6cyclictest0-21swapper/320:23:043
180761399111,6cyclictest0-21swapper/319:45:003
180761399111,6cyclictest0-21swapper/319:45:003
180761399111,5cyclictest0-21swapper/322:40:313
180761399111,5cyclictest0-21swapper/300:10:043
180761399104,4cyclictest0-21swapper/319:16:093
180761399103,4cyclictest0-21swapper/321:00:003
180761399103,4cyclictest0-21swapper/321:00:003
180761399102,5cyclictest0-21swapper/321:00:543
180761399102,5cyclictest0-21swapper/321:00:543
180761399101,6cyclictest0-21swapper/320:45:553
18076139981,5cyclictest0-21swapper/320:35:013
18076139981,5cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional