You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-09 - 10:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sun Aug 09, 2026 00:43:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1804707998071,6cyclictest0-21swapper/123:40:431
1804707998071,6cyclictest0-21swapper/123:40:431
1804708995041,6cyclictest1950904-21latency_hist00:10:022
1804708994840,5cyclictest1869905-21timerandwakeup21:56:062
1804708994840,5cyclictest1869905-21timerandwakeup21:56:062
1804708994536,6cyclictest1810714-21systemd-run19:30:012
1804707994336,4cyclictest1883561-21awk22:20:011
1804708994132,5cyclictest1830201-21systemd-run20:35:012
1804708994132,5cyclictest1830201-21systemd-run20:35:012
1804707994134,4cyclictest1002-21mta-sts-daemon21:55:541
1804707994134,4cyclictest1002-21mta-sts-daemon21:55:541
1804707994134,3cyclictest0-21swapper/123:40:011
1804707994134,3cyclictest0-21swapper/123:40:011
1804707994132,5cyclictest1911268-21latency_hist23:05:021
1804708994033,4cyclictest1906701-21grep22:56:052
1804708994031,5cyclictest1842409-21unixbench_multi21:11:062
1804708994029,8cyclictest1862231-21cut21:45:022
1804707994033,4cyclictest1929397-21munin-plugin-st23:35:011
1804707993933,3cyclictest0-21swapper/100:20:011
1804707993933,3cyclictest0-21swapper/100:20:011
1804707993929,7cyclictest1831680-21sendmail20:40:011
1804707993929,7cyclictest1831680-21sendmail20:40:011
1804707993928,8cyclictest1917204-21ssh23:15:011
1804706993933,4cyclictest1788955-21kworker/u8:1-writeback19:55:010
1804706993931,5cyclictest420-21avahi-daemon21:52:140
1804706993924,10cyclictest108580-21cupsd21:36:040
1804708993830,5cyclictest1849933-21ssh21:25:012
1804707993832,2cyclictest0-21swapper/100:00:011
1804707993831,5cyclictest1804705-21cyclictest00:08:261
1804707993829,5cyclictest1828712-21systemd-run20:30:011
1804708993727,8cyclictest1840692-21basename21:10:012
1804707993729,5cyclictest1901895-21systemd-run22:50:011
1804707993728,6cyclictest1815216-21true19:45:011
1804707993727,7cyclictest1846871-21sendmail21:20:011
1804706993733,2cyclictest0-21swapper/020:20:010
1804706993731,4cyclictest1854555-21threads21:31:040
1804706993731,3cyclictest1002-21mta-sts-daemon22:36:530
1804706993730,4cyclictest1932474-21systemd-run23:40:020
1804706993730,4cyclictest1932474-21systemd-run23:40:020
1804706993730,4cyclictest1874428-21cron22:05:000
1804706993729,5cyclictest1-21systemd00:00:010
1804706993725,10cyclictest1812479-21aten_r2power_en19:35:450
1804708993629,3cyclictest0-21swapper/222:36:042
1804708993628,5cyclictest1960191-21latency_hist00:25:022
1804708993628,5cyclictest1960191-21latency_hist00:25:022
1804708993628,5cyclictest1885234-21unixbench-2d22:21:062
180470899360,27cyclictest1854186-21ssh21:30:542
1804707993630,4cyclictest1804705-21cyclictest23:00:011
1804707993628,5cyclictest1898946-21latency_hist22:45:031
1804707993628,3cyclictest0-21swapper/119:40:011
1804707993627,6cyclictest1947761-21cron00:05:011
1804706993631,3cyclictest1809194-21cron19:25:000
1804706993631,3cyclictest1809194-21cron19:25:000
1804706993631,2cyclictest0-21swapper/023:25:450
1804706993629,3cyclictest1000-21mta-sts-daemon00:38:240
1804706993628,5cyclictest1847852-21latency21:20:500
1804706993622,9cyclictest12650irq/32-eth022:20:570
1804708993527,5cyclictest420-21avahi-daemon00:12:172
1804708993527,5cyclictest1874433-21sh22:05:002
1804708993525,7cyclictest265-21systemd-journal22:43:032
1804708993525,6cyclictest108584-21cups-browsed21:15:202
1804708993524,8cyclictest420-21avahi-daemon23:40:082
1804708993524,8cyclictest420-21avahi-daemon23:40:082
1804707993526,6cyclictest1836711-21fschecks_time20:55:471
1804707993526,6cyclictest1836711-21fschecks_time20:55:471
1804707993526,6cyclictest1824205-21systemd-run20:15:011
1804707993526,6cyclictest108584-21cups-browsed23:16:051
1804707993524,8cyclictest1891313-21timerwakeupswit22:31:041
1804707993523,9cyclictest1964409-21/usr/sbin/munin00:31:011
180470799351,27cyclictest1863395-21ssh21:45:541
1804708993428,3cyclictest0-21swapper/200:00:012
1804708993427,4cyclictest1904931-21latency_hist22:55:012
1804708993427,4cyclictest1904931-21latency_hist22:55:012
1804708993426,5cyclictest1821190-21cron20:05:012
180470899341,23cyclictest1863324-21ssh21:45:512
1804707993429,3cyclictest0-21swapper/122:30:011
1804707993425,6cyclictest1-21systemd19:10:011
1804707993425,6cyclictest108584-21cups-browsed23:05:491
1804706993429,3cyclictest1950314-21sshd00:08:260
1804706993428,4cyclictest1917181-21cron23:15:010
1804706993428,4cyclictest1863355-21ntp_kernel_pll_21:45:530
1804706993427,2cyclictest0-21swapper/000:20:020
1804706993427,2cyclictest0-21swapper/000:20:020
1804706993422,9cyclictest0-21swapper/022:26:020
1804708993327,3cyclictest0-21swapper/223:15:552
1804708993326,3cyclictest0-21swapper/200:30:542
1804708993325,5cyclictest108584-21cups-browsed20:35:362
1804708993325,5cyclictest108584-21cups-browsed20:35:362
1804708993325,5cyclictest108584-21cups-browsed19:51:222
1804708993324,6cyclictest1833183-21cron20:45:002
1804708993323,8cyclictest1805591-21ntp_kernel_pll_19:11:002
1804707993325,4cyclictest0-21swapper/100:35:591
1804707993324,7cyclictest1921315-21needreboot23:20:491
1804707993324,6cyclictest1952202-21sendmail_mailst00:11:011
1804707993324,6cyclictest1874428-21cron22:05:001
1804707993324,6cyclictest108580-21cupsd22:11:071
1804707993324,5cyclictest1944774-21cups-browsed00:22:131
1804707993324,5cyclictest1944774-21cups-browsed00:22:131
1804706993328,4cyclictest1804705-21cyclictest19:40:480
1804706993328,3cyclictest1934007-21switchtime23:41:040
1804706993328,3cyclictest1934007-21switchtime23:41:040
1804706993326,4cyclictest12650irq/32-eth021:15:530
1804706993325,6cyclictest0-21swapper/022:17:010
1804706993322,8cyclictest1900533-21unixbench_multi22:46:050
1804706993318,5cyclictest421-21dbus-daemon21:11:180
1804708993225,4cyclictest1833715-21fschecks_time20:45:472
1804708993224,5cyclictest1956930-21grep00:20:012
1804708993224,5cyclictest1956930-21grep00:20:012
1804708993224,5cyclictest1930153-21fschecks_count23:35:482
1804708993224,5cyclictest1930153-21fschecks_count23:35:482
1804708993224,5cyclictest1850457-21aten_r2power_en21:25:442
1804707993224,5cyclictest1925030-21unixbench_multi23:26:041
1804707993224,5cyclictest1851736-21unixbench_singl21:26:071
1804707993223,6cyclictest108584-21cups-browsed20:00:561
1804707993223,5cyclictest420-21avahi-daemon21:15:011
1804707993222,7cyclictest1836176-21sh20:55:001
1804707993222,7cyclictest1836176-21sh20:55:001
1804707993221,8cyclictest1939607-21missed_timers23:50:491
1804706993225,5cyclictest12650irq/32-eth023:05:570
1804706993225,4cyclictest1832217-21fschecks_count20:40:460
1804706993225,4cyclictest108584-21cups-browsed22:51:300
1804706993225,4cyclictest108584-21cups-browsed22:51:300
1804708993126,2cyclictest0-21swapper/223:55:002
1804708993126,2cyclictest0-21swapper/223:30:012
1804708993125,3cyclictest0-21swapper/222:30:442
1804708993124,4cyclictest1883547-21grep22:20:012
1804708993124,4cyclictest1002-21mta-sts-daemon20:20:542
1804708993124,3cyclictest0-21swapper/219:15:562
1804708993123,5cyclictest1937381-21ssh23:46:122
1804708993123,5cyclictest1937381-21ssh23:46:122
1804708993122,3cyclictest1840691-21diskmemload21:54:312
1804708993121,7cyclictest1879036-21timerandwakeup22:11:102
1804708993119,9cyclictest1001-21mta-sts-daemon22:28:022
1804707993123,5cyclictest1937137-21timerandwakeup23:46:071
1804707993123,5cyclictest1937137-21timerandwakeup23:46:071
1804706993125,3cyclictest0-21swapper/023:45:490
1804706993125,3cyclictest0-21swapper/023:45:490
1804706993125,3cyclictest0-21swapper/022:55:530
1804706993125,3cyclictest0-21swapper/000:01:030
1804706993124,5cyclictest1944774-21cups-browsed00:34:130
1804706993124,4cyclictest1840680-21cron21:10:010
1804706993124,4cyclictest1816693-21grep19:50:010
1804706993123,5cyclictest108584-21cups-browsed23:15:550
1804708993024,3cyclictest0-21swapper/222:10:002
1804708993023,4cyclictest1837664-21cron21:00:002
1804708993023,4cyclictest1837664-21cron21:00:002
1804708993022,5cyclictest1928762-21ssh23:33:032
1804708993022,5cyclictest1003-21mta-sts-daemon00:00:512
1804708993021,6cyclictest485-21in:imuxsock19:40:012
1804708993021,5cyclictest1824109-21sed20:11:052
1804707993022,5cyclictest1840691-21diskmemload22:10:011
1804707993022,5cyclictest1825689-21run-parts20:20:011
1804707993022,5cyclictest1806719-21fschecks_time19:15:471
1804707993022,5cyclictest1806187-21idleruntime-cro19:15:001
1804707993021,6cyclictest265-21systemd-journal20:10:011
1804707993021,6cyclictest0-21swapper/122:40:011
1804707993020,6cyclictest1810708-21systemd-run19:30:011
180470799301,22cyclictest1884699-21ssh22:20:541
1804706993025,3cyclictest1002-21mta-sts-daemon21:44:010
1804706993025,3cyclictest1001-21mta-sts-daemon00:12:340
1804706993024,3cyclictest0-21swapper/021:55:540
1804706993024,3cyclictest0-21swapper/021:55:540
1804706993023,5cyclictest1827187-21cron20:25:010
1804706993022,6cyclictest1898922-21latency_hist22:45:020
1804708992924,2cyclictest0-21swapper/219:50:022
1804708992922,3cyclictest0-21swapper/200:25:532
1804708992922,3cyclictest0-21swapper/200:25:532
180470899291,19cyclictest1857369-21ssh21:36:022
1804707992923,3cyclictest0-21swapper/121:45:011
1804707992923,3cyclictest0-21swapper/120:45:001
1804707992921,5cyclictest1853464-21/usr/sbin/munin21:30:541
1804707992921,5cyclictest1816569-21unixbench_multi19:46:051
1804706992922,3cyclictest151ktimers/022:05:590
1804706992921,6cyclictest1892713-21cron22:35:010
1804706992921,5cyclictest108584-21cups-browsed22:13:090
1804706992920,6cyclictest1941649-21cut23:55:010
1804706992917,9cyclictest0-21swapper/020:06:030
1804708992823,2cyclictest0-21swapper/222:45:432
1804708992822,3cyclictest0-21swapper/223:10:582
180470899282,21cyclictest108584-21cups-browsed23:20:542
1804708992821,4cyclictest1003-21mta-sts-daemon23:05:132
1804708992820,5cyclictest1911225-21latency_hist23:05:022
1804708992819,6cyclictest439-21systemd-logind20:30:012
1804708992816,5cyclictest1003-21mta-sts-daemon21:00:582
1804707992819,4cyclictest0-21swapper/119:25:011
1804707992819,4cyclictest0-21swapper/119:25:011
1804706992823,3cyclictest0-21swapper/021:00:000
1804706992823,3cyclictest0-21swapper/021:00:000
1804706992822,4cyclictest1834088-21ntp_kernel_pll_20:45:550
1804706992822,3cyclictest1000-21mta-sts-daemon23:21:320
1804708992721,3cyclictest0-21swapper/220:20:012
1804708992720,4cyclictest1821713-21cut20:05:472
180470799272,19cyclictest665-21snmpd00:25:571
180470799272,19cyclictest665-21snmpd00:25:571
1804707992720,4cyclictest1866949-21unixbench_multi21:51:021
1804707992720,4cyclictest1856888-21gltestperf21:35:511
1804707992720,4cyclictest1819675-21munin-plugin-st20:00:011
1804707992719,5cyclictest1838345-21kernelversion21:00:491
1804706992720,3cyclictest1830209-21kworker/u8:12-xprtiod21:25:450
1804706992718,7cyclictest1944774-21cups-browsed00:21:070
1804706992718,7cyclictest1944774-21cups-browsed00:21:070
1804706992718,7cyclictest0-21swapper/023:30:450
1804706992718,6cyclictest151ktimers/023:00:540
180470899263,14cyclictest428-21mta-sts-daemon19:20:522
180470899263,14cyclictest428-21mta-sts-daemon19:20:522
1804708992619,3cyclictest0-21swapper/200:36:102
1804708992618,5cyclictest1812183-21sh19:35:012
1804708992618,5cyclictest108584-21cups-browsed20:50:522
1804708992618,5cyclictest108584-21cups-browsed20:50:522
1804707992611,12cyclictest271ktimers/120:46:001
1804706992620,3cyclictest0-21swapper/020:05:020
1804706992619,5cyclictest1838092-21aten_r2power_vo21:00:450
180470699261,21cyclictest265-21systemd-journal20:32:500
180470699261,21cyclictest265-21systemd-journal20:32:500
180470699261,14cyclictest1811203-21sort19:30:470
180470999254,10cyclictest0-21swapper/321:25:593
1804708992519,3cyclictest0-21swapper/219:55:442
1804708992518,4cyclictest0-21swapper/219:40:592
1804707992520,3cyclictest0-21swapper/122:50:501
1804707992520,3cyclictest0-21swapper/122:50:501
1804707992520,2cyclictest0-21swapper/119:35:011
1804707992518,3cyclictest0-21swapper/121:24:371
1804706992520,3cyclictest151ktimers/020:10:580
1804706992517,6cyclictest1810472-21timerwakeupswit19:26:060
1804706992517,5cyclictest1960333-21ssh00:25:230
1804706992517,5cyclictest1960333-21ssh00:25:230
180470699251,15cyclictest1828375-21wc20:26:030
180470999243,12cyclictest0-21swapper/321:55:543
180470999243,12cyclictest0-21swapper/321:55:543
1804708992418,3cyclictest0-21swapper/219:10:012
1804707992418,4cyclictest0-21swapper/121:10:021
1804707992418,3cyclictest0-21swapper/119:51:051
1804706992419,3cyclictest151ktimers/020:35:520
1804706992419,3cyclictest151ktimers/020:35:520
1804706992419,3cyclictest151ktimers/019:55:530
180470999233,12cyclictest0-21swapper/322:06:043
1804707992315,5cyclictest1827187-21cron20:25:011
1804706992318,3cyclictest191rcuc/019:11:010
180470999222,12cyclictest0-21swapper/300:26:003
180470999222,12cyclictest0-21swapper/300:26:003
180470999222,11cyclictest0-21swapper/323:36:023
180470999222,11cyclictest0-21swapper/323:36:023
1804706992216,3cyclictest0-21swapper/020:50:550
1804706992216,3cyclictest0-21swapper/020:50:550
180470699221,18cyclictest1807160-21ntpq19:15:590
180470999214,9cyclictest0-21swapper/300:15:533
180470999214,9cyclictest0-21swapper/300:15:533
180470999213,10cyclictest0-21swapper/322:55:523
180470999212,12cyclictest0-21swapper/322:20:553
180470999212,11cyclictest0-21swapper/323:05:593
180470999212,11cyclictest0-21swapper/322:30:553
180470999204,9cyclictest0-21swapper/322:15:533
180470999204,9cyclictest0-21swapper/300:05:593
180470999204,8cyclictest0-21swapper/323:20:453
180470999204,11cyclictest0-21swapper/321:42:033
180470999203,9cyclictest0-21swapper/321:45:523
180470999203,9cyclictest0-21swapper/300:00:553
180470999203,11cyclictest0-21swapper/323:25:593
180470999202,9cyclictest0-21swapper/322:40:523
180470999202,10cyclictest0-21swapper/300:10:573
180470999201,13cyclictest0-21swapper/321:36:003
1804707992015,3cyclictest0-21swapper/120:35:011
1804707992015,3cyclictest0-21swapper/120:35:011
180470999193,10cyclictest0-21swapper/321:20:523
180470999192,11cyclictest0-21swapper/322:35:553
180470999192,10cyclictest0-21swapper/323:15:543
180470999192,10cyclictest0-21swapper/322:00:573
180470999192,10cyclictest0-21swapper/321:15:583
180470999191,11cyclictest0-21swapper/321:50:453
180470999191,11cyclictest0-21swapper/321:05:513
180470999183,9cyclictest0-21swapper/322:45:523
180470999183,8cyclictest0-21swapper/323:10:523
180470999183,10cyclictest0-21swapper/323:40:443
180470999183,10cyclictest0-21swapper/323:40:443
180470999182,8cyclictest0-21swapper/319:20:523
180470999182,8cyclictest0-21swapper/319:20:523
180470999182,10cyclictest0-21swapper/322:25:533
180470999181,9cyclictest0-21swapper/320:00:553
180470999181,9cyclictest0-21swapper/319:45:553
180470999181,8cyclictest0-21swapper/321:15:023
180470999181,11cyclictest0-21swapper/323:31:003
180470699182,12cyclictest1804661-21munin-run19:10:010
180470999175,8cyclictest0-21swapper/300:30:463
180470999174,8cyclictest0-21swapper/300:20:483
180470999174,8cyclictest0-21swapper/300:20:483
180470999173,7cyclictest0-21swapper/319:26:003
180470999173,6cyclictest0-21swapper/320:25:533
180470999173,10cyclictest0-21swapper/323:50:463
180470999172,8cyclictest0-21swapper/323:45:583
180470999172,8cyclictest0-21swapper/323:45:583
180470999172,8cyclictest0-21swapper/319:15:563
180470999172,11cyclictest0-21swapper/322:10:493
180470999172,10cyclictest0-21swapper/300:35:533
180470999171,8cyclictest0-21swapper/320:21:043
180470999171,11cyclictest0-21swapper/323:00:453
180470999164,4cyclictest0-21swapper/319:55:443
180470999163,7cyclictest0-21swapper/320:55:553
180470999163,7cyclictest0-21swapper/320:55:553
180470999163,7cyclictest0-21swapper/320:15:593
180470999162,9cyclictest0-21swapper/323:55:523
180470999162,8cyclictest0-21swapper/322:50:543
180470999162,8cyclictest0-21swapper/322:50:543
180470999162,8cyclictest0-21swapper/319:50:533
180470999162,7cyclictest0-21swapper/320:35:513
180470999162,7cyclictest0-21swapper/320:35:513
180470999162,7cyclictest0-21swapper/319:35:543
180470999161,9cyclictest0-21swapper/321:30:523
180470999161,8cyclictest0-21swapper/320:46:003
180470999161,7cyclictest0-21swapper/319:10:593
180470999153,6cyclictest0-21swapper/321:00:533
180470999153,6cyclictest0-21swapper/320:50:563
180470999153,6cyclictest0-21swapper/320:50:563
180470999142,7cyclictest0-21swapper/320:30:553
180470999142,7cyclictest0-21swapper/320:30:553
180470999142,7cyclictest0-21swapper/319:30:573
180470999142,6cyclictest0-21swapper/319:40:503
180470999142,5cyclictest0-21swapper/320:05:523
180470999141,7cyclictest0-21swapper/320:40:543
180470999133,6cyclictest0-21swapper/320:10:533
180470999101,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional