You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 16:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Fri Sep 04, 2026 12:42:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1970010997759,9cyclictest12650irq/32-eth009:31:100
1970010997759,9cyclictest12650irq/32-eth009:31:100
1970011995040,7cyclictest2024396-21sendmail09:40:011
1970011994939,6cyclictest420-21avahi-daemon07:36:111
1970012994839,6cyclictest2005989-21diskmemload11:15:022
1970011994630,5cyclictest203049250kworker/u8:1+xprtiod10:51:031
1970011994535,6cyclictest1-21systemd12:30:011
1970010994535,7cyclictest2009131-21ssh09:15:020
1970012994436,5cyclictest2130728-21cron12:35:002
1970011994437,4cyclictest2063863-21cron10:45:011
1970011994435,5cyclictest2133786-21basename12:40:011
1970010994436,5cyclictest2121657-21ssh12:20:010
1970010994436,5cyclictest2057768-21cron10:35:010
1970011994335,5cyclictest2099137-21unixbench_multi11:41:201
197001099431,36cyclictest2086438-21sshd11:21:100
197001099431,36cyclictest2086438-21sshd11:21:100
1970011994234,5cyclictest2033508-21munin-plugin-st09:55:011
1970012994133,5cyclictest2024417-21systemd09:40:012
1970011994134,4cyclictest2027469-21latency_hist09:45:021
1970011994131,6cyclictest420-21avahi-daemon11:48:541
1970012994032,5cyclictest2107755-21ssh11:56:122
1970012993930,6cyclictest1186442-21cups-browsed11:54:462
1970012993929,7cyclictest421-21dbus-daemon12:05:512
1970011993928,8cyclictest2081528-21sh11:13:141
1970010993932,4cyclictest2051839-21latency_hist10:25:020
1970012993833,2cyclictest0-21swapper/207:20:012
1970012993830,5cyclictest2063909-21latency_hist10:45:022
1970012993828,7cyclictest694-21cron07:45:012
1970011993831,4cyclictest2042626-21cut10:10:011
1970011993829,5cyclictest2071696-21unixbench-2d10:56:221
1970010993832,4cyclictest2069974-21systemd-run10:55:020
1970010993831,4cyclictest2098027-21aten_r2power_vo11:40:580
1970012993729,5cyclictest265-21systemd-journal12:30:012
1970012993729,5cyclictest2066921-21idleruntime-cro10:50:012
1970012993729,5cyclictest2057768-21cron10:35:012
1970012993727,7cyclictest2121747-21latency_hist12:20:022
1970012993727,7cyclictest2092813-21switchtime11:31:202
1970012993727,6cyclictest420-21avahi-daemon12:01:592
1970011993730,4cyclictest2066913-21grep10:50:011
1970011993729,5cyclictest1976514-21fschecks_count07:30:581
1970010993729,5cyclictest567-21gdbus07:20:020
1970010993729,5cyclictest2130755-21systemd-run12:35:010
1970012993628,5cyclictest2059604-21grep10:36:192
1970012993628,5cyclictest1186442-21cups-browsed08:30:412
1970011993629,4cyclictest2055462-21aten_r2power_vo10:30:561
1970011993628,5cyclictest2117026-21sendmail_mailtr12:11:171
1970011993627,5cyclictest1001-21mta-sts-daemon10:01:181
1970010993624,9cyclictest0-21swapper/009:20:010
1970010993613,13cyclictest151ktimers/010:36:070
1970012993528,4cyclictest1989558-21latency_hist08:15:022
1970012993528,4cyclictest1989558-21latency_hist08:15:022
1970012993527,6cyclictest2047533-21vmstat10:16:222
1970012993527,5cyclictest265-21systemd-journal10:12:142
1970012993527,5cyclictest1186442-21cups-browsed08:58:332
1970012993527,5cyclictest1186442-21cups-browsed08:01:212
1970012993526,6cyclictest2118784-21latency_hist12:15:032
1970012993526,6cyclictest1-21systemd09:55:012
1970012993526,5cyclictest2100356-21cron11:45:012
1970012993521,7cyclictest1997112-21latency_hist08:40:032
1970011993527,5cyclictest420-21avahi-daemon09:16:111
1970011993527,5cyclictest420-21avahi-daemon08:11:161
1970011993527,5cyclictest420-21avahi-daemon08:11:161
1970011993527,5cyclictest2128520-21fschecks_time12:30:581
1970011993527,5cyclictest1972009-21fschecks_count07:15:591
1970011993525,6cyclictest1186442-21cups-browsed12:24:151
1970011993525,6cyclictest1186442-21cups-browsed12:24:151
1970011993523,9cyclictest1186442-21cups-browsed09:49:231
1970011993512,3cyclictest171rcu_preempt12:06:111
1970010993530,3cyclictest0-21swapper/012:21:210
1970010993530,3cyclictest0-21swapper/012:21:210
1970010993529,3cyclictest0-21swapper/009:25:020
1970010993528,4cyclictest2106460-21systemd-run11:55:010
1970010993524,8cyclictest2089761-21switchtime11:26:170
1970010993524,8cyclictest2089761-21switchtime11:26:170
1970012993426,5cyclictest2041368-21unixbench_multi10:06:192
1970012993426,5cyclictest2021295-21ssh09:35:002
1970012993426,5cyclictest2021295-21ssh09:35:002
1970012993426,5cyclictest2005989-21diskmemload10:01:062
1970012993426,5cyclictest1977510-21systemd-run07:35:012
1970012993425,5cyclictest2027590-21latency_hist09:45:022
1970011993429,2cyclictest0-21swapper/110:20:011
1970011993426,5cyclictest2000008-21systemd-run08:50:021
1970011993425,6cyclictest2044457-21grep10:11:231
1970010993427,5cyclictest2131542-21fschecks_count12:35:570
1970010993423,8cyclictest2074582-21switchtime11:01:190
1970012993327,3cyclictest0-21swapper/210:25:022
1970012993326,4cyclictest1974523-21systemd-run07:25:012
1970012993325,5cyclictest2133816-21systemd-run12:40:012
1970012993325,5cyclictest2001495-21systemd-run08:55:012
1970012993325,5cyclictest1186442-21cups-browsed09:26:382
1970012993325,5cyclictest1186442-21cups-browsed09:26:382
1970011993327,3cyclictest0-21swapper/109:35:001
1970011993327,3cyclictest0-21swapper/109:35:001
1970011993326,4cyclictest2050548-21unixbench_singl10:21:191
1970011993325,5cyclictest2105218-21unixbench_multi11:51:221
1970011993325,5cyclictest1980080-21mailstats07:41:131
1970011993324,3cyclictest0-21swapper/109:10:011
1970011993322,7cyclictest2036542-21latency_hist10:00:021
1970010993327,4cyclictest2094276-21munin-plugin-st11:35:000
1970010993326,5cyclictest2000502-21fschecks_count08:50:590
1970010993324,6cyclictest2097311-21munin-run11:40:010
1970012993225,4cyclictest2101958-21ssh11:46:162
1970012993225,4cyclictest1989889-21aten_r2power_vo08:15:552
1970012993225,4cyclictest1002-21mta-sts-daemon11:16:112
1970012993224,5cyclictest2004497-21grep09:05:012
1970012993224,5cyclictest1993030-21fschecks_time08:25:592
1970011993225,4cyclictest2111252-21tail12:01:201
1970011993225,4cyclictest1974510-21munin-plugin-st07:25:011
1970011993224,5cyclictest2085127-21cron11:20:011
1970011993224,5cyclictest1186442-21cups-browsed12:16:221
1970011993224,5cyclictest1186442-21cups-browsed09:26:501
1970011993224,5cyclictest1186442-21cups-browsed09:26:501
1970011993223,6cyclictest421-21dbus-daemon10:30:011
1970011993223,6cyclictest421-21dbus-daemon10:30:011
1970010993225,4cyclictest1989479-21cron08:15:000
1970010993225,4cyclictest1989479-21cron08:15:000
1970010993223,7cyclictest2047502-21unixbench_singl10:16:200
1970012993124,4cyclictest2096071-21unixbench_multi11:36:202
1970012993124,4cyclictest2012127-21awk09:20:012
1970012993123,5cyclictest2013503-21ssh09:21:122
1970012993123,5cyclictest1987931-21head08:06:172
1970012993122,6cyclictest1186442-21cups-browsed09:12:202
1970012993121,7cyclictest2068484-21switchtime10:51:222
1970011993124,4cyclictest2092621-21ssh11:31:171
1970011993123,5cyclictest439-21systemd-logind08:25:011
1970011993123,5cyclictest439-21systemd-logind08:25:011
1970011993121,7cyclictest2089164-21kernelversion11:26:011
1970011993121,7cyclictest2089164-21kernelversion11:26:011
1970011993121,7cyclictest2075271-21sh11:02:411
1970011993121,7cyclictest2013756-21timerandwakeup09:21:171
1970010993125,4cyclictest421-21dbus-daemon10:50:020
1970010993124,4cyclictest2016981-21tail09:26:170
1970010993124,4cyclictest2016981-21tail09:26:170
1970010993122,6cyclictest420-21avahi-daemon09:49:230
1970012993024,3cyclictest0-21swapper/211:21:142
1970012993024,3cyclictest0-21swapper/211:21:142
1970012993023,5cyclictest0-21swapper/211:00:022
1970012993023,4cyclictest2122657-21latency12:21:042
1970012993023,4cyclictest2122657-21latency12:21:042
1970012993022,5cyclictest1976002-21cut07:30:012
1970012993022,5cyclictest1186442-21cups-browsed11:28:222
1970012993022,5cyclictest1186442-21cups-browsed11:28:222
1970012993022,5cyclictest1186442-21cups-browsed08:47:442
1970011993024,4cyclictest1985406-21aten_r2power_vo08:00:581
1970011993022,5cyclictest1982531-21fschecks_time07:50:591
1970010993023,4cyclictest2107201-21cpuspeed11:55:580
1970010993023,4cyclictest0-21swapper/011:46:170
1970010993022,5cyclictest567-21gdbus09:00:010
1970010993020,7cyclictest2025887-21sshd09:41:150
1970010993013,12cyclictest151ktimers/008:26:100
1970012992923,3cyclictest0-21swapper/211:06:122
1970012992922,4cyclictest2034322-21fschecks_count09:55:592
1970012992922,4cyclictest1971433-21unixbench_singl07:11:182
1970012992922,4cyclictest1186442-21cups-browsed07:36:122
1970012992920,7cyclictest2005379-21ntp_kernel_pll_09:06:082
1970011992924,2cyclictest0-21swapper/111:05:581
1970011992922,4cyclictest2108209-21unixbench_multi11:56:201
1970011992921,5cyclictest420-21avahi-daemon11:40:021
1970011992921,5cyclictest2000515-21fschecks_time08:51:001
1970011992920,7cyclictest2085419-21sh11:20:101
1970011992920,7cyclictest2085419-21sh11:20:101
1970011992919,4cyclictest0-21swapper/110:40:021
1970010992924,2cyclictest0-21swapper/010:06:190
1970010992922,4cyclictest2113104-21aten_r2power_en12:05:550
1970010992921,5cyclictest421-21dbus-daemon09:05:010
1970010992919,7cyclictest2039124-21ssh10:03:340
1970010992916,11cyclictest0-21swapper/010:41:180
1970010992911,15cyclictest567-21gdbus07:21:110
1970012992821,4cyclictest1992390-21unixbench_multi08:21:172
1970012992821,4cyclictest1992390-21unixbench_multi08:21:172
1970012992820,5cyclictest1998485-21cron08:45:012
1970012992818,6cyclictest2076049-21unin-run11:05:012
1970011992819,7cyclictest2004496-21munin-plugin-st09:05:011
197001199281,8cyclictest2007268-21ntpq09:11:091
1970010992822,4cyclictest0-21swapper/010:25:570
1970010992822,4cyclictest0-21swapper/010:25:570
1970010992821,5cyclictest1001-21mta-sts-daemon09:51:000
1970010992821,4cyclictest2076724-21cpu11:05:560
1970010992813,11cyclictest151ktimers/012:11:110
1970012992720,3cyclictest0-21swapper/210:26:122
1970012992720,3cyclictest0-21swapper/210:26:122
1970012992719,5cyclictest1186442-21cups-browsed07:49:132
1970012992719,4cyclictest0-21swapper/209:46:052
1970011992720,4cyclictest2002882-21unixbench_multi08:56:161
1970011992719,5cyclictest1975990-21cron07:30:011
1970011992718,6cyclictest1984824-21unixbench-2d07:56:161
1970011992718,6cyclictest1980850-21aten_r2power_po07:45:561
197001099275,14cyclictest151ktimers/007:46:040
1970010992722,3cyclictest1002-21mta-sts-daemon12:03:070
1970010992720,5cyclictest694-21cron11:00:010
1970010992720,4cyclictest2035242-21sed09:56:180
1970010992719,5cyclictest1979122-21latency_hist07:40:020
1970010992717,7cyclictest1997812-21needreboot08:41:030
197001099271,22cyclictest151ktimers/008:06:060
1970011992619,4cyclictest427-21polkitd08:18:471
197001199261,23cyclictest271ktimers/108:36:101
1970010992622,2cyclictest0-21swapper/011:15:010
1970010992622,2cyclictest0-21swapper/008:25:000
1970010992622,2cyclictest0-21swapper/008:25:000
197001099262,20cyclictest1984507-21ps07:56:120
1970010992619,4cyclictest1996979-21idleruntime-cro08:40:010
1970010992618,6cyclictest1995492-21cut08:35:010
1970010992618,6cyclictest1000-21mta-sts-daemon07:11:070
1970010992616,5cyclictest1285202-21munin-node11:16:270
1970010992613,11cyclictest1002-21mta-sts-daemon12:26:040
197001099261,22cyclictest2022492-21ntpq09:36:040
1970011992519,3cyclictest0-21swapper/108:40:581
1970011992510,10cyclictest421-21dbus-daemon08:06:081
197001099255,16cyclictest428-21mta-sts-daemon07:41:120
1970010992519,4cyclictest2043813-21sh10:11:090
1970010992517,5cyclictest1999882-21unixbench_multi08:46:180
1970010992517,5cyclictest1975036-21fschecks_time07:26:000
1970012992419,2cyclictest0-21swapper/208:00:012
1970012992415,6cyclictest1983363-21grep07:51:172
1970011992419,3cyclictest0-21swapper/108:35:021
1970011992418,3cyclictest0-21swapper/108:26:181
1970010992418,4cyclictest1970009-21cyclictest07:31:150
1970010992414,8cyclictest1970023-21unin-run07:10:010
1970010992413,8cyclictest0-21swapper/008:20:020
1970012992212,7cyclictest1-21systemd07:10:012
1970011992216,3cyclictest0-21swapper/107:11:011
1970010992217,2cyclictest0-21swapper/008:01:220
197001399211,13cyclictest0-21swapper/309:16:073
1970010992111,7cyclictest665-21snmpd09:06:090
197001399203,9cyclictest0-21swapper/312:06:053
197001399203,7cyclictest0-21swapper/309:01:063
197001399202,8cyclictest0-21swapper/308:11:053
197001399202,8cyclictest0-21swapper/308:11:053
197001099201,16cyclictest1982961-21ntpq07:51:100
197001399192,9cyclictest0-21swapper/310:06:103
197001399183,8cyclictest0-21swapper/311:15:553
197001399183,8cyclictest0-21swapper/310:00:013
197001399183,7cyclictest0-21swapper/308:56:043
197001399183,7cyclictest0-21swapper/308:41:093
197001399182,9cyclictest0-21swapper/311:31:093
197001399182,6cyclictest0-21swapper/308:16:053
197001399181,9cyclictest0-21swapper/311:06:093
197001399173,8cyclictest0-21swapper/312:36:083
197001399172,9cyclictest0-21swapper/312:11:083
197001399172,9cyclictest0-21swapper/311:26:183
197001399172,9cyclictest0-21swapper/311:26:183
197001399172,9cyclictest0-21swapper/311:21:103
197001399172,9cyclictest0-21swapper/311:21:103
197001399172,9cyclictest0-21swapper/310:31:063
197001399172,9cyclictest0-21swapper/310:11:163
197001399172,9cyclictest0-21swapper/309:21:053
197001399172,8cyclictest0-21swapper/311:36:143
197001399172,8cyclictest0-21swapper/310:16:133
197001399164,7cyclictest0-21swapper/312:16:123
197001399164,5cyclictest0-21swapper/307:36:073
197001399163,7cyclictest0-21swapper/310:41:073
197001399162,9cyclictest0-21swapper/312:26:153
197001399162,9cyclictest0-21swapper/309:26:013
197001399162,9cyclictest0-21swapper/309:26:013
197001399162,8cyclictest0-21swapper/309:51:003
197001399162,8cyclictest0-21swapper/309:36:083
197001399162,6cyclictest0-21swapper/310:25:553
197001399162,6cyclictest0-21swapper/310:25:553
197001399161,9cyclictest0-21swapper/311:01:103
197001399161,9cyclictest0-21swapper/309:46:123
197001399161,7cyclictest0-21swapper/308:36:103
197001199165,8cyclictest1-21systemd07:10:011
197001399154,5cyclictest0-21swapper/308:01:173
197001399153,7cyclictest0-21swapper/311:11:073
197001399152,8cyclictest0-21swapper/310:21:063
197001399152,7cyclictest0-21swapper/309:30:553
197001399152,7cyclictest0-21swapper/309:30:553
197001399152,7cyclictest0-21swapper/308:00:013
197001399152,7cyclictest0-21swapper/307:20:023
197001399152,6cyclictest0-21swapper/307:46:033
197001399152,6cyclictest0-21swapper/307:41:053
197001399152,5cyclictest0-21swapper/307:21:033
197001399151,9cyclictest0-21swapper/312:21:153
197001399151,9cyclictest0-21swapper/312:21:153
197001399151,9cyclictest0-21swapper/311:55:583
197001399151,8cyclictest0-21swapper/308:06:083
197001399151,7cyclictest0-21swapper/307:11:063
197001399151,6cyclictest0-21swapper/308:26:073
197001399151,6cyclictest0-21swapper/307:51:103
197001399151,10cyclictest0-21swapper/310:56:093
197001399151,10cyclictest0-21swapper/309:11:053
197001399142,8cyclictest0-21swapper/312:30:593
197001399142,8cyclictest0-21swapper/311:46:063
197001399142,7cyclictest0-21swapper/311:51:033
197001399142,7cyclictest0-21swapper/310:36:013
197001399142,7cyclictest0-21swapper/309:40:473
197001399141,9cyclictest0-21swapper/310:51:013
197001399141,8cyclictest0-21swapper/311:41:153
197001399141,8cyclictest0-21swapper/310:46:043
197001399141,8cyclictest0-21swapper/310:00:553
197001399141,6cyclictest0-21swapper/309:06:053
197001399141,6cyclictest0-21swapper/308:21:053
197001399141,6cyclictest0-21swapper/308:21:053
197001399132,8cyclictest0-21swapper/312:01:053
197001399131,7cyclictest0-21swapper/308:46:103
197001399131,6cyclictest0-21swapper/307:31:043
197001399122,5cyclictest0-21swapper/308:31:073
197001399121,7cyclictest0-21swapper/308:51:083
197001399111,5cyclictest0-21swapper/307:10:023
197001399101,6cyclictest0-21swapper/307:26:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional