You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 00:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun May 24, 2026 12:43:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2027895995039,8cyclictest421-21dbus-daemon12:10:561
2027895995039,6cyclictest0-21swapper/112:21:011
2027895995039,6cyclictest0-21swapper/112:21:011
2027895994938,5cyclictest0-21swapper/112:01:081
2027894994939,8cyclictest2229286-21mailstats12:36:040
2027894994935,10cyclictest2209501-21switchtime12:11:100
2027896994840,5cyclictest2045899-21systemd-run08:10:012
2027896994634,8cyclictest2065543-21mailstats09:11:072
202789699462,35cyclictest2149622-21ssh10:55:572
2027894994639,5cyclictest2071976-21ssh09:20:010
2027894994630,10cyclictest12350irq/32-eth009:55:580
2027895994537,5cyclictest2136195-21munin-run10:40:011
2027895994537,5cyclictest2059391-21systemd-run08:55:011
2027894994537,5cyclictest2065896-21head09:11:100
2027896994434,6cyclictest2164101-21systemd-run11:15:022
2027896994433,7cyclictest2231649-21cron12:40:002
2027895994435,4cyclictest0-21swapper/110:00:581
2027894994431,10cyclictest2124175-21munin-run10:25:010
2027896994334,6cyclictest2120107-21sendmail10:20:002
2027896994334,6cyclictest2120107-21sendmail10:20:002
2027895994335,4cyclictest0-21swapper/110:56:001
2027895994333,6cyclictest2100153-21latency_hist09:55:021
2027894994337,4cyclictest2207828-21systemd-run12:10:010
2027894994328,10cyclictest12350irq/32-eth011:56:020
2027896994234,5cyclictest2183995-21systemd-run11:40:012
2027896994233,4cyclictest0-21swapper/211:26:022
2027896994231,7cyclictest421-21dbus-daemon11:44:402
2027896994231,7cyclictest261-21systemd-journal09:54:532
2027896994228,10cyclictest2109772-21mailstats10:06:132
2027895994233,6cyclictest439-21systemd-logind10:25:011
2027894994228,12cyclictest2141494-21needreboot10:45:550
202789699414,28cyclictest1225104-21cupsd12:32:542
2027895994132,6cyclictest1225109-21cups-browsed11:59:541
2027895994130,8cyclictest2187952-21sh11:45:011
2027894994134,4cyclictest2053447-21latency_hist08:35:020
2027894994133,5cyclictest2038409-21systemd-run07:45:010
2027894994131,4cyclictest2180894-21fschecks_time11:35:500
2027896994033,4cyclictest2170094-21awk11:21:112
202789699403,32cyclictest2209294-21sh12:11:052
2027896994032,5cyclictest2195914-21cron11:55:012
2027896994030,6cyclictest421-21dbus-daemon09:21:072
2027896994029,7cyclictest2062409-21true09:05:012
2027895994032,5cyclictest2225644-21unixbench_singl12:31:101
2027895994031,5cyclictest0-21swapper/111:36:011
2027894994028,9cyclictest548-21gdbus09:41:040
2027894994025,8cyclictest12350irq/32-eth012:31:000
2027894994025,12cyclictest2097857-21threads09:51:080
2027896993933,3cyclictest0-21swapper/209:00:012
2027896993931,4cyclictest0-21swapper/209:36:072
2027896993931,4cyclictest0-21swapper/209:36:072
2027896993930,5cyclictest2071977-21systemctl09:20:012
2027896993927,8cyclictest1225109-21cups-browsed10:31:552
2027896993911,11cyclictest2105649-21ntp_states10:01:042
2027895993930,5cyclictest2060872-21cut09:00:011
2027896993831,3cyclictest0-21swapper/208:35:022
2027896993829,6cyclictest2205804-21unixbench_singl12:06:102
2027895993829,6cyclictest1225104-21cupsd10:11:311
2027895993829,4cyclictest0-21swapper/111:05:581
2027894993832,4cyclictest2088050-21idleruntime-cro09:40:000
2027894993832,4cyclictest2088050-21idleruntime-cro09:40:000
2027894993831,5cyclictest1-21systemd11:00:010
202789699372,23cyclictest2165398-21ssh11:15:572
2027896993716,9cyclictest0-21swapper/212:28:232
202789699371,30cyclictest2213110-21ssh12:15:582
202789699371,30cyclictest2213110-21ssh12:15:582
2027896993710,8cyclictest2037869-21ntpq07:41:022
202789599373,26cyclictest2228984-21ssh12:35:551
2027895993729,5cyclictest2166055-21unixbench_singl11:16:101
2027896993628,5cyclictest421-21dbus-daemon09:50:012
2027896993613,15cyclictest421-21dbus-daemon09:41:042
2027895993630,3cyclictest0-21swapper/110:00:011
2027895993630,3cyclictest0-21swapper/108:00:001
2027895993630,3cyclictest0-21swapper/108:00:001
2027895993628,5cyclictest2073945-21unixbench_multi09:21:121
2027895993628,5cyclictest2056918-21fschecks_time08:45:501
2027895993628,4cyclictest0-21swapper/112:05:571
2027895993627,7cyclictest2108861-21aten_r2power_fr10:05:491
2027895993627,6cyclictest421-21dbus-daemon11:12:301
2027895993627,5cyclictest2050409-21systemd-run08:25:011
2027895993624,9cyclictest2092165-21latency_hist09:45:021
202789499363,27cyclictest2217052-21ssh12:20:570
202789499363,27cyclictest2217052-21ssh12:20:570
2027894993631,3cyclictest0-21swapper/011:05:010
2027894993612,18cyclictest2113427-21ssh10:11:000
202789699359,14cyclictest2157629-21ssh11:06:052
202789699356,17cyclictest2113566-21ssh10:11:052
2027896993528,4cyclictest1054-21mta-sts-daemon10:44:012
2027896993527,5cyclictest2141293-21modprobe10:45:532
2027896993527,5cyclictest2036875-21munin-plugin-st07:40:012
2027896993525,6cyclictest2033910-21systemd-run07:30:012
2027896993524,8cyclictest2080633-21/usr/sbin/munin09:30:522
2027896993512,14cyclictest2124138-21cron10:25:002
2027895993529,3cyclictest0-21swapper/111:21:101
2027895993528,4cyclictest0-21swapper/110:30:571
2027895993528,3cyclictest0-21swapper/109:16:051
2027895993527,5cyclictest439-21systemd-logind09:35:011
202789499354,26cyclictest2165464-21ssh11:15:590
2027894993527,5cyclictest2125970-21timerandwakeup10:26:140
2027894993527,5cyclictest1225109-21cups-browsed11:33:040
2027896993427,3cyclictest0-21swapper/211:35:012
2027896993425,6cyclictest2218391-21ssh12:22:262
2027896993425,6cyclictest2218391-21ssh12:22:262
2027896993421,8cyclictest2040825-21ntpq07:51:012
202789599344,25cyclictest2193270-21ssh11:50:571
2027895993428,3cyclictest0-21swapper/109:11:101
2027895993426,4cyclictest0-21swapper/109:36:041
2027895993426,4cyclictest0-21swapper/109:36:041
202789499345,6cyclictest171rcu_preempt09:06:060
2027894993426,5cyclictest0-21swapper/010:30:520
2027894993425,5cyclictest2145328-21kernelversion10:50:520
2027894993424,8cyclictest2136184-21cron10:40:010
2027894993423,8cyclictest2201166-21netstat12:01:010
202789699335,9cyclictest2197216-21ntpq11:56:012
2027896993321,8cyclictest421-21dbus-daemon08:00:012
2027896993321,8cyclictest421-21dbus-daemon08:00:012
2027896993319,10cyclictest2077617-21mailstats09:26:052
2027896993319,10cyclictest2030784-21/usr/sbin/munin07:16:072
2027895993326,4cyclictest1054-21mta-sts-daemon10:40:471
2027895993325,5cyclictest2096121-21latency_hist09:50:011
2027894993326,5cyclictest2221764-21sshd12:26:130
2027894993326,4cyclictest2044314-21head08:01:090
2027894993326,4cyclictest2044314-21head08:01:090
2027894993326,3cyclictest0-21swapper/009:21:100
2027894993324,6cyclictest2185949-21unixbench_singl11:41:090
2027894993316,15cyclictest2156887-21aten_r2power_po11:05:490
202789699329,7cyclictest2127169-21ssh10:28:082
2027896993224,5cyclictest2029373-21cron07:15:012
2027896993222,5cyclictest2201290-21ntpq12:01:052
202789699321,23cyclictest2101611-21ssh09:56:032
2027895993225,4cyclictest0-21swapper/111:45:591
2027895993225,4cyclictest0-21swapper/110:20:001
2027895993225,4cyclictest0-21swapper/110:20:001
2027895993224,5cyclictest439-21systemd-logind07:40:011
2027895993224,5cyclictest421-21dbus-daemon08:05:011
2027895993224,5cyclictest421-21dbus-daemon08:05:011
2027895993224,5cyclictest2153401-21missed_timers11:00:561
2027895993224,5cyclictest2062277-21unixbench_multi09:01:141
2027895993224,5cyclictest2030884-21munin-run07:20:011
202789599322,24cyclictest2221182-21ssh12:26:031
202789599321,23cyclictest2173353-21ssh11:25:571
202789499322,25cyclictest2161457-21ssh11:10:590
202789499322,24cyclictest2137615-21ssh10:40:590
202789499321,27cyclictest2105524-21ssh10:00:590
202789499321,22cyclictest2109722-21diskmemload10:06:120
2027896993124,3cyclictest0-21swapper/208:01:022
2027896993124,3cyclictest0-21swapper/208:01:022
2027896993122,6cyclictest2154655-21ssh11:02:062
2027896993117,9cyclictest2051365-21ntpq08:26:042
2027896993117,10cyclictest2063238-21ntpq09:05:592
2027895993125,3cyclictest0-21swapper/107:10:011
2027895993124,4cyclictest2054874-21munin-plugin-st08:40:011
2027895993124,3cyclictest0-21swapper/112:15:561
2027895993124,3cyclictest0-21swapper/112:15:561
2027895993122,6cyclictest2048865-21cron08:20:001
2027895993122,5cyclictest420-21avahi-daemon08:26:071
202789599312,22cyclictest2077498-21ssh09:26:031
2027894993124,5cyclictest439-21systemd-logind07:20:010
2027894993123,4cyclictest0-21swapper/011:26:050
2027894993116,9cyclictest12350irq/32-eth009:26:000
2027896993019,7cyclictest2145667-21ntpq10:50:592
2027896993018,8cyclictest2058807-21ntpq08:51:002
2027896993018,8cyclictest2034733-21ntpq07:30:562
2027896993018,7cyclictest2055823-21ntpq08:40:592
2027896993018,7cyclictest2048339-21ntpq08:16:022
2027896993017,8cyclictest2049757-21ntpq08:21:062
2027896993016,11cyclictest2188389-21ssh11:45:262
2027896993015,11cyclictest2133147-21ssh10:35:512
2027896993011,13cyclictest2057345-21ntpq08:46:012
2027895993024,3cyclictest0-21swapper/110:26:341
2027895993022,5cyclictest547-21gmain07:40:471
202789599302,24cyclictest2141655-21ssh10:46:001
202789599302,21cyclictest2145653-21ssh10:50:591
2027894993024,4cyclictest2120110-21basename10:20:000
2027894993024,4cyclictest2120110-21basename10:20:000
2027894993020,7cyclictest0-21swapper/011:50:470
2027894993018,5cyclictest420-21avahi-daemon11:48:200
2027894993010,14cyclictest151ktimers/007:45:590
2027896992912,12cyclictest2054353-21ntpq08:36:022
2027895992922,4cyclictest2045031-21kernelversion08:05:521
2027895992922,3cyclictest0-21swapper/111:30:591
2027894992925,2cyclictest0-21swapper/012:20:020
2027894992925,2cyclictest0-21swapper/012:20:020
2027894992919,5cyclictest2059901-21fschecks_time08:55:520
2027896992819,4cyclictest2031878-21ntpq07:21:022
2027896992818,6cyclictest2046742-21ntpq08:11:002
2027896992815,7cyclictest2039354-21ntpq07:46:022
202789599281,22cyclictest1017-21sendmail-mta08:30:541
2027894992822,3cyclictest0-21swapper/009:50:010
2027894992821,3cyclictest0-21swapper/008:46:010
2027894992813,10cyclictest2172032-21latency_hist11:25:000
202789599277,3cyclictest171rcu_preempt08:11:061
2027895992721,3cyclictest0-21swapper/107:10:571
2027895992719,5cyclictest2035402-21systemd-run07:35:011
2027895992719,5cyclictest1225109-21cups-browsed08:44:111
2027895992719,5cyclictest1225109-21cups-browsed07:25:051
2027894992721,4cyclictest2042048-21latency07:55:540
2027894992721,4cyclictest2042048-21latency07:55:540
2027894992720,4cyclictest0-21swapper/008:45:010
2027894992718,5cyclictest0-21swapper/008:26:020
202789499271,23cyclictest2054882-21awk08:40:010
2027894992621,3cyclictest2058484-21if_wlan008:50:520
2027894992619,3cyclictest0-21swapper/009:33:590
2027894992618,6cyclictest2035161-21timerwakeupswit07:31:080
2027895992513,9cyclictest1054-21mta-sts-daemon07:45:491
2027894992518,5cyclictest421-21dbus-daemon07:52:250
2027894992510,10cyclictest1225104-21cupsd08:16:040
2027895992418,3cyclictest0-21swapper/109:10:001
202789499249,11cyclictest151ktimers/008:21:070
2027894992419,3cyclictest2053495-21kworker/u8:8-xprtiod09:01:140
2027894992417,3cyclictest151ktimers/008:06:000
2027894992416,3cyclictest2029366-21cron07:15:010
202789599231,17cyclictest1-21systemd07:20:531
2027894992318,3cyclictest2029396-21kworker/u8:9-xprtiod07:35:520
2027894992311,10cyclictest151ktimers/008:11:000
2027896992212,7cyclictest2027906-21munin-run07:10:012
2027895992114,5cyclictest2040729-21collect207:50:571
202789499212,14cyclictest2033342-21ntpq07:26:000
2027894992115,3cyclictest0-21swapper/007:25:030
2027894992015,3cyclictest2027908-21systemd07:10:010
202789799163,7cyclictest0-21swapper/309:55:593
202789799162,9cyclictest0-21swapper/312:21:013
202789799162,9cyclictest0-21swapper/312:21:013
202789799161,5cyclictest0-21swapper/308:40:563
202789799153,7cyclictest0-21swapper/312:26:033
202789799153,7cyclictest0-21swapper/310:40:583
202789799153,7cyclictest0-21swapper/309:10:573
202789799152,7cyclictest0-21swapper/311:00:593
202789799152,7cyclictest0-21swapper/309:36:033
202789799152,7cyclictest0-21swapper/309:36:033
202789799151,7cyclictest0-21swapper/312:16:043
202789799151,7cyclictest0-21swapper/312:16:043
202789799151,7cyclictest0-21swapper/309:06:013
202789799144,5cyclictest0-21swapper/311:50:573
202789799143,7cyclictest0-21swapper/309:30:583
202789799142,8cyclictest0-21swapper/312:36:013
202789799142,7cyclictest0-21swapper/311:05:583
202789799142,7cyclictest0-21swapper/310:56:013
202789799142,7cyclictest0-21swapper/310:30:583
202789799142,7cyclictest0-21swapper/310:11:043
202789799142,7cyclictest0-21swapper/309:50:583
202789799142,6cyclictest0-21swapper/310:15:573
202789799142,6cyclictest0-21swapper/310:15:573
202789799135,6cyclictest0-21swapper/311:45:573
202789799133,6cyclictest0-21swapper/311:25:573
202789799133,6cyclictest0-21swapper/310:06:013
202789799133,4cyclictest0-21swapper/307:51:053
202789799132,7cyclictest0-21swapper/312:01:023
202789799132,7cyclictest0-21swapper/309:15:583
202789799132,6cyclictest0-21swapper/312:05:593
202789799132,6cyclictest0-21swapper/311:40:573
202789799132,6cyclictest0-21swapper/311:35:573
202789799131,9cyclictest0-21swapper/312:11:043
202789799131,7cyclictest0-21swapper/311:30:563
202789799131,7cyclictest0-21swapper/310:26:003
202789799131,6cyclictest0-21swapper/308:46:013
202789799131,6cyclictest0-21swapper/308:25:593
202789799124,5cyclictest0-21swapper/309:20:583
202789799123,5cyclictest0-21swapper/311:11:013
202789799122,6cyclictest0-21swapper/311:20:563
202789799122,6cyclictest0-21swapper/310:50:523
202789799122,6cyclictest0-21swapper/310:45:573
202789799122,5cyclictest0-21swapper/308:55:533
202789799122,5cyclictest0-21swapper/307:20:563
202789799122,4cyclictest0-21swapper/309:01:033
202789799121,7cyclictest0-21swapper/310:01:043
202789799121,7cyclictest0-21swapper/309:25:593
202789799121,6cyclictest0-21swapper/310:35:563
202789799121,6cyclictest0-21swapper/308:35:563
202789799121,6cyclictest0-21swapper/307:30:533
202789799121,4cyclictest0-21swapper/308:50:513
202789799112,6cyclictest0-21swapper/311:15:563
202789799112,6cyclictest0-21swapper/309:45:563
202789799112,5cyclictest0-21swapper/309:40:563
202789799112,5cyclictest0-21swapper/308:20:003
202789799112,5cyclictest0-21swapper/308:01:023
202789799112,5cyclictest0-21swapper/308:01:023
202789799112,4cyclictest0-21swapper/308:05:553
202789799111,6cyclictest0-21swapper/312:31:013
202789799111,6cyclictest0-21swapper/311:56:023
202789799111,5cyclictest0-21swapper/308:11:003
202789799111,5cyclictest0-21swapper/307:45:523
202789799111,5cyclictest0-21swapper/307:36:003
202789799102,5cyclictest0-21swapper/310:20:563
202789799102,5cyclictest0-21swapper/308:30:483
202789799102,4cyclictest0-21swapper/308:20:533
202789799101,5cyclictest0-21swapper/307:40:563
202789799101,5cyclictest0-21swapper/307:25:563
202789799101,5cyclictest0-21swapper/307:15:543
20278979991,5cyclictest0-21swapper/307:55:523
20278979991,5cyclictest0-21swapper/307:55:523
20278979991,5cyclictest0-21swapper/307:10:573
20278979971,3cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional