You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 07:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Mon Aug 31, 2026 00:42:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1734667995040,7cyclictest421-21dbus-daemon21:31:132
173466799494,39cyclictest1857386-21ssh23:31:042
1734666994739,5cyclictest1828721-21cron22:45:011
1734665994740,4cyclictest1840933-21grep23:05:010
1734666994638,5cyclictest1772475-21head21:11:171
1734666994537,5cyclictest1749672-21systemd-run20:00:011
1734665994536,5cyclictest1874562-40systemctl00:00:010
1734666994437,4cyclictest1786021-21kworker/u8:2-rpciod21:50:011
1734666994436,5cyclictest1825710-21systemd22:40:021
1734665994438,4cyclictest1877613-21munin-run00:05:010
173466799434,33cyclictest1823944-21ssh22:36:132
1734666994336,3cyclictest0-21swapper/123:06:041
1734665994327,14cyclictest0-21swapper/020:20:010
1734666994214,8cyclictest171rcu_preempt21:41:061
1734665994236,4cyclictest1894650-21tail00:31:190
173466599422,21cyclictest171rcu_preempt20:51:090
173466599422,21cyclictest171rcu_preempt20:51:090
1734667994131,6cyclictest1769173-21systemd-run21:05:012
1734667994131,6cyclictest1-21systemd00:10:012
1734666994133,5cyclictest1804414-21latency_hist22:05:011
1734666994133,5cyclictest1804414-21latency_hist22:05:011
1734666994132,5cyclictest1847055-21systemd-run23:15:011
1734665994135,3cyclictest0-21swapper/021:36:090
1734667994035,2cyclictest0-21swapper/220:00:012
1734667994032,5cyclictest1792268-21latency_hist21:45:022
1734666994033,3cyclictest0-21swapper/123:16:041
1734666994032,5cyclictest1878106-21/usr/sbin/munin00:06:031
1734666994032,5cyclictest1776797-21systemd-run21:20:011
1734666994032,5cyclictest1776797-21systemd-run21:20:011
1734666994030,6cyclictest1744179-21sshd19:40:571
1734665994019,5cyclictest12750irq/33-eth022:56:090
1734667993931,5cyclictest1865446-21latency_hist23:45:022
1734667993931,5cyclictest1865446-21latency_hist23:45:022
1734666993932,4cyclictest1874534-21sendmail00:00:011
1734666993931,5cyclictest1868392-21cron23:50:001
1734666993931,5cyclictest1868392-21cron23:50:001
1734666993931,5cyclictest1769149-21cron21:05:001
1734666993931,5cyclictest1766133-21cron20:55:001
1734666993931,5cyclictest1766133-21cron20:55:001
1734666993930,5cyclictest0-21swapper/100:40:011
1734666993929,8cyclictest1894144-21ntp_offset00:31:101
1734665993926,10cyclictest0-21swapper/020:30:020
1734667993833,2cyclictest0-21swapper/222:56:092
1734666993832,3cyclictest0-21swapper/122:55:011
1734666993831,5cyclictest0-21swapper/122:20:551
1734666993829,6cyclictest1811141-21cpuspeed22:15:551
1734665993832,4cyclictest1813451-21munin-plugin-st22:20:010
1734665993829,7cyclictest0-21swapper/023:10:010
173466599382,32cyclictest151ktimers/000:06:040
1734665993822,6cyclictest4095404-21cups-browsed22:42:070
1734667993729,5cyclictest1767649-21sendmail21:00:002
1734667993728,6cyclictest1869523-21needreboot23:51:012
1734667993728,6cyclictest1869523-21needreboot23:51:012
1734667993728,6cyclictest1813667-21latency_hist22:20:032
1734666993731,3cyclictest0-21swapper/100:26:061
1734666993729,5cyclictest1871628-21latency_hist23:55:021
1734666993729,5cyclictest1871628-21latency_hist23:55:021
1734666993729,5cyclictest1860648-21processes23:36:131
1734665993733,2cyclictest0-21swapper/023:41:170
1734665993733,2cyclictest0-21swapper/023:41:170
1734665993732,3cyclictest0-21swapper/022:00:570
1734665993732,3cyclictest0-21swapper/022:00:570
1734665993732,2cyclictest0-21swapper/021:31:230
1734665993731,4cyclictest1845112-21needreboot23:11:010
1734665993729,5cyclictest420-21avahi-daemon21:56:570
1734665993728,7cyclictest1749637-21grep20:00:010
1734665993726,8cyclictest0-21swapper/022:10:020
1734665993715,8cyclictest12750irq/33-eth000:21:070
1734665993713,21cyclictest1777999-21ssh21:21:070
1734667993630,3cyclictest0-21swapper/221:40:012
1734667993629,3cyclictest0-21swapper/223:46:092
1734667993629,3cyclictest0-21swapper/223:46:092
1734667993628,5cyclictest1848800-21grep23:16:162
1734667993628,5cyclictest1799031-21uniq21:55:572
1734667993628,5cyclictest1782929-21grep21:30:012
1734667993627,6cyclictest1773912-21latency_hist21:15:032
1734667993626,6cyclictest1876358-21unixbench_multi00:01:222
1734666993630,3cyclictest0-21swapper/100:11:051
1734666993628,5cyclictest1795852-21telnet21:50:571
1734666993627,6cyclictest421-21dbus-daemon00:04:011
1734666993626,6cyclictest1003-21mta-sts-daemon21:21:051
1734665993628,4cyclictest1002-21mta-sts-daemon23:53:080
1734665993628,4cyclictest1002-21mta-sts-daemon23:53:080
1734665993620,12cyclictest151ktimers/019:41:030
1734667993527,3cyclictest0-21swapper/200:16:082
173466799351,29cyclictest1832999-21ssh22:51:052
1734666993524,7cyclictest4095404-21cups-browsed20:16:101
1734665993528,4cyclictest421-21dbus-daemon00:40:020
1734665993527,5cyclictest0-21swapper/021:30:010
1734665993523,9cyclictest0-21swapper/020:05:020
1734667993428,3cyclictest0-21swapper/221:10:012
1734667993427,4cyclictest1000-21mta-sts-daemon23:06:092
1734667993426,3cyclictest0-21swapper/222:11:072
1734666993427,4cyclictest1000-21mta-sts-daemon23:31:181
1734666993426,5cyclictest421-21dbus-daemon21:10:021
1734666993426,5cyclictest421-21dbus-daemon00:20:021
1734666993426,5cyclictest1743645-21munin-plugin-st19:40:001
1734666993426,5cyclictest1743645-21munin-plugin-st19:40:001
1734666993426,5cyclictest1736143-21cron19:15:011
1734666993421,5cyclictest1000-21mta-sts-daemon23:44:581
1734666993421,5cyclictest1000-21mta-sts-daemon23:44:581
173466599349,18cyclictest1881863-21ssh00:11:050
1734665993427,4cyclictest1868520-21latency_hist23:50:020
1734665993427,4cyclictest1868520-21latency_hist23:50:020
173466599342,28cyclictest1820743-21ssh22:31:060
1734667993327,3cyclictest0-21swapper/223:15:012
1734667993326,4cyclictest1889808-21cron00:25:012
1734667993326,3cyclictest0-21swapper/220:01:062
1734667993325,3cyclictest0-21swapper/223:56:042
1734667993324,5cyclictest421-21dbus-daemon00:40:022
1734666993327,3cyclictest0-21swapper/122:31:191
1734666993326,4cyclictest1760147-21ls20:35:011
1734666993326,3cyclictest0-21swapper/122:10:561
1734666993325,4cyclictest0-21swapper/121:36:071
173466599333,24cyclictest665-21snmpd19:36:090
173466599333,24cyclictest665-21snmpd19:36:090
173466599333,22cyclictest1001-21mta-sts-daemon21:41:070
1734665993326,5cyclictest0-21swapper/020:45:010
1734665993326,4cyclictest1003-21mta-sts-daemon21:53:330
1734665993317,11cyclictest12650irq/32-eth021:11:110
1734665993315,10cyclictest12650irq/32-eth000:26:100
173466799323,22cyclictest1851405-21ssh23:21:092
1734667993224,5cyclictest1806120-21unixbench_multi22:06:232
1734667993223,6cyclictest4095404-21cups-browsed20:46:062
1734667993223,6cyclictest4095404-21cups-browsed20:46:062
1734666993224,5cyclictest1836631-21unixbench_singl22:56:171
1734666993224,5cyclictest1806110-21unixbench_multi22:06:231
1734666993224,5cyclictest1799022-21fschecks_count21:55:571
1734665993227,2cyclictest0-21swapper/023:40:010
1734665993227,2cyclictest0-21swapper/022:10:560
1734665993226,3cyclictest1884290-21aten2.4-expect00:15:560
1734665993224,5cyclictest4095404-21cups-browsed22:48:320
1734665993224,5cyclictest4095404-21cups-browsed22:48:320
1734665993221,9cyclictest4095404-21cups-browsed23:31:080
1734665993220,8cyclictest4095404-21cups-browsed21:46:070
1734665993214,12cyclictest12650irq/32-eth023:16:070
1734667993124,4cyclictest1001-21mta-sts-daemon22:27:412
1734667993123,4cyclictest0-21swapper/222:41:022
1734667993122,6cyclictest1754159-21systemd-run20:15:012
1734667993122,5cyclictest0-21swapper/222:46:052
1734667993122,5cyclictest0-21swapper/222:46:052
1734667993119,8cyclictest0-21swapper/221:16:042
1734667993119,8cyclictest0-21swapper/221:16:042
1734666993125,3cyclictest0-21swapper/119:31:161
1734666993125,3cyclictest0-21swapper/119:31:161
1734666993123,5cyclictest1839017-21needreboot23:01:021
1734666993123,5cyclictest0-21swapper/121:30:541
1734666993122,6cyclictest4095404-21cups-browsed23:21:571
1734666993122,6cyclictest1-21systemd21:00:001
1734666993117,11cyclictest271ktimers/121:26:141
1734665993125,3cyclictest0-21swapper/023:21:000
1734667993024,3cyclictest0-21swapper/220:40:012
1734667993024,3cyclictest0-21swapper/220:31:132
1734667993023,4cyclictest1754506-21aten_r2power_po20:15:542
1734667993022,5cyclictest1881540-21sh00:10:592
1734666993024,3cyclictest0-21swapper/122:46:141
1734666993024,3cyclictest0-21swapper/122:46:141
1734666993024,3cyclictest0-21swapper/100:21:181
1734666993022,5cyclictest1819555-21cron22:30:011
173466699301,9cyclictest1854447-21ntpq23:26:081
1734666993019,8cyclictest1756504-21ntp_kernel_err20:21:051
1734665993025,2cyclictest0-21swapper/019:50:000
1734665993024,4cyclictest1752500-21sed20:06:150
1734665993022,5cyclictest1003-21mta-sts-daemon22:26:110
1734665993017,9cyclictest12650irq/32-eth019:25:010
173466799294,21cyclictest750-21ntpd19:51:032
1734667992921,5cyclictest1862351-21unin-run23:40:022
1734667992921,5cyclictest1002-21mta-sts-daemon20:05:552
1734667992919,7cyclictest1816268-21ssh22:24:182
173466799291,21cyclictest1894150-21ssh00:31:102
1734666992923,3cyclictest0-21swapper/119:46:091
1734666992921,5cyclictest1739183-21systemd-run19:25:011
1734666992918,6cyclictest0-21swapper/120:10:581
173466599297,14cyclictest151ktimers/020:46:090
173466599297,14cyclictest151ktimers/020:46:090
1734665992922,5cyclictest0-21swapper/019:10:010
1734665992921,5cyclictest439-21systemd-logind21:05:010
1734665992916,9cyclictest0-21swapper/023:26:110
1734667992823,3cyclictest1681788-21kworker/u8:13-xprtiod19:36:012
1734667992823,3cyclictest1681788-21kworker/u8:13-xprtiod19:36:012
173466799282,22cyclictest1854382-21ssh23:26:052
1734667992821,4cyclictest1002-21mta-sts-daemon20:53:572
1734667992821,4cyclictest1002-21mta-sts-daemon20:53:572
1734667992821,3cyclictest0-21swapper/222:31:152
1734667992820,5cyclictest1742074-21unixbench_singl19:31:162
1734667992820,5cyclictest1742074-21unixbench_singl19:31:162
1734667992819,7cyclictest1793647-21sh21:46:192
1734666992821,4cyclictest1761506-21unixbench_multi20:36:141
1734665992820,5cyclictest1767624-21latency_hist21:00:020
1734667992722,2cyclictest0-21swapper/223:01:012
1734667992722,2cyclictest0-21swapper/200:26:162
1734667992718,6cyclictest1000-21mta-sts-daemon19:23:322
1734667992718,6cyclictest0-21swapper/221:21:182
1734666992720,4cyclictest1762193-21gltestperf20:40:571
1734666992720,4cyclictest1758682-21latency_hist20:30:021
173466599273,14cyclictest428-21mta-sts-daemon20:31:110
1734665992721,3cyclictest0-21swapper/022:35:550
1734665992720,4cyclictest151ktimers/020:36:050
1734665992716,8cyclictest151ktimers/019:11:090
173466599271,22cyclictest1816644-21sed22:25:030
1734665992712,10cyclictest1770643-21diskmemload21:15:210
1734665992712,10cyclictest1770643-21diskmemload21:15:210
1734667992619,4cyclictest1804347-21munin-plugin-st22:05:002
1734667992619,4cyclictest1804347-21munin-plugin-st22:05:002
1734667992619,4cyclictest1001-21mta-sts-daemon19:16:032
1734667992615,8cyclictest1796868-21timerwakeupswit21:51:172
1734666992619,4cyclictest1747252-21if_wlan019:50:571
173466599262,17cyclictest665-21snmpd19:31:080
173466599262,17cyclictest665-21snmpd19:31:080
1734665992621,3cyclictest1000-21mta-sts-daemon19:16:050
1734665992620,3cyclictest0-21swapper/019:51:050
1734667992517,6cyclictest1001-21mta-sts-daemon19:41:102
1734666992520,2cyclictest0-21swapper/120:06:151
1734666992520,2cyclictest0-21swapper/119:20:011
1734666992516,6cyclictest4095404-21cups-browsed20:00:551
1734666992514,8cyclictest1740660-21cut19:30:011
1734667992418,3cyclictest1003-21mta-sts-daemon19:11:062
1734667992415,5cyclictest351ktimers/220:21:042
1734666992418,3cyclictest0-21swapper/119:10:011
1734665992419,2cyclictest0-21swapper/022:51:150
1734665992412,3cyclictest0-21swapper/021:10:010
1734667992318,2cyclictest0-21swapper/219:30:012
1734667992317,3cyclictest0-21swapper/220:30:012
173466599232,16cyclictest1740053-21ntpq19:26:050
1734667992213,6cyclictest1762544-21ntpq20:41:052
1734666992214,5cyclictest1763673-21fschecks_count20:45:571
1734666992214,5cyclictest1763673-21fschecks_count20:45:571
173466599222,15cyclictest1756579-21ntpq20:21:080
173466599221,18cyclictest1753507-21ntpq20:11:060
173466799212,14cyclictest1746130-21ntpq19:46:082
173466899191,11cyclictest0-21swapper/300:00:013
173466899181,9cyclictest0-21swapper/323:16:083
173466899173,7cyclictest0-21swapper/321:41:073
173466899172,6cyclictest0-21swapper/322:36:053
173466899171,7cyclictest0-21swapper/322:46:063
173466899171,7cyclictest0-21swapper/322:46:063
173466899162,8cyclictest0-21swapper/321:56:063
173466899162,6cyclictest0-21swapper/300:11:033
173466899161,8cyclictest0-21swapper/323:01:083
173466899161,8cyclictest0-21swapper/322:56:113
173466899161,8cyclictest0-21swapper/321:36:063
173466899161,8cyclictest0-21swapper/300:26:033
1734667991611,2cyclictest0-21swapper/219:10:022
173466899153,6cyclictest0-21swapper/323:06:073
173466899152,7cyclictest0-21swapper/322:31:063
173466899151,8cyclictest0-21swapper/323:36:093
173466899151,8cyclictest0-21swapper/322:51:083
173466899151,8cyclictest0-21swapper/321:46:123
173466899151,8cyclictest0-21swapper/321:21:073
173466899151,8cyclictest0-21swapper/319:56:073
173466899151,8cyclictest0-21swapper/319:36:103
173466899151,8cyclictest0-21swapper/319:36:103
173466899151,7cyclictest0-21swapper/322:41:033
173466899151,6cyclictest0-21swapper/320:51:063
173466899151,6cyclictest0-21swapper/320:51:063
173466899151,6cyclictest0-21swapper/320:01:033
173466899151,5cyclictest0-21swapper/321:01:053
173466899143,5cyclictest0-21swapper/322:01:043
173466899143,5cyclictest0-21swapper/322:01:043
173466899143,5cyclictest0-21swapper/300:06:093
173466899142,6cyclictest0-21swapper/323:41:053
173466899142,6cyclictest0-21swapper/323:41:053
173466899141,8cyclictest0-21swapper/323:11:073
173466899141,7cyclictest0-21swapper/323:31:063
173466899141,7cyclictest0-21swapper/322:16:063
173466899141,7cyclictest0-21swapper/322:11:053
173466899141,7cyclictest0-21swapper/321:51:063
173466899141,7cyclictest0-21swapper/320:36:033
173466899141,7cyclictest0-21swapper/319:35:013
173466899141,7cyclictest0-21swapper/319:35:013
173466899141,6cyclictest0-21swapper/322:26:053
173466899141,6cyclictest0-21swapper/320:46:093
173466899141,6cyclictest0-21swapper/320:46:093
173466899141,6cyclictest0-21swapper/320:16:093
173466899141,4cyclictest0-21swapper/300:00:563
173466899134,6cyclictest0-21swapper/321:31:153
173466899132,6cyclictest0-21swapper/323:26:113
173466899132,6cyclictest0-21swapper/300:36:063
173466899131,7cyclictest0-21swapper/321:16:013
173466899131,7cyclictest0-21swapper/321:16:013
173466899131,7cyclictest0-21swapper/320:41:053
173466899131,7cyclictest0-21swapper/300:16:083
173466899131,6cyclictest0-21swapper/322:06:073
173466899131,6cyclictest0-21swapper/320:21:053
173466899131,6cyclictest0-21swapper/320:11:103
173466899131,6cyclictest0-21swapper/319:41:033
173466899131,5cyclictest0-21swapper/320:56:023
173466899123,4cyclictest0-21swapper/320:26:043
173466899122,5cyclictest0-21swapper/322:25:023
173466899122,5cyclictest0-21swapper/321:10:543
173466899122,5cyclictest0-21swapper/300:31:003
173466899122,4cyclictest0-21swapper/319:16:023
173466899121,8cyclictest0-21swapper/323:51:103
173466899121,8cyclictest0-21swapper/323:51:103
173466899121,8cyclictest0-21swapper/300:20:573
173466899121,7cyclictest0-21swapper/319:51:033
173466899121,6cyclictest0-21swapper/323:21:003
173466899121,6cyclictest0-21swapper/321:06:133
173466899121,6cyclictest0-21swapper/319:46:043
173466899121,5cyclictest0-21swapper/319:11:023
173466899112,6cyclictest0-21swapper/323:46:033
173466899112,6cyclictest0-21swapper/323:46:033
173466899112,5cyclictest0-21swapper/321:26:033
173466899112,4cyclictest0-21swapper/319:26:033
173466899111,6cyclictest0-21swapper/320:31:043
173466899111,6cyclictest0-21swapper/320:06:023
173466899111,6cyclictest0-21swapper/319:21:103
17346689971,3cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional