You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 18:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Thu Jun 04, 2026 12:42:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1886891995849,6cyclictest421-21dbus-daemon08:46:052
1886889995330,14cyclictest151ktimers/007:21:010
1886890995140,7cyclictest1903397-21systemd-run08:05:011
188689099505,37cyclictest1987161-21ssh10:51:081
188689199462,5cyclictest171rcu_preempt11:21:042
1886890994535,7cyclictest1051-21mta-sts-daemon12:31:041
188689099453,35cyclictest1961988-21ssh10:11:051
1886890994532,10cyclictest2047492-21unixbench_multi12:26:161
1886891994437,3cyclictest0-21swapper/211:31:092
1886890994436,5cyclictest1897386-21grep07:45:011
1886890994436,5cyclictest1889911-21systemd-run07:20:011
1886890994436,5cyclictest1889911-21systemd-run07:20:011
188689099434,27cyclictest1922872-21diskmemload11:01:041
1886890994336,3cyclictest0-21swapper/109:26:141
1886889994338,2cyclictest0-21swapper/011:40:010
1886889994335,5cyclictest2048855-21systemd-run12:30:010
1886889994334,6cyclictest1903407-21unin-run08:05:020
1886889994232,7cyclictest0-21swapper/011:26:190
188689199415,33cyclictest2034290-21ssh12:06:032
1886891994133,3cyclictest0-21swapper/210:41:062
1886890994133,5cyclictest1943592-21tail09:41:151
1886890994132,6cyclictest420-21avahi-daemon09:31:041
1886890994131,6cyclictest2004788-21ssh11:20:011
1886890994033,4cyclictest2029945-21munin-run12:00:011
1886890994032,5cyclictest1975101-21unixbench_multi10:31:151
1886890994032,5cyclictest1975101-21unixbench_multi10:31:151
1886890994030,6cyclictest1894566-21latency_hist07:35:031
1886891993931,4cyclictest1051-21mta-sts-daemon10:15:022
1886890993931,3cyclictest0-21swapper/109:51:141
1886890993930,6cyclictest1912392-21systemd-run08:35:011
1886890993930,5cyclictest1903908-21fschecks_count08:05:561
1886891993828,7cyclictest421-21dbus-daemon10:08:412
1886891993817,16cyclictest351ktimers/207:41:072
1886890993830,5cyclictest1960720-21grep10:10:011
1886890993829,5cyclictest1051-21mta-sts-daemon09:48:021
1886890993827,7cyclictest1895869-21munin-plugin-st07:40:011
1886889993831,5cyclictest1954398-21ssh10:00:010
1886889993831,4cyclictest1973309-21sh10:30:000
1886889993831,4cyclictest1973309-21sh10:30:000
1886889993831,4cyclictest1914033-21latency_hist08:40:020
188689199375,7cyclictest171rcu_preempt11:01:002
1886891993730,4cyclictest2017362-21sendmail11:40:012
1886890993729,5cyclictest1929174-21idleruntime-cro09:20:001
1886890993726,8cyclictest1907884-21idleruntime-cro08:20:011
1886890993723,10cyclictest2009134-21ntp_kernel_err11:26:041
1886889993728,6cyclictest1086143-21cups-browsed12:38:230
188688999371,34cyclictest0-21swapper/007:41:020
1886891993630,3cyclictest0-21swapper/212:20:002
1886891993630,3cyclictest0-21swapper/209:05:002
1886891993630,3cyclictest0-21swapper/209:05:002
1886891993630,3cyclictest0-21swapper/207:40:012
1886891993628,5cyclictest1888366-21cron07:15:002
1886891993628,5cyclictest1888366-21cron07:15:002
1886891993625,8cyclictest1922872-21diskmemload11:15:552
188689199362,27cyclictest2043793-21ssh12:21:042
188689199362,26cyclictest1936864-21ssh09:31:092
188689199362,25cyclictest1971540-21ssh10:26:112
188689199362,25cyclictest1971540-21ssh10:26:112
188689199362,24cyclictest1952615-21ssh09:56:092
1886890993627,6cyclictest1901914-21latency_hist08:00:021
1886889993627,6cyclictest421-21dbus-daemon09:45:570
1886889993623,8cyclictest2034309-21sh12:06:040
1886889993622,11cyclictest1921370-21cron09:05:000
1886889993622,11cyclictest1921370-21cron09:05:000
1886889993619,12cyclictest12350irq/32-eth011:41:090
188688999361,28cyclictest2021787-21sh11:46:050
1886891993529,3cyclictest0-21swapper/212:35:012
1886891993524,8cyclictest2023693-21systemd11:50:022
188689099353,22cyclictest2012393-21ssh11:31:071
188689099353,14cyclictest1997293-21ssh11:06:271
1886890993523,9cyclictest1898256-21ntp_kernel_pll_07:46:021
188689099352,26cyclictest1990279-21ssh10:56:041
1886890993516,14cyclictest421-21dbus-daemon10:26:071
1886890993516,14cyclictest421-21dbus-daemon10:26:071
1886890993514,4cyclictest28-21ksoftirqd/111:46:011
1886890993511,7cyclictest28-21ksoftirqd/109:35:271
1886889993530,3cyclictest1922750-21tail09:06:130
1886889993530,3cyclictest1922750-21tail09:06:130
1886889993529,3cyclictest2004796-21munin-plugin-st11:20:010
1886889993529,3cyclictest0-21swapper/009:25:010
1886889993528,4cyclictest1924083-21ntp_kernel_pll_09:11:020
1886889993527,6cyclictest1981460-21grep10:41:140
1886889993525,7cyclictest0-21swapper/012:20:000
1886891993428,2cyclictest0-21swapper/210:35:012
1886891993428,2cyclictest0-21swapper/210:35:012
1886891993427,3cyclictest0-21swapper/210:46:142
1886891993426,4cyclictest0-21swapper/209:16:012
1886891993424,4cyclictest1055-21mta-sts-daemon07:51:012
1886891993413,4cyclictest0-21swapper/208:11:062
188689099343,23cyclictest1977666-21ssh10:36:021
188689099343,23cyclictest1924224-21ssh09:11:081
1886890993425,5cyclictest2037210-21ssh12:11:011
1886890993424,7cyclictest1-21systemd07:10:011
1886890993424,7cyclictest1-21systemd07:10:011
188689099342,22cyclictest2043892-21diskmemload12:21:081
1886889993425,6cyclictest420-21avahi-daemon10:01:200
1886889993424,7cyclictest2045695-21cron12:25:000
188688999342,29cyclictest1983990-21ssh10:46:030
1886889993412,13cyclictest12350irq/32-eth011:31:070
1886891993325,5cyclictest421-21dbus-daemon09:10:012
1886891993325,5cyclictest421-21dbus-daemon09:10:012
1886891993324,6cyclictest1967653-21tail10:20:522
1886891993324,6cyclictest1967653-21tail10:20:522
188689199332,26cyclictest1977642-21ssh10:36:012
188689099339,8cyclictest1899758-21ntpq07:51:011
188689099336,8cyclictest1916869-21idleruntime-cro08:50:011
188689099336,7cyclictest1905122-21cut08:10:521
188689099334,20cyclictest2040601-21ssh12:16:021
1886890993324,5cyclictest1054-21mta-sts-daemon10:23:361
1886890993324,5cyclictest1054-21mta-sts-daemon10:23:361
1886890993324,5cyclictest0-21swapper/111:51:021
1886890993322,7cyclictest421-21dbus-daemon11:10:301
1886890993322,5cyclictest1914802-21ntpq08:41:061
1886890993322,5cyclictest0-21swapper/112:06:041
188689099332,24cyclictest1955616-21ssh10:01:031
1886890993314,4cyclictest28-21ksoftirqd/111:41:051
1886889993328,3cyclictest1964614-21cpuspeed10:15:530
1886889993328,3cyclictest1964614-21cpuspeed10:15:530
1886889993327,4cyclictest1915399-21systemd-run08:45:010
1886889993327,3cyclictest0-21swapper/012:11:090
1886889993322,8cyclictest0-21swapper/012:00:010
188689199328,7cyclictest171rcu_preempt10:01:062
1886891993226,2cyclictest0-21swapper/211:00:022
1886891993225,4cyclictest1891412-21kworker/u8:2-xprtiod08:05:012
1886891993225,3cyclictest0-21swapper/211:30:012
188689099327,6cyclictest1910285-21ntpq08:26:031
188689099323,24cyclictest2015466-21ssh11:36:061
1886890993225,3cyclictest0-21swapper/110:20:001
1886890993225,3cyclictest0-21swapper/110:20:001
188689099322,24cyclictest1980904-21ssh10:41:051
1886890993218,8cyclictest1919202-21ntpq08:55:591
1886890993218,8cyclictest1919202-21ntpq08:55:591
188688999324,19cyclictest428-21mta-sts-daemon08:46:060
1886889993226,4cyclictest1934123-21grep09:26:140
1886889993226,3cyclictest0-21swapper/007:46:080
1886889993224,6cyclictest1892900-21sh07:30:010
1886889993223,7cyclictest1996733-21sshd11:06:090
1886889993223,6cyclictest1900892-21fschecks_count07:55:550
1886889993212,15cyclictest151ktimers/008:56:060
1886889993212,15cyclictest151ktimers/008:56:060
1886891993126,3cyclictest0-21swapper/209:00:002
1886891993126,3cyclictest0-21swapper/209:00:002
1886891993125,3cyclictest0-21swapper/211:45:012
1886891993124,4cyclictest1903400-21kworker/u8:12+xprtiod09:46:142
1886891993124,4cyclictest0-21swapper/212:11:042
1886891993123,5cyclictest2001673-21systemctl11:15:012
188689199312,21cyclictest1930501-21ssh09:21:062
188689099313,21cyclictest1086143-21cups-browsed08:51:041
188689099313,20cyclictest2031275-21ssh12:01:071
1886890993125,3cyclictest0-21swapper/109:21:101
1886890993123,5cyclictest1891385-21munin-plugin-st07:25:001
188689099312,23cyclictest2006091-21ssh11:21:051
1886889993125,4cyclictest1951282-21kworker/u8:5-xprtiod10:05:560
1886889993124,5cyclictest0-21swapper/010:15:010
188688999312,25cyclictest2050166-21ssh12:31:070
1886889993119,8cyclictest12350irq/32-eth009:41:070
188689199303,22cyclictest1933577-21ssh09:26:032
1886891993024,3cyclictest0-21swapper/210:20:002
1886891993024,3cyclictest0-21swapper/210:20:002
1886891993024,3cyclictest0-21swapper/209:45:012
1886891993023,4cyclictest1051-21mta-sts-daemon11:59:262
1886891993022,5cyclictest428-21mta-sts-daemon08:06:072
1886891993022,5cyclictest1987415-21systemctl10:51:162
1886891993018,8cyclictest1086143-21cups-browsed09:39:102
1886890993025,2cyclictest0-21swapper/112:36:011
1886890993021,5cyclictest0-21swapper/108:20:591
1886890993012,4cyclictest28-21ksoftirqd/107:26:031
1886890993011,5cyclictest28-21ksoftirqd/109:06:011
1886890993011,5cyclictest28-21ksoftirqd/109:06:011
1886890993010,5cyclictest28-21ksoftirqd/110:46:061
1886889993026,2cyclictest0-21swapper/009:40:010
1886889993024,4cyclictest1919902-21kworker/u8:8-xprtiod09:15:590
1886889993024,3cyclictest261-21systemd-journal11:05:010
1886889993020,7cyclictest1987590-21timerandwakeup10:51:180
188689199295,15cyclictest1949380-21ssh09:51:052
1886891992924,2cyclictest0-21swapper/208:51:042
1886891992923,3cyclictest0-21swapper/208:00:012
1886891992922,4cyclictest2048916-21latency_hist12:30:022
1886891992921,5cyclictest1926016-21cron09:15:002
1886891992921,4cyclictest0-21swapper/211:06:032
1886891992920,5cyclictest2024929-21sh11:51:042
1886891992918,8cyclictest1894219-21unixbench-2d07:31:142
1886891992918,6cyclictest2031314-21ssh12:01:092
1886890992923,3cyclictest0-21swapper/109:05:001
1886890992923,3cyclictest0-21swapper/109:05:001
1886889992924,2cyclictest0-21swapper/009:31:040
1886889992923,4cyclictest1903902-21fschecks_count08:05:560
1886889992922,4cyclictest1051-21mta-sts-daemon10:34:070
1886889992922,4cyclictest1051-21mta-sts-daemon10:34:070
188688999292,23cyclictest1990269-21ssh10:56:040
188688999292,23cyclictest1977629-21ssh10:36:010
1886889992921,5cyclictest2006586-21tail11:21:150
1886889992920,6cyclictest0-21swapper/011:15:020
1886889992918,9cyclictest1908350-21cpuspeed08:20:540
1886891992823,2cyclictest0-21swapper/207:20:002
1886891992823,2cyclictest0-21swapper/207:20:002
1886891992821,3cyclictest0-21swapper/212:40:012
188689099285,7cyclictest1953612-21ssh09:57:411
1886890992821,4cyclictest1888368-21cron07:15:001
1886890992821,4cyclictest1888368-21cron07:15:001
188688999285,20cyclictest1949302-21ssh09:51:010
1886889992820,6cyclictest1968045-21latency10:20:570
1886889992820,6cyclictest1968045-21latency10:20:570
1886889992812,11cyclictest151ktimers/007:36:010
188689199277,15cyclictest351ktimers/208:16:022
1886891992721,3cyclictest0-21swapper/208:40:012
1886891992721,3cyclictest0-21swapper/207:50:022
1886891992720,4cyclictest1886910-21latency_hist07:10:012
1886891992720,4cyclictest1886910-21latency_hist07:10:012
1886891992717,7cyclictest1914724-21ntp_kernel_err08:41:022
1886891992717,6cyclictest439-21systemd-logind08:30:022
1886889992719,6cyclictest2026803-21cron11:55:010
1886889992719,5cyclictest439-21systemd-logind07:10:010
1886889992719,5cyclictest439-21systemd-logind07:10:010
1886889992719,5cyclictest420-21avahi-daemon12:03:200
1886891992619,4cyclictest1912359-21cron08:35:012
1886891992619,4cyclictest1909332-21unixbench_singl08:21:152
1886891992618,5cyclictest421-21dbus-daemon07:30:012
188689099269,12cyclictest1913327-21ntpq08:36:071
1886889992617,7cyclictest1907180-21needreboot08:15:590
1886889992617,3cyclictest0-21swapper/007:30:560
1886889992616,3cyclictest0-21swapper/008:11:140
1886891992419,2cyclictest0-21swapper/207:25:012
188688999245,13cyclictest1187135-21munin-node07:11:020
188688999245,13cyclictest1187135-21munin-node07:11:020
188688999234,13cyclictest428-21mta-sts-daemon07:51:000
1886889992318,3cyclictest1886888-21cyclictest08:30:010
188688999222,14cyclictest1917719-21ntpq08:51:020
188688999221,18cyclictest1889247-21ntp_kernel_pll_07:16:060
188688999221,18cyclictest1889247-21ntp_kernel_pll_07:16:060
1886889992211,8cyclictest0-21swapper/008:31:080
188689299182,8cyclictest0-21swapper/311:21:093
188689299174,7cyclictest0-21swapper/311:56:063
188689299172,8cyclictest0-21swapper/310:16:043
188689299172,8cyclictest0-21swapper/310:16:043
188689299172,7cyclictest0-21swapper/311:51:013
188689299172,7cyclictest0-21swapper/309:26:013
188689299171,8cyclictest0-21swapper/309:11:043
188689299171,7cyclictest0-21swapper/309:16:063
188689299163,6cyclictest0-21swapper/312:16:053
188689299162,9cyclictest0-21swapper/312:15:023
188689299162,9cyclictest0-21swapper/310:01:093
188689299162,6cyclictest0-21swapper/311:26:043
188689299161,9cyclictest0-21swapper/310:31:093
188689299161,9cyclictest0-21swapper/310:31:093
188689299153,7cyclictest0-21swapper/311:41:093
188689299153,7cyclictest0-21swapper/310:46:083
188689299153,5cyclictest0-21swapper/312:01:033
188689299152,8cyclictest0-21swapper/311:31:103
188689299152,8cyclictest0-21swapper/309:31:133
188689299152,7cyclictest0-21swapper/311:11:003
188689299152,7cyclictest0-21swapper/309:46:093
188689299151,8cyclictest0-21swapper/312:31:053
188689299151,8cyclictest0-21swapper/310:06:023
188689299151,7cyclictest0-21swapper/312:36:063
188689299142,7cyclictest0-21swapper/309:41:033
188689299142,6cyclictest0-21swapper/310:51:063
188689299141,8cyclictest0-21swapper/312:06:063
188689299141,7cyclictest0-21swapper/311:16:033
188689299141,7cyclictest0-21swapper/310:56:093
188689299141,7cyclictest0-21swapper/308:26:053
188689299141,6cyclictest0-21swapper/308:36:023
188689299141,6cyclictest0-21swapper/308:01:063
188689299141,6cyclictest0-21swapper/307:56:033
188689299141,6cyclictest0-21swapper/307:21:013
188689299141,5cyclictest0-21swapper/308:11:013
188689299141,5cyclictest0-21swapper/307:16:053
188689299141,5cyclictest0-21swapper/307:16:053
188689299133,7cyclictest0-21swapper/311:01:003
188689299132,7cyclictest0-21swapper/312:21:093
188689299132,7cyclictest0-21swapper/310:20:533
188689299132,7cyclictest0-21swapper/310:20:533
188689299132,6cyclictest0-21swapper/309:56:023
188689299132,6cyclictest0-21swapper/309:20:523
188689299131,7cyclictest0-21swapper/312:26:073
188689299131,7cyclictest0-21swapper/311:06:093
188689299131,7cyclictest0-21swapper/310:41:043
188689299131,7cyclictest0-21swapper/310:26:073
188689299131,7cyclictest0-21swapper/310:26:073
188689299131,7cyclictest0-21swapper/309:36:013
188689299131,6cyclictest0-21swapper/308:31:003
188689299131,6cyclictest0-21swapper/308:21:023
188689299131,6cyclictest0-21swapper/308:06:073
188689299131,6cyclictest0-21swapper/307:11:063
188689299131,6cyclictest0-21swapper/307:11:063
188689299125,5cyclictest0-21swapper/310:11:053
188689299123,5cyclictest0-21swapper/309:51:043
188689299122,6cyclictest0-21swapper/311:46:013
188689299122,6cyclictest0-21swapper/311:36:053
188689299122,6cyclictest0-21swapper/308:20:013
188689299122,6cyclictest0-21swapper/307:41:003
188689299121,7cyclictest0-21swapper/310:35:533
188689299121,7cyclictest0-21swapper/309:06:063
188689299121,7cyclictest0-21swapper/309:06:063
188689299121,6cyclictest0-21swapper/308:46:053
188689299121,6cyclictest0-21swapper/308:41:043
188689299121,6cyclictest0-21swapper/307:36:023
188689299121,5cyclictest0-21swapper/307:31:013
188689299112,6cyclictest0-21swapper/309:01:003
188689299112,6cyclictest0-21swapper/309:01:003
188689299112,5cyclictest0-21swapper/308:56:013
188689299112,5cyclictest0-21swapper/308:56:013
188689299111,6cyclictest0-21swapper/308:51:043
188689299111,6cyclictest0-21swapper/307:51:003
188689299102,5cyclictest0-21swapper/307:26:033
188689299101,5cyclictest0-21swapper/307:45:563
18868929981,4cyclictest0-21swapper/307:10:023
18868929981,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional