You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 01:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sun Sep 13, 2026 12:42:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1844396996048,7cyclictest0-21swapper/209:06:122
1844396996048,7cyclictest0-21swapper/209:06:122
1844394995140,8cyclictest1914010-21systemd-run10:05:010
1844396994941,4cyclictest0-21swapper/210:11:192
1844396994840,5cyclictest1995973-21ssh12:20:012
184439599483,36cyclictest1890800-21ssh09:26:101
1844395994736,4cyclictest0-21swapper/109:46:151
1844395994736,4cyclictest0-21swapper/109:46:151
1844395994334,6cyclictest1923061-21cron10:20:011
1844395994333,7cyclictest1877374-21munin-plugin-st09:00:001
1844395994333,7cyclictest1877374-21munin-plugin-st09:00:001
1844395994333,6cyclictest1977785-21systemd11:50:021
1844395994233,6cyclictest1883456-21munin-run09:15:001
1844394994233,6cyclictest1957921-21ssh11:16:180
1844394994227,12cyclictest1066030-21cups-browsed11:46:220
1844395994133,5cyclictest1862396-21systemd-run08:10:011
184439699405,26cyclictest1936523-21sh10:41:142
1844396994033,4cyclictest1941314-21sh10:50:002
1844396993931,5cyclictest1848900-21munin-plugin-st07:25:002
1844396993931,5cyclictest1848900-21munin-plugin-st07:25:002
1844396993927,9cyclictest1872209-21needreboot08:41:102
1844395993931,5cyclictest420-21avahi-daemon08:55:011
1844395993931,5cyclictest420-21avahi-daemon08:55:011
1844395993931,5cyclictest1003-21mta-sts-daemon10:03:471
1844395993928,7cyclictest0-21swapper/109:20:011
1844394993932,4cyclictest1856384-21/usr/sbin/munin07:50:020
1844396993833,2cyclictest0-21swapper/208:00:012
1844396993830,5cyclictest1932217-21systemd-run10:35:012
1844396993830,5cyclictest1932217-21systemd-run10:35:012
184439699381,7cyclictest1969833-21ssh11:36:112
1844395993830,5cyclictest1938295-21systemd-run10:45:011
1844395993830,5cyclictest1901821-21cron09:45:011
1844395993830,5cyclictest1901821-21cron09:45:011
1844395993829,6cyclictest420-21avahi-daemon09:30:461
1844395993829,6cyclictest1003-21mta-sts-daemon10:58:491
1844395993829,5cyclictest1920073-21systemd-run10:15:021
1844395993816,5cyclictest28-21ksoftirqd/112:36:121
1844394993825,5cyclictest171rcu_preempt11:56:240
184439699373,26cyclictest1066030-21cups-browsed09:11:092
1844396993730,4cyclictest1853398-21idleruntime-cro07:40:012
1844396993729,5cyclictest1910943-21sendmail10:00:002
1844396993729,5cyclictest1868402-21systemd-run08:30:012
184439699371,30cyclictest1948773-21ssh11:01:162
184439699371,29cyclictest1975967-21ssh11:46:112
1844395993729,5cyclictest1889590-21cron09:25:011
1844395993728,5cyclictest1871376-21munin-run08:40:001
1844395993728,5cyclictest1871376-21munin-run08:40:001
1844394993729,4cyclictest0-21swapper/010:55:010
1844394993727,7cyclictest2005098-21munin-plugin-st12:35:010
1844394993727,7cyclictest1987765-21idleruntime12:06:050
1844394993727,7cyclictest1987765-21idleruntime12:06:050
184439699365,23cyclictest2000362-21ssh12:26:162
1844396993630,4cyclictest1844393-21cyclictest12:38:392
1844396993629,4cyclictest0-21swapper/210:01:042
1844396993628,4cyclictest0-21swapper/212:15:002
1844395993629,4cyclictest1962651-21latency_hist11:25:021
1844395993629,4cyclictest1929184-21systemd10:30:011
1844395993629,4cyclictest1929184-21systemd10:30:011
1844395993628,5cyclictest421-21dbus-daemon08:35:011
1844395993628,5cyclictest1951295-21fschecks_count11:06:031
184439599362,29cyclictest1933409-21ssh10:36:111
184439599362,29cyclictest1933409-21ssh10:36:111
1844395993614,6cyclictest28-21ksoftirqd/111:20:011
1844395993614,5cyclictest28-21ksoftirqd/110:21:151
1844395993614,5cyclictest28-21ksoftirqd/110:21:151
1844394993625,8cyclictest1886543-21sendmail09:20:010
1844394993624,9cyclictest420-21avahi-daemon10:07:390
1844394993618,12cyclictest12650irq/32-eth012:01:160
1844394993618,12cyclictest12650irq/32-eth012:01:160
184439499361,32cyclictest0-21swapper/010:16:140
184439699353,27cyclictest1966857-21ssh11:31:132
1844396993527,3cyclictest0-21swapper/211:45:022
1844396993526,6cyclictest1871396-21idleruntime-cro08:40:002
1844396993526,6cyclictest1871396-21idleruntime-cro08:40:002
1844396993526,6cyclictest1860905-21systemd-run08:05:012
1844396993525,7cyclictest1894506-21unixbench_singl09:31:272
184439599353,22cyclictest1954775-21ssh11:11:171
1844395993529,3cyclictest0-21swapper/112:21:161
1844395993528,4cyclictest1002-21mta-sts-daemon07:51:151
1844395993527,5cyclictest1847372-21cron07:20:001
1844395993511,5cyclictest28-21ksoftirqd/112:26:241
1844394993529,4cyclictest1875872-21latency_hist08:55:000
1844394993529,4cyclictest1875872-21latency_hist08:55:000
1844394993528,4cyclictest1946198-21head10:56:220
1844394993528,4cyclictest191rcuc/008:10:010
1844394993518,10cyclictest171rcu_preempt09:36:050
184439499351,24cyclictest1848312-21sshd07:21:160
184439499351,24cyclictest1848312-21sshd07:21:160
184439699343,24cyclictest1887775-21ssh09:21:112
184439699343,21cyclictest1942716-21ssh10:51:212
1844396993428,3cyclictest0-21swapper/207:20:012
1844396993428,2cyclictest0-21swapper/209:05:002
1844396993426,5cyclictest1933047-21fschecks_time10:36:042
1844396993426,5cyclictest1933047-21fschecks_time10:36:042
1844396993426,4cyclictest1003-21mta-sts-daemon11:06:172
1844396993425,5cyclictest0-21swapper/212:21:232
1844395993428,3cyclictest0-21swapper/108:20:011
1844395993427,5cyclictest0-21swapper/112:20:011
1844395993427,4cyclictest1968647-21systemd-run11:35:021
1844395993424,7cyclictest1930182-21latency10:31:061
1844395993424,7cyclictest1930182-21latency10:31:061
184439599342,27cyclictest1982001-21ssh11:56:121
1844395993421,9cyclictest1-21systemd11:26:021
1844394993427,4cyclictest1860961-21latency_hist08:05:020
1844394993426,5cyclictest151ktimers/011:51:530
1844394993411,10cyclictest191rcuc/011:24:090
184439699333,23cyclictest1979091-21ssh11:51:172
1844396993325,5cyclictest1869879-21cron08:35:002
1844396993325,5cyclictest1866895-21munin-plugin-st08:25:012
1844396993324,6cyclictest1066030-21cups-browsed12:04:552
1844396993324,6cyclictest1066030-21cups-browsed12:04:552
1844396993322,8cyclictest1863256-21ntp_kernel_pll_08:11:122
1844395993327,3cyclictest0-21swapper/109:56:221
1844395993326,4cyclictest0-21swapper/109:36:241
184439499336,18cyclictest1904884-21munin-run09:50:010
184439499336,18cyclictest1904884-21munin-run09:50:010
1844394993326,5cyclictest1869952-21latency_hist08:35:020
1844394993326,4cyclictest1932346-21latency_hist10:35:020
1844394993326,4cyclictest1932346-21latency_hist10:35:020
1844394993325,6cyclictest1880393-21diskmemload12:40:010
1844394993325,4cyclictest0-21swapper/012:21:070
1844394993322,8cyclictest1952278-21unixbench_multi11:06:220
184439499332,23cyclictest1887737-21ssh09:21:090
1844394993311,12cyclictest1991531-21switchtime12:11:240
1844396993225,3cyclictest0-21swapper/210:21:072
1844396993225,3cyclictest0-21swapper/210:21:072
1844396993223,6cyclictest1066030-21cups-browsed11:13:322
1844395993225,4cyclictest1939795-21sendmail_mailtr10:46:201
1844395993225,3cyclictest0-21swapper/112:06:141
1844395993225,3cyclictest0-21swapper/112:06:141
1844395993224,5cyclictest1849419-21fschecks_count07:26:041
1844395993213,4cyclictest28-21ksoftirqd/112:01:121
1844395993213,4cyclictest28-21ksoftirqd/112:01:121
1844395993212,14cyclictest1066030-21cups-browsed07:56:081
1844395993210,4cyclictest28-21ksoftirqd/108:11:061
1844395993210,10cyclictest28-21ksoftirqd/108:30:011
184439499329,3cyclictest0-21swapper/010:48:190
184439499326,9cyclictest14-21ksoftirqd/008:46:140
1844394993227,3cyclictest1968622-21cron11:35:010
1844394993224,5cyclictest1872794-21sed08:41:230
1844394993224,4cyclictest0-21swapper/007:35:270
1844394993223,3cyclictest171rcu_preempt07:16:210
1844394993222,7cyclictest265-21systemd-journal10:11:170
1844394993214,3cyclictest0-21swapper/009:52:080
1844394993214,3cyclictest0-21swapper/009:52:080
1844394993210,4cyclictest0-21swapper/009:33:200
184439699314,19cyclictest1963910-21ssh11:26:192
1844396993125,4cyclictest1844393-21cyclictest10:55:192
1844396993124,3cyclictest0-21swapper/210:06:212
1844396993123,5cyclictest1988695-21unixbench_multi12:06:232
1844396993123,5cyclictest1988695-21unixbench_multi12:06:232
1844396993123,4cyclictest0-21swapper/212:31:202
1844396993122,6cyclictest1066030-21cups-browsed09:28:132
184439699311,17cyclictest1961032-21ps11:21:192
1844395993123,5cyclictest1001-21mta-sts-daemon10:05:131
1844395993122,6cyclictest0-21swapper/110:51:141
184439599312,24cyclictest1969894-21ssh11:36:141
1844395993110,5cyclictest28-21ksoftirqd/112:32:031
184439499317,11cyclictest151ktimers/011:26:160
1844394993126,3cyclictest1821572-21kworker/u8:13-xprtiod11:15:010
1844394993117,3cyclictest171rcu_preempt09:06:140
1844394993117,3cyclictest171rcu_preempt09:06:140
1844394993117,3cyclictest171rcu_preempt09:03:590
1844394993116,12cyclictest1909775-21head09:56:240
184439499311,4cyclictest1926126-21munin-run10:25:010
184439499311,4cyclictest1926126-21munin-run10:25:010
1844396993023,4cyclictest1981589-21ssh11:56:032
1844396993022,5cyclictest1066030-21cups-browsed07:41:052
1844396993015,3cyclictest171rcu_preempt07:10:012
184439599309,4cyclictest28-21ksoftirqd/111:01:231
184439599309,4cyclictest28-21ksoftirqd/107:31:011
1844395993024,3cyclictest0-21swapper/111:51:081
1844395993023,4cyclictest1874268-21tail08:46:251
1844395993023,3cyclictest0-21swapper/107:45:011
1844395993022,5cyclictest1844405-21systemd-run07:10:011
1844395993010,5cyclictest28-21ksoftirqd/112:11:091
1844395993010,4cyclictest28-21ksoftirqd/111:41:121
1844395993010,4cyclictest28-21ksoftirqd/107:11:161
1844395993010,3cyclictest28-21ksoftirqd/109:01:091
184439499307,7cyclictest14-21ksoftirqd/007:26:150
184439499305,8cyclictest14-21ksoftirqd/012:28:120
184439499304,9cyclictest14-21ksoftirqd/011:42:370
184439499304,9cyclictest14-21ksoftirqd/010:40:320
1844394993024,3cyclictest1840305-21kworker/u8:16+rpciod07:15:010
1844394993022,5cyclictest1926685-21aten_r2power_cu10:26:010
1844394993022,5cyclictest1926685-21aten_r2power_cu10:26:010
1844394993022,4cyclictest1002-21mta-sts-daemon11:00:500
1844394993021,6cyclictest1066030-21cups-browsed09:10:250
1844394993018,3cyclictest171rcu_preempt10:36:110
1844394993018,3cyclictest171rcu_preempt10:36:110
1844394993017,11cyclictest151ktimers/008:16:150
1844394993016,3cyclictest171rcu_preempt08:26:110
1844396992922,4cyclictest1876825-21ntp_offset08:56:142
1844396992922,4cyclictest1876825-21ntp_offset08:56:142
1844396992921,5cyclictest421-21dbus-daemon09:16:282
1844396992920,6cyclictest1845744-21/usr/sbin/munin07:11:232
1844396992919,7cyclictest1921915-21head10:16:232
1844396992919,4cyclictest0-21swapper/211:19:122
184439699291,22cyclictest1897001-21ssh09:36:142
184439599298,5cyclictest28-21ksoftirqd/107:46:101
1844395992922,4cyclictest1001-21mta-sts-daemon08:41:141
184439599292,19cyclictest1906107-21ssh09:51:151
184439599292,19cyclictest1906107-21ssh09:51:151
1844395992910,4cyclictest28-21ksoftirqd/108:21:161
184439499296,7cyclictest14-21ksoftirqd/008:00:010
184439499295,9cyclictest14-21ksoftirqd/009:41:060
184439499295,9cyclictest14-21ksoftirqd/009:41:060
184439499295,9cyclictest14-21ksoftirqd/008:36:140
184439499295,9cyclictest14-21ksoftirqd/008:36:140
184439499295,14cyclictest14-21ksoftirqd/012:16:160
184439499293,10cyclictest14-21ksoftirqd/008:56:170
184439499293,10cyclictest14-21ksoftirqd/008:56:170
1844394992923,4cyclictest191rcuc/011:40:010
1844394992922,4cyclictest485-21in:imuxsock08:25:010
184439499291,9cyclictest151ktimers/009:26:170
1844394992917,3cyclictest171rcu_preempt08:11:050
1844394992912,4cyclictest151ktimers/007:10:020
1844394992910,10cyclictest191rcuc/007:51:220
1844396992821,4cyclictest1874396-21kworker/u8:3+xprtiod08:50:012
1844396992821,3cyclictest0-21swapper/209:51:052
1844396992821,3cyclictest0-21swapper/209:51:052
1844396992821,3cyclictest0-21swapper/209:41:042
1844396992821,3cyclictest0-21swapper/209:41:042
184439599289,4cyclictest28-21ksoftirqd/109:06:081
184439599289,4cyclictest28-21ksoftirqd/109:06:081
184439599289,4cyclictest28-21ksoftirqd/107:21:131
184439599289,4cyclictest28-21ksoftirqd/107:21:131
1844395992810,5cyclictest271ktimers/107:36:031
1844395992810,4cyclictest28-21ksoftirqd/108:01:191
1844394992811,7cyclictest171rcu_preempt07:41:140
1844396992720,4cyclictest1902722-21idleruntime09:46:072
1844396992720,4cyclictest1902722-21idleruntime09:46:072
1844396992716,8cyclictest1862567-21latency_hist08:10:022
1844394992714,6cyclictest171rcu_preempt07:31:040
184439699267,11cyclictest1002-21mta-sts-daemon08:51:152
184439699267,11cyclictest1002-21mta-sts-daemon08:51:152
184439699252,18cyclictest265-21systemd-journal08:16:172
1844396992419,2cyclictest0-21swapper/210:26:022
1844396992419,2cyclictest0-21swapper/210:26:022
1844396992418,3cyclictest0-21swapper/207:30:002
184439699241,4cyclictest171rcu_preempt07:51:132
1844396992317,3cyclictest0-21swapper/207:46:202
1844396992314,6cyclictest567-21gdbus07:31:162
184439799192,8cyclictest0-21swapper/310:36:163
184439799192,8cyclictest0-21swapper/310:36:163
184439799182,8cyclictest0-21swapper/312:36:123
184439799181,8cyclictest0-21swapper/311:56:183
184439799172,8cyclictest0-21swapper/312:01:123
184439799172,8cyclictest0-21swapper/312:01:123
184439799172,7cyclictest0-21swapper/312:26:133
184439799172,7cyclictest0-21swapper/312:06:153
184439799172,7cyclictest0-21swapper/312:06:153
184439799172,6cyclictest0-21swapper/310:16:123
184439799171,7cyclictest0-21swapper/309:56:173
184439799163,7cyclictest0-21swapper/310:11:183
184439799163,6cyclictest0-21swapper/309:21:143
184439799162,8cyclictest0-21swapper/309:01:163
184439799162,7cyclictest0-21swapper/311:26:193
184439799162,7cyclictest0-21swapper/311:21:233
184439799162,6cyclictest0-21swapper/312:21:073
184439799161,8cyclictest0-21swapper/308:01:163
184439799161,7cyclictest0-21swapper/310:31:033
184439799161,7cyclictest0-21swapper/310:31:033
184439799161,4cyclictest0-21swapper/309:26:103
184439799152,7cyclictest0-21swapper/311:46:163
184439799152,7cyclictest0-21swapper/311:31:203
184439799152,7cyclictest0-21swapper/309:51:143
184439799152,7cyclictest0-21swapper/309:51:143
184439799152,6cyclictest0-21swapper/307:21:143
184439799152,6cyclictest0-21swapper/307:21:143
184439799151,8cyclictest0-21swapper/311:51:163
184439799151,7cyclictest0-21swapper/311:06:103
184439799151,7cyclictest0-21swapper/310:26:113
184439799151,7cyclictest0-21swapper/310:26:113
184439799151,7cyclictest0-21swapper/309:46:163
184439799151,7cyclictest0-21swapper/309:46:163
184439799151,7cyclictest0-21swapper/307:11:143
184439799151,6cyclictest0-21swapper/308:56:163
184439799151,6cyclictest0-21swapper/308:56:163
184439799143,7cyclictest0-21swapper/309:31:163
184439799142,7cyclictest0-21swapper/312:31:133
184439799142,7cyclictest0-21swapper/312:16:243
184439799142,6cyclictest0-21swapper/311:01:053
184439799142,6cyclictest0-21swapper/308:36:133
184439799142,6cyclictest0-21swapper/308:36:133
184439799142,5cyclictest0-21swapper/308:31:093
184439799141,8cyclictest0-21swapper/311:41:143
184439799141,8cyclictest0-21swapper/310:56:163
184439799141,7cyclictest0-21swapper/312:11:123
184439799141,6cyclictest0-21swapper/308:51:113
184439799141,6cyclictest0-21swapper/308:51:113
184439799141,6cyclictest0-21swapper/308:21:133
184439799141,6cyclictest0-21swapper/308:06:183
184439799141,6cyclictest0-21swapper/307:46:143
184439799141,6cyclictest0-21swapper/307:26:093
184439799141,6cyclictest0-21swapper/307:16:113
184439799141,5cyclictest0-21swapper/307:41:093
184439799134,6cyclictest0-21swapper/310:51:123
184439799133,6cyclictest0-21swapper/310:21:063
184439799133,6cyclictest0-21swapper/310:21:063
184439799132,7cyclictest0-21swapper/311:16:053
184439799132,6cyclictest0-21swapper/311:11:053
184439799132,6cyclictest0-21swapper/310:06:013
184439799132,6cyclictest0-21swapper/309:11:093
184439799131,8cyclictest0-21swapper/310:41:033
184439799131,7cyclictest0-21swapper/309:16:133
184439799131,7cyclictest0-21swapper/308:46:123
184439799131,5cyclictest0-21swapper/307:56:073
184439799122,6cyclictest0-21swapper/311:36:113
184439799122,6cyclictest0-21swapper/310:46:013
184439799122,5cyclictest0-21swapper/309:06:113
184439799122,5cyclictest0-21swapper/309:06:113
184439799122,4cyclictest0-21swapper/308:26:103
184439799121,6cyclictest0-21swapper/310:00:593
184439799121,6cyclictest0-21swapper/309:41:003
184439799121,6cyclictest0-21swapper/309:41:003
184439799121,6cyclictest0-21swapper/308:41:143
184439799121,6cyclictest0-21swapper/307:31:083
184439799114,5cyclictest0-21swapper/308:11:123
184439799111,6cyclictest0-21swapper/308:16:153
184439799111,5cyclictest0-21swapper/309:36:013
184439799111,5cyclictest0-21swapper/307:51:113
184439799102,7cyclictest0-21swapper/307:10:013
184439799102,5cyclictest0-21swapper/307:40:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional