You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-15 - 13:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Wed Apr 15, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
219861322040,4sleep375-21kworker/3:1+mm_percpu_wq00:11:263
192485422040,3sleep375-21kworker/3:1-events19:36:233
1895084995848,4cyclictest1966871-21kworker/u8:3+flush-8:021:01:311
1895084995549,5cyclictest1966871-21kworker/u8:3+flush-8:000:26:321
21557002540,4sleep3481ktimers/323:31:183
1895084995442,4cyclictest1932038-21kworker/u8:2+flush-8:020:01:311
1895084995347,5cyclictest1838326-21kworker/u8:0+events_unbound21:15:011
1895084995346,5cyclictest1966871-21kworker/u8:3+flush-8:000:36:321
1895084995243,6cyclictest2124997-21latency_hist23:00:011
1895084995142,7cyclictest1966871-21kworker/u8:3+flush-8:000:16:321
1895080995142,7cyclictest1895302-21/usr/sbin/munin19:11:310
21611082490,3chrt0-21swapper/023:36:200
19917122490,5sleep20-21swapper/220:46:162
1895084994944,4cyclictest2015766-21kworker/u8:1+events_unbound22:00:001
1895084994944,4cyclictest2015766-21kworker/u8:1+events_unbound22:00:001
1895084994943,5cyclictest2036075-21kworker/u8:2+events_unbound22:06:271
1895084994939,7cyclictest2199498-21latency_hist00:15:001
1895084994843,3cyclictest1838326-21kworker/u8:0+events_unbound20:00:001
1895084994842,5cyclictest2159643-21kworker/u8:2+events_unbound23:46:131
1895086994740,5cyclictest1966515-21gpgv20:20:012
1895086994740,5cyclictest1966515-21gpgv20:20:012
1895084994742,4cyclictest1838326-21kworker/u8:0+events_unbound20:36:161
1895084994741,4cyclictest1916989-21kworker/u8:3+events_unbound19:51:311
1895084994740,5cyclictest1946895-21kworker/u8:1+flush-8:020:26:261
1895080994738,7cyclictest1922215-21/usr/sbin/munin19:36:310
1895084994641,4cyclictest1838326-21kworker/u8:0+flush-8:019:46:211
1895084994640,4cyclictest2015766-21kworker/u8:1+flush-8:022:31:311
1895084994639,5cyclictest1838326-21kworker/u8:0+flush-8:021:26:271
1895084994539,5cyclictest1838326-21kworker/u8:0+events_unbound21:06:301
1895084994440,3cyclictest1966871-21kworker/u8:3+events_unbound23:36:291
1895084994440,3cyclictest1838326-21kworker/u8:0+events_unbound19:41:211
1895084994440,3cyclictest1838326-21kworker/u8:0+events_unbound19:15:051
1895084994439,3cyclictest2125244-21kworker/u8:0+events_unbound23:31:191
1895084994435,6cyclictest2154972-21latency_hist23:30:011
1895084994338,4cyclictest2036075-21kworker/u8:2+events_unbound22:46:171
1895084994338,4cyclictest2015766-21kworker/u8:1+flush-8:022:41:281
1895084994338,4cyclictest2015766-21kworker/u8:1+events_unbound22:25:011
1895084994337,4cyclictest2036075-21kworker/u8:2+events_unbound22:01:201
1895084994337,4cyclictest1900094-21kworker/u8:1+flush-8:019:36:321
1895080994338,4cyclictest1838326-21kworker/u8:0+flush-8:020:21:180
1895080994337,4cyclictest2036075-21kworker/u8:2+flush-8:021:41:170
1895084994237,4cyclictest1966871-21kworker/u8:3+events_unbound20:46:171
1895084994137,3cyclictest1966871-21kworker/u8:3+flush-8:023:21:241
1895084994137,3cyclictest1966871-21kworker/u8:3+flush-8:021:36:351
1895084994136,4cyclictest2015766-21kworker/u8:1+events_unbound22:36:211
1895084994136,4cyclictest2015766-21kworker/u8:1+events_unbound21:31:261
1895084994136,4cyclictest1838326-21kworker/u8:0+events_unbound19:21:171
1895084994135,5cyclictest2036075-21kworker/u8:2+events_unbound22:16:141
1895084994135,4cyclictest1966871-21kworker/u8:3+events_unbound23:11:281
1895084994132,3cyclictest2169548-21kworker/u8:0+flush-8:023:56:251
1895080994134,5cyclictest2105623-21apt-get22:41:120
1895080994131,7cyclictest1907078-21latency_hist19:20:010
1895080994130,9cyclictest2125607-21diskstats23:01:170
1895084994033,5cyclictest1838326-21kworker/u8:0+flush-8:020:51:281
1895084994030,7cyclictest280-21systemd-journal19:30:101
1895080994031,7cyclictest2110535-21/usr/sbin/munin22:46:250
1895080994031,6cyclictest2085515-21latency_hist22:20:010
1895084993934,4cyclictest2218454-21kworker/u8:1+flush-8:000:31:331
1895084993933,4cyclictest2130338-21apt-get23:06:131
1895084993932,5cyclictest1838326-21kworker/u8:0+flush-8:020:31:321
1895084993931,6cyclictest1838326-21kworker/u8:0+flush-8:020:21:191
1895084993931,6cyclictest1838326-21kworker/u8:0+events_unbound21:21:281
1895080993933,5cyclictest2015766-21kworker/u8:1+flush-8:022:21:180
1895080993932,5cyclictest280-21systemd-journal00:03:120
1895086993831,5cyclictest2149917-21store23:25:002
1895084993833,4cyclictest1900094-21kworker/u8:1+events_unbound19:26:211
1895084993833,4cyclictest1838326-21kworker/u8:0+flush-8:021:16:281
1895084993832,5cyclictest2125244-21kworker/u8:0+flush-8:023:01:261
1895084993832,5cyclictest2036075-21kworker/u8:2+flush-8:022:26:261
1895084993832,5cyclictest2036075-21kworker/u8:2+flush-8:021:41:271
1895084993831,5cyclictest1966871-21kworker/u8:3+flush-8:000:21:331
1895080993827,9cyclictest280-21systemd-journal21:10:000
1895084993732,4cyclictest1932038-21kworker/u8:2+flush-8:020:06:161
1895084993732,4cyclictest1932038-21kworker/u8:2+flush-8:020:06:161
1895084993729,6cyclictest2115493-21apt-get22:51:141
1895084993728,7cyclictest2075951-21/usr/sbin/munin22:11:241
189466723726,8sleep20-21swapper/219:06:572
1895084993631,4cyclictest2036075-21kworker/u8:2+flush-8:021:51:331
1895084993631,3cyclictest2169548-21kworker/u8:0+flush-8:000:06:181
1895084993631,3cyclictest2169548-21kworker/u8:0+flush-8:000:06:181
1895084993629,5cyclictest1838326-21kworker/u8:0+flush-8:020:56:271
1895080993630,3cyclictest2015766-21kworker/u8:1+flush-8:021:16:230
1895084993530,4cyclictest2184917-21kworker/u8:1+flush-8:000:01:221
1895084993528,5cyclictest2179311-21grep23:51:331
1895084993528,5cyclictest1956862-21apt-get20:11:121
1895086993423,9cyclictest2120087-21turbostat.cron22:55:002
1895084993428,4cyclictest2015766-21kworker/u8:1+events_unbound21:46:161
1895080993429,3cyclictest280-21systemd-journal21:15:000
1895080993429,3cyclictest2036075-21kworker/u8:2+flush-8:022:26:300
1895080993428,4cyclictest1966871-21kworker/u8:3+flush-8:020:45:410
1895086993318,13cyclictest2195168-21fschecks_count00:11:192
189508699331,12cyclictest1946996-21apt-get20:01:132
189508699331,12cyclictest1927219-21apt-get19:41:152
1895084993328,3cyclictest1838326-21kworker/u8:0+flush-8:019:11:331
1895080993327,4cyclictest2036075-21kworker/u8:2+flush-8:023:06:210
1895080993323,7cyclictest2041400-21diskstats21:36:170
1895080993323,7cyclictest1965527-21ntpq20:16:250
1895080993323,7cyclictest1965527-21ntpq20:16:250
1895080993226,4cyclictest1838326-21kworker/u8:0+flush-8:021:01:160
20712372310,2sleep0201rcuc/022:06:150
189508699313,22cyclictest2209726-21apt-get00:26:132
189508699313,22cyclictest2065880-21apt-get22:01:152
189508699313,21cyclictest1976879-21apt-get20:31:152
1895080993127,3cyclictest2015766-21kworker/u8:1+flush-8:021:46:290
1895080993126,3cyclictest1932038-21kworker/u8:2+flush-8:020:06:230
1895080993126,3cyclictest1932038-21kworker/u8:2+flush-8:020:06:220
1895080993124,5cyclictest2125244-21kworker/u8:0+flush-8:023:16:260
1895080993122,7cyclictest1982058-21diskstats20:36:150
21137432300,5sleep3451irq_work/322:46:233
20317242300,3sleep0141ktimers/021:26:190
189508699301,9cyclictest2041248-21/usr/sbin/munin21:36:172
1895086993018,10cyclictest2046345-21cpuspeed_turbos21:41:162
189508699301,11cyclictest2085871-21apt-get22:21:142
1895084993025,3cyclictest2140927-21irqrtprio23:16:201
1895084993024,4cyclictest1946895-21kworker/u8:1+flush-8:020:16:301
1895084993024,4cyclictest1946895-21kworker/u8:1+flush-8:020:16:291
1895084993023,5cyclictest1946895-21kworker/u8:1+events_unbound20:41:161
1895080993025,4cyclictest1838326-21kworker/u8:0+flush-8:021:22:430
1895080993025,3cyclictest2149797-21cron23:25:000
1895080993024,4cyclictest2149794-21cron23:25:010
189508699292,7cyclictest2020152-21ntpq21:11:272
1895080992924,4cyclictest1932038-21kworker/u8:2+flush-8:020:11:310
1895080992924,3cyclictest1966871-21kworker/u8:3+events_unbound20:51:220
1895080992921,6cyclictest280-21systemd-journal23:31:300
1895080992920,7cyclictest280-21systemd-journal20:45:010
189508699289,6cyclictest1959565-21tr20:11:222
189508699288,5cyclictest1999000-21tr20:51:242
189508699287,6cyclictest2205233-21if_enp4s0.200:21:192
189508699283,5cyclictest2114338-21ntpq22:46:272
189508699282,8cyclictest2015162-21ntpq21:06:262
1895086992822,4cyclictest2060231-21unixbench_multi21:51:332
189508699282,10cyclictest2136170-21ps23:11:212
1895086992819,7cyclictest2194561-21turbostat.cron00:10:002
1895086992819,7cyclictest2194561-21turbostat.cron00:10:002
1895086992817,9cyclictest1966386-21apt-get20:20:032
1895086992815,11cyclictest0-21swapper/219:15:002
1895086992810,15cyclictest2005705-21timerwakeupswit20:56:332
1895080992822,4cyclictest1838326-21kworker/u8:0+flush-8:019:32:560
1895080992810,7cyclictest280-21systemd-journal20:56:110
189508699279,15cyclictest0-21swapper/220:30:012
189508699277,6cyclictest2184051-21grep23:56:292
189508699277,2cyclictest0-21swapper/221:04:222
189508699276,7cyclictest2084962-21grep22:16:292
189508699274,6cyclictest1921327-21grep19:31:322
189508699272,8cyclictest1902454-21apt-get19:16:142
189508699272,7cyclictest2039931-21ntpq21:31:252
189508699272,6cyclictest1912370-21apt-get19:26:142
189508699272,4cyclictest2175099-21apt-get23:51:142
189508699271,9cyclictest2164353-21/usr/sbin/munin23:36:322
189508699271,8cyclictest2223453-21ps00:36:302
189508699271,8cyclictest2099482-21ntpq22:31:262
189508699271,8cyclictest2030234-21/usr/sbin/munin21:21:322
1895086992715,10cyclictest2120483-21aten2_r5power_e22:56:152
1895086992712,13cyclictest0-21swapper/222:36:312
189508699271,11cyclictest2131061-21ps23:06:202
189508699271,11cyclictest1942041-21apt-get19:56:122
1895086992711,14cyclictest0-21swapper/221:46:142
1895080992722,4cyclictest1838326-21kworker/u8:0+flush-8:020:31:330
1895080992722,3cyclictest2184917-21kworker/u8:1+flush-8:000:11:240
1895080992722,3cyclictest1966871-21kworker/u8:3+flush-8:000:28:500
1895080992721,4cyclictest1946895-21kworker/u8:1+events_unbound20:30:010
189508099271,24cyclictest2219252-21munin-run00:35:010
189508699267,7cyclictest1923142-21sed19:36:212
189508699267,3cyclictest0-21swapper/221:16:202
189508699265,7cyclictest1932943-21sed19:46:212
189508699261,9cyclictest2035852-21munin-run21:30:002
189508699261,8cyclictest2174337-21/usr/sbin/munin23:46:332
189508699260,9cyclictest2185206-21/usr/sbin/munin00:01:162
189508699260,8cyclictest2200555-21/usr/sbin/munin00:16:212
1895080992622,3cyclictest1-21systemd21:35:000
1895080992622,2cyclictest2124985-21cron23:00:000
1895080992621,3cyclictest2070472-21latency_hist22:05:000
1895080992620,5cyclictest2015766-21kworker/u8:1+events_unbound22:11:130
18946652262,6sleep00-21swapper/019:06:550
21226892250,5sleep3451irq_work/322:56:223
189508699257,3cyclictest0-21swapper/222:06:172
189508699255,5cyclictest1909804-21tr19:21:232
189508699252,5cyclictest2106348-21ps22:41:202
189508699251,3cyclictest2159102-21ntpq23:31:272
189508699250,8cyclictest2214656-21/usr/sbin/munin00:31:132
189508699250,8cyclictest2155125-21grep23:30:012
1895084992519,3cyclictest0-21swapper/123:41:161
189508099252,22cyclictest0-21swapper/021:51:320
1895080992521,3cyclictest1966871-21kworker/u8:3+events_unbound00:36:310
1895080992520,3cyclictest2101238-21sort22:36:210
1895080992520,3cyclictest0-21swapper/019:41:330
1895080992516,7cyclictest280-21systemd-journal00:20:010
1895080992516,7cyclictest1937084-21/usr/sbin/munin19:51:250
189508699248,2cyclictest0-21swapper/223:16:192
189508699245,2cyclictest0-21swapper/221:56:202
189508699245,2cyclictest0-21swapper/221:56:202
189508699242,7cyclictest1954874-21cut20:06:232
189508699242,7cyclictest1954874-21cut20:06:232
1895086992420,2cyclictest2090894-21aten2_r5power_v22:26:162
189508699241,7cyclictest2125332-21/usr/sbin/munin23:01:122
189508699241,7cyclictest1990526-21ntpq20:41:282
189508699241,4cyclictest2165603-21smartctl23:41:172
189508699240,7cyclictest1940829-21sed19:51:252
1895080992418,4cyclictest0-21swapper/023:11:210
1895080992414,7cyclictest1912029-21latency_hist19:25:010
189508699231,8cyclictest2079699-21ntpq22:11:252
1895080992320,2cyclictest1916056-21needreboot19:26:250
1895080992319,2cyclictest0-21swapper/022:51:230
1895080992318,4cyclictest2209031-21unixbench_multi00:21:320
1895080992318,4cyclictest1838326-21kworker/u8:0+events_unbound19:46:520
1895080992318,3cyclictest1946895-21kworker/u8:1+flush-8:020:01:210
18939682233,13sleep11894051-21ntpq19:06:241
189508699220,7cyclictest1981996-21sort20:36:132
1895080992218,3cyclictest2169548-21kworker/u8:0+flush-8:023:51:180
1895080992218,3cyclictest1945784-21ntp_kernel_pll_19:56:250
1895080992216,4cyclictest876-21mta-sts-daemon23:41:190
189508099215,15cyclictest2149840-21kworker/0:0+events23:46:210
1895080992117,3cyclictest2097864-21kthreadcore22:31:220
1895080992116,4cyclictest2169548-21kworker/u8:0+events_unbound23:56:240
1895080992116,3cyclictest2061675-21irqrtprio21:56:200
1895080992116,3cyclictest2061675-21irqrtprio21:56:200
1895080992016,2cyclictest0-21swapper/000:06:140
1895080992016,2cyclictest0-21swapper/000:06:130
18900602193,7sleep30-21swapper/319:05:013
22143042176,5sleep30-21swapper/300:30:013
21452122152,8sleep30-21swapper/323:21:153
20114752156,5sleep30-21swapper/321:06:163
21846102143,7sleep30-21swapper/300:00:003
20806412132,6sleep30-21swapper/322:15:023
20605932132,7sleep30-21swapper/321:55:003
20350072132,7sleep30-21swapper/321:26:273
20230752132,7sleep30-21swapper/321:16:243
19913382132,6sleep30-21swapper/320:45:013
19816382132,6sleep30-21swapper/320:35:023
19509972132,7sleep30-21swapper/320:01:323
22184722122,6sleep30-21swapper/300:31:303
22044952122,6sleep30-21swapper/300:20:023
21899192122,6sleep30-21swapper/300:06:143
21899192122,6sleep30-21swapper/300:06:143
21850042122,6sleep30-21swapper/300:01:163
21649132122,5sleep30-21swapper/323:40:013
21603672122,6sleep30-21swapper/323:36:163
21398912122,5sleep30-21swapper/323:15:013
21151672122,5sleep30-21swapper/322:50:013
20904262122,5sleep30-21swapper/322:25:013
20714852123,5sleep30-21swapper/322:06:193
20701632122,6sleep30-21swapper/322:01:343
20614442122,6sleep30-21swapper/321:56:203
20614442122,6sleep30-21swapper/321:56:203
20408322122,5sleep30-21swapper/321:35:013
20286492122,6sleep30-21swapper/321:21:253
20160642122,5sleep30-21swapper/321:10:013
19867362122,6sleep30-21swapper/320:41:163
19764572122,6sleep30-21swapper/320:29:123
19676962122,6sleep30-21swapper/320:21:223
19653842122,6sleep30-21swapper/320:16:243
19653842122,6sleep30-21swapper/320:16:243
19515902122,6sleep30-21swapper/320:05:013
19515902122,6sleep30-21swapper/320:05:013
19424312122,6sleep30-21swapper/319:56:183
19367522122,7sleep30-21swapper/319:50:003
19169862122,5sleep30-21swapper/319:30:023
19128402122,6sleep30-21swapper/319:26:193
18953312122,6sleep30-21swapper/319:11:163
22199512112,5sleep30-21swapper/300:36:193
22038442112,5sleep30-21swapper/300:16:333
21756152112,5sleep30-21swapper/323:51:213
21723122112,5sleep30-21swapper/323:46:233
21509072112,5sleep30-21swapper/323:26:213
21353472112,5sleep30-21swapper/323:11:163
21307102112,5sleep30-21swapper/323:06:193
21256002112,5sleep30-21swapper/323:01:173
21054982112,5sleep30-21swapper/322:40:023
21046742112,5sleep30-21swapper/322:36:333
20997082112,5sleep30-21swapper/322:31:323
20858412112,5sleep30-21swapper/322:21:123
20762282112,5sleep30-21swapper/322:11:173
20521712112,5sleep30-21swapper/321:46:213
20461322112,5sleep30-21swapper/321:41:153
20363032112,5sleep30-21swapper/321:31:163
20063902112,5sleep30-21swapper/321:00:023
20016962112,5sleep30-21swapper/320:56:173
20003192112,5sleep30-21swapper/320:51:283
19808862112,5sleep30-21swapper/320:31:333
19567702112,6sleep30-21swapper/320:10:223
19370082112,6sleep30-21swapper/319:50:163
19310272112,5sleep30-21swapper/319:41:283
19073162112,5sleep30-21swapper/319:20:023
19008632112,5sleep30-21swapper/319:15:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional