You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 07:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Wed Dec 10, 2025 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
204068523040,2sleep3451irq_work/321:26:133
216808322040,3sleep3586-21kworker/3:2-events23:36:073
19297122760,2sleep3471rcuc/319:35:013
21354692590,6sleep20-21swapper/223:01:112
21737582580,4chrt521rcuop/323:41:090
1902696995646,8cyclictest2186223-21ntpq23:51:131
1902693995450,3cyclictest1860547-21kworker/u8:2+events_unbound21:06:070
1902693995346,3cyclictest2013248-21kworker/u8:0+events_unbound21:21:080
1902693995248,3cyclictest2013248-21kworker/u8:0+events_unbound21:26:260
1902696995132,15cyclictest282-21systemd-journal22:46:281
1902696995131,17cyclictest2009673-21irqrtprio20:56:101
1902693995046,3cyclictest2023134-21kworker/u8:1+events_unbound22:06:230
1902696994937,8cyclictest282-21systemd-journal23:16:291
1902696994829,16cyclictest2101999-21ntpq22:26:171
1902693994836,9cyclictest282-21systemd-journal00:35:010
1902696994737,8cyclictest2181321-21ntpq23:46:141
1902696994737,8cyclictest2181321-21ntpq23:46:131
1902696994730,14cyclictest2048159-21cron21:35:001
1902696994721,8cyclictest282-21systemd-journal20:00:011
1902700994640,4cyclictest1882982-21kworker/u8:3+flush-8:019:15:062
1902696994638,5cyclictest2107776-21latency_hist22:35:011
1902693994642,3cyclictest2192285-21kworker/u8:3+events_unbound00:18:280
1902693994642,3cyclictest1860547-21kworker/u8:2+events_unbound22:36:060
1902693994541,3cyclictest1924167-21kworker/u8:3+events_unbound20:05:290
1902696994435,7cyclictest2088260-21/usr/sbin/munin22:16:121
1902693994440,3cyclictest1860547-21kworker/u8:2+events_unbound20:14:310
1902693994437,3cyclictest1860547-21kworker/u8:2+events_unbound20:41:020
1902696994319,6cyclictest2172295-21latency_hist23:40:011
1902693994339,3cyclictest2117190-21kworker/u8:0+events_unbound23:02:300
1902693994339,3cyclictest2023134-21kworker/u8:1+events_unbound22:41:180
1902693994339,3cyclictest1924167-21kworker/u8:3+events_unbound19:36:540
1902693994339,3cyclictest1882982-21kworker/u8:3+events_unbound19:10:290
1902700994237,4cyclictest1974257-21kworker/u8:3+flush-8:020:26:202
1902693994238,3cyclictest2192285-21kworker/u8:3+events_unbound00:38:300
1902693994238,3cyclictest2117190-21kworker/u8:0+events_unbound23:37:300
1902693994238,3cyclictest2117190-21kworker/u8:0+events_unbound22:58:300
1902693994238,3cyclictest1974257-21kworker/u8:3+events_unbound20:21:380
1902693994238,3cyclictest1964354-21kworker/u8:1+events_unbound20:36:230
1902693994238,3cyclictest1860547-21kworker/u8:2+events_unbound19:15:290
1902696994135,4cyclictest1860547-21kworker/u8:2+events_unbound22:11:101
1902696994131,7cyclictest282-21systemd-journal21:10:281
1902696994120,6cyclictest2078394-21/usr/sbin/munin22:06:131
1902693994138,2cyclictest2192285-21kworker/u8:3+events_unbound00:04:300
1902693994138,2cyclictest1924167-21kworker/u8:3+events_unbound19:42:300
1902693994137,3cyclictest2092268-21kworker/u8:3+events_unbound22:31:310
1902693994137,3cyclictest2013248-21kworker/u8:0+events_unbound21:51:230
1902693994137,3cyclictest1860547-21kworker/u8:2+events_unbound21:34:080
1902693994133,6cyclictest282-21systemd-journal00:10:010
1902693994133,3cyclictest2156942-21kworker/u8:1+events_unbound23:46:530
1902693994133,3cyclictest2156942-21kworker/u8:1+events_unbound23:46:530
1902696994029,7cyclictest282-21systemd-journal21:56:281
1902693994037,2cyclictest2092268-21kworker/u8:3+events_unbound23:28:180
1902693994036,3cyclictest1924167-21kworker/u8:3+events_unbound19:48:510
1902696993933,4cyclictest1924897-21apt-get19:31:001
190269699392,15cyclictest2132855-21chrt23:01:021
1902693993936,2cyclictest1964354-21kworker/u8:1+events_unbound20:18:300
1902693993935,3cyclictest2117190-21kworker/u8:0+events_unbound23:51:240
1902693993935,3cyclictest2023134-21kworker/u8:1+events_unbound22:03:280
1902693993935,3cyclictest2023134-21kworker/u8:1+events_unbound21:47:310
1902693993935,3cyclictest1964354-21kworker/u8:1+events_unbound20:52:090
1902693993935,3cyclictest1860547-21kworker/u8:2+events_unbound23:15:300
1902693993935,3cyclictest1860547-21kworker/u8:2+events_unbound23:12:300
1902693993935,3cyclictest1860547-21kworker/u8:2+events_unbound20:34:070
1902693993934,4cyclictest1860547-21kworker/u8:2+events_unbound19:33:290
1902693993931,6cyclictest2053394-21latency_hist21:40:010
1902700993832,4cyclictest2152632-21apt-get23:21:012
1902700993829,7cyclictest282-21systemd-journal23:10:282
1902696993831,5cyclictest1860547-21kworker/u8:2+flush-8:022:41:101
1902696993829,7cyclictest1989170-21/usr/sbin/munin20:36:111
1902696993829,6cyclictest282-21systemd-journal22:35:281
1902693993834,3cyclictest2201182-21kworker/u8:0+events_unbound00:10:020
1902693993834,3cyclictest2156942-21kworker/u8:1+events_unbound23:55:020
1902693993834,3cyclictest2092268-21kworker/u8:3+events_unbound23:05:300
1902693993834,3cyclictest1974257-21kworker/u8:3+events_unbound20:59:300
1902693993733,3cyclictest2201182-21kworker/u8:0+events_unbound00:29:100
1902693993733,3cyclictest2117190-21kworker/u8:0+events_unbound22:45:020
1902693993733,3cyclictest2092268-21kworker/u8:3+events_unbound22:19:410
1902693993733,3cyclictest2013248-21kworker/u8:0+events_unbound21:15:010
1902693993733,3cyclictest1974257-21kworker/u8:3+events_unbound20:29:240
1902693993733,3cyclictest1924167-21kworker/u8:3+events_unbound19:58:230
1902700993626,8cyclictest2192031-21/usr/sbin/munin00:00:012
1902696993626,7cyclictest2202316-21/usr/sbin/munin00:11:141
1902696993624,9cyclictest282-21systemd-journal21:26:291
1902693993632,3cyclictest2117190-21kworker/u8:0+events_unbound23:24:410
1902693993632,3cyclictest2013248-21kworker/u8:0+events_unbound21:13:070
1902693993632,3cyclictest1964354-21kworker/u8:1+events_unbound20:47:060
1902693993632,3cyclictest1887891-21kworker/u8:0+events_unbound19:20:380
1902693993632,3cyclictest1860547-21kworker/u8:2+events_unbound21:04:090
1902700993523,8cyclictest2197059-21turbostat.cron00:05:012
1902696993529,3cyclictest1915742-21kthreadcore19:21:081
1902696993524,8cyclictest2002805-21ntpq20:46:131
1902696993519,5cyclictest2126706-21ntpq22:51:141
1902693993531,3cyclictest2092268-21kworker/u8:3+events_unbound22:27:580
1902693993531,3cyclictest2013248-21kworker/u8:0+events_unbound21:58:260
1902693993531,3cyclictest1909703-21kworker/u8:1+events_unbound19:29:460
1902696993428,4cyclictest2197389-21apt-get00:06:041
1902696993421,9cyclictest282-21systemd-journal21:39:591
1902693993430,3cyclictest2023134-21kworker/u8:1+events_unbound22:11:300
1902693993430,3cyclictest2023134-21kworker/u8:1+events_unbound21:43:250
1902693993430,3cyclictest2023134-21kworker/u8:1+events_unbound21:43:250
1902693993430,3cyclictest1939461-21kworker/u8:0+events_unbound19:50:420
19022562344,9sleep10-21swapper/119:06:301
1902700993324,7cyclictest282-21systemd-journal23:55:002
1902696993319,10cyclictest1924439-21cp19:30:001
1902693993329,3cyclictest2201182-21kworker/u8:0+events_unbound00:22:190
1902693993329,3cyclictest2156942-21kworker/u8:1+events_unbound23:30:490
1902693993329,3cyclictest2092268-21kworker/u8:3+events_unbound22:52:100
1902693993329,3cyclictest1924167-21kworker/u8:3+events_unbound20:04:010
1902700993228,3cyclictest1939461-21kworker/u8:0+flush-8:020:06:192
1902700993227,4cyclictest1860547-21kworker/u8:2+events_unbound21:01:142
1902696993226,4cyclictest1860547-21kworker/u8:2+flush-8:021:00:591
1902700993128,2cyclictest2177243-21awk23:45:002
1902700993121,8cyclictest1944675-21munin-node19:51:002
1902696993126,3cyclictest1971553-21kthreadcore20:16:141
1902696993124,5cyclictest282-21systemd-journal21:42:291
1902696993124,5cyclictest282-21systemd-journal21:42:291
1902693993127,3cyclictest2023134-21kworker/u8:1+events_unbound22:23:340
19004722310,6sleep01900476-21kthreadcore19:06:120
1902696993022,6cyclictest2097876-21latency_hist22:25:001
1902696992923,3cyclictest1965155-21irqrtprio20:11:101
1902696992921,5cyclictest2157223-21latency_hist23:25:011
1902700992824,2cyclictest2227153-21aten2_r5power_p00:36:032
1902700992824,2cyclictest1974661-21fschecks_time20:21:052
1902700992823,4cyclictest1860547-21kworker/u8:2+events_unbound19:31:132
1902700992822,4cyclictest2023134-21kworker/u8:1+events_unbound22:20:012
1902696992824,2cyclictest0-21swapper/100:00:001
1902696992823,3cyclictest2197048-21cron00:05:011
1902696992823,3cyclictest1964101-21cron20:10:011
21011492270,2sleep3481ktimers/322:26:113
1902700992723,2cyclictest2197447-21aten2_r5power_e00:06:072
1902700992723,2cyclictest2073599-21aten2_r5power_v22:01:042
1902696992721,4cyclictest2166714-21grep23:31:201
1902696992720,5cyclictest2023428-21latency_hist21:10:011
1902696992717,8cyclictest2063573-21/usr/sbin/munin21:51:121
1902696992717,8cyclictest2032497-21ntpq21:16:151
20396682260,4sleep2161rcu_preempt21:26:122
1902700992622,2cyclictest2217506-21fschecks_time00:26:072
1902700992621,3cyclictest2058887-21cpuspeed_turbos21:46:032
1902700992621,3cyclictest1924002-21unixbench_singl19:26:212
1902696992621,4cyclictest2216495-21unixbench_singl00:21:231
1902696992621,3cyclictest1939464-21sh19:45:001
1902696992620,5cyclictest2092268-21kworker/u8:3+flush-8:023:12:121
22040482250,4sleep3471rcuc/300:11:113
1902700992522,2cyclictest1988482-21unixbench_multi20:31:192
1902696992520,3cyclictest2221779-21systemd-run00:30:011
1902696992520,3cyclictest0-21swapper/120:05:011
1902696992517,4cyclictest0-21swapper/119:36:081
1902696992515,8cyclictest1906748-21ntpq19:11:161
1902700992421,2cyclictest2222201-21aten2_r5power_v00:31:032
1902700992421,2cyclictest2157808-21fschecks_count23:26:052
1902700992419,4cyclictest1964354-21kworker/u8:1+events_unbound20:11:262
1902700992418,4cyclictest1887891-21kworker/u8:0+events_unbound19:21:152
1902696992420,2cyclictest1978981-21ls20:25:001
1902696992420,2cyclictest0-21swapper/120:51:111
1902696992417,5cyclictest405-21dbus-daemon21:50:001
1902696992415,7cyclictest282-21systemd-journal23:41:291
1902696992413,3cyclictest0-21swapper/120:26:091
1902700992320,2cyclictest2043803-21cpuspeed_turbos21:31:042
1902700992319,3cyclictest1924167-21kworker/u8:3+events_unbound20:01:212
1902700992319,2cyclictest0-21swapper/221:40:002
1902696992318,3cyclictest0-21swapper/121:21:081
1902696992318,2cyclictest0-21swapper/119:15:051
1902696992316,5cyclictest1924167-21kworker/u8:3+flush-8:019:46:191
1902696992316,4cyclictest0-21swapper/119:51:201
1902696992314,7cyclictest2141503-21ntpq23:06:131
190270099222,14cyclictest2083329-21apt-get22:11:022
1902700992218,2cyclictest0-21swapper/220:56:142
190270099220,17cyclictest2207273-21apt-get00:16:002
1902696992217,4cyclictest2228999-21kthreadcore00:36:111
1902696992215,4cyclictest0-21swapper/122:01:191
190270099212,14cyclictest2127899-21apt-get22:56:042
1902700992117,2cyclictest2063683-21aten2_r5power_p21:51:022
1902696992115,4cyclictest2211494-21unixbench_multi00:16:201
1902696992115,4cyclictest0-21swapper/122:56:161
1902696992114,3cyclictest0-21swapper/120:31:011
1902700992016,2cyclictest1939852-21aten2_r5power_p19:46:032
1902700992016,2cyclictest1930074-21cpuspeed_turbos19:36:042
1902700992012,6cyclictest282-21systemd-journal19:15:012
190269699201,17cyclictest1997962-21ntpq20:41:161
1902700991916,2cyclictest2151901-21unixbench_multi23:16:212
1902700991916,2cyclictest2054655-21kthreadcore21:41:072
1902700991916,2cyclictest2054655-21kthreadcore21:41:072
1902700991915,2cyclictest587-21runrttasks21:08:282
1902700991915,2cyclictest0-21swapper/223:46:062
1902700991915,2cyclictest0-21swapper/223:46:062
1902700991915,2cyclictest0-21swapper/223:35:002
1902700991914,3cyclictest2104044-21sed22:31:112
1902700991914,3cyclictest2023843-21aten2_r5power_p21:11:042
1902700991914,3cyclictest2005110-21kthreadcore20:51:122
1902700991913,4cyclictest282-21systemd-journal20:00:012
1902700991913,4cyclictest0-21swapper/221:16:222
190270099191,13cyclictest2033693-21apt-get21:21:012
1902700991910,7cyclictest2205049-21tr00:11:122
1902696991913,4cyclictest0-21swapper/100:31:021
19021832194,9sleep20-21swapper/219:06:232
190270099182,1cyclictest0-21swapper/221:56:282
1902700991812,4cyclictest1934808-21apt-get19:40:582
190269699180,16cyclictest2162152-21cron23:30:001
19024752183,8sleep30-21swapper/319:09:213
1902700991713,3cyclictest0-21swapper/220:36:192
1902700991713,2cyclictest2122997-21aten2_r5power_e22:51:022
1902700991713,2cyclictest0-21swapper/222:21:062
1902700991713,2cyclictest0-21swapper/222:06:212
190270099171,12cyclictest2212222-21apt-get00:21:032
1902700991710,5cyclictest2117671-21cron22:44:592
22072762162,9sleep30-21swapper/300:16:013
19148202162,9sleep30-21swapper/319:21:013
190270099169,5cyclictest1994229-21hostname20:41:032
190270099162,13cyclictest0-21swapper/222:45:172
190270099162,13cyclictest0-21swapper/220:16:312
1902700991612,2cyclictest2101313-21kthreadcore22:26:132
1902700991612,2cyclictest0-21swapper/223:06:082
1902700991612,2cyclictest0-21swapper/222:39:202
1902700991611,3cyclictest0-21swapper/220:46:122
190270099152,12cyclictest0-21swapper/223:35:252
19347912142,7sleep30-21swapper/319:41:003
22309172132,7sleep30-21swapper/300:36:193
21184902133,6sleep30-21swapper/322:46:083
20135832132,6sleep30-21swapper/321:00:013
19504222132,7sleep30-21swapper/319:56:093
19394592132,6sleep30-21swapper/319:45:013
19218112132,7sleep30-21swapper/319:26:103
19066512132,7sleep30-21swapper/319:11:133
22220922122,6sleep30-21swapper/300:31:023
22175382123,6sleep30-21swapper/300:26:093
22124692122,6sleep30-21swapper/300:21:073
21926162122,6sleep30-21swapper/300:01:043
21908082122,6sleep30-21swapper/323:56:153
21833872122,6sleep30-21swapper/323:51:093
21815682122,6sleep30-21swapper/323:46:203
21815682122,6sleep30-21swapper/323:46:193
21723842122,5sleep30-21swapper/323:40:013
21669392124,5sleep30-21swapper/323:33:073
21608262122,6sleep30-21swapper/323:26:133
21473522122,5sleep30-21swapper/323:15:003
21226282122,6sleep30-21swapper/322:50:013
21130532122,6sleep30-21swapper/322:41:023
21118052122,6sleep30-21swapper/322:36:173
20978632122,6sleep30-21swapper/322:25:013
20740712122,6sleep30-21swapper/322:01:093
20589052120,8sleep30-21swapper/321:46:043
20440212122,6sleep30-21swapper/321:31:083
20369522122,6sleep30-21swapper/321:21:123
20275602122,6sleep30-21swapper/321:11:173
19979212123,6sleep30-21swapper/320:41:153
19888592122,5sleep30-21swapper/320:35:013
19888172122,6sleep30-21swapper/320:34:293
19748712122,6sleep30-21swapper/320:21:093
19662422122,6sleep30-21swapper/320:11:143
19450822122,6sleep30-21swapper/319:51:073
19144612122,5sleep30-21swapper/319:20:013
22011242112,5sleep30-21swapper/300:06:183
21526352112,6sleep30-21swapper/323:21:033
21464142112,5sleep30-21swapper/323:11:163
21381552112,5sleep30-21swapper/323:06:063
21328142113,5sleep30-21swapper/323:00:523
21278732112,5sleep30-21swapper/322:56:033
21030462112,6sleep30-21swapper/322:30:113
20882892112,5sleep30-21swapper/322:16:023
20874632112,5sleep30-21swapper/322:11:203
20821322112,5sleep30-21swapper/322:06:153
20684742112,5sleep30-21swapper/321:55:353
20649362112,5sleep30-21swapper/321:51:103
20581212112,5sleep30-21swapper/321:42:233
20581212112,5sleep30-21swapper/321:42:223
20481632112,5sleep30-21swapper/321:35:013
20331732112,5sleep30-21swapper/321:17:553
20192462112,5sleep30-21swapper/321:06:063
19995782111,6sleep30-21swapper/320:46:093
19733102112,5sleep30-21swapper/320:16:233
19598962112,5sleep30-21swapper/320:06:083
19585862112,5sleep30-21swapper/320:01:213
19307522112,5sleep30-21swapper/319:36:103
20132552102,5sleep30-21swapper/320:56:323
20040912101,5sleep30-21swapper/320:51:053
19794682102,5sleep30-21swapper/320:26:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional