You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 04:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Fri Nov 21, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18353792700,7chrt0-21swapper/119:06:311
1836029996459,4cyclictest2115914-21kworker/u8:3+events_unbound00:11:321
1836027996359,3cyclictest1011332-21kworker/u8:2+events_unbound19:16:300
1836029996156,3cyclictest1835860-21kworker/u8:0+events_unbound19:15:031
1836029995951,6cyclictest1922494-21kworker/u8:1+events_unbound22:41:251
1836027995955,3cyclictest1843118-21kworker/u8:3+events_unbound21:06:300
1836029995853,4cyclictest1857961-21kworker/u8:0+events_unbound19:36:311
1836029995853,4cyclictest1857961-21kworker/u8:0+events_unbound19:36:311
18784562570,4chrt0-21swapper/119:51:141
1836029995751,4cyclictest1011332-21kworker/u8:2+flush-8:019:11:311
1836027995742,5cyclictest1927668-21latency_hist20:40:010
1836029995651,3cyclictest2115914-21kworker/u8:3+events_unbound00:01:371
1836029995650,4cyclictest1922494-21kworker/u8:1+events_unbound23:31:251
1836029995650,4cyclictest1922494-21kworker/u8:1+events_unbound23:31:251
1836029995650,4cyclictest1887953-21kworker/u8:2+events_unbound20:46:291
1836027995652,3cyclictest1922494-21kworker/u8:1+events_unbound21:56:340
1836027995651,3cyclictest1843118-21kworker/u8:3+events_unbound21:01:290
1836029995550,4cyclictest1997138-21kworker/u8:2+events_unbound21:51:281
1836029995550,3cyclictest1967456-21kworker/u8:4+flush-8:021:26:261
1836029995549,4cyclictest1997138-21kworker/u8:2+flush-8:022:01:201
1836029995546,7cyclictest1843118-21kworker/u8:3+flush-8:019:26:251
1836029995546,7cyclictest1843118-21kworker/u8:3+flush-8:019:26:251
1836029995546,3cyclictest1778516-21kworker/u8:1+events_unbound19:31:261
1836029995546,3cyclictest1778516-21kworker/u8:1+events_unbound19:31:251
1836027995552,2cyclictest1922494-21kworker/u8:1+events_unbound21:21:300
1836029995450,3cyclictest1967456-21kworker/u8:4+events_unbound21:31:321
1836029995450,3cyclictest1843118-21kworker/u8:3+events_unbound21:36:261
1836029995450,3cyclictest1843118-21kworker/u8:3+events_unbound20:51:311
1836029995449,3cyclictest2120875-21kworker/u8:1+flush-8:000:06:311
1836029995448,4cyclictest2160739-21kworker/u8:2+flush-8:000:36:321
1836029995448,4cyclictest2071319-21kworker/u8:0+flush-8:023:15:011
1836029995448,4cyclictest1922494-21kworker/u8:1+events_unbound22:21:171
1836027995450,3cyclictest2154998-21kworker/u8:0+events_unbound00:26:290
20413282530,4sleep0201rcuc/022:31:320
1836027995349,3cyclictest1997138-21kworker/u8:2+events_unbound21:51:320
1836027995349,3cyclictest1922494-21kworker/u8:1+events_unbound22:41:290
1836029995247,3cyclictest1922494-21kworker/u8:1+events_unbound23:36:361
1836027995248,3cyclictest1887953-21kworker/u8:2+events_unbound21:16:290
1836027995248,3cyclictest1887953-21kworker/u8:2+events_unbound21:16:290
1836027995248,3cyclictest1857961-21kworker/u8:0+events_unbound19:41:280
1836027995248,3cyclictest1857961-21kworker/u8:0+events_unbound19:41:280
1836027995248,3cyclictest1843118-21kworker/u8:3+events_unbound21:36:330
1836027995248,3cyclictest1011332-21kworker/u8:2+events_unbound19:21:300
1836027995240,9cyclictest1921879-21ntpq20:31:310
1836029995147,3cyclictest1922494-21kworker/u8:1+events_unbound22:11:341
1836027995147,3cyclictest1887953-21kworker/u8:2+events_unbound20:41:340
1836027995147,3cyclictest1843118-21kworker/u8:3+events_unbound20:01:310
21227882500,4chrt371rcuc/223:56:222
1836029995046,3cyclictest1887953-21kworker/u8:2+flush-8:021:16:261
1836029995046,3cyclictest1887953-21kworker/u8:2+flush-8:021:16:261
1836029995045,4cyclictest2120875-21kworker/u8:1+events_unbound00:16:271
1836029995045,4cyclictest2071319-21kworker/u8:0+flush-8:023:46:301
1836029995045,4cyclictest1922494-21kworker/u8:1+flush-8:021:56:291
1836029995045,4cyclictest1843118-21kworker/u8:3+flush-8:020:01:281
1836029995044,4cyclictest1843118-21kworker/u8:3+events_unbound20:16:291
1836027995047,2cyclictest2115914-21kworker/u8:3+events_unbound00:31:190
1836027995046,3cyclictest2115914-21kworker/u8:3+events_unbound00:01:310
1836027995041,6cyclictest2071604-21latency_hist23:05:000
1836029994944,4cyclictest1843118-21kworker/u8:3+flush-8:020:21:281
1836029994944,4cyclictest1011332-21kworker/u8:2+flush-8:019:46:161
1836029994944,3cyclictest1843118-21kworker/u8:3+flush-8:021:06:251
1836029994943,4cyclictest1922494-21kworker/u8:1+flush-8:022:34:591
1836027994941,5cyclictest2085567-21ntpq23:16:230
18637122480,6sleep30-21swapper/319:36:193
18637122480,6sleep30-21swapper/319:36:193
1836029994844,3cyclictest2059361-21kworker/u8:3+events_unbound22:51:311
1836029994843,4cyclictest1857961-21kworker/u8:0+flush-8:019:56:251
1836029994843,4cyclictest1843118-21kworker/u8:3+events_unbound20:56:321
1836029994843,3cyclictest2059361-21kworker/u8:3+flush-8:022:56:281
1836029994838,3cyclictest2071319-21kworker/u8:0+events_unbound23:06:191
1836027994845,2cyclictest1922494-21kworker/u8:1+events_unbound22:11:260
1836027994844,3cyclictest2071319-21kworker/u8:0+events_unbound23:11:310
1836027994844,3cyclictest1982008-21kworker/u8:0+events_unbound21:41:150
1836027994837,3cyclictest2160739-21kworker/u8:2+events_unbound00:36:160
1836032994740,5cyclictest282-21systemd-journal19:15:002
1836029994741,5cyclictest1922494-21kworker/u8:1+flush-8:023:16:241
1836029994741,4cyclictest1887953-21kworker/u8:2+events_unbound20:41:281
1836027994744,2cyclictest2031985-21kworker/u8:0+events_unbound22:26:330
1836027994743,3cyclictest1843118-21kworker/u8:3+events_unbound20:11:350
1836027994743,3cyclictest1778516-21kworker/u8:1+events_unbound19:31:240
1836027994743,3cyclictest1778516-21kworker/u8:1+events_unbound19:31:240
1836029994642,3cyclictest2031985-21kworker/u8:0+events_unbound22:26:271
1836029994642,3cyclictest1922494-21kworker/u8:1+flush-8:022:06:161
1836029994641,4cyclictest1843118-21kworker/u8:3+events_unbound20:06:271
1836027994642,3cyclictest2120875-21kworker/u8:1+events_unbound23:56:200
1836027994642,3cyclictest2071319-21kworker/u8:0+events_unbound23:26:130
1836027994642,3cyclictest1922494-21kworker/u8:1+events_unbound23:31:310
1836027994642,3cyclictest1922494-21kworker/u8:1+events_unbound23:31:310
1836027994642,3cyclictest1843118-21kworker/u8:3+events_unbound20:56:240
1836027994642,3cyclictest1843118-21kworker/u8:3+events_unbound19:26:330
1836027994642,3cyclictest1843118-21kworker/u8:3+events_unbound19:26:330
1836027994642,3cyclictest1011332-21kworker/u8:2+events_unbound19:35:010
1836027994642,3cyclictest1011332-21kworker/u8:2+events_unbound19:35:010
1836029994541,3cyclictest1922494-21kworker/u8:1+events_unbound23:01:251
1836027994542,2cyclictest1887953-21kworker/u8:2+events_unbound20:51:190
1836027994541,3cyclictest2046880-21kworker/u8:2+events_unbound22:51:150
1836027994541,3cyclictest1011332-21kworker/u8:2+events_unbound19:46:310
18590892440,6sleep20-21swapper/219:31:202
18590892440,6sleep20-21swapper/219:31:202
1836029994439,4cyclictest1922494-21kworker/u8:1+events_unbound20:36:241
1836029994439,4cyclictest1857961-21kworker/u8:0+events_unbound20:11:281
1836029994438,5cyclictest1922494-21kworker/u8:1+events_unbound23:41:331
1836029994436,6cyclictest2115914-21kworker/u8:3+flush-8:000:21:271
1836027994440,3cyclictest2031985-21kworker/u8:0+events_unbound22:46:180
1836027994440,3cyclictest1922494-21kworker/u8:1+events_unbound23:40:400
1836027994440,3cyclictest1857961-21kworker/u8:0+events_unbound20:26:170
1836027994440,3cyclictest1843118-21kworker/u8:3+events_unbound21:46:260
1836027994440,3cyclictest1843118-21kworker/u8:3+events_unbound21:26:170
1836027994440,3cyclictest1843118-21kworker/u8:3+events_unbound20:16:170
1836027994435,7cyclictest1977460-21/usr/sbin/munin21:31:330
1836029994339,3cyclictest2076296-21kworker/u8:3+flush-8:023:26:311
1836029994336,5cyclictest1922494-21kworker/u8:1+flush-8:022:16:271
1836027994339,3cyclictest2059361-21kworker/u8:3+events_unbound22:58:500
1836027994339,3cyclictest1997138-21kworker/u8:2+events_unbound22:16:190
1836029994238,3cyclictest1887953-21kworker/u8:2+events_unbound20:35:011
1836029994238,3cyclictest1011332-21kworker/u8:2+events_unbound19:41:121
1836029994238,3cyclictest1011332-21kworker/u8:2+events_unbound19:41:121
1836029994236,4cyclictest2120875-21kworker/u8:1+flush-8:000:26:251
1836027994238,3cyclictest2115914-21kworker/u8:3+events_unbound00:11:220
1836027994238,3cyclictest2096081-21kworker/u8:2+events_unbound23:38:550
1836027994238,3cyclictest1922494-21kworker/u8:1+events_unbound22:09:420
1836029994137,3cyclictest1887953-21kworker/u8:2+flush-8:021:01:201
1836027994138,2cyclictest1922494-21kworker/u8:1+events_unbound22:38:480
1836027994137,3cyclictest1843118-21kworker/u8:3+events_unbound20:06:150
1836027994134,5cyclictest1840086-21ntpq19:11:260
1836027994037,2cyclictest1922494-21kworker/u8:1+events_unbound23:06:360
1836027994036,3cyclictest2120875-21kworker/u8:1+events_unbound00:16:190
1836027994036,3cyclictest1878230-21kworker/u8:1+events_unbound19:51:330
1836027993935,3cyclictest2071319-21kworker/u8:0+events_unbound23:21:370
1836029993834,3cyclictest2115914-21kworker/u8:3+flush-8:000:31:321
1836029993834,3cyclictest2115914-21kworker/u8:3+events_unbound00:00:011
1836029993834,3cyclictest1982008-21kworker/u8:0+events_unbound21:41:191
1836027993835,2cyclictest2120875-21kworker/u8:1+events_unbound00:21:400
1836027993834,3cyclictest2120875-21kworker/u8:1+events_unbound00:06:170
1836027993834,3cyclictest1997138-21kworker/u8:2+events_unbound22:22:410
1836029993732,4cyclictest1982008-21kworker/u8:0+flush-8:021:46:211
1836027993733,3cyclictest2115914-21kworker/u8:3+events_unbound23:50:410
1836027993733,3cyclictest1857961-21kworker/u8:0+events_unbound20:22:380
1836032993631,4cyclictest2016121-21kworker/u8:3+flush-8:022:31:222
1836032993627,6cyclictest2071600-21awk23:05:002
1836027993632,3cyclictest2071319-21kworker/u8:0+events_unbound23:46:110
1836027993632,3cyclictest1843118-21kworker/u8:3+events_unbound21:10:130
1836029993531,3cyclictest2096081-21kworker/u8:2+flush-8:023:51:171
1836029993529,5cyclictest1011332-21kworker/u8:2+events_unbound19:21:261
1836032993424,8cyclictest282-21systemd-journal23:30:402
1836032993424,8cyclictest282-21systemd-journal23:30:402
1836027993430,3cyclictest1843118-21kworker/u8:3+events_unbound19:58:230
1836032993324,6cyclictest2165940-21awk00:40:002
1836029993325,5cyclictest282-21systemd-journal22:36:391
1836027993329,3cyclictest1997138-21kworker/u8:2+events_unbound22:03:410
1836027993329,3cyclictest1887953-21kworker/u8:2+events_unbound20:47:400
1836029993228,3cyclictest2031985-21kworker/u8:0+flush-8:022:46:301
188509723020,6sleep30-21swapper/319:56:223
1836032993019,9cyclictest1947436-21awk21:00:002
1836032993019,8cyclictest1878007-21awk19:50:002
1836029993025,4cyclictest1857961-21kworker/u8:0+events_unbound20:26:301
1836032992925,2cyclictest1913552-21fschecks_time20:26:182
1836029992924,4cyclictest1922494-21kworker/u8:1+events_unbound21:21:281
1836029992824,3cyclictest2071319-21kworker/u8:0+flush-8:023:21:341
1836032992723,2cyclictest2146388-21aten2_r5power_v00:21:152
1836032992723,2cyclictest0-21swapper/222:19:592
1836032992718,7cyclictest282-21systemd-journal19:16:292
183603299270,25cyclictest1971827-21unixbench_multi21:21:342
1836032992623,2cyclictest1923132-21aten2_r5power_p20:36:142
1836032992622,3cyclictest2077205-21fschecks_time23:11:162
1836032992622,2cyclictest2091792-21aten2_r5power_c23:26:132
1836032992622,2cyclictest1883357-21aten2_r5power_e19:56:142
1836032992621,4cyclictest1987680-21sort21:41:162
1836032992617,7cyclictest2156061-21gpgv00:30:012
183603299261,23cyclictest1-21systemd23:25:002
1836029992620,5cyclictest1843118-21kworker/u8:3+events_unbound21:11:091
1836032992522,2cyclictest2062318-21fschecks_time22:56:182
1836032992522,2cyclictest1933335-21fschecks_time20:46:192
1836032992521,3cyclictest2131512-21aten2_r5power_p00:06:132
1836032992521,2cyclictest2012342-21aten2_r5power_c22:06:132
1836032992516,7cyclictest1912872-21awk20:25:002
1836032992421,2cyclictest1982422-21aten2_r5power_c21:36:142
1836032992420,3cyclictest2032183-21aten2_r5power_p22:26:162
1836032992419,4cyclictest412-21systemd-logind22:05:002
1836032992419,4cyclictest2044392-21kthreadcore22:36:232
1836032992414,8cyclictest2145925-21awk00:20:002
1836032992318,3cyclictest2113747-21kthreadcore23:46:202
1836032992219,2cyclictest1957714-21aten2_r5power_p21:11:142
1836032992218,2cyclictest2017284-21aten2_r5power_c22:11:192
183603299212,14cyclictest1972523-21apt-get21:26:132
1836032992118,2cyclictest2057123-21aten2_r5power_e22:51:142
1836032992117,3cyclictest1992554-21cpuspeed_turbos21:46:162
18309622212,9sleep30-21swapper/319:05:013
183603299203,2cyclictest19315362cut20:41:242
183603299201,8cyclictest18784582cpu19:51:142
1836032992017,2cyclictest1997551-21fschecks_time21:51:192
1836032992017,2cyclictest1888545-21aten2_r5power_e20:01:152
1836032992015,3cyclictest2086117-21unixbench_singl23:16:312
183603299200,6cyclictest21568492if_err_enp4s0.200:31:192
183603299200,6cyclictest21167132sort23:51:142
183603299199,8cyclictest1952382-21awk21:05:002
183603299199,7cyclictest2129656-21tr00:01:232
183603299198,3cyclictest0-21swapper/219:21:222
1836032991915,2cyclictest0-21swapper/220:31:352
1836032991914,3cyclictest0-21swapper/219:45:002
1836032991914,3cyclictest0-21swapper/219:45:002
1836032991913,2cyclictest0-21swapper/220:11:172
1836032991911,6cyclictest2029217-21tr22:21:232
1836032991910,7cyclictest1-21systemd20:10:012
183602799198,6cyclictest1835844-21latency_hist19:10:010
18357762194,11sleep20-21swapper/219:09:262
1836032991815,2cyclictest1867516-21unixbench_multi19:36:332
1836032991815,2cyclictest1867516-21unixbench_multi19:36:332
1836032991814,3cyclictest1922494-21kworker/u8:1+events_unbound22:45:082
1836032991814,2cyclictest2101981-21fschecks_time23:36:242
1836032991814,2cyclictest0-21swapper/221:31:352
1836032991814,2cyclictest0-21swapper/221:06:312
1836032991813,3cyclictest2002923-21cut21:56:222
1836032991713,2cyclictest0-21swapper/223:06:322
1836032991710,5cyclictest2051816-21turbostat.cron22:45:002
183603299162,8cyclictest1853136-21apt-get19:26:142
183603299162,8cyclictest1853136-21apt-get19:26:142
183603299162,2cyclictest0-21swapper/220:51:312
183603299162,13cyclictest0-21swapper/221:18:242
183603299162,13cyclictest0-21swapper/221:18:242
1836032991613,2cyclictest0-21swapper/223:43:302
1836032991611,3cyclictest0-21swapper/220:16:232
183603299161,11cyclictest2136387-21apt-get00:11:142
20039822150,4sleep3481ktimers/321:56:243
21502812142,7sleep30-21swapper/300:21:323
18732912142,8sleep30-21swapper/319:45:033
20176022134,5sleep30-21swapper/322:11:263
20009852132,7sleep30-21swapper/321:51:273
19865232132,7sleep30-21swapper/321:36:353
19672532132,6sleep30-21swapper/321:20:013
19425022132,6sleep30-21swapper/320:55:013
19229782135,5sleep30-21swapper/320:35:393
18679192132,6sleep30-21swapper/319:40:013
18679192132,6sleep30-21swapper/319:40:013
21563202122,6sleep30-21swapper/300:31:073
21412952123,6sleep30-21swapper/300:16:143
21305782122,6sleep30-21swapper/300:01:383
21119612122,6sleep30-21swapper/323:46:173
21013382122,6sleep30-21swapper/323:35:013
20966302122,6sleep30-21swapper/323:30:403
20966302122,6sleep30-21swapper/323:30:403
20917942124,5sleep30-21swapper/323:26:133
20910832122,6sleep30-21swapper/323:21:373
20806312122,6sleep30-21swapper/323:11:253
20760472121,7sleep30-21swapper/323:06:363
20574482122,6sleep30-21swapper/322:51:193
20524152122,6sleep30-21swapper/322:46:173
20472592122,6sleep30-21swapper/322:41:163
20416792122,6sleep30-21swapper/322:35:013
20369842122,6sleep30-21swapper/322:30:403
20272332122,6sleep30-21swapper/322:21:173
20226282122,6sleep30-21swapper/322:16:193
20114962122,6sleep30-21swapper/322:02:033
19914692122,6sleep30-21swapper/321:41:323
19731242122,6sleep30-21swapper/321:26:193
19576882122,6sleep30-21swapper/321:11:143
19564092122,6sleep30-21swapper/321:06:253
19508462123,6sleep30-21swapper/321:01:233
19365532122,6sleep30-21swapper/320:46:263
19177592122,6sleep30-21swapper/320:30:013
19125112122,6sleep30-21swapper/320:21:343
18983772122,6sleep30-21swapper/320:11:143
18980352122,6sleep30-21swapper/320:09:513
18580762123,5sleep30-21swapper/319:31:143
18580762123,5sleep30-21swapper/319:31:143
18533782122,6sleep30-21swapper/319:26:163
18533782122,6sleep30-21swapper/319:26:163
18524232122,6sleep30-21swapper/319:21:313
18409212122,5sleep30-21swapper/319:15:013
21612602113,5sleep30-21swapper/300:35:373
21532952112,5sleep30-21swapper/300:26:223
21368082112,6sleep30-21swapper/300:11:203
21322992112,5sleep30-21swapper/300:06:193
21214872112,5sleep30-21swapper/323:56:163
21168952112,5sleep30-21swapper/323:51:183
21070582112,5sleep30-21swapper/323:41:193
20817882112,6sleep30-21swapper/323:16:033
20670322112,5sleep30-21swapper/323:01:143
20660502112,5sleep30-21swapper/322:56:323
20358322112,6sleep30-21swapper/322:26:273
19963132112,5sleep30-21swapper/321:46:343
19774142112,6sleep30-21swapper/321:30:413
19628292112,5sleep30-21swapper/321:16:163
19628292112,5sleep30-21swapper/321:16:163
19377782112,5sleep30-21swapper/320:50:043
19318132112,5sleep30-21swapper/320:41:293
19180002112,5sleep30-21swapper/320:30:053
19037722112,6sleep30-21swapper/320:16:213
18910582112,5sleep30-21swapper/320:01:243
18820982112,6sleep30-21swapper/319:51:253
18362662112,5sleep30-21swapper/319:11:163
20122092102,5sleep30-21swapper/322:05:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional