You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-16 - 01:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot4s.osadl.org (updated Sun Sep 15, 2024 12:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1775758994639,2cyclictest1983286-21kworker/u8:1+events_unbound11:15:223
1775758994238,3cyclictest1848238-21kworker/u8:4+events_unbound08:55:263
1775744994237,4cyclictest2050937-21diskstats11:35:130
1775758994139,2cyclictest1822519-21kworker/u8:2+events_unbound08:10:133
1775754994035,3cyclictest1848238-21kworker/u8:4+flush-259:009:19:592
1775758993936,2cyclictest2029547-21kworker/u8:0+events_unbound11:25:253
1775758993935,3cyclictest2029547-21kworker/u8:0+events_unbound11:30:193
1775744993935,3cyclictest1962354-21kworker/u8:3+flush-259:010:50:230
1775758993834,3cyclictest1889972-21kworker/u8:2+events_unbound10:20:123
1775758993834,3cyclictest1848238-21kworker/u8:4+events_unbound09:30:203
1775758993834,3cyclictest1738143-21kworker/u8:1+events_unbound07:25:143
1775758993831,4cyclictest2029525-21latency_hist11:15:003
1775754993834,3cyclictest1925825-21kworker/u8:1+flush-259:010:05:012
1775758993733,3cyclictest1055381-21kworker/u8:3+flush-259:008:15:253
1775758993634,2cyclictest2029547-21kworker/u8:0+events_unbound11:20:123
1775758993633,2cyclictest1889972-21kworker/u8:2+events_unbound10:00:193
1775754993632,3cyclictest2008492-21kworker/u8:2+flush-259:011:00:162
1775754993630,4cyclictest2304165-21systemd-journal12:05:352
1775758993532,2cyclictest1796950-21kworker/u8:4+events_unbound07:55:223
1775749993531,3cyclictest1055381-21kworker/u8:3+flush-259:007:15:161
1775749993528,5cyclictest1801880-21latency_hist07:35:011
1775744993530,4cyclictest1780361-21ntpq07:10:200
1775758993431,2cyclictest1822519-21kworker/u8:2+events_unbound08:40:253
1775758993430,3cyclictest2008492-21kworker/u8:2+events_unbound11:00:213
1775754993431,2cyclictest1791789-21kworker/u8:2+events_unbound07:25:182
1775749993430,3cyclictest1848238-21kworker/u8:4+flush-259:009:30:231
1775758993330,2cyclictest1796950-21kworker/u8:4+flush-259:007:50:233
1775758993328,3cyclictest1791789-21kworker/u8:2+events_unbound07:30:243
1775749993326,5cyclictest2055695-21latency_hist11:40:001
1775744993330,2cyclictest2097116-21kworker/u8:3+flush-259:012:25:230
1775758993229,2cyclictest2059877-21kworker/u8:2+events_unbound11:55:153
1775758993229,2cyclictest1889972-21kworker/u8:2+flush-259:010:25:243
1775758993228,3cyclictest2059877-21kworker/u8:2+events_unbound11:45:023
1775758993228,3cyclictest1822519-21kworker/u8:2+events_unbound08:25:143
1775754993227,3cyclictest2045605-21kworker/u8:3+flush-259:011:40:172
1775754993227,3cyclictest1848238-21kworker/u8:4+flush-259:009:20:152
1775749993225,5cyclictest2044944-21latency_hist11:30:001
1775758993128,2cyclictest1983286-21kworker/u8:1+flush-259:012:00:213
1775758993125,4cyclictest1946610-21gpgv09:55:013
1775754993128,2cyclictest1889972-21kworker/u8:2+flush-259:009:30:002
1775749993125,4cyclictest2304165-21systemd-journal11:40:181
1775744993127,3cyclictest1962354-21kworker/u8:3+flush-259:010:25:210
1775744993127,3cyclictest1910632-21apt-get09:20:100
1775758993027,2cyclictest1962354-21kworker/u8:3+flush-259:010:40:233
1775758993027,2cyclictest1931202-21kworker/u8:0+events_unbound10:30:193
1775758993027,2cyclictest1889972-21kworker/u8:2+flush-259:010:15:243
1775758993027,2cyclictest1738143-21kworker/u8:1+events_unbound07:35:143
1775758993026,3cyclictest1931202-21kworker/u8:0+flush-259:010:05:213
1775758993024,4cyclictest1781273-21latency_hist07:15:013
1775749993026,3cyclictest1738143-21kworker/u8:1+flush-259:007:40:151
1775744993027,2cyclictest1848238-21kworker/u8:4+flush-259:009:00:200
1775758992926,2cyclictest1889972-21kworker/u8:2+events_unbound09:35:233
1775758992926,2cyclictest1848238-21kworker/u8:4+events_unbound08:20:223
1775758992925,3cyclictest1848238-21kworker/u8:4+flush-259:009:05:153
1775749992926,2cyclictest1983286-21kworker/u8:1+flush-259:012:10:261
1775749992923,4cyclictest2304165-21systemd-journal11:13:181
1775749992920,6cyclictest1775681-21latency_hist07:10:011
1775758992826,2cyclictest1738143-21kworker/u8:1+events_unbound07:40:213
1775758992825,2cyclictest1905208-21kworker/u8:3+events_unbound09:15:153
1775758992824,3cyclictest1983286-21kworker/u8:1+events_unbound12:10:183
1775758992823,2cyclictest1962354-21kworker/u8:3+events_unbound10:50:243
1775754992823,3cyclictest1988214-21kworker/u8:4+flush-259:011:35:162
1775754992822,4cyclictest2050517-21latency_hist11:35:002
1775749992824,3cyclictest1848238-21kworker/u8:4+flush-259:009:00:161
1775758992724,2cyclictest1738143-21kworker/u8:1+flush-259:007:20:163
1775749992721,4cyclictest1858771-21apt-get08:30:121
1775758992623,2cyclictest2059877-21kworker/u8:2+events_unbound11:50:123
1775758992623,2cyclictest2045605-21kworker/u8:3+events_unbound11:40:203
1775758992623,2cyclictest1988214-21kworker/u8:4+flush-259:010:55:213
1775758992622,3cyclictest1931202-21kworker/u8:0+events_unbound09:40:133
1775754992623,2cyclictest1796950-21kworker/u8:4+events_unbound07:30:182
1775754992620,4cyclictest2092248-21latency_hist12:15:012
1775754992620,4cyclictest2040216-21diskstats11:25:182
1775744992623,2cyclictest1822519-21kworker/u8:2+flush-259:008:25:240
1775744992623,2cyclictest1738143-21kworker/u8:1+flush-259:007:20:230
1775744992622,3cyclictest1848238-21kworker/u8:4+flush-259:008:55:270
1775744992622,3cyclictest1822519-21kworker/u8:2+flush-259:008:30:240
1775758992522,2cyclictest1055381-21kworker/u8:3+events_unbound08:05:183
1775754992522,2cyclictest1055381-21kworker/u8:3+flush-259:008:20:012
1775754992521,3cyclictest1931202-21kworker/u8:0+events_unbound10:50:202
1775749992520,4cyclictest1848238-21kworker/u8:4+flush-259:008:49:591
1775744992522,2cyclictest1738143-21kworker/u8:1+flush-259:007:25:150
1775744992521,3cyclictest1983286-21kworker/u8:1+events_unbound11:10:220
1775758992421,2cyclictest2029547-21kworker/u8:0+events_unbound12:09:203
1775758992421,2cyclictest1848238-21kworker/u8:4+events_unbound09:20:103
1775758992420,3cyclictest1889972-21kworker/u8:2+events_unbound09:10:253
1775758992416,6cyclictest1786442-21latency_hist07:20:003
1775749992418,4cyclictest2304165-21systemd-journal09:42:171
1775744992420,3cyclictest1848238-21kworker/u8:4+flush-259:008:35:250
1775744992420,3cyclictest1055381-21kworker/u8:3+flush-259:007:35:180
1775744992419,3cyclictest1931202-21kworker/u8:0+events_unbound09:45:230
1775744992418,4cyclictest1781514-21/usr/sbin/munin07:15:150
1775758992320,2cyclictest1931202-21kworker/u8:0+events_unbound10:45:223
1775758992320,2cyclictest1055381-21kworker/u8:3+flush-259:007:45:183
1775758992319,3cyclictest1931202-21kworker/u8:0+events_unbound09:45:163
1775758992319,3cyclictest1880044-21kworker/u8:0+flush-259:009:25:183
1775758992317,4cyclictest1993395-21latency_hist10:40:013
1775754992320,2cyclictest1931202-21kworker/u8:0+flush-259:010:40:012
1775749992319,3cyclictest1925825-21kworker/u8:1+flush-259:009:55:011
1775749992319,3cyclictest1055381-21kworker/u8:3+flush-259:007:45:261
1775744992321,1cyclictest2029547-21kworker/u8:0+events_unbound11:25:190
1775744992319,3cyclictest503-21systemd-logind10:35:010
1775758992219,2cyclictest1863875-21kworker/u8:0+events_unbound08:39:593
1775758992219,2cyclictest1848238-21kworker/u8:4+events_unbound08:45:213
1775758992216,4cyclictest2113095-21apt-get12:35:123
1775754992217,4cyclictest2304165-21systemd-journal10:00:002
1775744992219,2cyclictest1-21systemd09:35:000
1775744992219,2cyclictest1055381-21kworker/u8:3+flush-259:007:45:240
1775744992218,3cyclictest1822519-21kworker/u8:2+flush-259:008:40:160
1775754992118,2cyclictest1055381-21kworker/u8:3+events_unbound08:30:202
1775754992117,3cyclictest1822519-21kworker/u8:2+events_unbound08:10:172
1775754992116,4cyclictest1880044-21kworker/u8:0+events_unbound08:50:222
1775749992116,4cyclictest1962354-21kworker/u8:3+flush-259:010:40:191
1775749992116,3cyclictest2304165-21systemd-journal12:36:181
1775749992115,4cyclictest1843329-21apt-get08:15:111
1775749992114,5cyclictest2304165-21systemd-journal08:05:011
1775749992114,5cyclictest1738143-21kworker/u8:1+flush-259:007:20:161
1775744992118,2cyclictest1905183-21latency_hist09:14:590
1775744992118,2cyclictest1822519-21kworker/u8:2+flush-259:008:15:220
1775744992117,3cyclictest1988214-21kworker/u8:4+events_unbound11:17:180
1775744992117,3cyclictest1958803-21cat10:05:180
1775744992115,4cyclictest2304165-21systemd-journal11:55:310
1775758992017,2cyclictest1962354-21kworker/u8:3+events_unbound10:13:193
1775758992017,2cyclictest1848238-21kworker/u8:4+events_unbound09:00:283
1775754992017,2cyclictest1848238-21kworker/u8:4+flush-259:008:39:592
1775754992016,3cyclictest1889972-21kworker/u8:2+flush-259:009:30:152
1775754992016,3cyclictest1889972-21kworker/u8:2+flush-259:009:04:592
1775754992016,3cyclictest1822519-21kworker/u8:2+events_unbound08:40:182
1775754992016,3cyclictest1738143-21kworker/u8:1+events_unbound07:20:152
1775754992015,3cyclictest1930102-21ntpq09:35:202
1775744992016,3cyclictest1988214-21kworker/u8:4+events_unbound11:30:000
1775744992014,5cyclictest1827914-21/usr/sbin/munin08:00:150
1775744992014,4cyclictest509-21rs:main0
1775744992014,4cyclictest2304165-21systemd-journal11:24:180
1775744992014,4cyclictest1967789-21/usr/sbin/munin10:15:200
1775744992013,6cyclictest0-21swapper/009:20:000
1775758991916,2cyclictest2029547-21kworker/u8:0+events_unbound11:35:283
1775758991913,2cyclictest2097116-21kworker/u8:3+events_unbound12:30:243
1775758991911,5cyclictest1946877-21cat09:55:013
1775754991916,2cyclictest1848238-21kworker/u8:4+events_unbound08:45:252
1775754991915,3cyclictest1931202-21kworker/u8:0+flush-259:010:05:232
1775754991913,4cyclictest2102491-21latency_hist12:25:012
1775749991912,5cyclictest2008870-21latency_hist10:55:011
1775744991916,2cyclictest1983286-21kworker/u8:1+events_unbound12:15:160
1775744991916,2cyclictest1848238-21kworker/u8:4+flush-259:009:05:250
1775744991915,3cyclictest2059877-21kworker/u8:2+events_unbound11:50:330
1775744991915,3cyclictest1983286-21kworker/u8:1+flush-259:011:05:150
1775744991915,3cyclictest1807286-21kworker/u8:0+events_unbound07:53:170
1775758991815,2cyclictest1822519-21kworker/u8:2+events_unbound08:30:223
1775758991813,3cyclictest2019495-21apt-get11:05:123
1775754991816,1cyclictest1807286-21kworker/u8:0+events_unbound07:50:262
1775754991815,2cyclictest1936113-21idleruntime-cro09:45:002
1775754991814,3cyclictest2059877-21kworker/u8:2+events_unbound12:00:012
1775754991814,3cyclictest1962354-21kworker/u8:3+events_unbound10:40:152
1775754991813,4cyclictest2107698-21latency_hist12:30:012
1775754991812,4cyclictest1967545-21latency_hist10:15:002
1775749991815,2cyclictest1885300-21irqcore08:55:181
1775749991814,3cyclictest1988214-21kworker/u8:4+flush-259:011:05:261
1775749991813,3cyclictest1957465-21apt-get10:05:131
1775744991815,2cyclictest1848238-21kworker/u8:4+events_unbound09:40:040
1775744991815,2cyclictest1055381-21kworker/u8:3+flush-259:008:05:160
1775744991814,3cyclictest1983286-21kworker/u8:1+flush-259:012:05:240
1775744991814,3cyclictest1962354-21kworker/u8:3+events_unbound10:10:190
1775744991814,3cyclictest1848238-21kworker/u8:4+events_unbound08:24:190
1775744991812,5cyclictest2117992-21latency_hist12:40:010
1775758991715,2cyclictest2102717-21kworker/u8:4+events_unbound12:25:193
1775758991714,2cyclictest2059877-21kworker/u8:2+events_unbound12:24:593
1775758991713,3cyclictest1055381-21kworker/u8:3+events_unbound08:00:073
1775754991714,2cyclictest1946433-21cron09:55:002
1775754991714,2cyclictest1823184-21irqcore07:55:182
1775754991714,2cyclictest1807728-21irqcore07:40:142
1775754991713,3cyclictest1983286-21kworker/u8:1+flush-259:011:15:232
1775754991713,3cyclictest1983286-21kworker/u8:1+flush-259:011:10:252
1775749991714,2cyclictest2087118-21sh12:10:001
1775749991713,3cyclictest2029547-21kworker/u8:0+flush-259:011:25:001
1775749991711,4cyclictest1776118-21gpgv07:10:071
1775744991714,2cyclictest1962354-21kworker/u8:3+events_unbound10:20:190
1775744991714,2cyclictest1925825-21kworker/u8:1+flush-259:009:55:060
1775758991613,2cyclictest1848238-21kworker/u8:4+events_unbound08:50:163
1775754991613,2cyclictest1889972-21kworker/u8:2+events_unbound10:20:142
1775754991613,2cyclictest1786439-21sh07:20:002
1775754991611,3cyclictest2061347-21base6411:45:012
1775749991613,2cyclictest1947550-21irqcore09:55:151
1775749991612,3cyclictest1983286-21kworker/u8:1+flush-259:012:00:171
1775749991610,4cyclictest2304165-21systemd-journal08:42:181
1775749991610,4cyclictest1848238-21kworker/u8:4+flush-259:008:50:051
1775744991613,2cyclictest60650irq/133-enp0s29f1:rx-012:31:140
1775744991613,2cyclictest2077429-21irqcore12:00:170
1775744991613,2cyclictest1848238-21kworker/u8:4+events_unbound09:25:250
177574499160,15cyclictest1-21systemd07:34:180
1775754991511,3cyclictest1880044-21kworker/u8:0+flush-259:009:10:002
1775754991511,3cyclictest1796950-21kworker/u8:4+flush-259:008:00:212
1775754991511,3cyclictest1055381-21kworker/u8:3+events_unbound07:45:122
1775749991512,2cyclictest3002-21cron11:55:001
1775749991511,3cyclictest1907760-21sort09:15:161
1775749991511,3cyclictest1821652-21ntp_offset07:50:211
1775744991512,2cyclictest1889972-21kworker/u8:2+flush-259:009:35:160
1775744991511,3cyclictest1983286-21kworker/u8:1+flush-259:012:20:430
1775744991511,3cyclictest1931202-21kworker/u8:0+events_unbound09:50:220
1775754991411,2cyclictest1931202-21kworker/u8:0+flush-259:009:45:232
1775754991411,2cyclictest0-21swapper/210:30:002
1775754991410,3cyclictest1848238-21kworker/u8:4+flush-259:008:20:212
177574999141,3cyclictest1930959-21awk09:40:001
1775749991411,2cyclictest2065562-21ntp_kernel_pll_11:45:221
1775749991411,2cyclictest0-21swapper/109:25:001
1775749991410,3cyclictest1848238-21kworker/u8:4+flush-259:009:10:151
177574499149,3cyclictest1842246-21ntpq08:10:220
1775744991412,2cyclictest60650irq/133-enp0s29f1:rx-007:42:450
1775744991411,2cyclictest60650irq/133-enp0s29f1:rx-011:01:270
1775744991411,2cyclictest60650irq/133-enp0s29f1:rx-010:35:150
1775744991411,2cyclictest1962354-21kworker/u8:3+events_unbound10:40:170
1775744991410,3cyclictest1988214-21kworker/u8:4+flush-259:010:57:230
177575499139,3cyclictest1962354-21kworker/u8:3+flush-259:010:15:192
177574999139,3cyclictest1863875-21kworker/u8:0+flush-259:008:37:481
177574999138,3cyclictest2029790-21apt-get11:15:101
177574999133,2cyclictest1936933-21sed09:45:181
177574999131,1cyclictest0-21swapper/110:30:011
177574499139,3cyclictest2029547-21kworker/u8:0+events_unbound11:45:260
1775744991311,1cyclictest60650irq/133-enp0s29f1:rx-012:12:320
1775744991311,1cyclictest60650irq/133-enp0s29f1:rx-011:40:200
1775744991311,1cyclictest60650irq/133-enp0s29f1:rx-010:02:170
177575899128,3cyclictest1983286-21kworker/u8:1+events_unbound12:15:063
1775754991210,1cyclictest837-21snmpd12:31:292
177574999129,2cyclictest2108023-21cpuspeed_turbos12:30:141
177574999129,2cyclictest1967948-21cpuspeed_turbos10:15:131
177574999129,2cyclictest1962802-21fschecks_count10:10:121
177574999129,2cyclictest1895386-21fschecks_count09:05:141
177574999128,3cyclictest1988214-21kworker/u8:4+flush-259:011:04:491
177574999128,2cyclictest0-21swapper/108:10:161
177574499129,2cyclictest1796950-21kworker/u8:4+events_unbound07:55:290
177574999118,2cyclictest1796295-21unixbench_singl07:25:241
177575499106,3cyclictest1988214-21kworker/u8:4+flush-259:010:55:212
177574999105,4cyclictest2044968-21http11:30:001
177574999103,5cyclictest2101665-21ntpq12:20:221
17757549995,2cyclictest17250irq/130-nvme0q312:20:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional