You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 17:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Mon Nov 24, 2025 12:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1933405993429,4cyclictest1985657-21kworker/u8:0+flush-259:008:10:162
1933404993328,4cyclictest2078268-21latency_hist09:30:001
1933403993126,3cyclictest2119531-21kworker/u8:5+flush-259:010:10:190
1933406993023,5cyclictest2266895-21awk12:30:003
1933405993026,3cyclictest1938817-21kworker/u8:2+flush-259:007:45:182
1933405992925,3cyclictest2698871-21systemd-journal12:25:002
1933406992825,2cyclictest2214672-21kworker/u8:1+events_unbound12:15:003
1933406992724,2cyclictest2125975-21kworker/u8:4+events_unbound10:20:203
1933406992722,4cyclictest1994772-21kworker/u8:2+events_unbound08:25:173
1933403992721,4cyclictest2698871-21systemd-journal10:25:000
1933403992721,4cyclictest2041212-21ntpq08:50:190
1933403992721,4cyclictest2016620-21kworker/u8:1+flush-259:008:30:170
1933406992623,2cyclictest2199112-21kworker/u8:2+flush-259:011:25:153
1933406992623,2cyclictest2047904-21kworker/u8:4+flush-259:009:10:003
1933403992622,3cyclictest2225615-21apt-get11:50:120
1933403992621,3cyclictest2078280-21apt-get09:30:000
1933406992522,2cyclictest2147379-21kworker/u8:0+events_unbound10:35:173
1933406992522,2cyclictest2068189-21kworker/u8:3+flush-259:009:40:223
1933406992521,3cyclictest2266928-21kworker/u8:0+events_unbound12:35:203
1933406992521,3cyclictest2199112-21kworker/u8:2+events_unbound12:00:193
1933406992521,3cyclictest2000500-21kworker/u8:3+flush-259:008:25:003
1933406992518,5cyclictest1975095-21latency_hist07:50:013
1933405992522,2cyclictest2131459-21gpgv10:20:012
1933405992521,3cyclictest2167435-21kworker/u8:3+flush-259:010:55:172
1933404992520,4cyclictest2084117-21/usr/sbin/munin09:35:161
1933404992520,3cyclictest2172842-21apt-get11:00:121
1933403992521,3cyclictest2078956-21apt-get09:30:110
1933403992521,3cyclictest2078956-21apt-get09:30:110
193340399251,22cyclictest1-21systemd11:50:010
1933406992420,3cyclictest2141874-21kworker/u8:1+flush-259:010:30:203
1933405992418,4cyclictest2251070-21apt-get12:15:002
1933403992418,4cyclictest2698871-21systemd-journal11:34:210
1933403992418,4cyclictest2698871-21systemd-journal09:10:390
1933406992320,2cyclictest2084113-21kworker/u8:1+events_unbound09:35:153
1933406992319,3cyclictest2089253-21kworker/u8:2+flush-259:009:50:153
1933406992317,4cyclictest2698871-21systemd-journal09:00:003
1933406992316,5cyclictest2698871-21systemd-journal08:20:013
1933405992319,3cyclictest2000500-21kworker/u8:3+flush-259:008:35:212
1933404992320,2cyclictest2199112-21kworker/u8:2+flush-259:012:00:001
1933406992218,3cyclictest2167435-21kworker/u8:3+events_unbound10:55:213
1933406992218,3cyclictest2157132-21kworker/u8:2+events_unbound10:50:123
1933406992218,3cyclictest1913028-21kworker/u8:1+events_unbound07:25:173
1933405992220,1cyclictest2199122-21latency_hist11:25:012
1933405992216,4cyclictest2698871-21systemd-journal08:54:202
1933404992219,2cyclictest2078923-21kworker/u8:0+events_unbound09:45:251
1933403992215,5cyclictest2182875-21latency_hist11:10:010
1933406992118,2cyclictest2266928-21kworker/u8:0+events_unbound12:33:223
1933406992118,2cyclictest2041297-21kworker/u8:1+events_unbound09:10:193
1933406992117,3cyclictest1980488-21kworker/u8:1+flush-259:008:05:163
1933406992117,3cyclictest1970141-21kworker/u8:4+events_unbound07:50:193
1933406992116,3cyclictest2119531-21kworker/u8:5+flush-259:011:19:593
1933405992118,1cyclictest0-21swapper/208:45:002
1933404992117,3cyclictest1938856-21apt-get07:15:121
1933403992117,3cyclictest1980521-21apt-get07:55:120
1933406992017,2cyclictest3068800-21kworker/u8:0+flush-259:007:20:183
1933406992017,2cyclictest2147379-21kworker/u8:0+events_unbound10:40:233
1933406992017,2cyclictest2125975-21kworker/u8:4+flush-259:010:30:003
1933406992017,2cyclictest2000500-21kworker/u8:3+flush-259:008:50:203
1933406992016,3cyclictest1980488-21kworker/u8:1+events_unbound08:00:013
1933406992014,5cyclictest2119531-21kworker/u8:5+flush-259:011:09:593
1933405992017,2cyclictest2220019-21gpgv11:45:002
1933405992016,3cyclictest2104763-21apt-get09:55:122
1933405992016,3cyclictest2068189-21kworker/u8:3+flush-259:010:05:002
1933405992015,4cyclictest1995723-21latency_hist08:10:002
1933403992016,3cyclictest2251751-21apt-get12:15:100
1933403992016,3cyclictest2199112-21kworker/u8:2+flush-259:012:00:150
1933406991917,1cyclictest0-21swapper/307:44:593
1933406991915,3cyclictest2214672-21kworker/u8:1+events_unbound11:58:203
1933406991915,3cyclictest2199112-21kworker/u8:2+flush-259:011:35:233
1933406991915,3cyclictest2199112-21kworker/u8:2+events_unbound12:06:153
1933406991915,3cyclictest2000500-21kworker/u8:3+events_unbound09:04:223
1933406991915,3cyclictest1970141-21kworker/u8:4+flush-259:007:55:153
1933406991915,2cyclictest2199112-21kworker/u8:2+flush-259:011:50:163
1933404991915,3cyclictest2199112-21kworker/u8:2+flush-259:011:30:001
1933404991913,5cyclictest2109681-21latency_hist10:00:011
1933403991914,4cyclictest2125922-21apt-get10:15:090
1933403991913,4cyclictest2698871-21systemd-journal10:54:200
1933406991815,2cyclictest2084113-21kworker/u8:1+flush-259:009:50:003
1933406991814,3cyclictest2047904-21kworker/u8:4+events_unbound09:30:023
1933406991814,3cyclictest2047904-21kworker/u8:4+events_unbound09:30:023
1933405991814,3cyclictest2214672-21kworker/u8:1+flush-259:012:25:172
1933404991815,2cyclictest3000063-21rs:main1
1933404991815,2cyclictest2147379-21kworker/u8:0+flush-259:010:40:261
1933404991815,2cyclictest1985657-21kworker/u8:0+flush-259:008:30:011
1933403991815,2cyclictest2067942-21cron09:20:000
1933403991814,3cyclictest2245928-21kworker/u8:3+flush-259:012:30:010
1933403991812,4cyclictest2698871-21systemd-journal08:37:200
193340699179,6cyclictest2209486-21latency_hist11:35:003
193340699179,6cyclictest1999966-21ntpq08:10:203
1933406991714,2cyclictest2078923-21kworker/u8:0+flush-259:010:15:003
1933406991714,2cyclictest2016620-21kworker/u8:1+events_unbound08:30:183
1933406991713,3cyclictest2245928-21kworker/u8:3+events_unbound12:15:123
1933406991713,3cyclictest2041297-21kworker/u8:1+events_unbound09:25:223
1933406991713,3cyclictest2031863-21kworker/u8:0+events_unbound08:48:203
1933406991713,3cyclictest1933385-21kworker/u8:3+flush-259:007:15:153
1933405991714,2cyclictest3002-21cron10:15:002
1933405991714,2cyclictest2241459-21sed12:05:162
1933405991714,2cyclictest2147379-21kworker/u8:0+events_unbound10:35:232
1933405991714,2cyclictest2119531-21kworker/u8:5+flush-259:010:50:252
1933405991714,2cyclictest2031863-21kworker/u8:0+flush-259:009:15:002
1933405991714,2cyclictest2016620-21kworker/u8:1+flush-259:008:30:212
1933405991713,3cyclictest2245928-21kworker/u8:3+flush-259:012:15:272
1933405991713,2cyclictest0-21swapper/209:05:002
1933405991712,4cyclictest2698871-21systemd-journal08:05:012
1933405991712,4cyclictest2204312-21latency_hist11:30:002
1933404991715,2cyclictest1913028-21kworker/u8:1+flush-259:007:35:241
1933404991714,2cyclictest2125975-21kworker/u8:4+events_unbound10:25:181
1933404991713,3cyclictest2078787-21cat09:30:011
1933404991713,3cyclictest2078787-21cat09:30:011
1933404991713,3cyclictest2006314-21/usr/sbin/munin08:20:161
1933404991712,4cyclictest1970339-21diskstats07:45:131
1933403991712,3cyclictest2141874-21kworker/u8:1+flush-259:010:30:260
193340699169,5cyclictest2031846-21latency_hist08:45:003
1933406991613,2cyclictest2119531-21kworker/u8:5+flush-259:010:45:323
1933406991613,2cyclictest1933385-21kworker/u8:3+events_unbound07:30:153
1933406991612,3cyclictest1933385-21kworker/u8:3+events_unbound07:10:243
1933406991612,3cyclictest1913028-21kworker/u8:1+events_unbound07:39:563
1933406991611,4cyclictest2063012-21apt-get09:15:113
1933405991613,2cyclictest2078923-21kworker/u8:0+flush-259:009:40:002
1933405991613,2cyclictest2054985-21kthreadcore09:05:192
1933405991612,3cyclictest2078923-21kworker/u8:0+flush-259:009:45:192
1933405991611,3cyclictest2193791-21cat11:20:002
1933404991613,2cyclictest2199112-21kworker/u8:2+flush-259:011:35:171
1933404991613,2cyclictest2166972-21unixbench_singl10:50:241
1933404991613,2cyclictest2119531-21kworker/u8:5+flush-259:011:05:251
1933404991613,2cyclictest1933385-21kworker/u8:3+events_unbound07:55:001
1933403991614,1cyclictest60650irq/133-enp0s29f1:rx-011:35:070
1933403991614,1cyclictest60650irq/133-enp0s29f1:rx-009:55:120
1933403991614,1cyclictest60650irq/133-enp0s29f1:rx-007:50:120
1933403991614,1cyclictest60650irq/133-enp0s29f1:rx-007:37:250
1933403991613,2cyclictest60650irq/133-enp0s29f1:rx-011:20:010
1933403991613,2cyclictest60650irq/133-enp0s29f1:rx-011:00:220
1933403991613,2cyclictest60650irq/133-enp0s29f1:rx-010:43:060
1933403991612,3cyclictest3000061-21in:imuxsock12:40:000
1933403991612,3cyclictest2125975-21kworker/u8:4+flush-259:010:25:160
1933403991612,3cyclictest0-21swapper/010:50:000
1933403991611,3cyclictest2267168-21apt-get12:30:110
1933406991514,1cyclictest2000500-21kworker/u8:3+events_unbound08:35:233
1933406991512,2cyclictest2104728-21kworker/u8:4+events_unbound09:55:123
1933406991512,2cyclictest2047904-21kworker/u8:4+events_unbound09:20:193
1933406991511,3cyclictest2104728-21kworker/u8:4+events_unbound10:00:203
193340599159,5cyclictest2036108-21ntpq08:45:212
193340599159,4cyclictest2157118-21latency_hist10:45:012
1933405991513,1cyclictest0-21swapper/211:55:002
1933405991512,2cyclictest2219379-21unixbench_singl11:40:252
1933404991513,1cyclictest0-21swapper/110:25:001
1933404991512,2cyclictest2245928-21kworker/u8:3+flush-259:012:10:191
1933404991512,2cyclictest2214672-21kworker/u8:1+flush-259:011:45:001
1933404991512,2cyclictest2084113-21kworker/u8:1+flush-259:009:40:171
1933404991511,3cyclictest2240773-21cron12:05:011
1933404991510,4cyclictest2698871-21systemd-journal09:54:201
1933403991513,1cyclictest60650irq/133-enp0s29f1:rx-010:05:260
1933403991513,1cyclictest60650irq/133-enp0s29f1:rx-010:00:200
1933403991513,1cyclictest60650irq/133-enp0s29f1:rx-009:05:260
1933403991513,1cyclictest60650irq/133-enp0s29f1:rx-007:32:550
1933403991512,2cyclictest61450irq/134-enp0s29f1:tx-007:20:220
1933403991512,2cyclictest60650irq/133-enp0s29f1:rx-011:55:200
1933403991512,2cyclictest60650irq/133-enp0s29f1:rx-011:10:250
1933403991512,2cyclictest60650irq/133-enp0s29f1:rx-007:10:160
1933406991410,3cyclictest2193797-21apt-key11:20:003
1933406991410,3cyclictest2182882-21kworker/u8:0+events_unbound11:48:223
1933406991410,3cyclictest2182882-21kworker/u8:0+events_unbound11:10:193
1933405991411,2cyclictest2240377-21unixbench_singl12:00:242
1933405991411,2cyclictest2178090-21cpuspeed_turbos11:05:142
1933405991411,2cyclictest2093195-21needreboot09:40:202
1933405991410,3cyclictest2698871-21systemd-journal08:22:212
193340499148,5cyclictest2061889-21ntpq09:10:221
1933404991412,1cyclictest0-21swapper/108:00:011
1933404991411,2cyclictest2226024-21irqcore11:50:171
1933404991411,2cyclictest2119531-21kworker/u8:5+flush-259:011:15:251
1933404991411,2cyclictest2068189-21kworker/u8:3+flush-259:010:05:261
1933403991413,1cyclictest60650irq/133-enp0s29f1:rx-008:20:240
1933403991413,1cyclictest60650irq/133-enp0s29f1:rx-008:12:390
1933403991412,1cyclictest60650irq/133-enp0s29f1:rx-010:38:050
1933403991412,1cyclictest60650irq/133-enp0s29f1:rx-008:16:330
1933403991412,1cyclictest0-21swapper/008:55:160
1933403991411,2cyclictest61450irq/134-enp0s29f1:tx-011:44:200
1933403991411,2cyclictest60650irq/133-enp0s29f1:rx-007:25:140
1933403991410,3cyclictest2072640-21unixbench_multi09:20:250
1933403991410,2cyclictest2104503-21turbostat09:55:000
1933406991310,2cyclictest2259180-21kthreadcore12:20:183
1933406991310,2cyclictest2115097-21kworker/u8:1+events_unbound10:08:223
1933406991310,2cyclictest0-21swapper/311:04:593
193340599139,3cyclictest2068189-21kworker/u8:3+flush-259:009:30:162
193340599139,3cyclictest2068189-21kworker/u8:3+flush-259:009:30:162
1933405991311,1cyclictest2144221-21kthreadcore10:30:192
1933405991311,1cyclictest0-21swapper/211:15:002
193340499139,3cyclictest2256957-21cpuspeed_turbos12:20:131
193340499139,3cyclictest1985657-21kworker/u8:0+flush-259:008:00:251
193340499139,3cyclictest1913028-21kworker/u8:1+flush-259:007:43:251
193340499138,4cyclictest2698871-21systemd-journal09:20:431
193340499138,4cyclictest2125240-21latency_hist10:15:011
1933404991310,2cyclictest2000500-21kworker/u8:3+flush-259:008:55:001
1933404991310,2cyclictest1990912-21cpuspeed_turbos08:05:141
1933404991310,2cyclictest1935977-21kthreadcore07:10:181
193340399139,3cyclictest60650irq/133-enp0s29f1:rx-008:25:140
193340399138,3cyclictest1985684-21apt-get08:00:110
193340399137,5cyclictest2241050-21apt-get12:05:110
1933403991311,1cyclictest60650irq/133-enp0s29f1:rx-012:14:550
1933403991311,1cyclictest60650irq/133-enp0s29f1:rx-011:26:010
1933403991311,1cyclictest60650irq/133-enp0s29f1:rx-010:55:190
1933403991311,1cyclictest60650irq/133-enp0s29f1:rx-007:44:240
1933403991310,2cyclictest60650irq/133-enp0s29f1:rx-008:46:270
1933403991310,2cyclictest60650irq/133-enp0s29f1:rx-008:40:160
193340699129,2cyclictest2215933-21kthreadcore11:40:173
193340599129,2cyclictest1951375-21kthreadcore07:25:182
193340599128,3cyclictest2698871-21systemd-journal07:35:012
193340599127,4cyclictest2103572-21ntp_09:50:212
1933405991210,1cyclictest1933385-21kworker/u8:3+events_unbound07:10:132
193340499129,2cyclictest2267584-21irqcore12:30:161
193340499129,2cyclictest2141874-21kworker/u8:1+flush-259:010:30:171
193340499129,2cyclictest2104728-21kworker/u8:4+flush-259:010:00:261
193340499129,2cyclictest1994772-21kworker/u8:2+flush-259:008:40:261
193340499127,3cyclictest516-21gdbus07:30:001
193340499127,3cyclictest2698871-21systemd-journal10:35:201
193340399129,2cyclictest2047585-21cpuspeed_turbos09:00:120
1933403991210,1cyclictest60650irq/133-enp0s29f1:rx-011:15:150
1933403991210,1cyclictest60650irq/133-enp0s29f1:rx-009:35:250
193340699117,3cyclictest2125891-21kworker/u8:2+events_unbound10:18:553
193340599119,1cyclictest2047110-21runrttasks08:58:202
193340599119,1cyclictest0-21swapper/212:35:012
193340599118,2cyclictest2212050-21kthreadcore11:35:182
193340599118,2cyclictest1948900-21munin-plugin-st07:25:002
193340599118,2cyclictest0-21swapper/207:50:202
193340599117,3cyclictest2231177-21irqcore11:55:172
193340599117,2cyclictest1954969-21sort07:30:162
193340499119,1cyclictest0-21swapper/112:20:001
193340399118,2cyclictest60650irq/133-enp0s29f1:rx-009:45:140
193340399118,2cyclictest60650irq/133-enp0s29f1:rx-007:19:000
193340599107,2cyclictest2157446-21cpuspeed_turbos10:45:142
193340499108,1cyclictest0-21swapper/111:15:001
193340399106,3cyclictest2257089-21fschecks_count12:20:140
19334049997,1cyclictest0-21swapper/108:35:191
19334049996,2cyclictest2119531-21kworker/u8:5+flush-259:010:45:171
19334049994,4cyclictest2698871-21systemd-journal09:05:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional