You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 20:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Sep 23, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1924497995147,3cyclictest2033259-21kworker/u8:0+events_unbound09:20:022
1924494995140,9cyclictest2194818-21apt-get11:30:141
1924497994944,4cyclictest2205557-21apt-get11:40:122
1924492994942,5cyclictest2142054-21latency_hist10:40:020
1924502994743,3cyclictest2189605-21kworker/u8:3+flush-259:011:50:303
1924502994743,3cyclictest2173080-21kworker/u8:0+events_unbound11:08:213
1924497994743,3cyclictest2064574-21kworker/u8:2+events_unbound10:55:252
1924502994642,3cyclictest2064574-21kworker/u8:2+flush-259:010:15:253
1924492994642,3cyclictest2189605-21kworker/u8:3+flush-259:011:50:010
1924497994541,3cyclictest1945739-21kworker/u8:0+events_unbound08:20:132
1924497994541,3cyclictest1929096-21kworker/u8:1+flush-259:008:15:012
1924502994441,2cyclictest2033259-21kworker/u8:0+flush-259:010:00:263
1924502994440,2cyclictest1870726-21kworker/u8:0+events_unbound07:10:173
1924497994441,2cyclictest2199743-21kworker/u8:0+flush-259:011:55:132
1924497994441,2cyclictest2173080-21kworker/u8:0+flush-259:011:10:282
1924497994441,2cyclictest2012801-21kworker/u8:3+flush-259:008:35:252
1924494994237,3cyclictest2137161-21apt-get10:35:121
1924497994138,2cyclictest2226069-21kworker/u8:2+events_unbound12:00:232
1924497994138,2cyclictest2226069-21kworker/u8:2+events_unbound12:00:232
1924502993934,3cyclictest2147405-21gpgv10:45:013
1924497993935,3cyclictest2251905-21kworker/u8:3+flush-259:012:25:202
1924497993935,3cyclictest2189605-21kworker/u8:3+events_unbound11:50:272
1924502993835,2cyclictest2022951-21kworker/u8:2+flush-259:009:10:253
1924494993834,3cyclictest2163375-21kworker/u8:1+flush-259:011:20:171
1924492993830,6cyclictest2094720-21ntpq09:50:250
1924497993734,2cyclictest2064574-21kworker/u8:2+events_unbound09:25:142
1924497993734,2cyclictest2012801-21kworker/u8:3+flush-259:008:50:192
1924497993733,3cyclictest2163375-21kworker/u8:1+flush-259:011:30:272
1924497993731,4cyclictest1954968-21ntpq07:35:232
1924502993632,3cyclictest2033259-21kworker/u8:0+flush-259:010:20:233
1924497993633,2cyclictest2257085-21kworker/u8:4+events_unbound12:30:282
1924497993633,2cyclictest2064574-21kworker/u8:2+events_unbound10:10:182
1924497993633,2cyclictest2022951-21kworker/u8:2+events_unbound09:10:232
1924492993631,3cyclictest2199970-21apt-get11:35:130
1924502993532,2cyclictest2251905-21kworker/u8:3+flush-259:012:25:183
1924502993532,2cyclictest2226069-21kworker/u8:2+flush-259:012:10:183
1924502993531,3cyclictest2267443-21latency_hist12:40:003
1924497993532,2cyclictest2022951-21kworker/u8:2+flush-259:008:55:212
1924494993528,5cyclictest1934358-21ntpq07:15:221
1924502993431,2cyclictest2131929-21kworker/u8:3+flush-259:011:00:213
1924502993427,5cyclictest2047818-21ntpq09:05:223
1924497993431,2cyclictest1929096-21kworker/u8:1+flush-259:007:50:212
1924502993329,3cyclictest2064574-21kworker/u8:2+flush-259:009:30:253
1924502993328,3cyclictest2059499-21base6409:20:003
1924497993330,2cyclictest2033259-21kworker/u8:0+flush-259:009:15:252
1924497993329,3cyclictest1929096-21kworker/u8:1+flush-259:007:35:002
1924494993329,3cyclictest2064574-21kworker/u8:2+flush-259:010:05:231
1924492993328,4cyclictest2168582-21apt-get11:05:110
1924492993328,3cyclictest2221132-21apt-get11:55:120
1924492993328,3cyclictest2178885-21apt-get11:15:140
1924502993229,2cyclictest2033259-21kworker/u8:0+flush-259:009:15:233
1924502993228,3cyclictest1945739-21kworker/u8:0+events_unbound07:50:123
1924497993228,3cyclictest2226069-21kworker/u8:2+events_unbound12:10:252
1924497993228,3cyclictest2131929-21kworker/u8:3+events_unbound10:40:212
1924497993228,3cyclictest2033259-21kworker/u8:0+events_unbound09:35:212
1924497993226,4cyclictest335557-21systemd-journal09:52:422
1924492993228,3cyclictest335557-21systemd-journal09:48:250
1924492993228,3cyclictest335557-21systemd-journal09:48:250
1924492993228,3cyclictest2173080-21kworker/u8:0+flush-259:011:10:210
1924492993228,3cyclictest1987020-21apt-get08:10:110
1924497993128,2cyclictest2126569-21kworker/u8:1+flush-259:010:30:262
1924497993128,2cyclictest2033259-21kworker/u8:0+events_unbound09:40:172
1924497993127,3cyclictest2033259-21kworker/u8:0+events_unbound09:05:232
1924494993126,3cyclictest335557-21systemd-journal08:21:421
1924502993027,2cyclictest2033259-21kworker/u8:0+flush-259:009:35:163
1924502993027,2cyclictest2022951-21kworker/u8:2+flush-259:008:45:213
1924502993026,3cyclictest2226069-21kworker/u8:2+flush-259:012:20:203
1924497993027,2cyclictest2163375-21kworker/u8:1+flush-259:012:05:422
1924497993027,2cyclictest2163375-21kworker/u8:1+flush-259:011:15:202
1924497993027,2cyclictest2064574-21kworker/u8:2+events_unbound10:00:142
1924497993027,2cyclictest1945739-21kworker/u8:0+events_unbound08:05:202
1924497993027,2cyclictest1935434-21kworker/u8:3+flush-259:007:20:242
1924497993026,3cyclictest2163375-21kworker/u8:1+events_unbound11:38:442
1924497993026,3cyclictest2064574-21kworker/u8:2+events_unbound10:21:432
1924497993026,3cyclictest2022951-21kworker/u8:2+events_unbound09:00:222
1924502992925,3cyclictest1945739-21kworker/u8:0+flush-259:007:40:263
1924502992925,3cyclictest1929096-21kworker/u8:1+flush-259:007:25:213
1924502992925,3cyclictest1929096-21kworker/u8:1+flush-259:007:25:203
1924502992924,3cyclictest2064874-21apt-get09:25:113
1924497992926,2cyclictest2142073-21kworker/u8:0+flush-259:010:50:162
1924497992926,2cyclictest2064574-21kworker/u8:2+events_unbound10:50:002
1924497992926,2cyclictest1945739-21kworker/u8:0+flush-259:008:15:202
1924497992926,2cyclictest1929096-21kworker/u8:1+flush-259:007:25:192
1924497992926,2cyclictest1929096-21kworker/u8:1+flush-259:007:25:192
1924492992924,3cyclictest1945739-21kworker/u8:0+flush-259:008:05:190
1924502992823,3cyclictest1945739-21kworker/u8:0+flush-259:008:05:003
1924497992825,2cyclictest2163375-21kworker/u8:1+flush-259:012:15:202
1924497992825,2cyclictest2064574-21kworker/u8:2+events_unbound09:30:192
1924497992823,3cyclictest1956030-21/usr/sbin/munin07:40:192
1924494992825,2cyclictest1997296-21kworker/u8:2+events_unbound08:30:221
1924494992824,3cyclictest2147867-21/usr/sbin/munin10:45:141
1924494992824,3cyclictest1929096-21kworker/u8:1+flush-259:007:49:591
1924502992724,2cyclictest2163375-21kworker/u8:1+flush-259:011:15:253
1924502992724,2cyclictest2064574-21kworker/u8:2+events_unbound10:55:013
1924502992724,2cyclictest1945739-21kworker/u8:0+events_unbound08:10:153
1924502992723,3cyclictest2189605-21kworker/u8:3+flush-259:011:45:203
1924502992722,3cyclictest2033259-21kworker/u8:0+flush-259:010:10:153
1924497992724,2cyclictest2033259-21kworker/u8:0+events_unbound09:55:182
1924494992721,4cyclictest1981607-21latency_hist08:05:011
1924492992721,4cyclictest2063693-21ntpq09:20:210
1924502992623,2cyclictest1997296-21kworker/u8:2+flush-259:008:20:163
1924502992621,4cyclictest2199709-21latency_hist11:35:003
1924502992621,4cyclictest1930448-21diskstats07:15:143
1924497992623,2cyclictest2163375-21kworker/u8:1+events_unbound11:25:432
1924497992623,2cyclictest1929096-21kworker/u8:1+events_unbound07:47:592
1924497992623,2cyclictest1929096-21kworker/u8:1+events_unbound07:14:592
1924494992621,3cyclictest509-21rs:main1
1924494992620,4cyclictest2043544-21latency_hist09:05:001
1924502992522,2cyclictest2064574-21kworker/u8:2+events_unbound09:55:003
1924502992521,3cyclictest2163375-21kworker/u8:1+flush-259:011:20:203
1924502992521,3cyclictest2163375-21kworker/u8:1+events_unbound11:40:283
1924502992520,3cyclictest2064574-21kworker/u8:2+flush-259:010:05:153
1924497992522,2cyclictest2163375-21kworker/u8:1+events_unbound11:20:262
1924497992522,2cyclictest2131929-21kworker/u8:3+flush-259:011:00:182
1924497992522,2cyclictest1997296-21kworker/u8:2+events_unbound08:25:202
1924497992521,3cyclictest2007420-21kworker/u8:1+events_unbound08:41:442
1924497992521,3cyclictest1929096-21kworker/u8:1+events_unbound07:15:152
1924494992521,3cyclictest335557-21systemd-journal10:28:431
1924494992519,4cyclictest2158230-21/usr/sbin/munin10:55:151
1924492992521,3cyclictest335480-21sshd12:26:410
1924502992421,2cyclictest2226069-21kworker/u8:2+events_unbound12:00:193
1924502992421,2cyclictest2226069-21kworker/u8:2+events_unbound12:00:183
1924502992421,2cyclictest2022951-21kworker/u8:2+events_unbound09:03:163
1924502992421,2cyclictest1929096-21kworker/u8:1+events_unbound07:45:273
1924497992421,2cyclictest2064574-21kworker/u8:2+flush-259:010:15:212
1924494992420,3cyclictest2210733-21apt-get11:45:121
1924494992419,3cyclictest2048912-21/usr/sbin/munin09:10:151
1924494992418,4cyclictest2028507-21diskstats08:50:151
192449499240,22cyclictest1-21systemd08:40:011
1924492992420,3cyclictest2142073-21kworker/u8:0+flush-259:010:50:220
1924502992320,2cyclictest1935434-21kworker/u8:3+events_unbound07:20:213
1924502992319,3cyclictest1945739-21kworker/u8:0+events_unbound08:05:443
1924497992320,2cyclictest2163375-21kworker/u8:1+flush-259:011:05:182
1924492992320,2cyclictest2163375-21kworker/u8:1+flush-259:011:40:170
1924492992320,2cyclictest2147231-21gpgv10:45:000
1924492992317,4cyclictest335557-21systemd-journal10:35:000
1924502992219,2cyclictest2189605-21kworker/u8:3+flush-259:011:35:163
1924502992219,2cyclictest2142073-21kworker/u8:0+events_unbound10:45:183
1924502992219,2cyclictest1929096-21kworker/u8:1+flush-259:007:35:243
1924502992217,3cyclictest1929096-21kworker/u8:1+flush-259:007:55:183
1924497992219,2cyclictest2189605-21kworker/u8:3+flush-259:011:45:182
1924497992219,2cyclictest1945739-21kworker/u8:0+flush-259:008:00:162
1924494992218,3cyclictest335557-21systemd-journal12:15:011
1924494992218,3cyclictest335557-21systemd-journal12:03:501
1924494992218,3cyclictest335557-21systemd-journal12:03:491
1924494992218,3cyclictest2033259-21kworker/u8:0+flush-259:010:25:001
1924494992217,3cyclictest2111337-21apt-get10:10:151
192449499221,19cyclictest1-21systemd08:15:421
1924492992219,1cyclictest1-21systemd08:21:420
1924492992217,4cyclictest2064574-21kworker/u8:2+flush-259:009:25:250
1924492992215,5cyclictest335557-21systemd-journal07:40:000
1924502992118,2cyclictest2257085-21kworker/u8:4+events_unbound12:30:203
1924502992118,2cyclictest2173080-21kworker/u8:0+events_unbound11:10:293
1924502992118,2cyclictest2064574-21kworker/u8:2+events_unbound09:42:423
1924502992117,3cyclictest2064574-21kworker/u8:2+events_unbound10:38:433
1924497992118,2cyclictest2226069-21kworker/u8:2+events_unbound12:20:072
1924497992115,4cyclictest335557-21systemd-journal08:34:422
1924494992117,3cyclictest1935434-21kworker/u8:3+flush-259:007:20:211
1924494992116,4cyclictest2157988-21latency_hist10:55:021
1924492992116,3cyclictest335557-21systemd-journal09:40:000
1924492992116,3cyclictest335557-21systemd-journal08:02:420
1924502992017,2cyclictest2163375-21kworker/u8:1+flush-259:011:55:193
1924502992017,2cyclictest2163375-21kworker/u8:1+events_unbound11:25:213
1924502992017,2cyclictest2033259-21kworker/u8:0+flush-259:010:25:243
1924502992016,3cyclictest2012801-21kworker/u8:3+events_unbound08:39:433
1924502992015,3cyclictest2038387-21turbostat09:00:003
1924494992017,2cyclictest2226069-21kworker/u8:2+flush-259:012:20:001
1924494992015,4cyclictest1945739-21kworker/u8:0+flush-259:008:10:281
1924494992015,3cyclictest335557-21systemd-journal08:45:421
1924492992015,4cyclictest335557-21systemd-journal08:15:420
1924492992015,3cyclictest2038650-21apt-get09:00:120
1924502991916,2cyclictest2033259-21kworker/u8:0+flush-259:009:55:153
1924502991916,2cyclictest1945739-21kworker/u8:0+flush-259:008:30:163
1924497991916,2cyclictest2033259-21kworker/u8:0+events_unbound10:27:442
1924497991916,2cyclictest2022951-21kworker/u8:2+events_unbound08:45:112
192449499191,16cyclictest335557-21systemd-journal10:04:551
1924492991915,3cyclictest2257085-21kworker/u8:4+flush-259:012:30:290
1924494991815,2cyclictest2033259-21kworker/u8:0+flush-259:009:55:181
1924494991814,3cyclictest2226069-21kworker/u8:2+flush-259:012:05:231
1924494991813,4cyclictest2116472-21/usr/sbin/munin10:15:191
1924502991714,2cyclictest1945739-21kworker/u8:0+events_unbound08:15:233
1924497991713,3cyclictest1929096-21kworker/u8:1+events_unbound07:58:432
1924494991713,3cyclictest1924717-21apt-get07:10:121
1924494991712,3cyclictest2017968-21/usr/sbin/munin08:40:131
1924492991714,2cyclictest61150irq/133-enp0s29f1:tx-010:47:420
1924492991714,2cyclictest61150irq/133-enp0s29f1:tx-008:29:420
1924492991714,2cyclictest60350irq/132-enp0s29f1:rx-012:35:210
1924492991714,2cyclictest60350irq/132-enp0s29f1:rx-012:20:130
1924492991714,2cyclictest60350irq/132-enp0s29f1:rx-012:11:290
1924492991713,3cyclictest2163375-21kworker/u8:1+flush-259:011:30:190
1924492991713,3cyclictest2158255-21apt-get10:55:120
1924492991713,3cyclictest1929096-21kworker/u8:1+flush-259:007:45:120
1924492991712,3cyclictest335557-21systemd-journal07:52:420
1924492991711,5cyclictest1939520-21ntpq07:20:200
1924492991711,4cyclictest2038385-21latency_hist09:00:000
192449299171,14cyclictest1-21systemd11:29:590
1924502991613,2cyclictest2033259-21kworker/u8:0+flush-259:009:45:443
1924502991613,2cyclictest2033259-21kworker/u8:0+flush-259:009:45:443
1924502991613,2cyclictest2012801-21kworker/u8:3+flush-259:008:40:163
192449499169,5cyclictest2226046-21latency_hist12:00:001
1924494991614,1cyclictest2163831-21irqcore11:00:161
1924494991612,3cyclictest2189605-21kworker/u8:3+flush-259:011:45:001
1924494991612,3cyclictest2131929-21kworker/u8:3+flush-259:010:40:231
1924494991612,3cyclictest2022951-21kworker/u8:2+flush-259:008:56:481
1924494991610,4cyclictest2059675-21apt-get09:20:121
1924492991614,1cyclictest60350irq/132-enp0s29f1:rx-009:35:200
1924492991614,1cyclictest60350irq/132-enp0s29f1:rx-007:40:240
1924492991613,2cyclictest60350irq/132-enp0s29f1:rx-010:10:000
1924492991613,2cyclictest60350irq/132-enp0s29f1:rx-008:35:200
1924492991613,2cyclictest60350irq/132-enp0s29f1:rx-007:55:130
1924492991613,2cyclictest60350irq/132-enp0s29f1:rx-007:32:040
1924492991613,2cyclictest60350irq/132-enp0s29f1:rx-007:25:150
1924492991613,2cyclictest60350irq/132-enp0s29f1:rx-007:25:150
1924492991612,3cyclictest2188979-21gpgv11:25:000
1924502991512,2cyclictest2163375-21kworker/u8:1+events_unbound12:15:183
1924502991512,2cyclictest1929096-21kworker/u8:1+events_unbound07:30:183
1924502991511,3cyclictest2064574-21kworker/u8:2+flush-259:010:30:103
1924497991513,2cyclictest2033259-21kworker/u8:0+events_unbound10:07:422
1924497991512,2cyclictest2064574-21kworker/u8:2+events_unbound09:47:012
1924497991512,2cyclictest2064574-21kworker/u8:2+events_unbound09:47:012
1924497991511,3cyclictest2131929-21kworker/u8:3+events_unbound10:37:432
1924494991512,2cyclictest2174133-21irqcore11:10:211
1924494991512,2cyclictest2054534-21irqcore09:15:171
1924494991512,2cyclictest0-21swapper/107:25:161
1924494991512,2cyclictest0-21swapper/107:25:161
1924494991511,3cyclictest2064574-21kworker/u8:2+flush-259:009:30:271
1924494991511,3cyclictest1945739-21kworker/u8:0+flush-259:007:53:051
1924494991510,4cyclictest2262286-21latency_hist12:35:001
1924492991513,1cyclictest60350irq/132-enp0s29f1:rx-009:15:160
1924492991513,1cyclictest60350irq/132-enp0s29f1:rx-008:50:210
1924492991513,1cyclictest60350irq/132-enp0s29f1:rx-007:17:120
1924492991512,2cyclictest60350irq/132-enp0s29f1:rx-012:15:210
1924492991512,2cyclictest60350irq/132-enp0s29f1:rx-012:02:560
1924492991512,2cyclictest60350irq/132-enp0s29f1:rx-012:02:550
1924502991411,2cyclictest2142073-21kworker/u8:0+events_unbound10:57:213
1924502991410,2cyclictest2012801-21kworker/u8:3+events_unbound08:50:203
192449499149,4cyclictest1945775-21apt-get07:30:121
192449499141,3cyclictest2220875-21awk11:55:001
1924494991412,1cyclictest0-21swapper/112:30:011
1924494991410,3cyclictest2189605-21kworker/u8:3+flush-259:011:39:121
192449299149,4cyclictest335557-21systemd-journal08:50:010
1924492991412,1cyclictest60350irq/132-enp0s29f1:rx-012:06:030
1924492991412,1cyclictest60350irq/132-enp0s29f1:rx-011:50:070
1924492991412,1cyclictest60350irq/132-enp0s29f1:rx-010:25:130
1924492991412,1cyclictest60350irq/132-enp0s29f1:rx-010:15:160
1924492991412,1cyclictest60350irq/132-enp0s29f1:rx-009:55:040
1924492991412,1cyclictest60350irq/132-enp0s29f1:rx-008:30:270
192450299139,3cyclictest1945739-21kworker/u8:0+events_unbound08:25:123
1924502991310,2cyclictest2226069-21kworker/u8:2+events_unbound12:05:263
192449799139,3cyclictest2251905-21kworker/u8:3+flush-259:012:35:452
1924494991310,1cyclictest0-21swapper/112:40:011
1924492991312,1cyclictest60350irq/132-enp0s29f1:rx-010:14:490
1924492991311,2cyclictest61150irq/133-enp0s29f1:tx-011:00:230
1924492991311,1cyclictest60350irq/132-enp0s29f1:rx-009:06:300
1924492991311,1cyclictest60350irq/132-enp0s29f1:rx-008:43:370
192449499129,1cyclictest0-21swapper/111:25:181
192449499128,3cyclictest2064574-21kworker/u8:2+flush-259:009:25:251
1924492991210,1cyclictest60350irq/132-enp0s29f1:rx-007:10:230
192449299120,4cyclictest2048912-21/usr/sbin/munin09:10:200
192449299119,2cyclictest60350irq/132-enp0s29f1:rx-010:20:240
192449499101,7cyclictest2172520-21ntpq11:05:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional