You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 06:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Tue Apr 21, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2064551994336,5cyclictest252-21systemd-journal22:30:013
2064543994340,2cyclictest2194423-21kworker/u8:2+flush-259:021:20:211
2064543994239,2cyclictest2194423-21kworker/u8:2+events_unbound23:25:251
2064543994138,2cyclictest2235831-21kworker/u8:1+flush-259:022:00:161
2064543994138,2cyclictest2142555-21kworker/u8:1+flush-259:021:25:251
2064543994138,2cyclictest2090343-21kworker/u8:4+flush-259:020:10:221
2064547994033,5cyclictest252-21systemd-journal20:40:002
2064543994037,2cyclictest2194423-21kworker/u8:2+events_unbound23:35:241
2064543994037,2cyclictest2032048-21kworker/u8:2+flush-259:019:35:211
2064543994037,2cyclictest2016610-21kworker/u8:1+flush-259:019:25:011
2064543994030,2cyclictest2272770-21kworker/u8:3+events_unbound22:35:241
2064543993936,2cyclictest2309274-21kworker/u8:1+events_unbound23:15:241
2064543993936,2cyclictest2173771-21kworker/u8:3+flush-259:021:20:011
2064541993925,3cyclictest2324792-21apt-get23:20:120
2064551993833,3cyclictest2163500-21/usr/sbin/munin20:45:143
2064543993734,2cyclictest2345064-21kworker/u8:0+events_unbound00:20:231
2064543993733,3cyclictest2209600-21kworker/u8:3+flush-259:021:55:011
2064543993633,2cyclictest2392097-21kworker/u8:1+events_unbound00:30:181
2064551993530,3cyclictest2246369-21/usr/sbin/munin22:05:203
2064543993532,2cyclictest2132636-21kworker/u8:2+events_unbound20:20:191
2064543993531,3cyclictest2178987-21apt-get21:00:111
2064543993529,4cyclictest252-21systemd-journal19:18:151
2064543993430,2cyclictest2064493-21kworker/u8:0+events_unbound19:10:201
2064543993330,2cyclictest2235831-21kworker/u8:1+events_unbound22:05:171
2064543993329,3cyclictest2194423-21kworker/u8:2+flush-259:021:50:001
2064543993329,3cyclictest2142555-21kworker/u8:1+flush-259:020:35:251
2064551993229,2cyclictest2183701-21kworker/u8:0+flush-259:021:05:263
2064543993229,2cyclictest2329688-21kworker/u8:3+events_unbound23:30:151
2064543993229,2cyclictest2194423-21kworker/u8:2+flush-259:000:15:211
2064543993229,2cyclictest2147844-21kworker/u8:0+events_unbound20:30:241
2064543993229,2cyclictest2032048-21kworker/u8:2+events_unbound19:40:151
2064543993228,3cyclictest2288490-21kworker/u8:4+flush-259:023:10:001
2064543993228,2cyclictest2163491-21kworker/u8:0+flush-259:020:50:161
2064541993226,4cyclictest2225279-21store21:45:010
2064543993128,2cyclictest2194423-21kworker/u8:2+flush-259:023:20:151
2064543993127,3cyclictest2064493-21kworker/u8:0+events_unbound19:50:191
2064543993126,3cyclictest2350329-21kworker/u8:3+events_unbound23:45:221
2064541993125,4cyclictest252-21systemd-journal22:10:010
2064541993125,4cyclictest2111749-21latency_hist19:55:010
2064543993027,2cyclictest2288490-21kworker/u8:4+flush-259:022:55:201
2064543993026,3cyclictest2219913-21kworker/u8:0+events_unbound21:40:151
2064543993026,2cyclictest2392097-21kworker/u8:1+flush-259:000:35:201
2064543993026,2cyclictest2132636-21kworker/u8:2+events_unbound20:25:211
2064543993024,4cyclictest2178722-21latency_hist21:00:011
2064543993024,4cyclictest2178722-21latency_hist21:00:011
2064551992925,3cyclictest2288490-21kworker/u8:4+flush-259:023:10:273
2064551992925,3cyclictest2288490-21kworker/u8:4+flush-259:023:10:273
2064551992924,4cyclictest2194423-21kworker/u8:2+events_unbound22:17:413
2064551992923,4cyclictest2064466-21latency_hist19:10:003
2064543992926,2cyclictest2116535-21kworker/u8:3+flush-259:020:00:261
2064543992925,3cyclictest2345064-21kworker/u8:0+flush-259:000:05:161
2064543992923,2cyclictest2241588-21kworker/u8:0+events_unbound22:45:201
2064541992926,2cyclictest2142555-21kworker/u8:1+flush-259:020:55:200
2064541992926,2cyclictest2142555-21kworker/u8:1+flush-259:020:55:200
2064541992925,3cyclictest2096152-21apt-get19:40:130
2064543992825,2cyclictest2194423-21kworker/u8:2+events_unbound21:55:141
2064543992823,3cyclictest2345064-21kworker/u8:0+flush-259:023:40:181
2064541992821,5cyclictest2371012-21latency_hist00:05:020
2064551992723,3cyclictest2309303-21apt-get23:05:123
2064551992723,3cyclictest2235831-21kworker/u8:1+events_unbound22:10:153
2064551992722,4cyclictest2387000-21cat00:20:013
2064551992722,4cyclictest2016610-21kworker/u8:1+flush-259:019:25:163
2064551992721,4cyclictest2298703-21latency_hist22:55:003
2064543992724,2cyclictest2262045-21kworker/u8:1+events_unbound22:20:231
2064543992724,2cyclictest2132636-21kworker/u8:2+events_unbound20:40:121
2064543992722,3cyclictest2350329-21kworker/u8:3+flush-259:023:50:261
2064543992721,4cyclictest2255613-21ntpq22:10:221
2064543992623,2cyclictest2132636-21kworker/u8:2+flush-259:020:45:251
2064543992622,2cyclictest2194423-21kworker/u8:2+events_unbound22:15:131
2064543992621,3cyclictest2194423-21kworker/u8:2+flush-259:000:25:181
2064551992522,2cyclictest2194423-21kworker/u8:2+flush-259:023:30:213
2064551992521,3cyclictest2392097-21kworker/u8:1+events_unbound00:30:133
2064551992521,3cyclictest2345064-21kworker/u8:0+events_unbound00:05:243
2064551992521,3cyclictest2309274-21kworker/u8:1+flush-259:023:25:213
2064547992521,3cyclictest2262117-21gpgv22:20:012
2064547992519,4cyclictest252-21systemd-journal22:25:002
2064543992521,3cyclictest2173771-21kworker/u8:3+events_unbound21:10:251
2064541992520,3cyclictest2194423-21kworker/u8:2+flush-259:022:30:160
2064541992518,5cyclictest2406765-21ntpq00:35:220
2064551992418,4cyclictest2127664-21diskstats20:10:143
2064547992419,3cyclictest2407624-21latency_hist00:40:012
2064543992421,2cyclictest2142555-21kworker/u8:1+events_unbound21:35:141
2064543992421,2cyclictest2064493-21kworker/u8:0+events_unbound19:55:221
2064541992422,1cyclictest1-21systemd22:44:150
2064541992420,3cyclictest2184105-21apt-get21:05:120
2064541992419,3cyclictest490-21dbus-daemon22:25:000
2064551992320,2cyclictest2350329-21kworker/u8:3+flush-259:023:50:003
2064551992319,3cyclictest2032048-21kworker/u8:2+events_unbound19:34:163
2064547992317,4cyclictest2209638-21latency_hist21:30:012
2064543992320,2cyclictest2194423-21kworker/u8:2+events_unbound23:58:161
2064543992320,2cyclictest2183701-21kworker/u8:0+flush-259:021:05:201
2064543992320,2cyclictest2089863-21kworker/u8:3+events_unbound19:30:231
2064543992319,3cyclictest2194423-21kworker/u8:2+flush-259:021:30:191
2064541992316,5cyclictest2089832-21ntpq19:30:210
2064551992220,2cyclictest2194423-21kworker/u8:2+events_unbound00:25:133
2064551992219,2cyclictest2350146-21fwupd23:43:183
2064551992219,2cyclictest2235831-21kworker/u8:1+flush-259:022:00:213
2064551992218,3cyclictest2194423-21kworker/u8:2+flush-259:021:55:233
2064551992218,3cyclictest2116535-21kworker/u8:3+events_unbound20:00:183
2064551992217,4cyclictest2225039-21apt-get21:45:023
2064547992217,3cyclictest2272987-21/usr/sbin/munin22:30:142
2064547992216,4cyclictest2297854-21ntpq22:50:222
2064543992219,2cyclictest2288490-21kworker/u8:4+events_unbound23:00:241
2064543992218,3cyclictest2194423-21kworker/u8:2+events_unbound22:30:291
2064551992118,2cyclictest2194423-21kworker/u8:2+flush-259:021:35:263
2064551992118,2cyclictest2194423-21kworker/u8:2+events_unbound23:20:223
2064551992117,3cyclictest2194423-21kworker/u8:2+events_unbound21:30:223
2064551992116,4cyclictest2194423-21kworker/u8:2+flush-259:000:10:213
2064547992117,2cyclictest252-21systemd-journal21:36:152
2064547992115,4cyclictest252-21systemd-journal21:15:002
2064543992117,3cyclictest2090343-21kworker/u8:4+events_unbound20:07:151
2064543992116,3cyclictest2376432-21apt-get00:10:121
2064551992017,2cyclictest2142555-21kworker/u8:1+flush-259:020:35:153
2064551992016,3cyclictest2309274-21kworker/u8:1+events_unbound23:19:163
2064551992016,3cyclictest2194423-21kworker/u8:2+events_unbound22:20:163
2064551992016,3cyclictest2132636-21kworker/u8:2+flush-259:020:15:253
2064551992015,4cyclictest2241588-21kworker/u8:0+flush-259:022:30:203
2064543992017,2cyclictest2288490-21kworker/u8:4+events_unbound23:10:251
2064543992017,2cyclictest2288490-21kworker/u8:4+events_unbound23:10:241
2064543992016,3cyclictest2032048-21kworker/u8:2+events_unbound19:25:181
2064541992015,4cyclictest2380490-21ntpq00:10:220
2064551991916,2cyclictest2392097-21kworker/u8:1+flush-259:000:35:263
2064551991915,3cyclictest2345064-21kworker/u8:0+events_unbound23:50:293
2064551991915,3cyclictest2142555-21kworker/u8:1+events_unbound20:34:163
2064551991915,3cyclictest2090343-21kworker/u8:4+events_unbound20:05:223
2064551991915,3cyclictest2090343-21kworker/u8:4+events_unbound19:47:143
2064551991913,4cyclictest2288490-21kworker/u8:4+flush-259:023:05:003
2064547991915,3cyclictest2085144-21apt-get19:30:022
2064543991914,4cyclictest2366099-21apt-get00:00:111
2064541991914,3cyclictest252-21systemd-journal23:31:150
2064551991815,2cyclictest2173771-21kworker/u8:3+flush-259:021:10:243
2064551991814,3cyclictest2106879-21apt-get19:50:143
2064551991813,4cyclictest2064493-21kworker/u8:0+flush-259:019:15:253
2064547991815,2cyclictest2299989-21kthreadcore22:55:162
2064547991815,2cyclictest2283303-21kworker/u8:1+flush-259:022:40:172
2064547991815,2cyclictest2179413-21irqcore21:00:162
2064541991816,1cyclictest0-21swapper/020:30:000
2064541991814,3cyclictest2064493-21kworker/u8:0+flush-259:019:20:160
2064541991813,3cyclictest252-21systemd-journal21:54:550
2064541991813,3cyclictest2386810-21base6400:20:020
2064551991715,1cyclictest2350329-21kworker/u8:3+events_unbound00:00:253
2064551991714,2cyclictest2361940-21kthreadcore23:55:173
2064551991714,2cyclictest2309274-21kworker/u8:1+events_unbound23:36:163
2064551991713,3cyclictest2163491-21kworker/u8:0+events_unbound20:50:253
2064551991713,3cyclictest2116902-21cron20:00:013
2064551991713,3cyclictest2090343-21kworker/u8:4+flush-259:020:20:263
2064551991712,3cyclictest2225055-21http21:45:013
2064547991715,1cyclictest0-21swapper/221:20:002
2064547991714,2cyclictest2194423-21kworker/u8:2+flush-259:023:45:002
2064547991713,3cyclictest1-21systemd00:31:502
2064547991711,4cyclictest2387112-21cp00:20:022
2064543991714,2cyclictest2194423-21kworker/u8:2+events_unbound22:29:201
2064541991714,2cyclictest61150irq/133-enp0s29f1:tx-023:10:230
2064541991714,2cyclictest61150irq/133-enp0s29f1:tx-023:10:230
2064541991714,2cyclictest2241588-21kworker/u8:0+flush-259:022:55:160
2064541991714,2cyclictest2090343-21kworker/u8:4+flush-259:019:45:150
2064541991710,5cyclictest2240080-21ntpq21:55:220
2064551991613,2cyclictest2272770-21kworker/u8:3+events_unbound22:35:253
2064551991613,2cyclictest2132636-21kworker/u8:2+events_unbound20:55:193
2064551991613,2cyclictest2132636-21kworker/u8:2+events_unbound20:55:183
2064551991612,3cyclictest2241588-21kworker/u8:0+events_unbound22:45:243
2064551991611,4cyclictest2194423-21kworker/u8:2+events_unbound00:15:183
206454799169,5cyclictest2329673-21latency_hist23:25:002
2064547991613,1cyclictest0-21swapper/219:50:012
2064547991612,3cyclictest2142555-21kworker/u8:1+flush-259:021:33:202
2064547991611,3cyclictest2225279-21store21:45:012
2064543991612,3cyclictest2090343-21kworker/u8:4+events_unbound19:49:021
2064543991611,4cyclictest2194423-21kworker/u8:2+events_unbound22:42:501
2064541991614,1cyclictest60350irq/132-enp0s29f1:rx-022:10:240
2064541991614,1cyclictest60350irq/132-enp0s29f1:rx-020:08:070
2064541991613,2cyclictest61150irq/133-enp0s29f1:tx-022:50:170
2064541991613,2cyclictest60350irq/132-enp0s29f1:rx-023:40:180
2064541991613,2cyclictest60350irq/132-enp0s29f1:rx-021:15:140
2064541991613,2cyclictest60350irq/132-enp0s29f1:rx-019:13:220
2064541991613,1cyclictest2309044-21turbostat23:05:000
2064541991613,1cyclictest0-21swapper/020:40:210
2064541991612,3cyclictest2194423-21kworker/u8:2+events_unbound00:20:170
2064551991512,2cyclictest2209600-21kworker/u8:3+events_unbound21:54:153
2064551991512,2cyclictest2064493-21kworker/u8:0+flush-259:019:14:453
2064551991511,3cyclictest2157885-21kworker/u8:3+events_unbound20:42:233
2064551991510,4cyclictest2147817-21latency_hist20:30:003
2064547991513,1cyclictest509-21rs:main2
2064547991512,2cyclictest0-21swapper/223:55:012
2064547991511,2cyclictest2241588-21kworker/u8:0+flush-259:022:35:202
2064547991510,4cyclictest2256634-21gpgv22:15:012
2064541991514,1cyclictest60350irq/132-enp0s29f1:rx-000:06:250
2064541991513,2cyclictest60350irq/132-enp0s29f1:rx-000:25:280
2064541991513,1cyclictest60350irq/132-enp0s29f1:rx-023:55:100
2064541991513,1cyclictest60350irq/132-enp0s29f1:rx-021:21:560
2064541991513,1cyclictest60350irq/132-enp0s29f1:rx-021:10:270
2064541991513,1cyclictest60350irq/132-enp0s29f1:rx-020:50:210
2064541991513,1cyclictest60350irq/132-enp0s29f1:rx-020:38:490
2064541991513,1cyclictest60350irq/132-enp0s29f1:rx-020:23:000
2064541991513,1cyclictest60350irq/132-enp0s29f1:rx-019:55:190
2064541991513,1cyclictest60350irq/132-enp0s29f1:rx-000:32:100
2064541991512,2cyclictest60350irq/132-enp0s29f1:rx-023:50:210
2064541991512,2cyclictest60350irq/132-enp0s29f1:rx-020:13:430
2064541991512,2cyclictest2241588-21kworker/u8:0+events_unbound22:15:170
2064541991512,2cyclictest2152072-21ntp_kernel_err20:30:230
206454799149,4cyclictest2142555-21kworker/u8:1+flush-259:020:33:472
2064547991411,2cyclictest2142555-21kworker/u8:1+flush-259:020:55:252
2064547991411,2cyclictest2142555-21kworker/u8:1+flush-259:020:55:252
206454799140,2cyclictest22413552cut22:00:132
2064541991413,1cyclictest60350irq/132-enp0s29f1:rx-023:16:040
2064541991413,1cyclictest60350irq/132-enp0s29f1:rx-023:05:300
2064541991413,1cyclictest60350irq/132-enp0s29f1:rx-022:36:500
2064541991412,1cyclictest61150irq/133-enp0s29f1:tx-020:00:120
2064541991412,1cyclictest60350irq/132-enp0s29f1:rx-021:48:390
2064541991412,1cyclictest60350irq/132-enp0s29f1:rx-021:30:330
2064541991412,1cyclictest60350irq/132-enp0s29f1:rx-021:25:260
2064541991412,1cyclictest60350irq/132-enp0s29f1:rx-019:19:120
2064541991411,2cyclictest60350irq/132-enp0s29f1:rx-022:00:170
2064541991411,2cyclictest60350irq/132-enp0s29f1:rx-021:00:190
2064541991411,2cyclictest2194423-21kworker/u8:2+flush-259:021:35:240
2064541991410,3cyclictest2350568-21apt-get23:45:100
206455199139,3cyclictest2132636-21kworker/u8:2+flush-259:021:01:203
206455199139,3cyclictest2032048-21kworker/u8:2+events_unbound19:37:123
206454799138,4cyclictest2386482-21apt-get00:20:012
206454799137,4cyclictest2314201-21latency_hist23:10:002
206454799135,0cyclictest0-21swapper/200:15:002
2064547991311,1cyclictest2371000-21cron00:05:002
2064547991311,1cyclictest2334842-21turbostat.cron23:30:012
2064547991311,1cyclictest0-21swapper/223:10:182
2064547991311,1cyclictest0-21swapper/223:10:182
2064547991311,1cyclictest0-21swapper/219:40:012
2064547991310,2cyclictest0-21swapper/220:20:002
2064543991310,2cyclictest2293842-21cpuspeed_turbos22:50:131
2064541991312,1cyclictest60350irq/132-enp0s29f1:rx-019:36:520
2064541991311,1cyclictest60350irq/132-enp0s29f1:rx-023:25:120
2064541991311,1cyclictest60350irq/132-enp0s29f1:rx-022:45:130
2064541991310,2cyclictest2168407-21turbostat20:50:000
206455199129,2cyclictest2288490-21kworker/u8:4+events_unbound22:55:203
206455199128,3cyclictest2142555-21kworker/u8:1+flush-259:021:20:263
206455199127,4cyclictest2142555-21kworker/u8:1+events_unbound21:25:143
206454799129,2cyclictest2137852-21cpuspeed_turbos20:20:142
2064541991211,1cyclictest60350irq/132-enp0s29f1:rx-020:15:150
206455199117,3cyclictest2194423-21kworker/u8:2+flush-259:021:15:363
206454799107,2cyclictest2355473-21sh23:50:002
206454799107,1cyclictest0-21swapper/223:35:162
206454799106,3cyclictest2163491-21kworker/u8:0+flush-259:020:50:312
206454799105,4cyclictest2143093-21df20:25:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional