You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 20:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sun Apr 12, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2009145994237,4cyclictest2014511-21kworker/u8:4+flush-259:007:35:003
2009135994238,3cyclictest509-21rs:main0
2009135993832,4cyclictest2262349-21ntpq11:10:210
2009137993733,3cyclictest2148960-21kworker/u8:2+flush-259:009:25:171
2009145993631,4cyclictest2242085-21apt-get10:55:013
2009135993428,4cyclictest2118231-21apt-get08:55:120
2009137993329,3cyclictest2102733-21kworker/u8:2+flush-259:008:45:261
2009137993328,4cyclictest2014511-21kworker/u8:4+flush-259:007:35:191
2009137993327,4cyclictest2195752-21latency_hist10:10:001
2009141993226,4cyclictest2206034-21latency_hist10:20:012
2009137993225,5cyclictest2268343-21latency_hist11:20:011
2009145993127,3cyclictest2283823-21kworker/u8:0+flush-259:012:10:153
2009145993125,4cyclictest2201155-21apt-get10:15:123
2009141993127,3cyclictest1955614-21kworker/u8:3+flush-259:007:45:172
2009145993024,4cyclictest2014511-21kworker/u8:4+flush-259:007:40:193
2009145993023,4cyclictest509-21rs:main3
2009141993024,3cyclictest252-21systemd-journal07:41:172
2009135993026,3cyclictest2325888-21apt-get12:15:110
2009135993025,4cyclictest2045595-21apt-get07:45:120
2009141992923,4cyclictest509-21rs:main2
2009137992927,1cyclictest2236573-21fwupd10:46:001
2009145992824,3cyclictest2185043-21kworker/u8:2+flush-259:010:10:203
2009145992823,4cyclictest2211238-21kworker/u8:0+flush-259:010:35:173
2009145992822,4cyclictest2294173-21kworker/u8:1+events_unbound11:50:183
2009141992823,4cyclictest2226898-21apt-get10:40:122
2009135992822,4cyclictest2018620-21ntpq07:15:200
2009135992819,7cyclictest252-21systemd-journal08:44:590
2009145992720,5cyclictest2242295-21apt-key10:55:003
2009137992722,4cyclictest252-21systemd-journal12:20:001
2009135992723,3cyclictest2035215-21apt-get07:35:140
2009135992722,4cyclictest2325455-21gpgv12:15:010
2009145992620,4cyclictest252-21systemd-journal08:25:173
2009141992619,5cyclictest252-21systemd-journal08:40:002
2009145992521,3cyclictest2294173-21kworker/u8:1+events_unbound11:45:163
2009145992520,4cyclictest2252897-21kworker/u8:3+events_unbound11:25:123
2009145992518,5cyclictest252-21systemd-journal07:50:173
2009137992521,3cyclictest2195777-21kworker/u8:1+events_unbound10:55:001
2009137992521,3cyclictest2076922-21apt-get08:15:101
2009137992521,3cyclictest1955614-21kworker/u8:3+flush-259:007:45:181
2009137992520,4cyclictest1955614-21kworker/u8:3+flush-259:007:15:161
2009137992519,4cyclictest2097627-21apt-get08:35:121
2009135992519,4cyclictest2060898-21cat08:00:010
200913599251,12cyclictest1-21systemd12:02:180
2009145992419,3cyclictest2014511-21kworker/u8:4+flush-259:007:25:163
2009141992416,7cyclictest0-21swapper/209:45:002
2009137992420,3cyclictest2331044-21/usr/sbin/munin12:20:141
2009137992420,3cyclictest2061410-21/usr/sbin/munin08:00:201
2009137992418,4cyclictest2164688-21apt-get09:40:121
2009145992320,2cyclictest2283823-21kworker/u8:0+events_unbound12:15:183
2009145992319,3cyclictest2159470-21kworker/u8:4+flush-259:010:20:193
2009145992318,3cyclictest1955614-21kworker/u8:3+flush-259:007:35:223
2009141992318,4cyclictest2050733-21apt-get07:50:112
2009137992319,3cyclictest509-21rs:main1
2009137992319,3cyclictest2283823-21kworker/u8:0+flush-259:012:30:211
2009137992319,3cyclictest1955614-21kworker/u8:3+flush-259:008:25:241
2009137992316,5cyclictest2189331-21ntpq10:00:211
2009137992316,5cyclictest2024629-21latency_hist07:25:011
200913599239,13cyclictest2288518-21unixbench_multi11:35:270
200913599239,13cyclictest2014290-21turbostat07:15:000
2009135992319,3cyclictest2190703-21cat10:05:010
2009145992219,2cyclictest2195777-21kworker/u8:1+flush-259:011:20:193
2009145992215,5cyclictest2173927-21ntpq09:45:223
2009145992215,5cyclictest2059880-21ntpq07:55:213
2009137992218,3cyclictest2336216-21kworker/u8:4+flush-259:012:25:241
2009137992216,4cyclictest252-21systemd-journal10:34:181
2009137992215,5cyclictest252-21systemd-journal09:00:201
2009135992216,4cyclictest2117967-21latency_hist08:55:000
2009145992117,3cyclictest2195777-21kworker/u8:1+flush-259:011:05:003
2009145992117,2cyclictest2324820-21kworker/u8:2+flush-259:012:20:243
2009145992116,3cyclictest1955614-21kworker/u8:3+flush-259:007:10:213
2009145992115,4cyclictest2283823-21kworker/u8:0+events_unbound11:55:183
2009137992118,2cyclictest2273509-21kworker/u8:2+events_unbound11:55:001
2009137992115,4cyclictest252-21systemd-journal10:29:201
2009135992117,3cyclictest2211238-21kworker/u8:0+flush-259:011:15:170
2009135992115,4cyclictest2045336-21latency_hist07:45:010
2009145992017,2cyclictest2309294-21kworker/u8:4+events_unbound12:05:133
2009145992016,2cyclictest2226659-21kworker/u8:3+flush-259:010:50:003
2009145992015,4cyclictest2102733-21kworker/u8:2+flush-259:008:40:203
200914199200,9cyclictest2216095-21sshd10:27:202
2009137992017,2cyclictest2242789-21apt-get10:55:111
2009137992014,4cyclictest2289217-21/usr/sbin/munin11:40:201
2009137992014,4cyclictest2289217-21/usr/sbin/munin11:40:191
2009135992015,3cyclictest2268771-21diskstats11:20:140
2009135992011,8cyclictest61150irq/133-enp0s29f1:tx-011:30:140
200913599200,7cyclictest2335552-21sed12:20:250
2009145991915,3cyclictest2273509-21kworker/u8:2+flush-259:011:35:003
2009145991915,3cyclictest2143623-21kworker/u8:3+flush-259:009:20:233
2009145991915,3cyclictest2102733-21kworker/u8:2+events_unbound08:50:233
2009145991913,4cyclictest2341213-21latency_hist12:30:013
2009145991912,5cyclictest252-21systemd-journal08:34:593
2009145991912,4cyclictest2076897-21/usr/sbin/munin08:15:133
2009141991913,4cyclictest2314816-21latency_hist12:05:012
200914199191,16cyclictest2252856-21cron11:04:592
200914199190,9cyclictest2137736-21ntpq09:10:192
2009137991916,2cyclictest2283823-21kworker/u8:0+flush-259:012:10:211
2009137991916,2cyclictest2283823-21kworker/u8:0+events_unbound12:35:201
2009137991916,2cyclictest1955614-21kworker/u8:3+flush-259:008:10:011
2009137991914,4cyclictest2128569-21apt-get09:05:121
2009137991914,4cyclictest2102733-21kworker/u8:2+flush-259:008:50:251
200913599191,4cyclictest2278628-21cron11:29:590
2009135991914,3cyclictest2149202-21apt-get09:25:120
200913599190,6cyclictest2029814-21turbostat.cron07:30:000
2009145991814,3cyclictest2091975-21kworker/u8:1+flush-259:009:15:243
2009145991813,4cyclictest2091975-21kworker/u8:1+flush-259:009:00:173
2009145991813,3cyclictest2159470-21kworker/u8:4+events_unbound09:50:173
2009145991812,4cyclictest252-21systemd-journal08:58:183
200914199184,6cyclictest2211198-21munin-plugin-st10:25:002
2009141991814,3cyclictest507-21in:imuxsock07:35:002
2009141991812,4cyclictest252-21systemd-journal11:45:002
2009141991812,4cyclictest252-21systemd-journal11:45:002
2009141991812,4cyclictest252-21systemd-journal07:30:012
200914199181,10cyclictest2174925-21cat09:50:012
2009137991815,2cyclictest2316104-21kthreadcore12:05:171
2009137991814,3cyclictest2143623-21kworker/u8:3+flush-259:009:20:011
2009137991814,3cyclictest1955614-21kworker/u8:3+flush-259:007:55:391
200913599188,2cyclictest60350irq/132-enp0s29f1:rx-009:30:130
200913599185,3cyclictest2341823-21if_enp0s30f412:30:130
200913599183,4cyclictest2077361-21sed08:15:160
2009135991814,3cyclictest2252897-21kworker/u8:3+flush-259:011:45:000
2009135991814,3cyclictest2252897-21kworker/u8:3+flush-259:011:45:000
2009135991813,4cyclictest60350irq/132-enp0s29f1:rx-012:05:230
2009145991714,2cyclictest2314816-21sh12:05:003
2009145991714,2cyclictest2211871-21irqcore10:25:143
2009145991713,3cyclictest2252897-21kworker/u8:3+flush-259:011:35:213
2009145991713,3cyclictest2211238-21kworker/u8:0+flush-259:010:30:283
2009145991713,3cyclictest1955614-21kworker/u8:3+flush-259:008:10:203
2009145991712,3cyclictest252-21systemd-journal12:31:203
2009145991712,3cyclictest2273509-21kworker/u8:2+flush-259:011:40:203
2009145991712,3cyclictest2273509-21kworker/u8:2+flush-259:011:40:193
200914199177,8cyclictest2236921-21latency_hist10:49:592
200914199175,10cyclictest2347089-21sed12:35:172
2009141991711,4cyclictest2014283-21latency_hist07:15:002
200914199170,15cyclictest2300087-21irqcore11:50:152
200913799172,8cyclictest2200900-21awk10:15:011
2009137991713,3cyclictest1955614-21kworker/u8:3+flush-259:007:10:251
200913599177,9cyclictest0-21swapper/009:40:130
200913599174,3cyclictest2073997-21cut08:10:170
200913599170,3cyclictest2171220-21kthreadcore09:45:160
2009145991612,3cyclictest2091975-21kworker/u8:1+events_unbound09:05:203
2009145991612,3cyclictest2034973-21kworker/u8:2+flush-259:008:00:243
2009145991611,4cyclictest2159470-21kworker/u8:4+events_unbound09:35:183
200914199166,8cyclictest2106454-21cut08:40:182
2009141991613,2cyclictest500-21systemd-logind09:05:012
2009141991612,3cyclictest2252897-21kworker/u8:3+flush-259:011:35:272
200914199161,13cyclictest2149349-21df09:25:132
200914199161,13cyclictest2060743-21latency_hist08:00:012
200914199161,12cyclictest2159634-21cpuspeed_turbos09:35:132
200914199160,9cyclictest2195547-21taskset10:07:432
200914199160,14cyclictest2279569-21cut11:30:132
200914199160,14cyclictest2226746-21latency_hist10:40:002
200914199160,14cyclictest2144133-21grep09:20:122
200914199160,13cyclictest2320514-21grep12:10:132
200914199160,13cyclictest2216602-21apt-get10:30:132
200913799161,7cyclictest2211196-21awk10:25:001
2009137991611,3cyclictest2164418-21awk09:40:001
2009135991614,1cyclictest60350irq/132-enp0s29f1:rx-011:46:280
2009135991614,1cyclictest60350irq/132-enp0s29f1:rx-010:31:240
2009135991614,1cyclictest60350irq/132-enp0s29f1:rx-010:15:150
2009135991614,1cyclictest60350irq/132-enp0s29f1:rx-007:20:030
2009135991613,2cyclictest60350irq/132-enp0s29f1:rx-012:25:230
2009135991613,2cyclictest60350irq/132-enp0s29f1:rx-011:55:150
2009135991613,2cyclictest60350irq/132-enp0s29f1:rx-010:10:230
2009135991612,3cyclictest2143779-21latency_hist09:20:010
2009135991610,4cyclictest2138608-21latency_hist09:15:010
2009145991512,2cyclictest2133063-21kworker/u8:4+flush-259:009:10:153
2009145991511,3cyclictest2123338-21kworker/u8:0+flush-259:009:30:253
200914199159,4cyclictest252-21systemd-journal12:35:012
200914199157,6cyclictest0-21swapper/211:55:252
200914199153,11cyclictest2039280-21ntpq07:35:232
2009141991511,3cyclictest2283823-21kworker/u8:0+flush-259:011:45:202
200914199151,12cyclictest2075764-21ntpq08:10:212
200914199150,14cyclictest2261373-21kthreadcore11:10:192
200914199150,13cyclictest2242643-21cat10:55:012
200914199150,13cyclictest2111199-21grep08:45:192
200914199150,13cyclictest2097360-21latency_hist08:35:002
200914199150,13cyclictest2082201-21grep08:20:142
200914199150,13cyclictest2079195-21cut08:15:172
200914199150,12cyclictest2331021-21cut12:20:012
2009137991511,3cyclictest2283823-21kworker/u8:0+flush-259:012:00:051
2009137991511,3cyclictest2252897-21kworker/u8:3+flush-259:011:15:001
2009137991511,3cyclictest2154330-21kworker/u8:1+events_unbound09:30:201
2009137991511,3cyclictest2117815-21kworker/u8:3+flush-259:008:55:231
2009137991511,2cyclictest0-21swapper/110:45:001
2009135991513,2cyclictest60350irq/132-enp0s29f1:rx-009:05:250
2009135991513,2cyclictest60350irq/132-enp0s29f1:rx-008:03:550
2009135991513,1cyclictest60350irq/132-enp0s29f1:rx-010:25:170
2009135991513,1cyclictest60350irq/132-enp0s29f1:rx-009:00:170
2009135991513,1cyclictest60350irq/132-enp0s29f1:rx-008:05:250
2009135991513,1cyclictest2009127-21cyclictest08:50:000
2009135991512,2cyclictest60350irq/132-enp0s29f1:rx-011:03:030
2009135991512,2cyclictest60350irq/132-enp0s29f1:rx-010:35:130
2009135991512,2cyclictest60350irq/132-enp0s29f1:rx-009:50:200
2009135991512,2cyclictest60350irq/132-enp0s29f1:rx-007:54:530
2009145991410,3cyclictest2211238-21kworker/u8:0+flush-259:011:17:363
2009145991410,3cyclictest2067637-21kthreadcore08:05:173
2009145991410,1cyclictest0-21swapper/307:20:173
200914199143,9cyclictest2155476-21cut09:30:172
200914199143,9cyclictest2020877-21kthreadcore07:20:172
200914199141,11cyclictest2087428-21fschecks_count08:25:132
200914199141,11cyclictest2014666-21cpuspeed_turbos07:15:122
200914199140,12cyclictest2332216-21cat12:20:162
200914199140,12cyclictest2317631-21cut12:05:172
200914199140,12cyclictest2276741-21cut11:25:162
200914199140,12cyclictest2269079-21cut11:20:162
200914199140,12cyclictest2253798-21sed11:05:142
200914199140,12cyclictest2196203-21fschecks_count10:10:132
200914199140,12cyclictest2184325-21expr09:55:232
200914199140,12cyclictest2142178-21grep09:15:192
200914199140,12cyclictest2116054-21grep08:50:182
200914199140,12cyclictest2066869-21cut08:05:152
200914199140,12cyclictest2062745-21cut08:00:172
200914199140,12cyclictest2009209-21grep07:10:002
200914199140,11cyclictest2336368-21grep12:25:122
200914199140,11cyclictest2128788-21tune2fs09:05:162
200913799149,3cyclictest2102733-21kworker/u8:2+flush-259:008:40:161
2009137991412,1cyclictest2040807-21irqcore07:40:151
2009137991411,2cyclictest2258015-21idleruntime-cro11:10:001
2009137991411,2cyclictest2252863-21idleruntime-cro11:04:591
2009137991410,3cyclictest507-21in:imuxsock11:50:001
200913599147,2cyclictest60350irq/132-enp0s29f1:rx-010:55:230
2009135991413,1cyclictest60350irq/132-enp0s29f1:rx-010:40:250
2009135991412,1cyclictest60350irq/132-enp0s29f1:rx-011:50:160
2009135991412,1cyclictest60350irq/132-enp0s29f1:rx-010:00:220
2009135991411,2cyclictest60350irq/132-enp0s29f1:rx-010:20:010
2009135991411,2cyclictest60350irq/132-enp0s29f1:rx-009:57:290
2009135991411,2cyclictest60350irq/132-enp0s29f1:rx-009:35:200
2009135991411,2cyclictest60350irq/132-enp0s29f1:rx-008:35:200
2009135991411,2cyclictest60350irq/132-enp0s29f1:rx-008:25:110
2009135991411,2cyclictest2148945-21turbostat09:25:000
200914599139,2cyclictest0-21swapper/309:55:233
200914599138,4cyclictest2258299-21apt-get11:10:123
200914599138,4cyclictest2159470-21kworker/u8:4+events_unbound10:05:133
200914199130,11cyclictest2185288-21apt10:00:112
200914199130,11cyclictest2175003-21cat09:50:102
200914199130,11cyclictest2119249-21grep08:55:172
200913799139,3cyclictest2211238-21kworker/u8:0+flush-259:011:20:241
200913799139,3cyclictest2174772-21kworker/u8:1+flush-259:009:55:001
200913799138,3cyclictest2184113-21ntpq09:55:201
2009137991310,2cyclictest2071849-21cpuspeed_turbos08:10:131
2009137991310,1cyclictest0-21swapper/112:00:001
2009135991311,1cyclictest60350irq/132-enp0s29f1:rx-010:50:170
2009135991311,1cyclictest60350irq/132-enp0s29f1:rx-007:30:170
2009135991310,2cyclictest60350irq/132-enp0s29f1:rx-008:20:120
2009135991310,2cyclictest2346696-21cpuspeed_turbos12:35:150
200914599129,2cyclictest2169115-21unixbench_multi09:40:233
200914599128,3cyclictest2195777-21kworker/u8:1+flush-259:010:40:253
200914599128,3cyclictest2148960-21kworker/u8:2+flush-259:009:25:243
200914599128,3cyclictest2091975-21kworker/u8:1+flush-259:008:35:213
200914599128,3cyclictest2040159-21kworker/u8:5+flush-259:007:50:003
200913799129,2cyclictest2288546-21unixbench_singl11:35:271
200913799128,3cyclictest2211238-21kworker/u8:0+flush-259:010:35:261
200914599118,1cyclictest0-21swapper/310:05:013
200913799118,2cyclictest2279164-21fschecks_time11:30:121
200913799118,2cyclictest2055621-21idleruntime-cro07:55:011
200914599101,6cyclictest2086982-21idleruntime-cro08:24:593
200913799107,1cyclictest17450irq/129-nvme0q207:35:001
200913799106,3cyclictest2159470-21kworker/u8:4+flush-259:010:15:201
200913799106,2cyclictest2174734-21cron09:50:011
200913799106,2cyclictest2028903-21ntpq07:25:191
200913799105,3cyclictest2133682-21/usr/sbin/munin09:10:171
20091379997,1cyclictest0-21swapper/108:30:211
20091379994,3cyclictest2086823-21sshd08:23:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional