You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 14:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sun Nov 16, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1998192995047,2cyclictest75-21kswapd022:49:503
1998188994639,5cyclictest2293704-21latency_hist23:55:012
1998181993730,5cyclictest2220827-21latency_hist22:45:000
1998188993530,3cyclictest2147268-21kworker/u8:4+events_unbound21:35:002
1998185993533,1cyclictest2225979-21fwupd22:49:481
1998181993427,5cyclictest2091245-21latency_hist20:40:010
1998185993328,3cyclictest2205138-21kworker/u8:3+flush-259:022:40:001
1998181993229,2cyclictest75-21kswapd022:49:500
1998188993127,3cyclictest2091265-21kworker/u8:1+flush-259:021:35:182
1998185993127,3cyclictest2070329-21apt-get20:16:551
1998181993026,3cyclictest2091265-21kworker/u8:1+events_unbound21:50:170
1998181993023,5cyclictest2002386-21ntpq19:10:250
1998188992926,2cyclictest2091265-21kworker/u8:1+events_unbound22:00:252
1998188992925,3cyclictest2013587-21kworker/u8:2+events_unbound20:05:192
1998188992925,3cyclictest2013587-21kworker/u8:2+events_unbound20:05:192
1998192992825,2cyclictest2262890-21kworker/u8:4+flush-259:023:40:173
1998192992823,4cyclictest2335601-21gpgv00:35:013
1998192992823,4cyclictest2335601-21gpgv00:35:003
1998188992723,3cyclictest2003493-21kworker/u8:0+flush-259:019:20:182
1998192992623,2cyclictest2075840-21kworker/u8:3+flush-259:020:35:003
1998188992622,3cyclictest2060063-21kworker/u8:0+flush-259:020:20:152
1998188992522,2cyclictest2075840-21kworker/u8:3+flush-259:020:30:292
1998188992521,3cyclictest2210784-21apt-get22:35:132
1998188992521,3cyclictest2153334-21kworker/u8:2+flush-259:021:45:202
1998188992521,3cyclictest2101415-21kworker/u8:0+events_unbound20:50:192
1998188992519,4cyclictest2142845-21cat21:30:012
1998185992520,4cyclictest2204443-21ntpq22:25:241
1998185992520,4cyclictest2101415-21kworker/u8:0+flush-259:021:25:011
1998181992521,3cyclictest2168387-21kworker/u8:3+flush-259:022:05:000
1998181992519,4cyclictest2698871-21systemd-journal22:10:000
1998192992417,5cyclictest2034540-21diskstats19:45:173
1998188992421,2cyclictest2262676-21kworker/u8:1+events_unbound23:25:202
1998188992420,3cyclictest2091265-21kworker/u8:1+events_unbound21:50:252
1998185992419,4cyclictest2319978-21latency_hist00:20:001
1998181992420,3cyclictest2075840-21kworker/u8:3+events_unbound20:25:000
1998181992418,5cyclictest2126392-21ntpq21:10:210
1998188992320,2cyclictest2168387-21kworker/u8:3+events_unbound22:10:002
1998188992319,3cyclictest2262676-21kworker/u8:1+events_unbound23:30:222
1998188992319,3cyclictest2091265-21kworker/u8:1+events_unbound22:52:472
1998188992319,3cyclictest2091265-21kworker/u8:1+events_unbound20:55:312
1998185992317,4cyclictest2278167-21latency_hist23:40:011
1998185992317,4cyclictest2194647-21apt-get22:20:001
1998185992316,5cyclictest2112232-21diskstats21:00:151
1998181992318,4cyclictest2060075-21/usr/sbin/munin20:10:170
1998192992219,2cyclictest0-21swapper/322:45:003
1998188992219,2cyclictest2241631-21kworker/u8:2+events_unbound23:10:172
1998188992219,2cyclictest2091265-21kworker/u8:1+flush-259:021:20:162
1998188992219,2cyclictest2091265-21kworker/u8:1+events_unbound20:50:002
1998188992219,2cyclictest2091265-21kworker/u8:1+events_unbound20:40:282
1998188992218,3cyclictest2054740-21kworker/u8:1+flush-259:020:15:282
1998185992219,2cyclictest2054713-21cron20:05:001
1998185992216,4cyclictest2261771-21ntpq23:20:231
1998188992117,3cyclictest2262890-21kworker/u8:4+events_unbound23:40:472
1998188992115,4cyclictest2698871-21systemd-journal22:20:022
1998185992116,3cyclictest2148542-21apt-get21:35:111
1998181992117,3cyclictest2091265-21kworker/u8:1+flush-259:021:15:160
1998192992015,3cyclictest2065664-21/usr/sbin/munin20:15:183
1998192992012,5cyclictest2034109-21latency_hist19:45:003
1998188992017,2cyclictest2315002-21kworker/u8:0+flush-259:000:35:182
1998188992016,3cyclictest2205138-21kworker/u8:3+flush-259:023:35:192
1998185992018,1cyclictest2153334-21kworker/u8:2+flush-259:021:50:001
1998185992017,2cyclictest2153334-21kworker/u8:2+flush-259:021:45:001
1998185992014,4cyclictest2287693-21ntpq23:45:241
1998181992015,4cyclictest2273236-21apt-get23:35:130
1998181992015,3cyclictest2101415-21kworker/u8:0+events_unbound22:00:000
1998181992013,5cyclictest2199299-21ntpq22:20:220
1998181992012,6cyclictest2012711-21ntpq19:20:230
1998188991916,2cyclictest2205138-21kworker/u8:3+events_unbound00:00:122
1998188991916,2cyclictest2091265-21kworker/u8:1+events_unbound22:40:242
1998188991915,3cyclictest2298877-21kworker/u8:2+events_unbound00:07:462
1998188991912,5cyclictest2054720-21latency_hist20:05:012
1998185991916,2cyclictest2075840-21kworker/u8:3+flush-259:020:25:191
1998181991916,2cyclictest2091265-21kworker/u8:1+events_unbound21:35:200
1998181991915,3cyclictest2315002-21kworker/u8:0+events_unbound00:20:160
1998181991915,3cyclictest2168387-21kworker/u8:3+flush-259:022:15:010
1998181991912,5cyclictest2698871-21systemd-journal19:48:470
1998181991911,6cyclictest2111810-21latency_hist21:00:010
1998192991815,2cyclictest2091228-21cron20:40:003
1998192991814,3cyclictest2039267-21kworker/u8:4+flush-259:020:10:003
1998192991814,3cyclictest2039267-21kworker/u8:4+flush-259:020:10:003
1998192991814,2cyclictest0-21swapper/321:30:013
1998192991813,3cyclictest2147268-21kworker/u8:4+flush-259:021:35:013
1998188991815,2cyclictest2298877-21kworker/u8:2+events_unbound00:17:482
1998188991815,2cyclictest2091265-21kworker/u8:1+events_unbound22:58:482
1998188991815,2cyclictest2091265-21kworker/u8:1+events_unbound21:14:562
1998188991815,2cyclictest2075840-21kworker/u8:3+flush-259:020:25:152
1998188991815,2cyclictest2039267-21kworker/u8:4+flush-259:019:55:262
1998188991815,2cyclictest0-21swapper/221:45:002
1998188991814,2cyclictest2091265-21kworker/u8:1+events_unbound21:15:142
1998188991812,2cyclictest1993010-21kworker/u8:3+events_unbound19:47:132
1998188991811,4cyclictest2200453-21/usr/sbin/munin22:25:212
1998185991814,3cyclictest2091265-21kworker/u8:1+events_unbound20:55:011
1998181991815,2cyclictest1993010-21kworker/u8:3+flush-259:019:25:250
1998181991814,3cyclictest2205138-21kworker/u8:3+flush-259:023:05:000
1998181991813,4cyclictest2117208-21/usr/sbin/munin21:05:210
1998181991813,3cyclictest2698871-21systemd-journal23:49:560
1998181991811,5cyclictest2049597-21latency_hist20:00:010
1998192991714,2cyclictest3000063-21rs:main3
1998192991711,4cyclictest2698871-21systemd-journal20:45:003
1998188991714,2cyclictest1993010-21kworker/u8:3+events_unbound19:10:312
1998188991714,2cyclictest0-21swapper/222:35:002
1998188991712,3cyclictest2194686-21kworker/u8:0+events_unbound23:00:182
1998188991711,5cyclictest2101415-21kworker/u8:0+flush-259:021:00:172
1998188991711,4cyclictest2194766-21gpgv22:20:002
1998185991714,2cyclictest2292864-21ntp_offset23:50:231
1998185991714,2cyclictest2039267-21kworker/u8:4+events_unbound19:55:001
1998185991713,3cyclictest2325126-21idleruntime-cro00:24:591
1998185991712,3cyclictest2013815-21/usr/sbin/munin19:25:191
1998185991710,6cyclictest2698871-21systemd-journal19:35:001
1998185991710,5cyclictest2028105-21ntpq19:35:241
1998181991714,2cyclictest2205138-21kworker/u8:3+events_unbound23:50:240
1998181991713,3cyclictest2241631-21kworker/u8:2+events_unbound23:20:160
1998192991613,2cyclictest2196362-21kthreadcore22:20:193
1998192991613,1cyclictest0-21swapper/323:10:183
1998192991611,4cyclictest2314351-21apt-get00:15:033
1998188991613,2cyclictest2091265-21kworker/u8:1+events_unbound22:45:172
1998188991613,2cyclictest2075840-21kworker/u8:3+events_unbound20:35:152
1998188991613,2cyclictest2013587-21kworker/u8:2+flush-259:019:35:192
1998188991613,2cyclictest1950650-21kworker/u8:1+events_unbound19:29:562
1998188991612,2cyclictest2013587-21kworker/u8:2+events_unbound19:40:212
1998188991611,4cyclictest2205138-21kworker/u8:3+flush-259:023:45:052
1998185991613,2cyclictest0-21swapper/121:34:591
1998181991614,1cyclictest60650irq/133-enp0s29f1:rx-022:20:000
1998181991614,1cyclictest60650irq/133-enp0s29f1:rx-000:05:240
1998181991613,2cyclictest61450irq/134-enp0s29f1:tx-000:00:270
1998181991613,2cyclictest0-21swapper/022:30:000
1998181991612,3cyclictest2029650-21irqcore19:40:200
1998181991610,4cyclictest2142666-21apt-get21:30:000
1998192991513,1cyclictest0-21swapper/320:50:003
1998192991512,2cyclictest75-21kswapd023:20:033
199818899159,2cyclictest2013587-21kworker/u8:2+events_unbound19:50:182
199818899158,5cyclictest2314354-21latency_hist00:15:002
1998188991513,2cyclictest2315002-21kworker/u8:0+events_unbound00:28:522
1998188991512,2cyclictest2168387-21kworker/u8:3+flush-259:022:05:252
1998188991511,3cyclictest2283558-21kworker/u8:0+events_unbound23:57:512
1998188991511,3cyclictest2256173-21ntp_kernel_err23:15:222
1998185991512,2cyclictest2241631-21kworker/u8:2+flush-259:023:15:001
1998185991510,4cyclictest2299108-21/usr/sbin/munin00:00:161
199818199159,4cyclictest2235636-21ntpq22:55:260
1998181991513,1cyclictest60650irq/133-enp0s29f1:rx-023:59:190
1998181991513,1cyclictest60650irq/133-enp0s29f1:rx-023:30:280
1998181991513,1cyclictest60650irq/133-enp0s29f1:rx-023:08:200
1998181991513,1cyclictest60650irq/133-enp0s29f1:rx-021:30:250
1998181991513,1cyclictest60650irq/133-enp0s29f1:rx-000:15:270
1998181991513,1cyclictest60650irq/133-enp0s29f1:rx-000:10:290
1998181991512,2cyclictest60650irq/133-enp0s29f1:rx-023:25:230
1998181991511,3cyclictest2013587-21kworker/u8:2+flush-259:019:50:240
1998181991510,4cyclictest2256230-21ntp_kernel_pll_23:15:240
199819299149,4cyclictest2121861-21journal-offline21:07:313
199819299148,4cyclictest2174281-21/usr/sbin/munin22:00:173
1998192991411,2cyclictest2174032-21idleruntime-cro22:00:003
1998192991411,2cyclictest1950650-21kworker/u8:1+flush-259:019:30:203
1998192991410,3cyclictest0-21swapper/321:10:163
1998188991411,2cyclictest2205138-21kworker/u8:3+events_unbound00:23:482
1998188991410,3cyclictest2241631-21kworker/u8:2+events_unbound23:22:542
1998185991411,2cyclictest2178786-21unixbench_singl22:00:281
1998185991411,2cyclictest2059416-21unixbench_singl20:05:261
1998185991411,2cyclictest2059416-21unixbench_singl20:05:261
1998185991410,3cyclictest2226292-21/usr/sbin/munin22:50:231
1998185991410,3cyclictest2205138-21kworker/u8:3+flush-259:023:45:001
199818199148,5cyclictest2282428-21ntpq23:40:210
1998181991413,1cyclictest60650irq/133-enp0s29f1:rx-021:22:420
1998181991413,1cyclictest60650irq/133-enp0s29f1:rx-020:53:410
1998181991412,1cyclictest60650irq/133-enp0s29f1:rx-023:10:140
1998181991412,1cyclictest60650irq/133-enp0s29f1:rx-022:38:300
1998181991412,1cyclictest60650irq/133-enp0s29f1:rx-020:25:260
199819299139,3cyclictest2194686-21kworker/u8:0+flush-259:023:00:173
199819299138,4cyclictest2091265-21kworker/u8:1+flush-259:021:40:203
199819299137,4cyclictest2698871-21systemd-journal20:05:003
199818899136,5cyclictest2698871-21systemd-journal23:10:002
1998188991310,2cyclictest2019256-21fschecks_time19:30:152
199818599139,3cyclictest2698871-21systemd-journal20:39:591
199818599139,3cyclictest2336174-21apt-get00:35:141
199818599139,3cyclictest2293970-21apt-get23:55:121
199818599139,3cyclictest2075840-21kworker/u8:3+flush-259:020:45:051
199818599137,4cyclictest2304334-21apt-get00:05:131
1998185991311,1cyclictest0-21swapper/121:15:011
1998185991310,2cyclictest1993010-21kworker/u8:3+flush-259:019:40:181
199818199139,3cyclictest2153334-21kworker/u8:2+events_unbound21:45:250
199818199139,3cyclictest2028075-21ntp_kernel_pll_19:35:240
1998181991312,1cyclictest60650irq/133-enp0s29f1:rx-021:41:230
1998181991311,1cyclictest60650irq/133-enp0s29f1:rx-022:50:280
1998181991311,1cyclictest60650irq/133-enp0s29f1:rx-020:19:540
1998181991311,1cyclictest60650irq/133-enp0s29f1:rx-020:05:170
1998181991311,1cyclictest60650irq/133-enp0s29f1:rx-020:05:160
1998181991311,1cyclictest60650irq/133-enp0s29f1:rx-019:30:110
1998181991311,1cyclictest60650irq/133-enp0s29f1:rx-019:16:460
1998181991310,2cyclictest60650irq/133-enp0s29f1:rx-020:45:180
1998181991310,2cyclictest60650irq/133-enp0s29f1:rx-020:30:290
199819299129,2cyclictest2293215-21unixbench_multi23:50:263
199819299128,3cyclictest2013587-21kworker/u8:2+flush-259:019:50:163
199819299127,3cyclictest2205356-21systemd-run22:30:003
199819299127,3cyclictest2091265-21kworker/u8:1+flush-259:021:15:203
199819299126,4cyclictest2698871-21systemd-journal22:55:003
1998192991210,1cyclictest0-21swapper/300:25:003
199818899129,2cyclictest2008426-21sh19:20:002
199818899128,3cyclictest2315002-21kworker/u8:0+events_unbound00:30:082
199818899128,3cyclictest2315002-21kworker/u8:0+events_unbound00:30:072
199818599129,2cyclictest2096393-21cron20:45:001
199818199129,2cyclictest60650irq/133-enp0s29f1:rx-000:30:270
199818199129,2cyclictest60650irq/133-enp0s29f1:rx-000:30:260
199819299118,2cyclictest2003598-21cpuspeed_turbos19:15:173
199819299118,2cyclictest0-21swapper/300:25:203
199819299118,1cyclictest0-21swapper/323:30:013
199819299117,2cyclictest3000063-21rs:main3
199819299116,4cyclictest2309245-21latency_hist00:10:013
199819299116,4cyclictest2091265-21kworker/u8:1+flush-259:021:50:183
199819299116,3cyclictest2168387-21kworker/u8:3+flush-259:022:05:173
199819299114,5cyclictest2008981-21fw_forwarded_lo19:20:163
199819299110,9cyclictest2698871-21systemd-journal20:00:003
199818899118,2cyclictest2091265-21kworker/u8:1+events_unbound21:06:422
199818599118,2cyclictest2233826-21kthreadcore22:55:211
199818599118,2cyclictest2036663-21kthreadcore19:45:201
199818199119,2cyclictest60650irq/133-enp0s29f1:rx-000:25:160
199818199115,4cyclictest2053865-21ntpq20:00:210
199819299108,1cyclictest0-21swapper/319:35:143
199819299107,2cyclictest2299200-21cpuspeed_turbos00:00:133
199819299107,2cyclictest2078392-21tr20:25:203
199819299106,2cyclictest2235591-21ntpq22:55:253
199819299105,3cyclictest2242012-21diskstats23:05:123
199819299105,3cyclictest2206184-21sed22:30:173
199818599101,8cyclictest2246763-21awk23:10:011
19981929996,2cyclictest2091265-21kworker/u8:1+flush-259:020:55:223
19981859995,2cyclictest61-21kcompactd020:30:231
19981859990,2cyclictest2241588-21latency_hist23:05:001
19981859986,1cyclictest0-21swapper/122:05:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional