You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 03:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Fri Mar 20, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4003799993933,4cyclictest22818-21/usr/sbin/munin10:35:152
4003791993935,3cyclictest69653-21apt-get11:20:110
4003799993634,1cyclictest4019256-21kworker/u8:2+flush-259:007:45:222
4003791993631,4cyclictest115967-21kworker/u8:1+flush-259:012:20:160
4003799993532,2cyclictest4071140-21kworker/u8:3+flush-259:008:30:182
4003801993429,3cyclictest12567-21apt-get10:25:113
4003794993329,3cyclictest3998796-21kworker/u8:1+events_unbound07:15:141
4003794993327,4cyclictest509-21rs:main1
4003799993229,2cyclictest4019256-21kworker/u8:2+flush-259:008:00:212
4003791993228,3cyclictest4060655-21apt-get08:05:110
4003799993129,2cyclictest4019256-21kworker/u8:2+events_unbound08:15:232
4003799993128,2cyclictest101036-21kworker/u8:3+flush-259:012:00:262
4003794993127,3cyclictest4138721-21apt-get09:20:111
4003801993025,4cyclictest38324-21/usr/sbin/munin10:50:153
4003801993023,5cyclictest4128603-21diskstats09:10:143
4003799993027,2cyclictest4019256-21kworker/u8:2+events_unbound08:35:142
4003799993027,2cyclictest125691-21kworker/u8:0+flush-259:012:15:232
4003799993024,5cyclictest142257-21/usr/sbin/munin12:30:132
4003794992927,1cyclictest2239-21/usr/sbin/munin10:15:141
4003799992825,2cyclictest4081441-21kworker/u8:1+flush-259:009:40:222
4003794992825,2cyclictest69317-21kworker/u8:1+flush-259:011:20:161
4003794992824,3cyclictest4019256-21kworker/u8:2+flush-259:007:50:121
4003794992824,3cyclictest3987439-21kworker/u8:2+flush-259:007:10:241
4003791992823,3cyclictest4164455-21apt-get09:45:100
4003791992821,6cyclictest4158185-21ntpq09:35:230
4003799992723,3cyclictest38110-21kworker/u8:2+flush-259:011:30:212
4003799992722,4cyclictest22604-21kworker/u8:0+flush-259:011:10:232
4003794992723,3cyclictest115967-21kworker/u8:1+flush-259:012:05:101
4003794992723,3cyclictest101036-21kworker/u8:3+flush-259:011:55:161
4003794992722,4cyclictest4081681-21apt-get08:25:111
4003799992623,2cyclictest4179634-21kworker/u8:3+events_unbound10:15:242
4003799992623,2cyclictest106618-21kworker/u8:2+flush-259:012:00:002
4003799992622,3cyclictest22604-21kworker/u8:0+flush-259:011:15:252
4003799992620,5cyclictest4019231-21latency_hist07:25:012
4003791992622,3cyclictest43488-21apt-get10:55:100
4003801992515,9cyclictest507-21in:imuxsock08:55:003
4003799992521,3cyclictest4019256-21kworker/u8:2+flush-259:008:45:202
4003799992520,4cyclictest115967-21kworker/u8:1+events_unbound12:11:462
4003794992521,3cyclictest4081441-21kworker/u8:1+events_unbound09:00:141
4003794992521,3cyclictest4019256-21kworker/u8:2+events_unbound08:40:131
4003801992419,3cyclictest85606-21apt-get11:35:123
4003799992421,3cyclictest101036-21kworker/u8:3+events_unbound12:05:242
4003799992421,2cyclictest4019256-21kworker/u8:2+events_unbound07:25:152
4003799992420,3cyclictest38110-21kworker/u8:2+events_unbound11:45:462
4003799992419,3cyclictest4071140-21kworker/u8:3+flush-259:008:20:162
4003799992419,3cyclictest22604-21kworker/u8:0+events_unbound11:10:012
4003794992421,2cyclictest75563-21kworker/u8:0+flush-259:011:25:161
4003794992420,3cyclictest4019256-21kworker/u8:2+events_unbound07:55:231
4003794992419,3cyclictest131372-21kworker/u8:2+events_unbound12:25:181
4003791992420,3cyclictest33140-21/usr/sbin/munin10:45:140
4003799992319,3cyclictest75563-21kworker/u8:0+flush-259:011:40:152
4003799992318,3cyclictest4050307-21/usr/sbin/munin07:55:142
4003794992320,2cyclictest4117874-21kworker/u8:0+events_unbound09:30:121
4003794992320,2cyclictest4013765-21kworker/u8:0+events_unbound08:10:231
4003794992319,3cyclictest48646-21apt-get11:00:101
4003794992319,3cyclictest115967-21kworker/u8:1+flush-259:012:30:231
4003794992319,2cyclictest4190344-21kworker/u8:1+events_unbound10:40:221
4003791992318,4cyclictest121690-21apt-get12:10:120
4003799992219,2cyclictest4179634-21kworker/u8:3+flush-259:010:05:262
4003799992218,3cyclictest4190344-21kworker/u8:1+events_unbound10:50:172
4003794992218,3cyclictest4179634-21kworker/u8:3+flush-259:010:05:221
4003794992218,3cyclictest4179624-21gpgv10:00:001
4003794992218,3cyclictest4013765-21kworker/u8:0+flush-259:007:45:161
4003794992218,3cyclictest38110-21kworker/u8:2+events_unbound11:05:211
4003794992218,2cyclictest4013765-21kworker/u8:0+flush-259:007:40:241
4003801992117,3cyclictest75236-21/usr/sbin/munin11:25:143
4003801992116,3cyclictest116503-21apt-get12:05:083
4003799992118,2cyclictest4117874-21kworker/u8:0+flush-259:009:20:232
4003799992117,3cyclictest101036-21kworker/u8:3+events_unbound11:50:232
4003794992117,3cyclictest4081441-21kworker/u8:1+events_unbound09:10:131
4003791992118,2cyclictest4071100-21cron08:15:000
4003791992117,3cyclictest4034881-21/usr/sbin/munin07:40:140
4003791992117,3cyclictest4024397-21sh07:30:000
4003791992116,4cyclictest252-21systemd-journal09:30:000
4003799992017,2cyclictest4081441-21kworker/u8:1+flush-259:009:45:102
4003794992017,2cyclictest75563-21kworker/u8:0+events_unbound11:40:001
4003794992016,3cyclictest4081441-21kworker/u8:1+events_unbound08:50:191
4003794992015,3cyclictest4112930-21/usr/sbin/munin08:55:141
4003794992015,3cyclictest252-21systemd-journal08:39:251
4003794992014,5cyclictest4184284-21ntpq10:00:211
4003791992016,3cyclictest4118070-21/usr/sbin/munin09:00:240
4003801991916,2cyclictest4019256-21kworker/u8:2+flush-259:007:35:163
4003801991915,3cyclictest252-21systemd-journal10:20:003
4003799991917,2cyclictest4117874-21kworker/u8:0+events_unbound10:00:232
4003799991916,2cyclictest4050068-21idleruntime-cro07:55:002
4003799991916,2cyclictest38110-21kworker/u8:2+flush-259:011:35:212
4003799991916,2cyclictest1615-21kworker/u8:4+events_unbound10:26:452
4003799991915,3cyclictest4117874-21kworker/u8:0+events_unbound09:15:192
4003799991915,3cyclictest4013765-21kworker/u8:0+events_unbound07:32:472
4003799991915,3cyclictest3987439-21kworker/u8:2+flush-259:007:10:262
4003799991915,3cyclictest101036-21kworker/u8:3+events_unbound12:29:472
4003794991916,2cyclictest4190344-21kworker/u8:1+events_unbound10:45:231
4003794991916,2cyclictest4013765-21kworker/u8:0+events_unbound07:20:211
4003794991915,3cyclictest4081441-21kworker/u8:1+events_unbound09:25:231
4003794991914,3cyclictest4019495-21apt-get07:25:101
4003794991914,3cyclictest126828-21apt-get12:15:091
4003791991915,2cyclictest4029768-21apt-get07:35:110
4003799991815,2cyclictest69317-21kworker/u8:1+flush-259:011:20:212
4003799991815,2cyclictest4071140-21kworker/u8:3+events_unbound08:25:162
4003799991815,2cyclictest101036-21kworker/u8:3+events_unbound12:21:462
4003799991814,3cyclictest75563-21kworker/u8:0+flush-259:011:25:452
4003799991814,3cyclictest4190344-21kworker/u8:1+events_unbound10:13:452
4003799991814,3cyclictest4081441-21kworker/u8:1+events_unbound08:52:452
4003794991814,3cyclictest1615-21kworker/u8:4+flush-259:010:34:571
4003801991714,2cyclictest500-21systemd-logind09:40:003
4003801991712,3cyclictest4066400-21diskstats08:10:133
400379999174,7cyclictest4158993-21cron09:40:002
4003799991714,2cyclictest4117874-21kworker/u8:0+flush-259:009:05:262
4003799991714,2cyclictest4117874-21kworker/u8:0+flush-259:009:05:252
4003799991714,2cyclictest4013765-21kworker/u8:0+events_unbound08:10:282
4003799991713,3cyclictest4117874-21kworker/u8:0+flush-259:009:25:112
4003799991713,3cyclictest4097490-21/usr/sbin/munin08:40:132
4003799991713,3cyclictest4013765-21kworker/u8:0+events_unbound07:39:462
4003799991713,3cyclictest151410-21ntp_kernel_pll_12:35:202
4003794991714,2cyclictest115967-21kworker/u8:1+flush-259:012:35:201
4003791991714,2cyclictest4190344-21kworker/u8:1+flush-259:010:50:150
4003791991713,2cyclictest60350irq/132-enp0s29f1:rx-007:20:100
4003801991613,2cyclictest4119114-21kthreadcore09:00:163
4003801991612,3cyclictest4133553-21/usr/sbin/munin09:15:133
4003801991612,2cyclictest105943-21cron11:55:013
4003801991610,4cyclictest116232-21latency_hist12:05:013
4003799991613,2cyclictest4190344-21kworker/u8:1+events_unbound10:46:472
4003799991613,2cyclictest4179634-21kworker/u8:3+events_unbound10:44:462
4003799991613,2cyclictest4019256-21kworker/u8:2+events_unbound07:40:222
4003799991613,2cyclictest22604-21kworker/u8:0+events_unbound10:55:232
4003799991613,2cyclictest12329-21kworker/u8:0+flush-259:010:25:012
4003794991612,3cyclictest500-21systemd-logind10:15:011
4003794991612,3cyclictest4081441-21kworker/u8:1+flush-259:009:35:151
4003794991612,3cyclictest4081441-21kworker/u8:1+events_unbound09:40:211
4003794991612,3cyclictest22604-21kworker/u8:0+flush-259:010:55:251
4003794991612,3cyclictest115967-21kworker/u8:1+flush-259:012:20:201
4003794991610,4cyclictest4065561-21latency_hist08:10:011
4003791991614,1cyclictest60350irq/132-enp0s29f1:rx-008:00:250
4003791991613,2cyclictest61150irq/133-enp0s29f1:tx-011:05:170
4003791991613,2cyclictest60350irq/132-enp0s29f1:rx-011:45:140
4003791991613,2cyclictest60350irq/132-enp0s29f1:rx-010:10:130
4003791991613,2cyclictest60350irq/132-enp0s29f1:rx-010:05:230
4003791991613,2cyclictest60350irq/132-enp0s29f1:rx-009:10:070
4003791991613,2cyclictest60350irq/132-enp0s29f1:rx-008:35:160
4003791991613,2cyclictest60350irq/132-enp0s29f1:rx-008:30:130
4003791991611,4cyclictest252-21systemd-journal09:15:450
4003799991512,2cyclictest4117874-21kworker/u8:0+flush-259:009:55:272
4003799991512,2cyclictest4081441-21kworker/u8:1+events_unbound09:00:462
4003799991511,3cyclictest4081441-21kworker/u8:1+events_unbound09:54:382
4003799991511,3cyclictest4081441-21kworker/u8:1+events_unbound09:32:462
4003799991511,3cyclictest38110-21kworker/u8:2+events_unbound11:00:272
4003794991512,2cyclictest63801-21cron11:14:591
4003794991512,2cyclictest4078831-21kthreadcore08:20:171
4003794991511,3cyclictest75563-21kworker/u8:0+events_unbound11:40:161
4003791991514,1cyclictest60350irq/132-enp0s29f1:rx-009:40:310
4003791991513,2cyclictest61150irq/133-enp0s29f1:tx-012:35:170
4003791991513,2cyclictest60350irq/132-enp0s29f1:rx-012:16:280
4003791991513,2cyclictest60350irq/132-enp0s29f1:rx-009:24:450
4003791991513,1cyclictest60350irq/132-enp0s29f1:rx-012:25:160
4003791991513,1cyclictest60350irq/132-enp0s29f1:rx-012:05:080
4003791991513,1cyclictest60350irq/132-enp0s29f1:rx-011:25:140
4003791991513,1cyclictest60350irq/132-enp0s29f1:rx-010:25:210
4003791991513,1cyclictest60350irq/132-enp0s29f1:rx-009:50:220
4003791991513,1cyclictest60350irq/132-enp0s29f1:rx-009:05:200
4003791991513,1cyclictest60350irq/132-enp0s29f1:rx-009:05:200
4003791991513,1cyclictest60350irq/132-enp0s29f1:rx-008:43:390
4003791991512,2cyclictest60350irq/132-enp0s29f1:rx-012:01:180
4003791991512,2cyclictest60350irq/132-enp0s29f1:rx-011:50:120
4003791991512,2cyclictest60350irq/132-enp0s29f1:rx-010:18:160
400380199149,4cyclictest4039766-21latency_hist07:45:013
400380199149,4cyclictest4019256-21kworker/u8:2+flush-259:008:45:233
4003801991412,1cyclictest0-21swapper/312:15:003
4003801991412,1cyclictest0-21swapper/307:19:593
4003801991411,2cyclictest4143605-21cron09:25:003
4003801991410,3cyclictest4190561-21/usr/sbin/munin10:10:153
4003801991410,3cyclictest252-21systemd-journal11:17:453
4003801991410,3cyclictest0-21swapper/312:35:013
400380199140,4cyclictest252-21systemd-journal07:55:003
4003799991411,2cyclictest4132397-21ntp_kernel_err09:10:202
4003799991411,2cyclictest4061669-21kthreadcore08:05:162
4003794991410,3cyclictest4190344-21kworker/u8:1+flush-259:010:20:161
4003794991410,3cyclictest4019256-21kworker/u8:2+events_unbound08:00:221
4003791991412,2cyclictest60350irq/132-enp0s29f1:rx-010:30:110
4003791991412,2cyclictest60350irq/132-enp0s29f1:rx-010:22:060
4003791991412,1cyclictest60350irq/132-enp0s29f1:rx-011:43:310
4003791991412,1cyclictest60350irq/132-enp0s29f1:rx-011:30:240
4003791991412,1cyclictest60350irq/132-enp0s29f1:rx-010:03:210
4003791991412,1cyclictest60350irq/132-enp0s29f1:rx-008:55:190
4003791991412,1cyclictest60350irq/132-enp0s29f1:rx-008:50:190
4003791991411,2cyclictest60350irq/132-enp0s29f1:rx-011:55:110
4003791991411,2cyclictest60350irq/132-enp0s29f1:rx-010:35:240
4003791991411,2cyclictest4014029-21turbostat07:20:000
400380199139,3cyclictest4123346-21cpuspeed_turbos09:05:143
400380199139,3cyclictest4123346-21cpuspeed_turbos09:05:143
4003801991311,1cyclictest0-21swapper/308:20:003
4003801991310,2cyclictest4168448-21ntp_kernel_pll_09:45:203
4003801991310,2cyclictest100374-21unixbench_singl11:45:243
400379499139,3cyclictest75563-21kworker/u8:0+events_unbound11:45:211
400379499139,3cyclictest4117874-21kworker/u8:0+flush-259:009:15:201
400379499139,3cyclictest4013765-21kworker/u8:0+flush-259:007:35:131
400379499139,3cyclictest115967-21kworker/u8:1+events_unbound12:14:451
4003794991310,2cyclictest80867-21sed11:30:171
4003794991310,2cyclictest0-21swapper/109:50:001
4003791991312,1cyclictest60350irq/132-enp0s29f1:rx-011:37:430
4003791991311,1cyclictest60350irq/132-enp0s29f1:rx-011:00:230
4003791991311,1cyclictest60350irq/132-enp0s29f1:rx-008:45:170
4003791991311,1cyclictest60350irq/132-enp0s29f1:rx-008:19:230
4003791991311,1cyclictest60350irq/132-enp0s29f1:rx-007:45:240
4003791991311,1cyclictest60350irq/132-enp0s29f1:rx-007:30:140
4003791991310,2cyclictest60350irq/132-enp0s29f1:rx-011:15:200
4003791991310,2cyclictest60350irq/132-enp0s29f1:rx-007:50:140
400380199129,2cyclictest53248-21kworker/u8:3+flush-259:011:10:013
400380199129,2cyclictest4076683-21cpuspeed_turbos08:20:123
400380199127,4cyclictest252-21systemd-journal09:55:003
4003801991210,1cyclictest0-21swapper/312:00:003
4003801991210,1cyclictest0-21swapper/308:00:163
400379999128,3cyclictest4081441-21kworker/u8:1+events_unbound08:55:122
400379499129,2cyclictest4179634-21kworker/u8:3+flush-259:010:39:551
400379199127,4cyclictest252-21systemd-journal11:15:000
4003791991211,1cyclictest60350irq/132-enp0s29f1:rx-009:55:240
4003791991210,1cyclictest60350irq/132-enp0s29f1:rx-012:30:220
4003791991210,1cyclictest60350irq/132-enp0s29f1:rx-010:40:220
400380199118,2cyclictest4113214-21fschecks_time08:55:143
400380199118,2cyclictest4087014-21fschecks_count08:30:143
400380199118,2cyclictest0-21swapper/310:45:123
400380199118,2cyclictest0-21swapper/310:00:133
400380199110,3cyclictest95431kthreadcore10:20:163
400379999118,2cyclictest22095-21unixbench_multi10:30:232
400379499118,2cyclictest4091679-21idleruntime-cro08:35:001
400379499118,2cyclictest4081441-21kworker/u8:1+events_unbound09:50:131
400380199100,2cyclictest4019231-21latency_hist07:25:013
400379499108,2cyclictest101036-21kworker/u8:3+flush-259:011:50:201
400379499107,2cyclictest4103064-21sort08:45:151
400379499107,2cyclictest12683-21cpuspeed_turbos10:25:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional