You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-18 - 16:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Mon May 18, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3967345998077,2cyclictest84340-21kworker/u8:0+events_unbound12:15:042
3967350994745,1cyclictest22026-21fwupd11:06:593
3967345993935,2cyclictest4128201-21kworker/u8:1+events_unbound10:50:242
3967345993835,2cyclictest3982558-21kworker/u8:1+flush-259:007:45:242
3967345993832,4cyclictest3903180-21kworker/u8:3+flush-259:007:50:232
3967345993734,2cyclictest4143643-21kworker/u8:3+events_unbound10:10:012
3967345993431,2cyclictest3982558-21kworker/u8:1+events_unbound08:15:242
3967345993428,4cyclictest79180-21latency_hist12:05:002
3967333993431,2cyclictest3903180-21kworker/u8:3+events_unbound07:15:200
3967350993329,3cyclictest4034918-21kworker/u8:2+flush-259:010:00:233
3967345993330,2cyclictest4169855-21kworker/u8:2+events_unbound10:35:252
3967345993227,3cyclictest4029235-21gpgv08:10:002
3967341993227,4cyclictest252-21systemd-journal12:24:591
3967341993226,4cyclictest4175256-21apt-get10:30:121
3967341993226,4cyclictest4175256-21apt-get10:30:111
3967341993226,4cyclictest4133336-21latency_hist09:50:001
3967333993226,4cyclictest4164664-21latency_hist10:20:010
3967345993027,2cyclictest4128201-21kworker/u8:1+events_unbound11:20:242
3967345993026,3cyclictest3952069-21kworker/u8:4+flush-259:007:20:232
3967333993024,4cyclictest252-21systemd-journal07:44:440
3967345992926,2cyclictest3913765-21kworker/u8:1+flush-259:007:10:252
3967345992925,3cyclictest7024-21kworker/u8:4+events_unbound10:55:262
3967333992924,3cyclictest4143643-21kworker/u8:3+flush-259:010:10:160
3967350992823,3cyclictest89498-21latency_hist12:14:593
3967350992822,4cyclictest4096917-21latency_hist09:15:013
3967345992825,2cyclictest48155-21kworker/u8:3+events_unbound11:45:242
3967345992821,5cyclictest3987885-21latency_hist07:30:002
3967333992822,4cyclictest6794-21latency_hist10:55:010
3967345992724,2cyclictest4024001-21kworker/u8:0+events_unbound08:25:142
3967345992720,5cyclictest4023130-21ntpq08:00:232
3967350992621,3cyclictest252-21systemd-journal07:29:563
3967345992623,2cyclictest4128201-21kworker/u8:1+flush-259:010:40:242
3967345992623,2cyclictest22035-21kworker/u8:2+flush-259:011:35:192
3967345992622,3cyclictest4128201-21kworker/u8:1+flush-259:012:05:152
3967341992622,3cyclictest4128201-21kworker/u8:1+flush-259:010:09:591
3967345992521,3cyclictest4091794-21kworker/u8:3+events_unbound09:20:002
3967345992521,3cyclictest4034918-21kworker/u8:2+flush-259:010:14:592
3967345992520,3cyclictest3982558-21kworker/u8:1+flush-259:007:40:002
3967341992520,4cyclictest95201-21gpgv12:20:001
3967350992420,3cyclictest4066289-21apt-get08:45:123
3967345992421,2cyclictest4143643-21kworker/u8:3+events_unbound10:20:582
3967345992421,2cyclictest4034918-21kworker/u8:2+events_unbound09:37:592
3967345992420,3cyclictest4128442-21apt-get09:45:112
3967345992418,4cyclictest4029122-21apt-get08:10:052
3967345992417,5cyclictest116228-21latency_hist12:40:002
3967345992416,3cyclictest7024-21kworker/u8:4+events_unbound11:00:212
3967333992421,2cyclictest4128201-21kworker/u8:1+flush-259:010:45:000
3967333992420,3cyclictest4034951-21apt-get08:15:110
3967350992318,4cyclictest4060456-21apt-get08:40:013
3967341992319,3cyclictest4169855-21kworker/u8:2+flush-259:010:40:161
3967341992317,4cyclictest58425-21latency_hist11:45:011
3967333992319,3cyclictest4128201-21kworker/u8:1+flush-259:009:55:230
3967333992319,3cyclictest4050445-21apt-get08:30:120
3967333992319,3cyclictest3913765-21kworker/u8:1+flush-259:007:10:170
3967333992318,4cyclictest4024001-21kworker/u8:0+flush-259:008:25:170
3967333992318,3cyclictest4091794-21kworker/u8:3+flush-259:009:15:170
3967333992318,3cyclictest4034918-21kworker/u8:2+events_unbound08:55:180
3967350992216,4cyclictest4153897-21latency_hist10:10:013
3967345992219,2cyclictest4066043-21kworker/u8:0+flush-259:008:50:052
3967345992218,3cyclictest4143643-21kworker/u8:3+events_unbound10:30:222
3967345992218,3cyclictest4143643-21kworker/u8:3+events_unbound10:30:212
3967345992218,3cyclictest3982558-21kworker/u8:1+events_unbound07:33:582
3967341992218,3cyclictest32894-21apt-get11:20:111
3967333992219,2cyclictest4034918-21kworker/u8:2+flush-259:009:40:210
3967333992216,4cyclictest4029767-21rred08:10:030
396733399221,19cyclictest1-21systemd11:48:570
3967350992118,2cyclictest4019077-21apt-get08:00:113
3967350992115,4cyclictest509-21rs:main3
3967350992115,4cyclictest4180353-21apt-get10:35:113
3967345992117,3cyclictest4034918-21kworker/u8:2+flush-259:009:30:002
3967341992117,3cyclictest89689-21cp12:15:001
3967341992117,3cyclictest4164906-21/usr/sbin/munin10:20:211
3967341992117,3cyclictest4045060-21kworker/u8:3+flush-259:008:30:261
3967333992117,3cyclictest500-21systemd-logind07:55:000
3967333992117,3cyclictest4143643-21kworker/u8:3+flush-259:010:30:250
3967333992117,3cyclictest4143643-21kworker/u8:3+flush-259:010:30:240
3967333992117,3cyclictest4091794-21kworker/u8:3+events_unbound09:20:150
3967345992017,2cyclictest7024-21kworker/u8:4+events_unbound11:05:252
3967345992017,2cyclictest37822-21kworker/u8:0+flush-259:011:55:012
3967345992012,3cyclictest4128201-21kworker/u8:1+events_unbound11:30:232
3967341992015,4cyclictest252-21systemd-journal11:06:581
3967350991915,3cyclictest53551-21apt-get11:40:093
3967345991916,2cyclictest4066043-21kworker/u8:0+events_unbound08:48:582
3967345991915,3cyclictest4034918-21kworker/u8:2+events_unbound08:40:132
3967341991915,3cyclictest4034918-21kworker/u8:2+flush-259:009:50:251
3967341991912,6cyclictest0-21swapper/109:15:011
3967333991915,3cyclictest7024-21kworker/u8:4+events_unbound10:55:270
3967333991915,3cyclictest4128201-21kworker/u8:1+flush-259:010:45:260
3967333991913,4cyclictest4170257-21diskstats10:25:140
3967333991913,4cyclictest4170257-21diskstats10:25:130
3967333991912,5cyclictest4029116-21latency_hist08:10:000
3967350991814,3cyclictest4128201-21kworker/u8:1+flush-259:010:25:003
3967345991816,2cyclictest3982558-21kworker/u8:1+events_unbound07:40:232
3967345991815,2cyclictest4034918-21kworker/u8:2+flush-259:009:40:202
3967345991814,3cyclictest4091794-21kworker/u8:3+events_unbound09:15:122
3967345991814,3cyclictest4045060-21kworker/u8:3+events_unbound08:30:232
3967345991814,3cyclictest4034918-21kworker/u8:2+flush-259:008:55:162
3967345991814,3cyclictest3982558-21kworker/u8:1+events_unbound07:55:122
3967345991813,4cyclictest4133604-21apt-get09:50:122
3967345991812,4cyclictest1649-21latency_hist10:50:002
3967341991814,3cyclictest4029635-21base6408:10:011
3967341991813,4cyclictest4065135-21ntpq08:40:211
3967341991813,4cyclictest252-21systemd-journal11:32:581
3967333991814,2cyclictest60350irq/132-enp0s29f1:rx-009:30:200
3967333991813,4cyclictest7024-21kworker/u8:4+events_unbound11:00:180
3967333991813,3cyclictest4128201-21kworker/u8:1+flush-259:012:05:190
3967333991813,3cyclictest252-21systemd-journal07:45:570
3967350991715,1cyclictest4029138-21https08:10:003
3967350991714,2cyclictest4113400-21irqcore09:30:153
3967345991714,2cyclictest4128201-21kworker/u8:1+events_unbound11:25:192
3967345991714,2cyclictest4034918-21kworker/u8:2+flush-259:009:30:182
3967345991713,3cyclictest4034918-21kworker/u8:2+flush-259:010:00:182
3967341991714,2cyclictest3977571-21idleruntime-cro07:20:001
3967341991712,4cyclictest252-21systemd-journal11:55:001
3967333991713,3cyclictest4060513-21apt-config08:40:000
3967333991710,5cyclictest4085747-21ntpq09:00:210
3967333991710,5cyclictest252-21systemd-journal08:05:000
3967350991613,2cyclictest90601-21irqcore12:15:163
3967350991611,3cyclictest4123036-21latency_hist09:40:003
3967345991613,2cyclictest4045060-21kworker/u8:3+events_unbound08:35:052
3967345991612,3cyclictest64280-21kworker/u8:4+flush-259:011:55:412
3967345991612,3cyclictest4159730-21kworker/u8:0+events_unbound10:17:572
3967345991612,3cyclictest4128201-21kworker/u8:1+flush-259:009:55:182
3967345991612,3cyclictest4128201-21kworker/u8:1+events_unbound11:15:242
3967345991612,3cyclictest3972040-21kworker/u8:0+events_unbound07:15:272
3967341991612,3cyclictest38065-21apt-get11:25:091
396733399169,5cyclictest3981842-21ntpq07:20:210
3967333991614,1cyclictest60350irq/132-enp0s29f1:rx-007:30:160
3967333991612,3cyclictest60350irq/132-enp0s29f1:rx-009:35:150
3967333991612,2cyclictest73987-21turbostat12:00:010
3967333991612,2cyclictest42965-21turbostat11:30:010
3967333991611,3cyclictest100927-21apt-get12:25:120
3967350991512,2cyclictest7241-21fschecks_count10:55:193
3967350991512,2cyclictest4169855-21kworker/u8:2+flush-259:010:25:253
3967350991512,2cyclictest4169855-21kworker/u8:2+flush-259:010:25:243
3967350991512,2cyclictest4081447-21sh09:00:003
3967350991512,2cyclictest4034918-21kworker/u8:2+flush-259:009:49:593
3967345991512,2cyclictest4169855-21kworker/u8:2+flush-259:010:25:192
3967345991512,2cyclictest4169855-21kworker/u8:2+flush-259:010:25:192
3967345991511,3cyclictest4066043-21kworker/u8:0+events_unbound09:13:582
3967345991511,3cyclictest4034918-21kworker/u8:2+events_unbound08:20:182
3967341991512,2cyclictest4034918-21kworker/u8:2+flush-259:009:40:161
3967341991511,3cyclictest4128201-21kworker/u8:1+flush-259:009:55:221
3967341991510,4cyclictest94831-21kworker/u8:1+flush-259:012:35:231
3967341991510,4cyclictest252-21systemd-journal10:59:561
3967333991513,1cyclictest60350irq/132-enp0s29f1:rx-011:50:220
3967333991513,1cyclictest60350irq/132-enp0s29f1:rx-009:05:160
3967333991513,1cyclictest60350irq/132-enp0s29f1:rx-008:52:440
3967333991512,2cyclictest64280-21kworker/u8:4+flush-259:012:15:240
3967333991512,2cyclictest60350irq/132-enp0s29f1:rx-011:20:100
3967333991512,2cyclictest60350irq/132-enp0s29f1:rx-009:45:120
3967333991512,2cyclictest4184452-21ntp_offset10:35:230
396735099149,4cyclictest252-21systemd-journal11:48:573
3967350991412,1cyclictest81745-21kthreadcore12:05:203
3967350991412,1cyclictest4018807-21cron08:00:003
3967350991411,2cyclictest99898-21proc_pri12:20:263
3967350991410,3cyclictest4034918-21kworker/u8:2+flush-259:008:20:163
3967350991410,3cyclictest3998447-21apt-get07:40:123
3967350991410,3cyclictest37822-21kworker/u8:0+flush-259:011:30:263
396734599149,4cyclictest48155-21kworker/u8:3+events_unbound12:13:172
3967345991410,3cyclictest48155-21kworker/u8:3+events_unbound11:40:122
396734199148,4cyclictest31753-21ntpq11:15:211
3967341991412,1cyclictest3986657-21kthreadcore07:25:181
3967341991412,1cyclictest104513-21kthreadcore12:25:181
3967341991411,2cyclictest4045563-21fschecks_time08:25:151
3967341991411,2cyclictest4016206-21kthreadcore07:55:171
3967341991411,2cyclictest4013612-21cron07:55:001
3967341991411,2cyclictest3971493-21ntp_kernel_err07:10:211
3967341991410,3cyclictest509-21rs:main1
3967333991413,1cyclictest60350irq/132-enp0s29f1:rx-008:20:040
3967333991412,1cyclictest60350irq/132-enp0s29f1:rx-012:37:330
3967333991412,1cyclictest60350irq/132-enp0s29f1:rx-012:20:580
3967333991412,1cyclictest60350irq/132-enp0s29f1:rx-011:10:110
3967333991412,1cyclictest60350irq/132-enp0s29f1:rx-008:46:530
3967333991412,1cyclictest60350irq/132-enp0s29f1:rx-007:55:160
3967333991411,2cyclictest60350irq/132-enp0s29f1:rx-012:30:120
3967333991411,2cyclictest60350irq/132-enp0s29f1:rx-012:10:170
3967333991411,2cyclictest60350irq/132-enp0s29f1:rx-008:40:110
3967333991411,2cyclictest4034918-21kworker/u8:2+events_unbound09:10:190
3967333991410,3cyclictest60350irq/132-enp0s29f1:rx-007:35:130
3967333991410,3cyclictest27681-21kworker/u8:3+events_unbound11:15:140
396735099139,3cyclictest64280-21kworker/u8:4+flush-259:011:50:243
396735099139,3cyclictest4029122-21apt-get08:10:013
396735099138,3cyclictest252-21systemd-journal10:17:003
396734599139,3cyclictest94831-21kworker/u8:1+events_unbound12:30:212
396734599139,3cyclictest4034918-21kworker/u8:2+events_unbound09:00:232
396734199138,4cyclictest4102715-21apt-get09:20:111
3967341991311,1cyclictest111034-21idleruntime-cro12:35:001
3967341991311,1cyclictest0-21swapper/108:10:001
3967341991310,2cyclictest4108268-21irqcore09:25:161
3967341991310,2cyclictest4076992-21sed08:55:191
3967333991312,1cyclictest60350irq/132-enp0s29f1:rx-010:20:140
3967333991311,1cyclictest60350irq/132-enp0s29f1:rx-011:35:130
3967333991311,1cyclictest60350irq/132-enp0s29f1:rx-009:50:130
3967333991311,1cyclictest60350irq/132-enp0s29f1:rx-009:25:240
3967333991310,2cyclictest60350irq/132-enp0s29f1:rx-011:40:170
3967333991310,2cyclictest60350irq/132-enp0s29f1:rx-010:00:130
3967333991310,2cyclictest60350irq/132-enp0s29f1:rx-007:25:130
3967333991310,2cyclictest43292-21cpuspeed_turbos11:30:140
396735099129,2cyclictest500-21systemd-logind08:55:003
396735099128,3cyclictest3982558-21kworker/u8:1+flush-259:007:45:253
396735099125,5cyclictest110178-21ntpq12:30:223
3967350991210,1cyclictest0-21swapper/309:45:003
3967350991210,1cyclictest0-21swapper/308:25:193
396734599129,2cyclictest94831-21kworker/u8:1+events_unbound12:25:132
396734599129,2cyclictest84340-21kworker/u8:0+events_unbound12:20:282
396734599128,3cyclictest7024-21kworker/u8:4+events_unbound11:10:592
396734599128,3cyclictest4066043-21kworker/u8:0+flush-259:009:05:412
396734199129,2cyclictest4060484-21gpgv08:40:001
396734199129,1cyclictest0-21swapper/111:50:011
396734199128,3cyclictest4143643-21kworker/u8:3+flush-259:010:15:171
396734199128,2cyclictest40908392sleep109:05:221
396734199127,4cyclictest252-21systemd-journal10:30:001
396734199127,4cyclictest252-21systemd-journal10:29:591
396733399129,2cyclictest60350irq/132-enp0s29f1:rx-010:09:590
396735099119,1cyclictest4133579-21/usr/sbin/munin09:50:183
396735099118,2cyclictest4192983-21kthreadcore10:45:183
396735099118,2cyclictest4008973-21fschecks_time07:50:143
396735099118,1cyclictest0-21swapper/309:55:193
396735099111,2cyclictest4060838-21cat08:40:013
396734199119,1cyclictest0-21swapper/107:25:001
396734199118,2cyclictest4143952-21cpuspeed_turbos10:00:141
396734199118,2cyclictest4097245-21cpuspeed_turbos09:15:131
396734199118,2cyclictest4067306-21kthreadcore08:45:171
396734199118,2cyclictest0-21swapper/112:05:241
396734199118,1cyclictest0-21swapper/110:10:171
396734199116,4cyclictest4040116-21/usr/sbin/munin08:20:221
396735099108,1cyclictest0-21swapper/310:15:003
396735099107,2cyclictest74649-21irqcore12:00:163
396735099107,2cyclictest42958-21latency_hist11:30:013
396735099107,2cyclictest3972040-21kworker/u8:0+flush-259:007:20:003
396735099107,1cyclictest0-21swapper/309:30:003
396735099107,1cyclictest0-21swapper/309:15:143
396735099107,1cyclictest0-21swapper/309:00:123
396735099105,3cyclictest273-21systemd-udevd08:30:203
396735099100,3cyclictest32637-21latency_hist11:20:003
396734199108,1cyclictest3993505-21fschecks_count07:35:141
396734199108,1cyclictest0-21swapper/111:55:221
396734199108,1cyclictest0-21swapper/108:05:001
396734199106,3cyclictest4190685-21apt-get10:45:141
396734199106,3cyclictest4112973-21apt-get09:30:101
396734199106,3cyclictest4034918-21kworker/u8:2+flush-259:008:50:261
396734199106,2cyclictest4002501-21ntpq07:40:231
396733399108,1cyclictest60350irq/132-enp0s29f1:rx-012:00:200
39673509996,2cyclictest491ktimers/310:50:203
39673509996,2cyclictest12338-21cpuspeed_turbos11:00:133
39673509995,3cyclictest491ktimers/307:35:133
39673509994,3cyclictest3967636-21grep07:10:143
39673509991,6cyclictest37808-21turbostat11:25:013
39673419997,1cyclictest0-21swapper/109:00:231
39673509986,1cyclictest0-21swapper/310:30:203
39673509986,1cyclictest0-21swapper/310:30:193
39673509985,2cyclictest491ktimers/311:55:203
39673509985,1cyclictest0-21swapper/307:30:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional