You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 16:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Mon Mar 16, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3711383994643,2cyclictest3652697-21kworker/u8:0+flush-259:007:45:012
3711387994037,2cyclictest3944300-21kworker/u8:0+flush-259:011:00:203
3711382994037,2cyclictest3777884-21kworker/u8:2+flush-259:009:55:001
3711387993835,2cyclictest3777884-21kworker/u8:2+flush-259:008:35:233
3711387993731,4cyclictest4011691-21latency_hist12:00:013
3711387993431,2cyclictest3658078-21kworker/u8:3+events_unbound07:40:173
3711383993431,2cyclictest3882038-21kworker/u8:0+events_unbound10:40:202
3711387993330,2cyclictest3652697-21kworker/u8:0+flush-259:007:15:003
3711387993326,5cyclictest3809425-21latency_hist08:45:013
3711387993229,2cyclictest3815267-21kworker/u8:3+flush-259:008:50:243
3711379993227,3cyclictest3970328-21apt-get11:20:100
3711387993129,2cyclictest3892334-21kworker/u8:3+events_unbound10:30:143
3711387993129,1cyclictest3882038-21kworker/u8:0+events_unbound10:15:233
3711387993128,2cyclictest3777884-21kworker/u8:2+flush-259:009:35:153
3711387993128,2cyclictest3777884-21kworker/u8:2+flush-259:009:25:223
3711383993125,4cyclictest3851571-21diskstats09:25:122
3711387993027,2cyclictest3969938-21kworker/u8:4+events_unbound11:30:133
3711383993027,2cyclictest3652697-21kworker/u8:0+events_unbound07:20:182
3711379993026,3cyclictest3928609-21kworker/u8:2+events_unbound11:10:170
3711379992925,3cyclictest3658078-21kworker/u8:3+events_unbound07:30:000
3711387992823,3cyclictest3944300-21kworker/u8:0+flush-259:011:15:163
3711387992823,3cyclictest3944300-21kworker/u8:0+flush-259:011:15:153
3711387992823,3cyclictest3658078-21kworker/u8:3+flush-259:007:15:173
3711387992822,4cyclictest3777884-21kworker/u8:2+events_unbound09:30:163
3711379992824,3cyclictest3652697-21kworker/u8:0+flush-259:007:20:160
3711387992723,3cyclictest3969938-21kworker/u8:4+flush-259:012:20:253
3711383992723,3cyclictest3969938-21kworker/u8:4+flush-259:011:55:262
3711382992721,4cyclictest3824469-21ntpq08:55:201
3711382992721,4cyclictest3824469-21ntpq08:55:201
3711379992721,5cyclictest3772236-21ntpq08:05:220
3711379992721,4cyclictest3798749-21apt-get08:35:000
3711387992623,2cyclictest3882038-21kworker/u8:0+flush-259:010:40:163
3711387992623,2cyclictest3777884-21kworker/u8:2+flush-259:009:15:203
3711387992620,4cyclictest3793610-21latency_hist08:30:013
3711387992618,6cyclictest3798753-21latency_hist08:35:003
3711382992623,2cyclictest3747367-21kworker/u8:2+flush-259:007:55:011
3711382992621,4cyclictest3969215-21ntpq11:15:201
3711382992621,4cyclictest3969215-21ntpq11:15:201
3711382992621,3cyclictest3892553-21/usr/sbin/munin10:05:151
3711379992623,2cyclictest3777884-21kworker/u8:2+flush-259:009:45:000
3711387992521,3cyclictest3658078-21kworker/u8:3+events_unbound08:10:193
3711387992521,3cyclictest3652697-21kworker/u8:0+flush-259:007:50:013
3711382992520,3cyclictest3825384-21kworker/u8:1+flush-259:009:20:161
3711379992522,2cyclictest3861704-21apt-get09:35:110
3711387992421,2cyclictest3652697-21kworker/u8:0+events_unbound07:30:233
3711387992420,3cyclictest3752726-21apt-get07:50:113
3711382992419,4cyclictest3757887-21apt-get07:55:111
3711382992419,3cyclictest3939379-21apt-get10:50:121
3711383992319,3cyclictest3767915-21kworker/u8:0+events_unbound08:40:222
3711383992319,3cyclictest1-21systemd07:15:002
3711382992319,3cyclictest3969938-21kworker/u8:4+flush-259:011:24:591
3711382992318,3cyclictest3856539-21/usr/sbin/munin09:30:131
3711379992319,3cyclictest4001080-21gpgv11:50:000
3711387992219,2cyclictest3825384-21kworker/u8:1+events_unbound09:15:003
3711387992219,2cyclictest3777884-21kworker/u8:2+flush-259:009:50:213
3711387992216,4cyclictest4011925-21/usr/sbin/munin12:00:143
3711383992218,3cyclictest3969938-21kworker/u8:4+events_unbound12:05:192
3711383992218,3cyclictest3825384-21kworker/u8:1+flush-259:009:40:152
3711382992216,4cyclictest3711327-21latency_hist07:10:011
3711379992217,3cyclictest3969938-21kworker/u8:4+events_unbound12:05:160
3711387992118,2cyclictest3980410-21kworker/u8:0+flush-259:012:10:223
3711387992117,3cyclictest3969938-21kworker/u8:4+events_unbound12:15:273
3711387992117,3cyclictest3658078-21kworker/u8:3+events_unbound08:05:173
3711387992115,4cyclictest4052011-21ntpq12:35:223
3711387992115,4cyclictest4001400-21gpgv11:50:013
3711387992115,4cyclictest3949441-21latency_hist11:00:013
3711383992117,3cyclictest3907505-21kworker/u8:1+events_unbound10:55:172
3711383992117,3cyclictest3882038-21kworker/u8:0+flush-259:010:20:002
3711383992116,3cyclictest252-21systemd-journal09:11:332
3711383992115,4cyclictest3892320-21latency_hist10:05:012
3711379992117,3cyclictest3777884-21kworker/u8:2+events_unbound10:00:160
3711379992117,3cyclictest3658078-21kworker/u8:3+events_unbound07:15:130
3711379992116,4cyclictest3969938-21kworker/u8:4+flush-259:012:25:010
3711379992116,3cyclictest3877607-21irqcore09:50:160
3711387992017,2cyclictest3969938-21kworker/u8:4+events_unbound12:05:513
3711387992017,2cyclictest3928609-21kworker/u8:2+flush-259:011:25:233
3711387992017,2cyclictest3928609-21kworker/u8:2+events_unbound11:08:323
3711387992017,2cyclictest3652697-21kworker/u8:0+events_unbound07:25:223
3711387992016,3cyclictest3825384-21kworker/u8:1+flush-259:010:05:343
3711387992016,3cyclictest3825384-21kworker/u8:1+flush-259:009:00:263
3711387992016,3cyclictest3777884-21kworker/u8:2+events_unbound08:49:343
3711383992013,4cyclictest252-21systemd-journal10:49:592
3711382992017,2cyclictest509-21rs:main1
3711382992017,2cyclictest3658078-21kworker/u8:3+flush-259:007:10:251
371138299200,18cyclictest1-21systemd08:30:001
3711379992015,4cyclictest3882279-21apt-get09:55:110
3711379992014,4cyclictest3933968-21latency_hist10:45:000
3711379992014,4cyclictest3928607-21store10:40:010
3711379992013,6cyclictest3979462-21ntpq11:25:210
3711387991916,2cyclictest3934650-21irqcore10:45:153
3711387991916,2cyclictest3882038-21kworker/u8:0+flush-259:010:15:013
3711387991915,2cyclictest3969938-21kworker/u8:4+flush-259:011:40:203
3711387991912,5cyclictest3944300-21kworker/u8:0+flush-259:011:15:003
3711383991916,2cyclictest3866609-21idleruntime-cro09:40:002
3711383991915,3cyclictest3825384-21kworker/u8:1+events_unbound09:00:182
3711382991916,2cyclictest3747611-21apt-get07:45:121
3711382991915,3cyclictest3985757-21kworker/u8:1+flush-259:011:45:161
3711382991912,4cyclictest3731954-21latency_hist07:30:021
3711379991916,2cyclictest3969938-21kworker/u8:4+flush-259:011:30:240
3711379991915,3cyclictest509-21rs:main0
3711379991915,3cyclictest3825384-21kworker/u8:1+events_unbound09:45:230
3711379991914,4cyclictest3777884-21kworker/u8:2+flush-259:009:25:280
371138799189,7cyclictest252-21systemd-journal08:00:013
3711387991815,2cyclictest3825384-21kworker/u8:1+flush-259:009:45:213
3711387991815,2cyclictest3777884-21kworker/u8:2+events_unbound08:20:183
3711387991814,3cyclictest3767915-21kworker/u8:0+events_unbound08:15:183
3711387991814,3cyclictest3658078-21kworker/u8:3+events_unbound07:36:343
3711382991814,3cyclictest3923450-21/usr/sbin/munin10:35:191
3711382991813,3cyclictest252-21systemd-journal09:01:341
371138299180,17cyclictest3995827-21unin-run11:45:011
3711379991815,2cyclictest3969938-21kworker/u8:4+flush-259:011:40:260
3711379991814,3cyclictest252-21systemd-journal11:05:000
3711387991714,2cyclictest3887883-21irqcore10:00:163
3711387991712,4cyclictest3908018-21apt-get10:20:113
3711383991713,3cyclictest3886270-21ntp_kernel_err09:55:202
3711383991713,2cyclictest0-21swapper/210:10:002
3711383991710,5cyclictest3928525-21cat10:40:012
3711383991710,5cyclictest3834686-21ntpq09:05:202
3711383991710,5cyclictest252-21systemd-journal07:56:342
3711382991714,2cyclictest3767915-21kworker/u8:0+flush-259:008:15:001
3711382991713,3cyclictest3767915-21kworker/u8:0+flush-259:008:40:161
3711382991712,3cyclictest3934203-21/usr/sbin/munin10:45:221
3711382991711,4cyclictest252-21systemd-journal09:10:001
3711379991714,2cyclictest3969938-21kworker/u8:4+flush-259:012:15:160
3711379991714,2cyclictest3907759-21cron10:20:000
3711379991714,2cyclictest3892334-21kworker/u8:3+flush-259:010:30:110
3711379991713,3cyclictest3658078-21kworker/u8:3+events_unbound07:35:230
3711379991711,4cyclictest3810089-21/usr/sbin/munin08:45:140
3711387991613,2cyclictest3658078-21kworker/u8:3+flush-259:008:00:203
3711387991613,2cyclictest0-21swapper/311:50:153
3711387991612,3cyclictest3969938-21kworker/u8:4+events_unbound11:37:353
3711387991612,3cyclictest252-21systemd-journal09:25:003
3711387991611,3cyclictest509-21rs:main3
371138399169,5cyclictest3778591-21diskstats08:15:132
3711383991612,3cyclictest3777884-21kworker/u8:2+flush-259:008:20:152
3711383991612,3cyclictest3658078-21kworker/u8:3+flush-259:008:00:252
3711383991610,5cyclictest4037688-21apt-get12:25:102
371138299169,5cyclictest4006972-21diskstats11:55:131
3711382991612,3cyclictest4000963-21apt-get11:50:021
3711382991610,5cyclictest3720774-21ntpq07:15:201
3711379991614,1cyclictest60350irq/132-enp0s29f1:rx-010:05:180
3711379991614,1cyclictest60350irq/132-enp0s29f1:rx-009:05:190
3711379991613,2cyclictest60350irq/132-enp0s29f1:rx-010:56:460
3711379991613,2cyclictest60350irq/132-enp0s29f1:rx-009:00:210
3711379991613,2cyclictest60350irq/132-enp0s29f1:rx-008:39:340
3711379991613,2cyclictest60350irq/132-enp0s29f1:rx-007:46:180
3711379991613,2cyclictest60350irq/132-enp0s29f1:rx-007:42:420
3711379991613,2cyclictest3783596-21apt-get08:20:120
3711379991612,3cyclictest3980410-21kworker/u8:0+events_unbound12:10:180
3711379991612,3cyclictest3652697-21kworker/u8:0+events_unbound07:30:130
3711387991513,1cyclictest3815267-21kworker/u8:3+events_unbound09:05:153
3711387991512,2cyclictest3777884-21kworker/u8:2+flush-259:009:42:533
3711387991511,3cyclictest3825384-21kworker/u8:1+events_unbound09:55:493
3711387991511,2cyclictest3767915-21kworker/u8:0+events_unbound08:55:143
3711387991511,2cyclictest3767915-21kworker/u8:0+events_unbound08:55:133
3711387991511,2cyclictest0-21swapper/311:25:003
371138399159,4cyclictest252-21systemd-journal09:50:292
3711383991513,1cyclictest3711378-21cyclictest07:15:172
3711383991512,2cyclictest3652697-21kworker/u8:0+flush-259:007:30:152
3711383991511,3cyclictest3777884-21kworker/u8:2+flush-259:009:50:002
3711383991511,3cyclictest3777884-21kworker/u8:2+flush-259:008:35:202
3711383991511,3cyclictest3777884-21kworker/u8:2+flush-259:008:30:242
3711383991510,4cyclictest252-21systemd-journal11:25:002
371138399150,13cyclictest1-21systemd07:54:332
3711382991511,2cyclictest0-21swapper/110:55:151
3711382991510,4cyclictest3777884-21kworker/u8:2+flush-259:009:10:211
3711379991513,1cyclictest60350irq/132-enp0s29f1:rx-012:30:240
3711379991513,1cyclictest60350irq/132-enp0s29f1:rx-010:13:020
3711379991513,1cyclictest60350irq/132-enp0s29f1:rx-007:50:250
3711379991512,2cyclictest60350irq/132-enp0s29f1:rx-011:55:380
3711379991512,2cyclictest60350irq/132-enp0s29f1:rx-010:50:240
3711379991512,2cyclictest60350irq/132-enp0s29f1:rx-010:25:110
3711379991512,2cyclictest60350irq/132-enp0s29f1:rx-008:51:010
3711379991512,2cyclictest3767915-21kworker/u8:0+flush-259:008:25:250
3711379991512,2cyclictest3757664-21kworker/u8:1+events_unbound08:00:050
3711379991512,2cyclictest3658078-21kworker/u8:3+events_unbound08:10:220
3711379991511,3cyclictest3777884-21kworker/u8:2+events_unbound09:30:260
3711379991511,3cyclictest3658078-21kworker/u8:3+events_unbound07:10:210
3711379991511,3cyclictest2971-21cron11:10:000
3711379991510,4cyclictest3985757-21kworker/u8:1+events_unbound11:35:240
3711387991411,2cyclictest4042591-21kworker/u8:2+events_unbound12:34:333
3711387991411,2cyclictest0-21swapper/310:40:003
371138399149,3cyclictest3928609-21kworker/u8:2+events_unbound11:25:182
371138399145,6cyclictest0-21swapper/208:50:002
371138399141,7cyclictest3944283-21munin-run10:55:002
3711383991412,1cyclictest0-21swapper/211:20:002
3711383991412,1cyclictest0-21swapper/211:19:592
3711383991411,2cyclictest4036600-21ntp_offset12:20:222
3711383991411,2cyclictest3951621-21kthreadcore11:00:162
3711383991410,2cyclictest0-21swapper/211:50:012
3711382991412,1cyclictest3954633-21grep11:04:591
3711382991411,2cyclictest0-21swapper/112:40:011
3711382991411,2cyclictest0-21swapper/108:05:001
3711382991411,1cyclictest0-21swapper/108:05:151
3711382991410,3cyclictest4021985-21latency_hist12:10:011
3711382991410,3cyclictest3777884-21kworker/u8:2+flush-259:010:00:261
3711379991412,1cyclictest60350irq/132-enp0s29f1:rx-010:20:240
3711379991411,2cyclictest60350irq/132-enp0s29f1:rx-012:25:190
3711379991411,2cyclictest60350irq/132-enp0s29f1:rx-011:50:130
3711379991411,2cyclictest60350irq/132-enp0s29f1:rx-010:45:110
3711379991411,2cyclictest60350irq/132-enp0s29f1:rx-009:22:200
3711379991411,2cyclictest60350irq/132-enp0s29f1:rx-008:55:120
3711379991411,2cyclictest60350irq/132-enp0s29f1:rx-008:55:110
371138399137,4cyclictest4042799-21/usr/sbin/munin12:30:122
3711383991310,2cyclictest3892334-21kworker/u8:3+events_unbound10:20:172
3711383991310,2cyclictest3778166-21sh08:14:592
371138399130,12cyclictest3918044-21awk10:30:002
371138299139,3cyclictest3969938-21kworker/u8:4+flush-259:012:00:241
371138299139,3cyclictest3793633-21kworker/u8:1+flush-259:008:45:251
371138299138,4cyclictest4027409-21apt-get12:15:111
371138299138,3cyclictest252-21systemd-journal08:19:591
3711382991311,1cyclictest0-21swapper/107:40:001
3711382991310,2cyclictest509-21rs:main1
3711382991310,2cyclictest4026277-21ntp_kernel_pll_12:10:221
371137999133,3cyclictest3760116-21cat07:55:170
3711379991312,1cyclictest60350irq/132-enp0s29f1:rx-012:00:160
3711379991312,1cyclictest60350irq/132-enp0s29f1:rx-009:19:500
3711379991311,1cyclictest0-21swapper/012:40:010
371138399129,2cyclictest3767915-21kworker/u8:0+flush-259:008:55:172
371138399129,2cyclictest3767915-21kworker/u8:0+flush-259:008:55:172
371138399128,3cyclictest3985757-21kworker/u8:1+flush-259:011:45:012
371138399127,4cyclictest3652697-21kworker/u8:0+flush-259:007:25:162
371138299129,2cyclictest3825384-21kworker/u8:1+flush-259:009:40:171
371138299128,3cyclictest3825384-21kworker/u8:1+events_unbound09:45:171
371138299127,4cyclictest3732195-21/usr/sbin/munin07:30:131
3711382991210,1cyclictest0-21swapper/108:20:181
371137999129,2cyclictest60350irq/132-enp0s29f1:rx-011:15:200
371137999129,2cyclictest60350irq/132-enp0s29f1:rx-011:15:190
371137999129,2cyclictest3778496-21cpuspeed_turbos08:15:120
371138399118,2cyclictest3919309-21kthreadcore10:30:162
371138399118,2cyclictest3841459-21irqcore09:15:152
371138399117,3cyclictest3969938-21kworker/u8:4+flush-259:011:30:212
371138399117,3cyclictest3928609-21kworker/u8:2+flush-259:011:10:162
371138399117,2cyclictest4026295-21ntpq12:10:222
371138299119,1cyclictest0-21swapper/107:20:171
371138299118,2cyclictest3954951-21cpuspeed_turbos11:05:131
371138299118,1cyclictest0-21swapper/110:44:591
371138399108,1cyclictest0-21swapper/212:15:142
371138399100,2cyclictest4052871-21latency_hist12:40:012
371138299108,1cyclictest0-21swapper/111:30:161
371138299108,1cyclictest0-21swapper/110:15:181
371138299108,1cyclictest0-21swapper/108:35:211
371138299105,3cyclictest252-21systemd-journal08:51:331
37113839997,1cyclictest0-21swapper/211:10:012
37113839994,4cyclictest3750798-21cut07:45:192
37113829997,1cyclictest0-21swapper/110:00:011
37113829997,1cyclictest0-21swapper/109:35:131
37113829995,3cyclictest3917492-21sed10:25:241
37113829995,3cyclictest3898021-21cut10:10:141
37113829990,7cyclictest3990689-21latency_hist11:40:001
37113829990,2cyclictest3798751-21awk08:35:001
37113829986,1cyclictest0-21swapper/112:25:141
37113829986,1cyclictest0-21swapper/110:20:161
37113829984,3cyclictest3919326-21tr10:30:161
37113829984,3cyclictest311ktimers/111:10:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional