You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-09 - 06:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Tue Jun 09, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3693588994538,4cyclictest3781638-21latency_hist20:35:010
3693591994441,2cyclictest3654268-21kworker/u8:4+events_unbound20:40:243
3693591994340,2cyclictest3974684-21kworker/u8:1+events_unbound23:55:013
3693589994339,3cyclictest3917078-21latency_hist22:45:001
3693591994240,1cyclictest3654268-21kworker/u8:4+flush-259:019:55:003
3693590994237,3cyclictest3750481-21/usr/sbin/munin20:05:252
3693591994037,2cyclictest3911892-21kworker/u8:0+flush-259:023:20:163
3693591993935,3cyclictest3654268-21kworker/u8:4+flush-259:020:20:143
3693591993935,3cyclictest3654268-21kworker/u8:4+flush-259:020:20:143
3693590993933,4cyclictest3891050-21kworker/u8:0+flush-259:022:19:592
3693591993835,2cyclictest3849382-21kworker/u8:4+flush-259:021:55:163
3693591993734,2cyclictest3677596-21kworker/u8:1+events_unbound19:35:233
3693591993734,2cyclictest3654268-21kworker/u8:4+flush-259:019:10:213
3693591993733,3cyclictest3957886-21kworker/u8:1+flush-259:023:25:213
3693588993731,4cyclictest4030609-21ntpq00:30:210
3693588993730,5cyclictest3964348-21latency_hist23:30:000
3693591993633,2cyclictest3683418-21kworker/u8:2+flush-259:019:25:163
3693591993632,3cyclictest3703868-21kworker/u8:3+events_unbound19:20:203
3693589993630,4cyclictest335557-21systemd-journal21:36:141
3693591993532,2cyclictest3781408-21kworker/u8:0+events_unbound21:30:233
3693590993432,1cyclictest3688499-21kworker/2:1+pm20:26:282
3693588993428,4cyclictest3859494-21apt-get21:50:000
3693590993329,3cyclictest3964575-21kworker/u8:0+flush-259:023:30:212
3693590993328,3cyclictest3771316-21latency_hist20:25:012
3693590993328,3cyclictest3771316-21latency_hist20:25:002
3693589993328,3cyclictest3745338-21/usr/sbin/munin20:00:141
3693591993228,3cyclictest4010911-21kworker/u8:0+flush-259:000:25:143
3693591993228,3cyclictest3943540-21kworker/u8:2+events_unbound23:10:223
3693591993228,3cyclictest3876008-21kworker/u8:3+events_unbound22:55:003
3693591993127,3cyclictest4010911-21kworker/u8:0+events_unbound00:15:233
3693590993126,4cyclictest3849382-21kworker/u8:4+flush-259:022:15:002
3693589993124,4cyclictest3906780-21latency_hist22:35:001
3693591993024,4cyclictest3922636-21latency_hist22:50:003
3693590993027,2cyclictest3990358-21kworker/u8:2+events_unbound00:10:002
3693590993023,5cyclictest3776931-21diskstats20:30:132
3693589993024,4cyclictest3938147-21latency_hist23:05:001
3693591992926,2cyclictest3755608-21kworker/u8:3+flush-259:020:35:243
3693591992923,4cyclictest335557-21systemd-journal23:35:003
3693590992922,5cyclictest3974667-21latency_hist23:40:012
3693591992825,2cyclictest3990358-21kworker/u8:2+flush-259:023:55:183
3693591992825,2cyclictest3802951-21kworker/u8:3+events_unbound21:10:143
3693591992824,3cyclictest3943540-21kworker/u8:2+events_unbound23:35:223
3693590992824,3cyclictest3876008-21kworker/u8:3+flush-259:022:55:162
3693590992823,3cyclictest3876008-21kworker/u8:3+events_unbound22:35:152
3693588992822,4cyclictest3755608-21kworker/u8:3+flush-259:020:40:170
3693591992724,2cyclictest4010911-21kworker/u8:0+events_unbound00:20:193
3693591992724,2cyclictest3922663-21kworker/u8:4+events_unbound00:00:203
3693591992724,2cyclictest3876008-21kworker/u8:3+events_unbound22:15:203
3693591992724,2cyclictest3875384-21kworker/u8:1+flush-259:022:20:153
3693591992723,3cyclictest3974684-21kworker/u8:1+events_unbound00:35:223
3693591992723,3cyclictest3849382-21kworker/u8:4+flush-259:021:47:283
3693589992721,5cyclictest3985229-21apt-get23:50:101
3693589992721,4cyclictest3886387-21/usr/sbin/munin22:15:111
3693591992623,2cyclictest3849382-21kworker/u8:4+flush-259:022:05:223
3693591992622,3cyclictest3802951-21kworker/u8:3+flush-259:021:00:183
3693591992619,2cyclictest3974684-21kworker/u8:1+events_unbound00:10:203
3693590992622,3cyclictest4026526-21kworker/u8:4+events_unbound00:30:202
3693590992622,3cyclictest3781408-21kworker/u8:0+events_unbound20:45:152
3693590992621,4cyclictest335557-21systemd-journal21:15:152
3693590992619,5cyclictest335557-21systemd-journal20:45:002
3693589992623,2cyclictest3813269-21apt-get21:05:111
3693589992622,3cyclictest3849382-21kworker/u8:4+flush-259:022:04:591
3693588992621,3cyclictest3802951-21kworker/u8:3+flush-259:021:10:250
3693588992621,3cyclictest335557-21systemd-journal20:37:220
3693591992522,2cyclictest3990358-21kworker/u8:2+flush-259:000:30:203
3693591992521,3cyclictest3849382-21kworker/u8:4+events_unbound22:12:173
3693590992522,2cyclictest3677596-21kworker/u8:1+events_unbound19:35:152
3693590992521,3cyclictest3781408-21kworker/u8:0+flush-259:021:40:252
3693590992520,4cyclictest3943540-21kworker/u8:2+events_unbound23:43:052
3693589992521,3cyclictest3797215-21gpgv20:50:011
3693588992520,3cyclictest3964582-21/usr/sbin/munin23:30:130
3693591992422,2cyclictest3911892-21kworker/u8:0+events_unbound23:20:003
3693591992421,2cyclictest3922663-21kworker/u8:4+events_unbound23:50:013
3693591992421,2cyclictest3875384-21kworker/u8:1+events_unbound22:30:183
3693591992421,2cyclictest3828234-21kworker/u8:1+flush-259:021:25:233
3693591992420,3cyclictest3844205-21kworker/u8:3+events_unbound21:35:183
3693591992420,3cyclictest3802951-21kworker/u8:3+events_unbound21:21:153
3693591992418,4cyclictest3750239-21latency_hist20:05:013
3693590992420,3cyclictest3733846-21kworker/u8:1+events_unbound19:54:162
3693590992420,3cyclictest3714420-21kworker/u8:0+events_unbound19:32:162
3693590992420,3cyclictest3654268-21kworker/u8:4+events_unbound19:44:172
3693590992419,4cyclictest3917640-21gpgv22:45:002
3693589992420,3cyclictest3980069-21apt-get23:45:121
3693589992418,4cyclictest3932080-21ntpq22:55:201
3693591992321,2cyclictest3876008-21kworker/u8:3+events_unbound22:37:503
3693591992320,2cyclictest3922663-21kworker/u8:4+flush-259:023:00:233
3693591992320,2cyclictest3875384-21kworker/u8:1+flush-259:022:25:243
3693591992320,2cyclictest3714420-21kworker/u8:0+flush-259:019:45:263
3693591992319,3cyclictest3943540-21kworker/u8:2+events_unbound23:40:123
3693590992320,2cyclictest3990358-21kworker/u8:2+events_unbound00:15:152
3693590992319,3cyclictest3849382-21kworker/u8:4+flush-259:021:55:182
3693590992319,2cyclictest3943540-21kworker/u8:2+flush-259:023:10:212
3693590992318,4cyclictest3859533-21gpgv21:49:592
3693590992317,4cyclictest3870634-21diskstats22:00:132
3693589992319,3cyclictest4031761-21apt-get00:35:111
3693589992318,3cyclictest3860193-21apt-get21:50:121
3693589992318,3cyclictest335557-21systemd-journal22:08:161
3693589992317,4cyclictest3969744-21/usr/sbin/munin23:35:141
3693590992219,2cyclictest3875384-21kworker/u8:1+flush-259:022:25:222
3693590992218,3cyclictest4010911-21kworker/u8:0+flush-259:000:20:202
3693590992218,3cyclictest3922663-21kworker/u8:4+flush-259:023:50:162
3693590992218,3cyclictest3781408-21kworker/u8:0+events_unbound21:03:182
3693590992218,2cyclictest3922663-21kworker/u8:4+events_unbound23:45:152
3693589992218,3cyclictest3943540-21kworker/u8:2+flush-259:023:40:151
3693589992217,4cyclictest3859523-21http21:50:001
3693589992217,3cyclictest3797760-21/usr/sbin/munin20:50:191
3693589992216,4cyclictest335557-21systemd-journal19:45:151
3693588992218,3cyclictest3958923-21gpgv23:25:010
3693588992218,3cyclictest3714420-21kworker/u8:0+flush-259:019:30:160
3693591992118,2cyclictest3813021-21kworker/u8:2+events_unbound21:16:163
3693591992118,2cyclictest3781408-21kworker/u8:0+events_unbound20:45:263
3693591992118,2cyclictest3755608-21kworker/u8:3+events_unbound20:15:263
3693591992117,3cyclictest3876008-21kworker/u8:3+events_unbound22:55:233
3693591992115,2cyclictest3693567-21kworker/u8:0+events_unbound19:15:143
3693591992114,5cyclictest3912109-21/usr/sbin/munin22:40:153
3693590992117,3cyclictest3828234-21kworker/u8:1+events_unbound21:35:142
3693590992117,3cyclictest3802951-21kworker/u8:3+flush-259:021:10:152
3693589992118,2cyclictest3677596-21kworker/u8:1+flush-259:019:25:221
3693589992116,3cyclictest3959414-21base6423:25:011
3693588992112,8cyclictest3917141-21apt-key22:45:000
3693591992017,2cyclictest3733846-21kworker/u8:1+events_unbound19:55:293
3693591992017,2cyclictest3654268-21kworker/u8:4+events_unbound20:15:003
3693591992014,4cyclictest335557-21systemd-journal20:50:153
3693590992017,2cyclictest3875384-21kworker/u8:1+flush-259:022:42:522
3693590992017,2cyclictest3654268-21kworker/u8:4+events_unbound19:55:252
3693590992016,3cyclictest3875384-21kworker/u8:1+events_unbound22:32:152
3693590992016,3cyclictest3802951-21kworker/u8:3+events_unbound21:05:202
3693590992016,3cyclictest3781408-21kworker/u8:0+flush-259:020:50:242
3693590992016,3cyclictest3714420-21kworker/u8:0+events_unbound19:45:222
3693589992017,2cyclictest3990358-21kworker/u8:2+flush-259:023:55:161
3693589992017,2cyclictest3781408-21kworker/u8:0+events_unbound21:40:231
3693589992016,3cyclictest3974684-21kworker/u8:1+flush-259:000:30:001
3693589992016,3cyclictest3802951-21kworker/u8:3+flush-259:021:10:271
3693589992014,4cyclictest3921745-21ntpq22:45:201
3693588992016,2cyclictest3875586-21gpgv22:05:020
369358899201,17cyclictest509-21rs:main0
3693591991916,2cyclictest3802951-21kworker/u8:3+events_unbound21:09:563
3693591991916,2cyclictest3654268-21kworker/u8:4+events_unbound19:40:243
3693590991916,2cyclictest3911892-21kworker/u8:0+events_unbound23:05:162
3693590991916,2cyclictest0-21swapper/219:20:012
3693590991916,1cyclictest0-21swapper/220:40:012
3693590991915,3cyclictest3974684-21kworker/u8:1+events_unbound00:14:162
3693590991915,3cyclictest3957886-21kworker/u8:1+events_unbound23:25:182
3693590991915,3cyclictest3802951-21kworker/u8:3+events_unbound21:23:162
3693590991915,3cyclictest3755608-21kworker/u8:3+events_unbound20:15:162
3693589991916,2cyclictest3875384-21kworker/u8:1+flush-259:022:29:591
3693589991915,3cyclictest3876008-21kworker/u8:3+events_unbound22:35:191
3693589991915,3cyclictest3802951-21kworker/u8:3+events_unbound21:00:221
3693589991915,3cyclictest3733846-21kworker/u8:1+events_unbound20:10:221
3693589991915,3cyclictest3693567-21kworker/u8:0+events_unbound19:15:161
3693589991914,4cyclictest335557-21systemd-journal20:34:161
3693589991914,3cyclictest3823620-21apt-get21:15:101
369358899190,16cyclictest509-21rs:main0
3693591991815,2cyclictest3849382-21kworker/u8:4+events_unbound21:40:183
3693591991815,2cyclictest3654268-21kworker/u8:4+events_unbound20:25:133
3693591991814,3cyclictest3781408-21kworker/u8:0+events_unbound20:57:163
3693591991814,3cyclictest3750475-21kworker/u8:0+flush-259:020:05:193
3693590991815,2cyclictest3922663-21kworker/u8:4+events_unbound23:00:142
3693590991812,5cyclictest3999639-21ntpq00:00:222
3693589991814,3cyclictest500-21systemd-logind19:40:001
3693589991814,2cyclictest3755608-21kworker/u8:3+flush-259:020:15:181
3693589991811,5cyclictest335557-21systemd-journal00:05:001
3693588991815,2cyclictest3922663-21kworker/u8:4+events_unbound00:00:180
3693588991814,3cyclictest3849382-21kworker/u8:4+flush-259:022:10:450
3693588991813,4cyclictest3797127-21gpgv20:50:000
3693591991714,2cyclictest3990358-21kworker/u8:2+events_unbound00:05:263
3693591991714,2cyclictest3745122-21kworker/u8:2+events_unbound20:32:163
3693590991714,2cyclictest3875384-21kworker/u8:1+flush-259:022:21:202
3693590991713,3cyclictest3990358-21kworker/u8:2+events_unbound23:59:142
3693590991713,3cyclictest3922663-21kworker/u8:4+events_unbound23:20:152
3693590991713,3cyclictest3911892-21kworker/u8:0+flush-259:023:15:192
3693589991714,2cyclictest4001203-21irqcore00:05:171
3693589991713,3cyclictest3876008-21kworker/u8:3+flush-259:022:50:201
3693589991713,3cyclictest3791018-21kworker/u8:1+events_unbound20:55:151
3693589991712,3cyclictest3735040-21/usr/sbin/munin19:50:141
3693588991715,1cyclictest3932984-21latency_hist23:00:000
3693588991714,2cyclictest60350irq/132-enp0s29f1:rx-022:02:050
3693588991714,2cyclictest60350irq/132-enp0s29f1:rx-021:22:510
3693588991714,2cyclictest3720062-21irqcore19:35:160
3693588991713,3cyclictest3724547-21kworker/u8:2+flush-259:019:45:200
3693588991711,4cyclictest4025460-21ntpq00:25:220
3693588991710,5cyclictest335557-21systemd-journal00:00:010
3693591991612,3cyclictest3781408-21kworker/u8:0+flush-259:021:50:003
3693590991613,2cyclictest4010911-21kworker/u8:0+flush-259:000:25:252
3693590991612,3cyclictest3828234-21kworker/u8:1+events_unbound21:25:252
3693590991612,3cyclictest3781408-21kworker/u8:0+events_unbound20:57:262
3693590991612,3cyclictest3703868-21kworker/u8:3+events_unbound19:20:122
3693589991612,3cyclictest3875384-21kworker/u8:1+flush-259:022:20:181
3693589991612,3cyclictest3849382-21kworker/u8:4+flush-259:022:10:211
3693589991612,3cyclictest3654268-21kworker/u8:4+flush-259:020:20:161
3693589991612,3cyclictest3654268-21kworker/u8:4+flush-259:020:20:161
3693589991612,3cyclictest3654268-21kworker/u8:4+flush-259:019:10:171
3693589991611,4cyclictest3750475-21kworker/u8:0+flush-259:020:05:151
3693589991610,4cyclictest335557-21systemd-journal23:05:161
3693588991614,1cyclictest60350irq/132-enp0s29f1:rx-021:50:180
3693588991614,1cyclictest60350irq/132-enp0s29f1:rx-000:20:170
3693588991613,2cyclictest60350irq/132-enp0s29f1:rx-023:48:590
3693588991613,2cyclictest60350irq/132-enp0s29f1:rx-023:05:210
3693588991613,2cyclictest60350irq/132-enp0s29f1:rx-022:30:040
3693588991613,2cyclictest60350irq/132-enp0s29f1:rx-021:05:160
3693588991613,2cyclictest60350irq/132-enp0s29f1:rx-020:57:170
3693588991613,2cyclictest60350irq/132-enp0s29f1:rx-020:03:390
3693588991613,2cyclictest3870221-21turbostat21:59:590
3693588991612,3cyclictest60350irq/132-enp0s29f1:rx-021:35:200
3693588991610,4cyclictest3760988-21diskstats20:15:140
3693591991512,2cyclictest3911892-21kworker/u8:0+events_unbound23:05:223
3693591991512,2cyclictest3866325-21kworker/u8:2+events_unbound22:03:163
3693590991512,3cyclictest3849382-21kworker/u8:4+events_unbound21:50:272
3693590991512,2cyclictest3974684-21kworker/u8:1+flush-259:000:35:252
3693590991511,3cyclictest3849382-21kworker/u8:4+flush-259:022:05:462
3693590991511,3cyclictest3654268-21kworker/u8:4+flush-259:020:04:512
3693589991512,2cyclictest3733846-21kworker/u8:1+flush-259:019:55:291
3693589991511,2cyclictest3943573-21apt-get23:10:111
3693588991513,2cyclictest60350irq/132-enp0s29f1:rx-019:27:530
3693588991513,1cyclictest60350irq/132-enp0s29f1:rx-023:40:250
3693588991513,1cyclictest60350irq/132-enp0s29f1:rx-022:53:390
3693588991513,1cyclictest60350irq/132-enp0s29f1:rx-021:17:590
3693588991513,1cyclictest60350irq/132-enp0s29f1:rx-020:10:150
3693588991513,1cyclictest60350irq/132-enp0s29f1:rx-019:40:190
3693588991513,1cyclictest60350irq/132-enp0s29f1:rx-000:11:080
3693588991512,2cyclictest60350irq/132-enp0s29f1:rx-023:35:130
3693588991512,2cyclictest60350irq/132-enp0s29f1:rx-022:46:120
3693588991512,2cyclictest60350irq/132-enp0s29f1:rx-020:51:070
3693591991411,2cyclictest3714420-21kworker/u8:0+events_unbound19:30:583
369359099149,3cyclictest3733846-21kworker/u8:1+events_unbound20:10:192
369358999148,4cyclictest3870208-21latency_hist22:00:011
3693588991412,1cyclictest60350irq/132-enp0s29f1:rx-022:25:230
3693588991412,1cyclictest60350irq/132-enp0s29f1:rx-020:20:230
3693588991412,1cyclictest60350irq/132-enp0s29f1:rx-020:20:220
3693588991412,1cyclictest60350irq/132-enp0s29f1:rx-000:15:150
3693588991412,1cyclictest60350irq/132-enp0s29f1:rx-000:05:250
3693588991411,2cyclictest60350irq/132-enp0s29f1:rx-023:00:190
3693588991411,2cyclictest60350irq/132-enp0s29f1:rx-021:30:120
3693588991411,2cyclictest60350irq/132-enp0s29f1:rx-021:27:480
3693588991411,2cyclictest60350irq/132-enp0s29f1:rx-019:55:290
3693588991411,2cyclictest60350irq/132-enp0s29f1:rx-019:51:080
3693588991410,3cyclictest3812041-21needreboot21:00:190
369359099139,3cyclictest3654268-21kworker/u8:4+events_unbound19:10:172
369358999139,3cyclictest3781408-21kworker/u8:0+events_unbound21:20:171
3693589991310,2cyclictest3782124-21fschecks_time20:35:151
369358899137,4cyclictest509-21rs:main0
3693588991312,1cyclictest60350irq/132-enp0s29f1:rx-019:14:300
3693588991311,1cyclictest60350irq/132-enp0s29f1:rx-022:35:190
3693588991311,1cyclictest60350irq/132-enp0s29f1:rx-022:15:030
3693588991311,1cyclictest60350irq/132-enp0s29f1:rx-020:25:150
3693588991311,1cyclictest60350irq/132-enp0s29f1:rx-020:05:180
3693588991310,2cyclictest60350irq/132-enp0s29f1:rx-021:40:230
3693588991310,2cyclictest3703868-21kworker/u8:3+flush-259:019:22:200
369358999127,3cyclictest335557-21systemd-journal00:35:001
3693588991210,2cyclictest60350irq/132-enp0s29f1:rx-000:38:120
369359099117,3cyclictest3876008-21kworker/u8:3+flush-259:022:53:252
369358999118,2cyclictest3755608-21kworker/u8:3+events_unbound20:40:141
369358999118,1cyclictest0-21swapper/119:40:261
369358899118,2cyclictest60350irq/132-enp0s29f1:rx-023:50:100
369358899118,2cyclictest60350irq/132-enp0s29f1:rx-019:15:150
369359099106,3cyclictest3683418-21kworker/u8:2+events_unbound19:27:072
369358999108,1cyclictest0-21swapper/123:20:181
369358999108,1cyclictest0-21swapper/121:30:171
369358999107,2cyclictest3964780-21fschecks_count23:30:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional