You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 15:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sat Jun 13, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3988543996864,3cyclictest4050614-21kworker/u8:4+flush-259:020:40:011
3988543994641,4cyclictest4107593-21latency_hist21:05:011
3988543993935,3cyclictest335557-21systemd-journal23:32:021
3988541993933,4cyclictest4013426-21ntpq19:30:190
3988547993834,3cyclictest4107611-21kworker/u8:4+flush-259:022:20:003
3988547993630,4cyclictest95629-21cp00:00:013
3988543993532,2cyclictest4107611-21kworker/u8:4+events_unbound21:55:171
3988543993428,4cyclictest3988476-21latency_hist19:10:011
3988547993327,4cyclictest4175434-21latency_hist22:10:003
3988544993329,2cyclictest4019658-21kworker/u8:4+flush-259:019:45:162
3988547993227,4cyclictest4075371-21ntpq20:30:213
3988543993228,3cyclictest4097533-21apt-get20:55:111
3988541993227,3cyclictest101109-21apt-get00:05:160
3988547993127,3cyclictest4107611-21kworker/u8:4+flush-259:022:10:143
3988544993126,3cyclictest4029981-21kworker/u8:3+flush-259:019:55:172
3988543993127,3cyclictest4029981-21kworker/u8:3+events_unbound20:30:211
3988547993028,1cyclictest127009-21fwupd00:30:013
3988543993027,2cyclictest3998828-21kworker/u8:1+flush-259:019:20:261
3988543993026,3cyclictest4019293-21kworker/u8:0+flush-259:019:49:591
3988543993025,3cyclictest75194-21kworker/u8:3+flush-259:023:50:211
3988543993025,3cyclictest3988737-21/usr/sbin/munin19:10:181
3988541992924,3cyclictest4191085-21kworker/u8:1+flush-259:022:25:160
3988544992821,5cyclictest335557-21systemd-journal23:00:002
3988541992821,5cyclictest335557-21systemd-journal00:35:000
3988547992724,2cyclictest81399-21kworker/u8:1+flush-259:000:05:203
3988547992723,3cyclictest335557-21systemd-journal23:10:283
3988547992723,3cyclictest132423-21apt-get00:35:103
3988544992720,5cyclictest95588-21latency_hist00:00:012
3988543992719,5cyclictest100838-21latency_hist00:04:591
3988541992723,3cyclictest2337-21kworker/u8:4+flush-259:023:05:010
3988543992622,3cyclictest4107611-21kworker/u8:4+events_unbound21:45:201
3988543992621,3cyclictest74687-21base6423:40:001
3988543992621,3cyclictest4040314-21kworker/u8:2+events_unbound23:25:141
3988547992521,3cyclictest4185763-21kworker/u8:0+events_unbound22:30:183
3988547992518,5cyclictest4081454-21cp20:40:013
3988544992521,3cyclictest2337-21kworker/u8:4+events_unbound23:00:122
3988544992520,4cyclictest335557-21systemd-journal00:30:002
3988543992518,5cyclictest6426-21ntpq22:30:181
3988543992518,5cyclictest335557-21systemd-journal22:25:001
3988543992518,5cyclictest116283-21latency_hist00:19:591
3988541992522,2cyclictest12493-21kworker/u8:1+flush-259:023:05:240
3988547992420,3cyclictest12493-21kworker/u8:1+flush-259:023:15:213
3988547992419,4cyclictest54116-21apt-get23:20:113
3988547992419,4cyclictest335557-21systemd-journal20:04:593
3988544992420,3cyclictest4102195-21kworker/u8:1+flush-259:021:20:212
3988544992418,4cyclictest4128841-21cat21:25:012
3988543992420,2cyclictest4029981-21kworker/u8:3+flush-259:019:55:191
3988547992320,2cyclictest4029981-21kworker/u8:3+events_unbound20:15:193
3988547992320,2cyclictest4019293-21kworker/u8:0+flush-259:019:40:203
3988544992318,4cyclictest335557-21systemd-journal22:09:592
3988543992319,3cyclictest4107611-21kworker/u8:4+flush-259:021:10:151
3988543992319,3cyclictest4107611-21kworker/u8:4+events_unbound21:36:371
3988543992319,3cyclictest4050614-21kworker/u8:4+flush-259:020:20:211
3988543992318,3cyclictest12493-21kworker/u8:1+flush-259:022:45:181
3988541992319,3cyclictest4040314-21kworker/u8:2+flush-259:022:15:140
3988541992318,3cyclictest116517-21/usr/sbin/munin00:20:200
3988541992317,4cyclictest84351-21ntpq23:45:180
3988541992317,4cyclictest509-21rs:main0
3988547992219,2cyclictest910899-21kworker/u8:3+flush-259:019:25:003
3988547992218,3cyclictest4102195-21kworker/u8:1+events_unbound21:30:003
3988547992216,4cyclictest28005-21latency_hist22:55:013
3988544992218,3cyclictest4107611-21kworker/u8:4+flush-259:021:35:202
3988543992218,3cyclictest95888-21kworker/u8:4+events_unbound00:09:331
3988543992218,3cyclictest4149462-21kworker/u8:3+flush-259:022:27:281
3988543992218,3cyclictest132391-21kworker/u8:0+flush-259:000:35:211
3988543992217,3cyclictest4128765-21base6421:25:021
3988543992216,4cyclictest4055571-21latency_hist20:15:001
3988547992118,2cyclictest4040314-21kworker/u8:2+events_unbound20:45:193
3988547992117,3cyclictest95888-21kworker/u8:4+flush-259:000:10:203
3988547992117,3cyclictest95888-21kworker/u8:4+flush-259:000:00:263
3988544992118,2cyclictest4191119-21apt-get22:25:122
3988544992117,3cyclictest4040314-21kworker/u8:2+events_unbound23:40:172
3988544992117,3cyclictest12493-21kworker/u8:1+flush-259:023:05:222
3988543992117,3cyclictest910899-21kworker/u8:3+flush-259:019:35:191
3988543992117,3cyclictest4040314-21kworker/u8:2+flush-259:020:05:201
3988543992115,4cyclictest23078-21apt-get22:50:121
3988541992116,3cyclictest4087195-21apt-get20:45:110
3988547992018,2cyclictest4107611-21kworker/u8:4+flush-259:021:35:243
3988547992017,2cyclictest4107611-21kworker/u8:4+events_unbound22:00:203
3988547992016,3cyclictest4107611-21kworker/u8:4+flush-259:021:45:243
3988547992016,3cyclictest4050614-21kworker/u8:4+flush-259:020:25:003
3988547992015,3cyclictest4040314-21kworker/u8:2+flush-259:023:30:163
3988544992015,4cyclictest335557-21systemd-journal20:27:262
3988544992014,5cyclictest4030015-21apt-get19:50:112
3988543992017,2cyclictest4040314-21kworker/u8:2+flush-259:020:45:211
3988543992017,2cyclictest12474-21sh22:40:001
3988543992017,2cyclictest12474-21sh22:40:001
3988543992016,3cyclictest90057-21kworker/u8:0+events_unbound23:58:101
3988543992014,4cyclictest4024617-21latency_hist19:45:001
3988543992013,5cyclictest4118181-21/usr/sbin/munin21:15:151
3988541992014,4cyclictest4107834-21/usr/sbin/munin21:05:140
3988544991916,2cyclictest4060960-21kworker/u8:0+events_unbound20:20:202
3988544991916,2cyclictest132391-21kworker/u8:0+flush-259:000:40:002
3988544991915,3cyclictest910899-21kworker/u8:3+flush-259:019:10:262
3988544991914,4cyclictest4040346-21apt-get20:00:112
3988543991915,3cyclictest4107611-21kworker/u8:4+events_unbound21:30:161
3988543991915,3cyclictest4040314-21kworker/u8:2+flush-259:022:40:201
3988543991915,3cyclictest121273-21kworker/u8:1+events_unbound00:25:181
3988543991913,4cyclictest4014332-21latency_hist19:35:001
3988541991916,2cyclictest7260-21idleruntime-cro22:34:590
3988541991916,2cyclictest4060960-21kworker/u8:0+flush-259:020:40:100
3988541991916,1cyclictest0-21swapper/023:30:010
3988541991915,3cyclictest95888-21kworker/u8:4+flush-259:000:15:170
3988541991914,3cyclictest335557-21systemd-journal19:15:250
3988541991913,4cyclictest75226-21apt-get23:40:110
3988541991912,5cyclictest137365-21latency_hist00:40:000
3988547991815,2cyclictest4029981-21kworker/u8:3+flush-259:020:05:203
3988547991815,2cyclictest4029753-21sh19:49:593
3988547991814,3cyclictest4040314-21kworker/u8:2+flush-259:021:10:263
3988547991814,3cyclictest4019293-21kworker/u8:0+events_unbound19:50:013
3988547991814,3cyclictest2337-21kworker/u8:4+events_unbound22:35:143
3988547991814,3cyclictest2337-21kworker/u8:4+events_unbound22:35:133
3988544991816,1cyclictest127009-21fwupd00:30:012
3988544991815,2cyclictest4149220-21idleruntime-cro21:45:012
3988544991815,2cyclictest4040314-21kworker/u8:2+flush-259:023:30:012
3988544991814,3cyclictest4107611-21kworker/u8:4+flush-259:021:45:212
3988544991814,3cyclictest4040314-21kworker/u8:2+events_unbound20:10:212
3988544991814,3cyclictest335557-21systemd-journal22:41:282
3988543991815,2cyclictest4107611-21kworker/u8:4+flush-259:022:00:231
3988543991815,2cyclictest4040314-21kworker/u8:2+events_unbound21:50:161
3988543991814,3cyclictest4107611-21kworker/u8:4+flush-259:022:05:281
3988543991814,3cyclictest4040314-21kworker/u8:2+flush-259:021:20:131
3988543991814,3cyclictest4019658-21kworker/u8:4+flush-259:019:50:271
3988543991814,3cyclictest12493-21kworker/u8:1+flush-259:023:05:171
3988543991814,3cyclictest121273-21kworker/u8:1+flush-259:000:30:241
3988541991815,2cyclictest4159540-21cron21:55:000
3988541991813,4cyclictest4029981-21kworker/u8:3+flush-259:020:00:250
3988541991813,3cyclictest4066153-21apt-get20:25:090
3988541991811,5cyclictest4128249-21apt-get21:25:010
3988541991811,5cyclictest4028860-21ntpq19:45:200
3988547991714,3cyclictest4009149-21kworker/u8:2+flush-259:019:35:243
3988547991714,2cyclictest4102195-21kworker/u8:1+events_unbound21:20:233
3988547991714,2cyclictest4040314-21kworker/u8:2+flush-259:020:10:183
3988547991713,3cyclictest4102195-21kworker/u8:1+flush-259:021:04:273
3988544991714,2cyclictest4092074-21idleruntime-cro20:50:012
3988544991714,2cyclictest4076224-21munin-plugin-st20:35:002
3988544991713,3cyclictest4045691-21diskstats20:05:132
3988543991714,2cyclictest910899-21kworker/u8:3+flush-259:019:25:251
3988543991714,2cyclictest4050614-21kworker/u8:4+flush-259:020:35:201
3988543991714,2cyclictest4040314-21kworker/u8:2+events_unbound23:40:291
3988543991714,2cyclictest28023-21kworker/u8:0+events_unbound23:20:191
3988543991713,3cyclictest75194-21kworker/u8:3+events_unbound23:48:291
3988543991713,3cyclictest2337-21kworker/u8:4+events_unbound23:00:201
3988543991712,4cyclictest75194-21kworker/u8:3+events_unbound00:10:251
3988541991715,1cyclictest0-21swapper/020:15:000
3988541991714,2cyclictest60350irq/132-enp0s29f1:rx-022:45:090
3988541991714,2cyclictest60350irq/132-enp0s29f1:rx-022:00:190
3988541991710,5cyclictest335557-21systemd-journal19:40:000
3988547991613,3cyclictest4149462-21kworker/u8:3+events_unbound22:29:293
3988547991613,2cyclictest910899-21kworker/u8:3+flush-259:019:10:153
3988547991613,1cyclictest0-21swapper/300:24:593
3988547991612,3cyclictest74554-21http23:40:023
3988544991613,2cyclictest95888-21kworker/u8:4+flush-259:000:05:002
3988544991613,2cyclictest4040314-21kworker/u8:2+events_unbound23:30:122
3988544991613,2cyclictest12493-21kworker/u8:1+flush-259:022:45:252
3988544991612,3cyclictest69610-21kworker/u8:1+flush-259:023:35:202
3988543991613,2cyclictest4180550-21idleruntime-cro22:15:001
3988543991612,3cyclictest3977872-21kworker/u8:0+events_unbound19:15:141
3988543991612,2cyclictest28023-21kworker/u8:0+flush-259:023:10:231
3988541991614,2cyclictest60350irq/132-enp0s29f1:rx-023:30:560
3988541991614,1cyclictest60350irq/132-enp0s29f1:rx-019:10:230
3988541991613,1cyclictest1-21systemd22:14:290
3988541991612,3cyclictest60350irq/132-enp0s29f1:rx-021:00:120
3988541991612,2cyclictest60350irq/132-enp0s29f1:rx-020:55:190
3988547991513,1cyclictest4008207-21ntp_kernel_err19:25:203
3988547991512,2cyclictest509-21rs:main3
3988547991512,2cyclictest12493-21kworker/u8:1+flush-259:022:45:203
3988544991513,1cyclictest0-21swapper/221:54:592
3988544991513,1cyclictest0-21swapper/220:40:012
3988544991512,2cyclictest4149462-21kworker/u8:3+flush-259:022:20:152
3988544991512,2cyclictest4107611-21kworker/u8:4+events_unbound21:55:192
398854399159,4cyclictest4092347-21apt-get20:50:091
3988543991511,3cyclictest4144332-21apt-get21:40:121
3988543991511,3cyclictest4029769-21kworker/u8:1+events_unbound20:25:161
3988543991511,3cyclictest12493-21kworker/u8:1+flush-259:023:15:261
3988543991510,3cyclictest4029981-21kworker/u8:3+flush-259:020:15:171
398854199158,5cyclictest4039184-21ntpq19:55:210
3988541991513,2cyclictest60350irq/132-enp0s29f1:rx-023:39:540
3988541991513,2cyclictest60350irq/132-enp0s29f1:rx-021:59:310
3988541991513,1cyclictest60350irq/132-enp0s29f1:rx-023:53:400
3988541991513,1cyclictest60350irq/132-enp0s29f1:rx-023:16:380
3988541991513,1cyclictest60350irq/132-enp0s29f1:rx-021:35:170
3988541991513,1cyclictest60350irq/132-enp0s29f1:rx-020:05:200
3988541991512,2cyclictest60350irq/132-enp0s29f1:rx-023:55:230
3988541991512,2cyclictest60350irq/132-enp0s29f1:rx-023:23:130
3988541991512,2cyclictest60350irq/132-enp0s29f1:rx-023:13:100
3988541991512,2cyclictest60350irq/132-enp0s29f1:rx-022:20:190
3988541991512,2cyclictest60350irq/132-enp0s29f1:rx-020:50:100
3988541991512,2cyclictest4050614-21kworker/u8:4+events_unbound20:35:010
3988541991511,3cyclictest60350irq/132-enp0s29f1:rx-021:25:020
3988547991412,1cyclictest4123089-21sh21:20:013
398854499149,4cyclictest335557-21systemd-journal21:16:282
3988544991411,3cyclictest4040314-21kworker/u8:2+events_unbound23:50:132
3988544991411,2cyclictest4107611-21kworker/u8:4+flush-259:021:10:202
3988544991411,2cyclictest28023-21kworker/u8:0+events_unbound23:10:182
3988543991411,2cyclictest2337-21kworker/u8:4+events_unbound22:55:131
3988541991413,1cyclictest60350irq/132-enp0s29f1:rx-019:52:570
3988541991412,2cyclictest60350irq/132-enp0s29f1:rx-019:25:560
3988541991412,1cyclictest60350irq/132-enp0s29f1:rx-021:48:420
3988541991412,1cyclictest17637-21turbostat22:45:000
3988541991411,2cyclictest60350irq/132-enp0s29f1:rx-022:55:460
3988541991411,2cyclictest60350irq/132-enp0s29f1:rx-020:15:160
3988541991410,2cyclictest60350irq/132-enp0s29f1:rx-000:00:150
398854799139,3cyclictest910899-21kworker/u8:3+flush-259:019:31:313
398854799138,4cyclictest335557-21systemd-journal23:28:273
3988547991310,2cyclictest4060960-21kworker/u8:0+flush-259:020:55:203
3988547991310,2cyclictest38631-21apt-get23:05:123
3988547991310,2cyclictest2337-21kworker/u8:4+flush-259:023:35:243
3988547991310,2cyclictest15027-21kthreadcore22:40:173
398854499137,4cyclictest4081984-21cp20:40:012
3988544991310,2cyclictest0-21swapper/222:20:002
398854399139,3cyclictest4107611-21kworker/u8:4+events_unbound22:15:161
398854199139,3cyclictest60350irq/132-enp0s29f1:rx-022:05:120
398854199139,2cyclictest60350irq/132-enp0s29f1:rx-022:35:100
398854199139,2cyclictest60350irq/132-enp0s29f1:rx-022:35:090
398854199138,4cyclictest335557-21systemd-journal21:41:280
3988541991312,1cyclictest60350irq/132-enp0s29f1:rx-019:24:290
3988541991311,1cyclictest60350irq/132-enp0s29f1:rx-021:15:250
3988541991311,1cyclictest60350irq/132-enp0s29f1:rx-021:10:470
3988541991311,1cyclictest60350irq/132-enp0s29f1:rx-020:35:230
3988541991310,2cyclictest60350irq/132-enp0s29f1:rx-022:54:420
3988541991310,2cyclictest60350irq/132-enp0s29f1:rx-019:41:250
3988541991310,2cyclictest4134501-21irqcore21:30:150
398854799129,2cyclictest4029769-21kworker/u8:1+flush-259:020:25:143
398854799127,4cyclictest335557-21systemd-journal21:45:003
398854799127,3cyclictest335557-21systemd-journal20:40:003
398854499129,2cyclictest4040314-21kworker/u8:2+flush-259:021:32:342
398854499129,2cyclictest4009149-21kworker/u8:2+flush-259:019:35:262
398854499129,2cyclictest112423-21tr00:15:182
398854399128,3cyclictest75194-21kworker/u8:3+events_unbound00:20:161
398854499119,1cyclictest0-21swapper/223:15:192
398854499118,2cyclictest95888-21kworker/u8:4+flush-259:000:20:222
398854499118,2cyclictest910899-21kworker/u8:3+flush-259:019:30:172
398854499118,2cyclictest4185763-21kworker/u8:0+flush-259:022:40:002
398854499118,2cyclictest4185763-21kworker/u8:0+flush-259:022:39:592
398854499118,2cyclictest4040314-21kworker/u8:2+flush-259:020:20:002
398854499118,2cyclictest3983601-21kworker/u8:2+flush-259:019:19:312
398854499118,2cyclictest0-21swapper/200:10:142
398854499117,3cyclictest4185763-21kworker/u8:0+events_unbound22:30:122
398854399118,2cyclictest4040314-21kworker/u8:2+events_unbound21:05:151
398854799108,2cyclictest4040314-21kworker/u8:2+events_unbound21:50:223
398854799107,2cyclictest80576-21fschecks_count23:45:133
398854499107,2cyclictest4107611-21kworker/u8:4+flush-259:022:10:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional