You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 02:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Fri Jul 03, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
339087995651,4cyclictest655535-21gpgv12:15:003
339076995448,4cyclictest479592-21latency_hist09:25:010
339087995249,2cyclictest608762-21kworker/u8:0+flush-259:012:10:003
339087994844,3cyclictest407681-21kworker/u8:1+flush-259:008:25:223
339084994640,4cyclictest335557-21systemd-journal12:40:002
339076994539,4cyclictest433311-21latency_hist08:40:010
339087994441,2cyclictest459259-21kworker/u8:1+events_unbound09:30:013
339076994437,5cyclictest624412-21cp11:44:590
339087994339,3cyclictest360400-21kworker/u8:3+flush-259:008:45:223
339087994137,3cyclictest531620-21kworker/u8:4+flush-259:011:25:013
339084994135,4cyclictest335557-21systemd-journal09:15:122
339087994036,3cyclictest552274-21kworker/u8:1+events_unbound10:35:233
339087994035,4cyclictest381468-21kworker/u8:4+events_unbound09:10:153
339076994034,4cyclictest370761-21apt-get07:40:110
339087993935,3cyclictest459259-21kworker/u8:1+flush-259:009:45:153
339087993935,3cyclictest417819-21kworker/u8:0+flush-259:008:40:263
339081993933,4cyclictest370494-21latency_hist07:40:001
339076993933,4cyclictest381448-21cp07:50:010
339087993835,2cyclictest531620-21kworker/u8:4+flush-259:011:15:193
339087993835,2cyclictest443820-21kworker/u8:1+flush-259:008:55:243
339087993834,3cyclictest614214-21/usr/sbin/munin11:35:163
339087993834,3cyclictest531620-21kworker/u8:4+flush-259:011:30:213
339087993833,3cyclictest474481-21kworker/u8:0+flush-259:009:20:213
339076993832,4cyclictest413060-21diskstats08:20:120
339087993734,2cyclictest645256-21kworker/u8:1+events_unbound12:15:183
339087993733,2cyclictest490098-21kworker/u8:2+flush-259:011:00:153
339076993731,4cyclictest344328-21gpgv07:15:000
339087993632,3cyclictest360400-21kworker/u8:3+flush-259:007:45:153
339087993632,3cyclictest344865-21kworker/u8:2+flush-259:007:15:153
339084993632,3cyclictest490098-21kworker/u8:2+flush-259:010:25:182
339084993629,5cyclictest365330-21latency_hist07:35:002
339081993632,3cyclictest350069-21apt-get07:20:121
339087993532,2cyclictest577982-21kworker/u8:1+events_unbound11:05:253
339087993531,3cyclictest613997-21kworker/u8:2+flush-259:012:00:003
339087993530,3cyclictest360400-21kworker/u8:3+flush-259:008:50:243
339084993531,3cyclictest515772-21apt-get10:00:022
339081993532,2cyclictest613997-21kworker/u8:2+flush-259:011:45:011
339081993529,4cyclictest448755-21latency_hist08:55:011
339087993431,2cyclictest417819-21kworker/u8:0+events_unbound08:35:123
339084993329,3cyclictest531620-21kworker/u8:4+flush-259:011:00:012
339084993327,4cyclictest640359-21/usr/sbin/munin12:00:132
339081993328,4cyclictest335557-21systemd-journal07:50:111
339087993230,1cyclictest515832-21kworker/u8:1+events_unbound10:00:143
339087993228,3cyclictest329437-21kworker/u8:1+flush-259:007:55:203
339084993226,4cyclictest660843-21kworker/u8:4+flush-259:012:29:592
339084993224,5cyclictest381468-21kworker/u8:4+flush-259:009:10:172
339087993127,3cyclictest459259-21kworker/u8:1+flush-259:009:15:233
339087993125,4cyclictest531620-21kworker/u8:4+events_unbound11:10:183
339084993126,3cyclictest329437-21kworker/u8:1+flush-259:007:55:262
339087993027,2cyclictest391530-21kworker/u8:0+flush-259:008:10:213
339087993026,3cyclictest505380-21kworker/u8:0+flush-259:009:50:223
339087993026,3cyclictest459259-21kworker/u8:1+events_unbound09:40:153
339087993026,3cyclictest329437-21kworker/u8:1+events_unbound07:40:243
339087993026,2cyclictest577982-21kworker/u8:1+flush-259:011:40:223
339084993025,3cyclictest505380-21kworker/u8:0+flush-259:009:55:232
339084993023,4cyclictest335557-21systemd-journal07:20:002
339081993025,4cyclictest583407-21apt-get11:05:141
339081993023,5cyclictest671461-21latency_hist12:30:011
339081993023,5cyclictest655677-21cat12:14:591
339087992926,2cyclictest660843-21kworker/u8:4+events_unbound12:20:113
339087992925,3cyclictest526472-21kworker/u8:3+flush-259:010:10:253
339081992922,5cyclictest464199-21latency_hist09:10:011
339076992923,4cyclictest396911-21kworker/u8:2+flush-259:008:15:170
339087992824,3cyclictest531620-21kworker/u8:4+events_unbound10:50:183
339084992821,4cyclictest666283-21latency_hist12:25:012
339081992824,3cyclictest490098-21kworker/u8:2+flush-259:010:40:011
339087992724,2cyclictest541934-21kworker/u8:0+events_unbound10:30:003
339087992723,3cyclictest613997-21kworker/u8:2+events_unbound12:00:173
339087992623,2cyclictest531620-21kworker/u8:4+flush-259:010:40:323
339087992623,2cyclictest490098-21kworker/u8:2+events_unbound10:15:163
339087992623,2cyclictest381468-21kworker/u8:4+flush-259:009:05:163
339087992622,3cyclictest505380-21kworker/u8:0+events_unbound10:06:163
339087992622,3cyclictest505380-21kworker/u8:0+events_unbound10:06:163
339087992622,3cyclictest381468-21kworker/u8:4+flush-259:008:04:153
339084992622,3cyclictest381468-21kworker/u8:4+flush-259:009:00:272
339084992622,3cyclictest306729-21kworker/u8:0+flush-259:007:10:272
339084992620,3cyclictest509-21rs:main2
339084992619,5cyclictest335557-21systemd-journal10:38:142
339081992621,3cyclictest547359-21apt-get10:30:111
339081992620,4cyclictest671733-21apt-get12:30:111
339076992622,3cyclictest660843-21kworker/u8:4+flush-259:012:25:100
339087992522,2cyclictest577982-21kworker/u8:1+events_unbound11:29:553
339087992521,3cyclictest608762-21kworker/u8:0+flush-259:011:55:003
339087992521,3cyclictest490098-21kworker/u8:2+flush-259:009:35:283
339087992521,3cyclictest360400-21kworker/u8:3+events_unbound08:15:023
339087992521,3cyclictest329437-21kworker/u8:1+events_unbound07:35:183
339087992521,2cyclictest396911-21kworker/u8:2+flush-259:008:05:153
339084992521,3cyclictest603935-21/usr/sbin/munin11:25:222
339084992519,4cyclictest656106-21cat12:15:012
339084992519,4cyclictest335557-21systemd-journal10:35:002
339081992521,3cyclictest562555-21kworker/u8:0+flush-259:010:45:011
339081992519,4cyclictest656026-21base6412:15:011
339076992521,3cyclictest344865-21kworker/u8:2+flush-259:007:20:010
339087992422,2cyclictest417819-21kworker/u8:0+flush-259:008:30:213
339087992421,2cyclictest531620-21kworker/u8:4+events_unbound10:45:143
339087992420,3cyclictest531620-21kworker/u8:4+events_unbound10:30:193
33908499240,22cyclictest1-21systemd12:14:592
339081992418,5cyclictest438666-21/usr/sbin/munin08:45:141
339076992421,2cyclictest505380-21kworker/u8:0+flush-259:010:15:010
339076992419,4cyclictest381468-21kworker/u8:4+flush-259:008:00:250
339087992319,3cyclictest531620-21kworker/u8:4+events_unbound11:00:013
339087992319,3cyclictest505380-21kworker/u8:0+flush-259:009:55:013
339087992319,3cyclictest381468-21kworker/u8:4+events_unbound09:00:273
339084992319,3cyclictest490098-21kworker/u8:2+flush-259:011:00:152
339084992315,5cyclictest355413-21diskstats07:25:142
339076992318,4cyclictest531620-21kworker/u8:4+flush-259:010:55:280
339087992219,2cyclictest676847-21apt-get12:35:113
339087992217,4cyclictest386397-21latency_hist07:55:013
339087992217,4cyclictest329437-21kworker/u8:1+events_unbound07:10:133
339084992216,4cyclictest635004-21latency_hist11:55:002
339084992216,4cyclictest423417-21diskstats08:30:162
339084992216,4cyclictest335557-21systemd-journal12:32:192
339084992214,5cyclictest335557-21systemd-journal08:36:132
339076992218,3cyclictest531620-21kworker/u8:4+flush-259:011:35:010
339076992217,3cyclictest531620-21kworker/u8:4+flush-259:011:20:060
339076992215,5cyclictest452990-21ntpq08:55:220
339087992118,2cyclictest329437-21kworker/u8:1+flush-259:007:20:193
339087992118,2cyclictest306729-21kworker/u8:0+flush-259:007:25:193
339087992117,3cyclictest490098-21kworker/u8:2+events_unbound10:20:423
339087992117,3cyclictest360400-21kworker/u8:3+events_unbound07:30:213
339084992114,4cyclictest588253-21latency_hist11:10:012
339076992116,3cyclictest500247-21cron09:44:590
339087992017,2cyclictest608762-21kworker/u8:0+flush-259:011:45:163
339084992016,3cyclictest531620-21kworker/u8:4+flush-259:011:10:172
339076992016,3cyclictest645256-21kworker/u8:1+flush-259:012:15:180
339076992015,3cyclictest490098-21kworker/u8:2+events_unbound09:55:000
339076992013,4cyclictest593420-21latency_hist11:15:000
339087991916,2cyclictest655550-21kworker/u8:2+events_unbound12:30:243
339084991916,2cyclictest635683-21irqcore11:55:172
339084991914,4cyclictest490098-21kworker/u8:2+flush-259:009:50:172
339084991914,3cyclictest459259-21kworker/u8:1+flush-259:009:45:182
339084991913,4cyclictest417819-21kworker/u8:0+flush-259:008:40:052
339076991917,1cyclictest438692-21apt-get08:45:110
339076991915,3cyclictest660843-21kworker/u8:4+flush-259:012:20:040
339076991915,3cyclictest360400-21kworker/u8:3+events_unbound07:50:000
339076991915,2cyclictest552258-21kworker/0:1-rcu_gp10:45:000
339087991814,3cyclictest474481-21kworker/u8:0+events_unbound09:30:243
339084991813,3cyclictest401872-21apt-get08:10:002
339084991812,4cyclictest412627-21idleruntime-cro08:20:012
339084991811,5cyclictest335557-21systemd-journal09:10:002
339081991812,4cyclictest344878-21/usr/sbin/munin07:15:201
339087991714,2cyclictest660843-21kworker/u8:4+flush-259:012:25:053
339084991712,4cyclictest645256-21kworker/u8:1+flush-259:012:05:282
339081991714,2cyclictest660843-21kworker/u8:4+flush-259:012:25:001
339081991712,3cyclictest417819-21kworker/u8:0+flush-259:008:40:181
339076991714,2cyclictest60350irq/132-enp0s29f1:rx-011:15:190
339076991714,2cyclictest60350irq/132-enp0s29f1:rx-010:20:210
339076991713,3cyclictest542177-21apt-get10:25:130
339076991711,5cyclictest509-21rs:main0
339084991612,3cyclictest490098-21kworker/u8:2+flush-259:010:45:252
339084991612,2cyclictest0-21swapper/210:50:172
339084991611,4cyclictest360400-21kworker/u8:3+flush-259:008:20:152
339084991611,3cyclictest381468-21kworker/u8:4+flush-259:008:45:212
339084991610,4cyclictest365599-21apt-get07:35:132
339081991611,4cyclictest526472-21kworker/u8:3+flush-259:010:10:181
339081991611,4cyclictest396947-21apt-get08:05:131
339081991610,4cyclictest335557-21systemd-journal08:27:141
339076991614,1cyclictest60350irq/132-enp0s29f1:rx-007:30:240
339076991614,1cyclictest60350irq/132-enp0s29f1:rx-007:20:130
339076991614,1cyclictest0-21swapper/009:15:000
339076991613,2cyclictest660843-21kworker/u8:4+flush-259:012:30:240
339076991613,2cyclictest60350irq/132-enp0s29f1:rx-010:47:160
339076991613,2cyclictest60350irq/132-enp0s29f1:rx-009:55:160
339076991610,4cyclictest645256-21kworker/u8:1+flush-259:012:05:240
339087991511,3cyclictest407681-21kworker/u8:1+events_unbound08:23:343
33908499158,5cyclictest335557-21systemd-journal11:16:182
339084991513,1cyclictest0-21swapper/209:25:182
33908199159,5cyclictest602773-21ntpq11:20:201
33908199154,6cyclictest380814-21apt07:50:011
33908199152,3cyclictest411752-21ntpq08:15:221
339076991513,2cyclictest60350irq/132-enp0s29f1:rx-008:32:420
339076991513,2cyclictest60350irq/132-enp0s29f1:rx-008:07:040
339076991513,1cyclictest60350irq/132-enp0s29f1:rx-009:48:290
339076991513,1cyclictest60350irq/132-enp0s29f1:rx-009:19:410
339076991513,1cyclictest60350irq/132-enp0s29f1:rx-008:10:170
339076991512,2cyclictest608762-21kworker/u8:0+flush-259:011:55:240
339076991512,2cyclictest60350irq/132-enp0s29f1:rx-010:05:250
339076991512,2cyclictest60350irq/132-enp0s29f1:rx-010:05:250
339076991512,2cyclictest60350irq/132-enp0s29f1:rx-009:35:170
339076991512,2cyclictest552217-21cron10:35:000
33908499149,4cyclictest531620-21kworker/u8:4+flush-259:010:20:182
33908499149,3cyclictest624299-21apt-get11:45:012
339084991412,1cyclictest0-21swapper/210:20:012
339084991411,2cyclictest531620-21kworker/u8:4+flush-259:011:35:262
339084991411,2cyclictest489408-21unixbench_multi09:30:262
339084991410,3cyclictest474481-21kworker/u8:0+flush-259:009:20:172
339084991410,3cyclictest329437-21kworker/u8:1+flush-259:007:40:242
339084991410,2cyclictest380801-21cron07:50:002
339084991410,2cyclictest0-21swapper/211:20:192
339084991410,2cyclictest0-21swapper/209:35:242
33908199144,3cyclictest593403-21cron11:15:001
33908199142,6cyclictest365337-21munin-run07:35:011
33908199141,4cyclictest517558-21grep10:00:191
33908199141,3cyclictest505437-21munin-run09:49:591
339081991410,3cyclictest603960-21apt-get11:25:111
339081991410,3cyclictest485016-21apt-get09:30:111
339081991410,2cyclictest0-21swapper/109:55:001
339076991412,1cyclictest60350irq/132-enp0s29f1:rx-012:01:210
339076991412,1cyclictest60350irq/132-enp0s29f1:rx-011:45:260
339076991412,1cyclictest60350irq/132-enp0s29f1:rx-010:37:500
339076991412,1cyclictest60350irq/132-enp0s29f1:rx-009:34:520
339076991412,1cyclictest60350irq/132-enp0s29f1:rx-009:05:160
339076991412,1cyclictest60350irq/132-enp0s29f1:rx-009:02:290
339076991412,1cyclictest60350irq/132-enp0s29f1:rx-007:35:450
339076991412,1cyclictest60350irq/132-enp0s29f1:rx-007:25:260
339076991411,2cyclictest60350irq/132-enp0s29f1:rx-011:00:200
339076991411,2cyclictest60350irq/132-enp0s29f1:rx-009:29:360
339076991411,2cyclictest60350irq/132-enp0s29f1:rx-008:43:310
339076991410,3cyclictest60350irq/132-enp0s29f1:rx-010:15:240
33908499139,3cyclictest577982-21kworker/u8:1+events_unbound11:30:002
33908499139,3cyclictest509-21rs:main2
339084991310,2cyclictest505380-21kworker/u8:0+events_unbound10:05:152
339084991310,2cyclictest505380-21kworker/u8:0+events_unbound10:05:142
33908199139,3cyclictest531620-21kworker/u8:4+flush-259:011:15:261
33908199138,3cyclictest531620-21kworker/u8:4+flush-259:010:50:171
33908199134,6cyclictest0-21swapper/110:45:151
33908199132,9cyclictest681861-21cat12:40:011
33908199131,7cyclictest514864-21ntpq09:55:191
339081991310,2cyclictest578395-21fschecks_count11:00:131
339076991312,1cyclictest60350irq/132-enp0s29f1:rx-010:00:350
339076991311,1cyclictest60350irq/132-enp0s29f1:rx-007:55:140
339076991310,2cyclictest60350irq/132-enp0s29f1:rx-011:05:220
339076991310,2cyclictest60350irq/132-enp0s29f1:rx-010:52:200
33908499128,3cyclictest329437-21kworker/u8:1+events_unbound08:00:152
33908499126,4cyclictest360400-21kworker/u8:3+flush-259:008:50:182
33908199129,2cyclictest417386-21unixbench_singl08:20:241
33908199129,2cyclictest0-21swapper/107:25:171
33908199128,3cyclictest381468-21kworker/u8:4+flush-259:009:15:011
33908199128,2cyclictest0-21swapper/108:05:011
33908199127,4cyclictest402500-21cp08:10:011
33908199127,3cyclictest475747-21sort09:20:181
33908199124,5cyclictest0-21swapper/107:55:271
33908199122,2cyclictest639164-21taskset11:55:191
33908199122,2cyclictest370732-21modprobe07:40:101
33908199121,6cyclictest469763-21df09:15:121
33908199121,3cyclictest644840-21users12:00:271
33908199121,3cyclictest541850-21runrttasks10:24:341
33908199121,3cyclictest32-21ksoftirqd/111:55:001
33908199121,3cyclictest32-21ksoftirqd/111:35:251
33908199121,2cyclictest546184-21ntpq10:25:231
33908199121,2cyclictest32-21ksoftirqd/111:30:161
33907699129,2cyclictest60350irq/132-enp0s29f1:rx-011:52:060
33907699129,2cyclictest60350irq/132-enp0s29f1:rx-011:25:180
33907699128,3cyclictest645256-21kworker/u8:1+flush-259:012:10:250
339076991210,2cyclictest60350irq/132-enp0s29f1:rx-011:37:390
339076991210,1cyclictest60350irq/132-enp0s29f1:rx-012:35:140
33908499119,1cyclictest0-21swapper/211:45:012
33908499118,2cyclictest557888-21fschecks_time10:40:182
33908199116,3cyclictest343293-21ntpq07:10:231
33908199114,5cyclictest459038-21turbostat09:05:001
33908199114,1cyclictest0-21swapper/111:40:161
33908199113,6cyclictest480059-21grep09:25:131
33908199113,6cyclictest433311-21latency_hist08:40:001
33908199113,2cyclictest0-21swapper/108:55:281
33908199111,3cyclictest832-21runrttasks12:07:021
33907699118,2cyclictest418138-21cpuspeed_turbos08:25:150
33908199103,5cyclictest490234-21cpuspeed_turbos09:35:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional