You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-16 - 23:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Tue Sep 16, 2025 12:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3966228994340,2cyclictest113436-21fwupd12:36:561
3966228993734,2cyclictest4017929-21kworker/u8:0+events_unbound08:35:201
3966225993528,4cyclictest103328-21latency_hist12:30:010
3966233993226,4cyclictest4012510-21latency_hist07:55:012
3966225993228,3cyclictest14996-21kworker/u8:0+flush-259:012:35:180
3966233993126,3cyclictest83046-21/usr/sbin/munin12:10:242
3966236992925,3cyclictest3981569-21apt-get07:24:313
3966233992924,3cyclictest2698871-21systemd-journal09:28:312
3966236992825,2cyclictest4017929-21kworker/u8:0+flush-259:009:05:003
3966233992824,3cyclictest103569-21/usr/sbin/munin12:30:162
3966228992824,3cyclictest4178266-21kworker/u8:1+flush-259:011:05:161
3966233992721,4cyclictest4007396-21latency_hist07:50:012
3966228992721,4cyclictest4002258-21latency_hist07:45:001
3966225992721,4cyclictest2698871-21systemd-journal08:25:220
3966236992623,2cyclictest4178266-21kworker/u8:1+flush-259:011:05:173
3966236992620,4cyclictest3997146-21latency_hist07:40:013
3966233992621,4cyclictest14965-21latency_hist11:05:002
3966228992623,2cyclictest4146746-21kworker/u8:0+events_unbound10:30:181
3966228992622,2cyclictest72759-21kworker/u8:1+flush-259:012:25:161
3966225992623,2cyclictest4178266-21kworker/u8:1+flush-259:010:45:180
3966236992520,4cyclictest41904-21/usr/sbin/munin11:30:193
3966228992522,2cyclictest3971516-21kworker/u8:0+flush-259:007:20:261
3966228992521,3cyclictest83071-21apt-get12:10:131
3966228992521,3cyclictest4017929-21kworker/u8:0+events_unbound09:36:301
3966228992519,5cyclictest2698871-21systemd-journal08:30:011
3966225992519,4cyclictest51068-21ntpq11:35:210
3966236992420,3cyclictest4017929-21kworker/u8:0+flush-259:008:05:173
3966228992421,2cyclictest4188522-21kworker/u8:4+flush-259:011:20:151
3966228992420,3cyclictest4178266-21kworker/u8:1+events_unbound10:50:191
3966236992320,2cyclictest3002-21cron10:45:013
3966233992319,3cyclictest3981637-21kworker/u8:4+events_unbound07:35:152
3966233992319,3cyclictest36756-21apt-get11:25:122
3966228992319,2cyclictest14996-21kworker/u8:0+flush-259:011:10:181
3966225992320,1cyclictest1-21systemd09:28:310
3966236992217,4cyclictest3951063-21kworker/u8:3+flush-259:007:45:203
3966236992217,3cyclictest72759-21kworker/u8:1+events_unbound12:25:003
3966236992216,4cyclictest2698871-21systemd-journal09:50:003
3966233992219,2cyclictest4100456-21kworker/u8:4+flush-259:009:25:002
3966228992219,2cyclictest4183195-21systemd-run10:40:011
3966228992218,3cyclictest4017929-21kworker/u8:0+flush-259:008:45:191
3966225992218,3cyclictest2698871-21systemd-journal11:05:000
3966225992217,3cyclictest41547-40mandb11:28:510
3966236992117,3cyclictest4146746-21kworker/u8:0+flush-259:010:35:213
3966233992117,3cyclictest36059-21apt-get11:25:012
3966233992116,4cyclictest4068279-21ntpq08:45:232
3966228992118,2cyclictest4058890-21kworker/u8:2+events_unbound08:55:191
3966228992117,3cyclictest4188522-21kworker/u8:4+events_unbound11:54:541
3966228992117,3cyclictest4178266-21kworker/u8:1+flush-259:011:30:001
3966228992117,3cyclictest4146746-21kworker/u8:0+flush-259:010:40:231
3966228992117,3cyclictest4012535-21kworker/u8:1+events_unbound08:30:231
3966228992117,3cyclictest3981637-21kworker/u8:4+flush-259:007:25:211
3966228992114,5cyclictest2698871-21systemd-journal11:19:511
3966225992118,2cyclictest4146746-21kworker/u8:0+flush-259:010:20:180
3966225992117,3cyclictest14996-21kworker/u8:0+flush-259:012:10:000
3966225992115,4cyclictest4675-21latency_hist10:55:000
3966225992115,4cyclictest4121049-21latency_hist09:40:010
3966236992016,3cyclictest82828-21kworker/u8:2+events_unbound12:30:163
3966233992015,3cyclictest4152310-21latency_hist10:10:012
3966228992016,3cyclictest72759-21kworker/u8:1+events_unbound12:16:521
3966228992016,3cyclictest4178266-21kworker/u8:1+events_unbound11:40:211
3966228992016,3cyclictest4146746-21kworker/u8:0+flush-259:010:05:261
3966228992016,3cyclictest4017929-21kworker/u8:0+events_unbound09:30:271
3966228992016,3cyclictest4012535-21kworker/u8:1+events_unbound08:20:151
3966228992016,3cyclictest3951063-21kworker/u8:3+flush-259:007:50:251
3966228992016,2cyclictest4017929-21kworker/u8:0+events_unbound08:50:221
3966228992015,3cyclictest31160-21kworker/u8:2+flush-259:011:30:221
3966228992013,5cyclictest4136446-21latency_hist09:55:011
3966228992013,5cyclictest2698871-21systemd-journal07:40:011
3966225992017,2cyclictest82828-21kworker/u8:2+flush-259:012:25:000
3966225992016,3cyclictest20824-21apt-get11:10:120
3966225992014,4cyclictest15590-21cat11:05:010
3966236991916,2cyclictest4017929-21kworker/u8:0+flush-259:008:45:263
3966236991915,3cyclictest3981637-21kworker/u8:4+events_unbound07:30:183
3966236991915,3cyclictest14996-21kworker/u8:0+events_unbound12:35:233
3966236991913,4cyclictest4157391-21latency_hist10:15:013
3966236991913,4cyclictest2698871-21systemd-journal10:23:303
3966236991913,4cyclictest2698871-21systemd-journal08:25:013
3966233991917,1cyclictest4146712-21idleruntime-cro10:05:002
3966233991916,2cyclictest31160-21kworker/u8:2+flush-259:011:30:252
3966233991915,3cyclictest4146746-21kworker/u8:0+flush-259:010:35:172
3966233991913,4cyclictest2698871-21systemd-journal09:45:342
3966228991916,2cyclictest82828-21kworker/u8:2+events_unbound12:30:571
3966228991916,2cyclictest4012535-21kworker/u8:1+events_unbound08:45:001
3966225991915,3cyclictest3971516-21kworker/u8:0+flush-259:007:20:180
3966225991913,5cyclictest88370-21diskstats12:15:150
3966236991815,2cyclictest3981637-21kworker/u8:4+flush-259:007:25:253
3966236991815,2cyclictest14996-21kworker/u8:0+flush-259:011:50:263
3966236991814,3cyclictest4146746-21kworker/u8:0+flush-259:010:30:313
3966236991813,4cyclictest4136705-21apt-get09:55:123
3966233991815,2cyclictest57109-21idleruntime-cro11:45:002
3966233991815,1cyclictest0-21swapper/212:40:002
3966233991813,4cyclictest2698871-21systemd-journal11:45:412
3966228991815,2cyclictest4100456-21kworker/u8:4+events_unbound09:40:181
3966228991815,2cyclictest4028000-21kworker/u8:2+flush-259:008:15:171
3966228991815,2cyclictest3951063-21kworker/u8:3+events_unbound07:45:241
3966228991814,3cyclictest72759-21kworker/u8:1+flush-259:012:05:201
3966228991814,3cyclictest4080080-21kworker/u8:1+flush-259:009:05:161
3966228991814,3cyclictest3966443-21apt-get07:10:111
3966228991813,3cyclictest3981608-21kworker/u8:1+events_unbound07:30:151
3966228991811,5cyclictest4084992-21latency_hist09:05:011
3966228991811,2cyclictest4146746-21kworker/u8:0+events_unbound10:20:251
3966225991814,3cyclictest2698871-21systemd-journal07:56:300
3966225991814,2cyclictest0-21swapper/007:50:010
3966225991812,5cyclictest4021962-21ntpq08:00:210
3966236991714,2cyclictest4058890-21kworker/u8:2+events_unbound08:50:313
3966236991714,2cyclictest4017929-21kworker/u8:0+events_unbound09:10:303
3966236991713,3cyclictest4141867-21apt-get10:00:133
3966236991713,3cyclictest4012535-21kworker/u8:1+flush-259:008:01:343
3966236991713,3cyclictest3950878-21kworker/u8:2+flush-259:007:10:163
3966236991711,4cyclictest4161638-21ntpq10:15:213
3966233991715,1cyclictest0-21swapper/210:15:002
3966228991714,2cyclictest4162789-21kworker/u8:3+flush-259:010:25:251
3966228991714,2cyclictest4126213-21kworker/u8:2+flush-259:010:10:211
3966228991713,3cyclictest4188522-21kworker/u8:4+flush-259:010:55:181
3966228991713,3cyclictest4090157-21kworker/u8:3+events_unbound09:15:211
3966228991713,3cyclictest4012535-21kworker/u8:1+events_unbound08:00:171
3966225991714,2cyclictest60650irq/133-enp0s29f1:rx-008:05:210
3966225991711,4cyclictest4121480-21diskstats09:40:170
3966225991710,5cyclictest8915-21ntpq10:55:200
3966236991613,2cyclictest68068-21irqcore11:55:173
3966236991613,2cyclictest4162789-21kworker/u8:3+flush-259:010:25:353
3966236991613,2cyclictest4080080-21kworker/u8:1+flush-259:009:20:233
3966236991613,2cyclictest4080080-21kworker/u8:1+events_unbound09:25:193
3966236991612,3cyclictest4188522-21kworker/u8:4+events_unbound10:55:163
3966236991612,3cyclictest4017929-21kworker/u8:0+events_unbound08:11:313
3966236991612,3cyclictest4012535-21kworker/u8:1+events_unbound08:41:313
3966236991612,3cyclictest3981637-21kworker/u8:4+events_unbound07:40:173
3966236991611,3cyclictest4188522-21kworker/u8:4+flush-259:011:00:253
3966233991612,3cyclictest3981608-21kworker/u8:1+flush-259:007:33:012
3966233991611,4cyclictest3970405-21ntpq07:10:222
3966228991613,2cyclictest4017929-21kworker/u8:0+flush-259:009:25:211
3966228991613,2cyclictest3000063-21rs:main1
3966228991612,3cyclictest4188966-21irqcore10:45:181
3966228991612,3cyclictest2698871-21systemd-journal09:56:301
3966228991612,2cyclictest4100456-21kworker/u8:4+events_unbound10:00:201
396622599169,5cyclictest4178461-21diskstats10:35:150
3966225991614,1cyclictest60650irq/133-enp0s29f1:rx-009:00:240
3966225991614,1cyclictest60650irq/133-enp0s29f1:rx-008:10:170
3966225991613,2cyclictest4178266-21kworker/u8:1+flush-259:011:40:160
3966225991612,2cyclictest2698871-21systemd-journal07:17:000
3966225991610,5cyclictest4104737-21ntpq09:20:210
3966225991610,4cyclictest2698871-21systemd-journal12:35:000
3966236991512,2cyclictest4188522-21kworker/u8:4+events_unbound11:20:203
3966236991512,2cyclictest4080080-21kworker/u8:1+flush-259:009:05:243
3966236991512,2cyclictest3950878-21kworker/u8:2+events_unbound07:10:003
3966236991511,3cyclictest72759-21kworker/u8:1+events_unbound12:10:193
3966236991511,3cyclictest4126213-21kworker/u8:2+flush-259:010:06:353
3966236991511,3cyclictest4100456-21kworker/u8:4+flush-259:009:50:243
3966236991511,3cyclictest4058890-21kworker/u8:2+events_unbound08:55:173
3966236991511,2cyclictest4178266-21kworker/u8:1+events_unbound11:40:173
3966233991512,2cyclictest4162789-21kworker/u8:3+flush-259:010:25:182
3966233991510,4cyclictest4012535-21kworker/u8:1+flush-259:008:00:162
3966228991512,2cyclictest31160-21kworker/u8:2+flush-259:012:00:001
3966228991511,3cyclictest72759-21kworker/u8:1+events_unbound12:20:021
3966228991511,3cyclictest3981637-21kworker/u8:4+flush-259:007:55:181
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-012:00:150
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-011:55:240
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-011:52:240
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-011:31:080
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-011:15:200
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-010:25:220
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-010:10:160
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-009:15:280
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-008:45:260
3966225991513,1cyclictest60650irq/133-enp0s29f1:rx-007:26:060
3966225991512,2cyclictest60650irq/133-enp0s29f1:rx-010:40:200
3966225991512,2cyclictest60650irq/133-enp0s29f1:rx-008:50:240
3966225991512,2cyclictest60650irq/133-enp0s29f1:rx-007:40:190
3966225991511,3cyclictest4126433-21apt-get09:45:130
3966225991511,3cyclictest31196-21apt-get11:20:120
3966225991511,2cyclictest60650irq/133-enp0s29f1:rx-009:30:130
396623699149,4cyclictest2698871-21systemd-journal07:55:013
3966236991412,1cyclictest4012535-21kworker/u8:1+events_unbound08:25:143
3966236991411,2cyclictest72759-21kworker/u8:1+events_unbound12:20:003
3966236991411,2cyclictest51958-21idleruntime-cro11:40:003
3966236991411,2cyclictest4017929-21kworker/u8:0+flush-259:009:33:353
3966236991411,2cyclictest4012535-21kworker/u8:1+flush-259:008:35:013
3966236991411,2cyclictest21233-21irqcore11:10:173
3966236991410,3cyclictest4178266-21kworker/u8:1+events_unbound10:47:303
3966236991410,3cyclictest14996-21kworker/u8:0+flush-259:011:48:573
396623399149,4cyclictest2698871-21systemd-journal10:45:012
3966233991411,2cyclictest4146746-21kworker/u8:0+flush-259:010:15:452
3966233991411,2cyclictest4049086-21fschecks_time08:30:162
3966233991410,3cyclictest4126213-21kworker/u8:2+flush-259:009:55:412
3966233991410,3cyclictest4017929-21kworker/u8:0+flush-259:009:30:262
3966233991410,3cyclictest4017929-21kworker/u8:0+flush-259:008:40:262
3966233991410,3cyclictest4017929-21kworker/u8:0+flush-259:008:10:172
3966228991410,3cyclictest4188522-21kworker/u8:4+events_unbound11:03:341
3966228991410,3cyclictest4126213-21kworker/u8:2+flush-259:009:45:161
3966228991410,3cyclictest4017929-21kworker/u8:0+events_unbound08:05:211
3966228991410,3cyclictest14996-21kworker/u8:0+events_unbound11:46:221
3966228991410,3cyclictest14996-21kworker/u8:0+events_unbound11:37:041
3966225991413,1cyclictest60650irq/133-enp0s29f1:rx-010:08:010
3966225991412,1cyclictest60650irq/133-enp0s29f1:rx-011:45:160
3966225991412,1cyclictest60650irq/133-enp0s29f1:rx-009:55:150
3966225991412,1cyclictest60650irq/133-enp0s29f1:rx-008:36:540
3966225991412,1cyclictest60650irq/133-enp0s29f1:rx-008:20:230
3966225991412,1cyclictest60650irq/133-enp0s29f1:rx-008:15:220
3966225991412,1cyclictest60650irq/133-enp0s29f1:rx-007:10:220
3966225991411,2cyclictest60650irq/133-enp0s29f1:rx-008:41:440
3966225991411,2cyclictest4178045-21turbostat.cron10:35:000
3966225991411,2cyclictest4075416-21irqcore08:55:170
3966225991411,2cyclictest0-21swapper/010:05:010
396623699139,3cyclictest14996-21kworker/u8:0+events_unbound12:00:273
396623699138,4cyclictest72759-21kworker/u8:1+events_unbound12:28:583
3966236991310,2cyclictest77930-21/usr/sbin/munin12:05:223
3966236991310,2cyclictest3981637-21kworker/u8:4+flush-259:007:57:353
3966236991310,2cyclictest31160-21kworker/u8:2+events_unbound11:25:223
3966233991311,1cyclictest0-21swapper/212:25:202
3966233991310,2cyclictest4178266-21kworker/u8:1+flush-259:010:52:012
3966233991310,2cyclictest4043048-21unixbench_singl08:20:252
396622899139,3cyclictest4100456-21kworker/u8:4+events_unbound10:15:151
3966228991310,2cyclictest4105152-21unixbench_multi09:20:241
3966225991312,1cyclictest60650irq/133-enp0s29f1:rx-007:37:220
3966225991311,2cyclictest60650irq/133-enp0s29f1:rx-009:10:160
3966225991311,1cyclictest60650irq/133-enp0s29f1:rx-012:14:570
3966225991310,2cyclictest60650irq/133-enp0s29f1:rx-010:16:310
3966225991310,2cyclictest60650irq/133-enp0s29f1:rx-008:30:020
3966225991310,2cyclictest4007727-21cpuspeed_turbos07:50:130
396623699129,2cyclictest4048988-21cpuspeed_turbos08:30:153
396623699128,3cyclictest4188522-21kworker/u8:4+events_unbound10:53:313
396623699128,3cyclictest4095534-21apt-get09:15:133
396623699128,3cyclictest4080080-21kworker/u8:1+flush-259:009:43:353
396623699128,3cyclictest4028000-21kworker/u8:2+events_unbound08:15:143
396623399129,2cyclictest97719-21unixbench_multi12:20:252
396623399129,2cyclictest4095956-21grep09:15:172
396623399129,2cyclictest4085320-21cpuspeed_turbos09:05:132
396623399129,2cyclictest3986429-21unixbench_singl07:25:242
3966233991210,1cyclictest0-21swapper/211:55:002
396622899128,3cyclictest72759-21kworker/u8:1+events_unbound12:03:231
3966228991210,2cyclictest4017929-21kworker/u8:0+events_unbound09:10:241
3966225991210,2cyclictest60650irq/133-enp0s29f1:rx-007:31:240
3966225991210,1cyclictest60650irq/133-enp0s29f1:rx-009:50:130
396623699118,2cyclictest4017929-21kworker/u8:0+events_unbound09:35:213
396623699117,3cyclictest4178266-21kworker/u8:1+flush-259:011:15:513
396623699117,3cyclictest3971516-21kworker/u8:0+flush-259:007:15:283
396623399119,1cyclictest0-21swapper/208:50:162
396623399118,2cyclictest3976791-21cpuspeed_turbos07:20:142
396623399118,2cyclictest3971629-21cpuspeed_turbos07:15:172
396623399117,3cyclictest4178266-21kworker/u8:1+flush-259:011:15:262
396623399116,3cyclictest9720-21sshd10:58:502
396623399107,2cyclictest48672-21kthreadcore11:35:182
396623399107,2cyclictest0-21swapper/208:25:192
396623399105,4cyclictest492-21dbus-daemon11:15:002
39662339996,2cyclictest4167242-21unixbench_multi10:20:262
39662339994,4cyclictest4091080-21sed09:10:182
39662339991,1cyclictest0-21swapper/209:40:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional