You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 17:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Dec 06, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2474120994233,2cyclictest2510135-21kworker/u8:2+events_unbound08:25:163
2474111994036,3cyclictest2441905-21kworker/u8:3+flush-259:007:50:011
2474106993936,2cyclictest2733560-21kworker/u8:3+flush-259:012:20:010
2474120993835,2cyclictest2598013-21kworker/u8:2+events_unbound10:00:173
2474106993831,3cyclictest2541435-21kworker/u8:3+flush-259:008:15:010
2474120993734,2cyclictest2525823-21kworker/u8:0+events_unbound08:00:243
2474120993733,3cyclictest2473609-21kworker/u8:0+flush-259:007:20:253
2474120993632,3cyclictest2723253-21kworker/u8:1+events_unbound11:10:173
2474116993627,8cyclictest2628870-21kworker/u8:3+flush-259:010:15:152
2474116993532,2cyclictest2441905-21kworker/u8:3+flush-259:007:35:002
2474120993431,2cyclictest2525823-21kworker/u8:0+flush-259:008:45:253
2474120993431,2cyclictest2510135-21kworker/u8:2+events_unbound08:15:173
2474120993330,2cyclictest2691252-21kworker/u8:3+events_unbound10:45:153
2474120993330,2cyclictest2660381-21kworker/u8:1+events_unbound10:25:173
2474120993330,2cyclictest2660381-21kworker/u8:1+events_unbound10:20:003
2474120993330,2cyclictest2510135-21kworker/u8:2+events_unbound07:50:153
2474120993229,2cyclictest2723253-21kworker/u8:1+events_unbound11:20:233
2474120993229,2cyclictest2525823-21kworker/u8:0+flush-259:008:40:203
2474120993229,2cyclictest2441905-21kworker/u8:3+events_unbound07:15:213
2474120993228,3cyclictest2577100-21kworker/u8:1+events_unbound09:00:143
2474120993226,5cyclictest2795689-21diskstats12:20:143
2474120993225,5cyclictest2805503-21latency_hist12:30:003
2474106993229,2cyclictest2691252-21kworker/u8:3+flush-259:010:55:280
2474120993128,2cyclictest2733560-21kworker/u8:3+events_unbound11:40:153
2474120993128,2cyclictest2489533-21kworker/u8:1+flush-259:007:30:213
2474120993125,4cyclictest2717647-21latency_hist11:05:003
2474111993125,4cyclictest2675183-21kworker/u8:0+flush-259:010:40:001
2474106993128,2cyclictest2738936-21kworker/u8:0+flush-259:011:25:380
2474120993027,2cyclictest2723253-21kworker/u8:1+events_unbound12:05:253
2474106993027,2cyclictest2525823-21kworker/u8:0+flush-259:008:35:010
2474120992925,3cyclictest2706950-21kworker/u8:2+events_unbound11:15:143
2474120992925,3cyclictest2587774-21kworker/u8:0+events_unbound09:05:213
2474116992924,3cyclictest2707447-21base6410:55:022
2474106992926,2cyclictest2805550-21kworker/u8:2+flush-259:012:35:310
2474116992823,4cyclictest2530728-21latency_hist08:05:012
2474120992724,2cyclictest2675183-21kworker/u8:0+events_unbound10:35:283
2474120992724,2cyclictest2525823-21kworker/u8:0+events_unbound08:35:253
2474120992721,4cyclictest2712480-21latency_hist11:00:003
2474120992623,2cyclictest2665877-21kworker/u8:2+flush-259:010:30:253
2474120992622,3cyclictest2805550-21kworker/u8:2+flush-259:012:30:193
2474120992620,3cyclictest2489533-21kworker/u8:1+events_unbound07:36:453
2474111992622,3cyclictest2441905-21kworker/u8:3+flush-259:007:35:261
2474106992622,3cyclictest2628870-21kworker/u8:3+flush-259:010:15:210
2474120992522,2cyclictest2598013-21kworker/u8:2+flush-259:009:45:243
2474120992522,2cyclictest2525823-21kworker/u8:0+events_unbound08:10:413
2474116992522,2cyclictest2510135-21kworker/u8:2+flush-259:007:55:252
2474116992521,3cyclictest2733560-21kworker/u8:3+flush-259:012:15:242
2474116992521,3cyclictest2723253-21kworker/u8:1+events_unbound11:20:212
2474111992521,3cyclictest2473609-21kworker/u8:0+flush-259:007:20:261
2474111992519,4cyclictest2701998-21/usr/sbin/munin10:50:161
2474106992522,2cyclictest2587774-21kworker/u8:0+flush-259:009:05:240
2474106992521,3cyclictest2660381-21kworker/u8:1+events_unbound10:10:240
2474106992521,3cyclictest2541435-21kworker/u8:3+flush-259:008:25:010
2474106992519,4cyclictest2700839-21ntpq10:45:200
2474120992421,2cyclictest2738936-21kworker/u8:0+flush-259:011:35:203
2474120992421,2cyclictest2733560-21kworker/u8:3+events_unbound12:15:133
2474120992421,2cyclictest2733560-21kworker/u8:3+events_unbound11:55:233
2474120992421,2cyclictest2441905-21kworker/u8:3+flush-259:007:25:193
2474120992421,2cyclictest2441905-21kworker/u8:3+events_unbound07:10:243
2474120992420,3cyclictest2691252-21kworker/u8:3+flush-259:010:40:223
2474120992420,2cyclictest2698871-21systemd-journal11:50:003
2474116992420,3cyclictest2675183-21kworker/u8:0+events_unbound10:25:012
2474116992418,4cyclictest2587741-21latency_hist09:00:002
2474111992420,3cyclictest2717378-21kworker/u8:0+flush-259:011:05:171
2474111992418,4cyclictest2660390-21/usr/sbin/munin10:10:151
2474106992421,2cyclictest2441905-21kworker/u8:3+flush-259:007:20:000
2474106992420,2cyclictest2598013-21kworker/u8:2+flush-259:009:55:240
2474120992320,2cyclictest2805550-21kworker/u8:2+events_unbound12:39:463
2474116992320,2cyclictest2510135-21kworker/u8:2+flush-259:008:35:172
2474116992319,3cyclictest2525823-21kworker/u8:0+events_unbound08:30:022
2474106992320,2cyclictest2577100-21kworker/u8:1+flush-259:009:15:000
2474106992319,3cyclictest2525823-21kworker/u8:0+flush-259:008:45:190
2474106992317,4cyclictest2623711-21latency_hist09:35:010
2474106992317,4cyclictest2623711-21latency_hist09:35:000
247410699230,22cyclictest1-21systemd07:26:430
2474120992220,2cyclictest2510135-21kworker/u8:2+events_unbound07:45:273
2474120992219,2cyclictest2587774-21kworker/u8:0+events_unbound09:35:213
2474116992219,2cyclictest2628870-21kworker/u8:3+flush-259:010:05:192
2474116992218,3cyclictest2541435-21kworker/u8:3+flush-259:008:20:252
2474111992217,4cyclictest2706927-21apt-get10:55:031
2474111992215,5cyclictest2698871-21systemd-journal08:05:011
2474106992219,2cyclictest2723253-21kworker/u8:1+flush-259:011:10:430
2474106992219,2cyclictest2717378-21kworker/u8:0+events_unbound11:05:230
2474106992219,2cyclictest2441905-21kworker/u8:3+flush-259:007:20:150
2474106992218,3cyclictest2598013-21kworker/u8:2+flush-259:010:05:250
2474106992218,3cyclictest2587774-21kworker/u8:0+events_unbound09:35:250
2474120992119,2cyclictest2587774-21kworker/u8:0+events_unbound09:16:443
2474120992118,2cyclictest2738936-21kworker/u8:0+events_unbound11:28:463
2474120992118,2cyclictest2541435-21kworker/u8:3+events_unbound08:55:163
2474116992118,2cyclictest2665877-21kworker/u8:2+events_unbound10:30:152
2474116992117,3cyclictest2738936-21kworker/u8:0+events_unbound12:25:182
2474116992117,3cyclictest2698871-21systemd-journal07:26:422
2474116992117,3cyclictest2587774-21kworker/u8:0+flush-259:009:05:002
2474116992116,4cyclictest2618567-21latency_hist09:30:002
2474111992117,3cyclictest2723253-21kworker/u8:1+flush-259:011:10:251
2474111992113,6cyclictest2520430-21latency_hist07:55:001
2474106992118,2cyclictest2723253-21kworker/u8:1+events_unbound12:05:280
2474106992118,2cyclictest2723253-21kworker/u8:1+events_unbound11:50:150
2474106992118,2cyclictest2623604-21kworker/u8:1+events_unbound09:40:230
2474106992118,2cyclictest2441905-21kworker/u8:3+flush-259:007:10:200
2474106992117,3cyclictest2733560-21kworker/u8:3+events_unbound11:49:460
2474120992017,2cyclictest2525823-21kworker/u8:0+events_unbound08:32:453
2474116992017,2cyclictest2598013-21kworker/u8:2+flush-259:009:15:192
2474116992016,3cyclictest2723253-21kworker/u8:1+flush-259:011:10:192
2474111992016,3cyclictest2598013-21kworker/u8:2+flush-259:009:25:251
2474111992016,3cyclictest2598013-21kworker/u8:2+flush-259:009:15:261
2474106992017,2cyclictest2733560-21kworker/u8:3+events_unbound11:40:160
2474106992017,2cyclictest2525823-21kworker/u8:0+flush-259:008:35:210
2474106992016,3cyclictest2685923-21kworker/u8:1+events_unbound11:00:250
2474106992016,3cyclictest2510135-21kworker/u8:2+events_unbound07:50:130
2474120991915,3cyclictest2654249-21kworker/u8:0+events_unbound10:14:563
2474116991917,1cyclictest0-21swapper/210:45:012
2474116991916,2cyclictest2706950-21kworker/u8:2+events_unbound11:19:452
2474116991915,3cyclictest2733560-21kworker/u8:3+events_unbound12:30:192
2474116991915,3cyclictest2525823-21kworker/u8:0+events_unbound08:12:442
2474116991913,5cyclictest2653671-21ntpq10:00:212
2474111991916,2cyclictest2628870-21kworker/u8:3+flush-259:009:50:131
2474111991916,1cyclictest0-21swapper/110:20:001
2474111991914,3cyclictest2733560-21kworker/u8:3+flush-259:011:30:171
2474106991916,2cyclictest2706950-21kworker/u8:2+flush-259:011:15:150
2474106991916,2cyclictest2675183-21kworker/u8:0+flush-259:010:25:200
2474106991916,2cyclictest2510135-21kworker/u8:2+events_unbound08:50:300
2474106991916,2cyclictest2489533-21kworker/u8:1+events_unbound07:35:170
2474120991815,2cyclictest2738936-21kworker/u8:0+events_unbound11:33:433
2474120991815,2cyclictest2723253-21kworker/u8:1+events_unbound11:50:253
2474120991815,2cyclictest2598013-21kworker/u8:2+events_unbound09:55:213
2474120991815,2cyclictest2577100-21kworker/u8:1+flush-259:008:50:213
2474120991814,2cyclictest0-21swapper/309:45:003
2474116991815,2cyclictest2733560-21kworker/u8:3+flush-259:011:40:182
2474116991815,2cyclictest2685923-21kworker/u8:1+flush-259:010:51:012
2474116991815,2cyclictest2587774-21kworker/u8:0+flush-259:009:35:282
2474116991814,3cyclictest2805550-21kworker/u8:2+events_unbound12:35:352
2474116991814,3cyclictest2685923-21kworker/u8:1+flush-259:011:00:162
2474111991814,3cyclictest2577100-21kworker/u8:1+flush-259:008:55:001
2474111991812,4cyclictest2698871-21systemd-journal07:30:001
2474111991811,5cyclictest2698871-21systemd-journal08:40:011
2474106991815,2cyclictest2733560-21kworker/u8:3+flush-259:012:00:220
2474106991815,2cyclictest2510135-21kworker/u8:2+events_unbound08:25:200
2474106991814,3cyclictest2805550-21kworker/u8:2+events_unbound12:30:270
2474120991714,2cyclictest2723253-21kworker/u8:1+events_unbound12:12:133
2474120991712,4cyclictest2774953-21apt-get12:00:143
2474116991714,3cyclictest2738936-21kworker/u8:0+events_unbound11:35:242
2474116991714,2cyclictest2733560-21kworker/u8:3+events_unbound11:55:262
2474116991714,2cyclictest2628870-21kworker/u8:3+flush-259:009:50:152
2474116991714,2cyclictest2577100-21kworker/u8:1+events_unbound09:10:152
2474116991714,2cyclictest2525823-21kworker/u8:0+flush-259:008:45:252
2474116991714,2cyclictest2441905-21kworker/u8:3+flush-259:007:45:192
2474116991713,3cyclictest2733560-21kworker/u8:3+flush-259:012:10:012
2474116991713,3cyclictest2587774-21kworker/u8:0+events_unbound09:06:442
2474111991714,2cyclictest2441905-21kworker/u8:3+flush-259:007:55:271
2474106991715,1cyclictest2691252-21kworker/u8:3+events_unbound10:40:260
2474106991714,2cyclictest2587774-21kworker/u8:0+events_unbound09:19:440
2474106991714,2cyclictest2525823-21kworker/u8:0+events_unbound08:05:200
2474120991614,2cyclictest2717378-21kworker/u8:0+flush-259:011:05:203
2474120991613,2cyclictest2525823-21kworker/u8:0+events_unbound08:05:123
2474120991610,4cyclictest2441905-21kworker/u8:3+flush-259:007:40:163
2474116991613,2cyclictest2723253-21kworker/u8:1+events_unbound11:46:232
2474116991613,2cyclictest2717378-21kworker/u8:0+flush-259:011:05:242
2474116991613,2cyclictest2623604-21kworker/u8:1+flush-259:009:40:202
2474116991613,2cyclictest2510135-21kworker/u8:2+flush-259:007:55:012
2474116991613,2cyclictest2441905-21kworker/u8:3+events_unbound07:10:132
2474116991612,3cyclictest2546625-21latency_hist08:20:012
247411199169,5cyclictest2698871-21systemd-journal08:10:001
2474111991614,1cyclictest0-21swapper/107:20:001
2474111991613,2cyclictest2805550-21kworker/u8:2+flush-259:012:35:341
2474111991613,2cyclictest2654585-21cron10:05:011
2474111991612,3cyclictest2712752-21apt-get11:00:141
2474111991611,4cyclictest2541435-21kworker/u8:3+flush-259:008:20:191
2474111991610,4cyclictest3000063-21rs:main1
2474111991610,4cyclictest2809802-21ntpq12:30:251
2474106991614,2cyclictest2525823-21kworker/u8:0+events_unbound08:00:190
2474106991614,1cyclictest60650irq/133-enp0s29f1:rx-007:55:180
2474106991614,1cyclictest2733560-21kworker/u8:3+events_unbound11:55:130
2474106991613,2cyclictest60650irq/133-enp0s29f1:rx-011:30:240
2474106991613,2cyclictest2723253-21kworker/u8:1+events_unbound12:10:250
2474106991613,2cyclictest2706956-21gpgv10:55:020
2474106991613,2cyclictest2665877-21kworker/u8:2+flush-259:010:35:190
2474106991613,2cyclictest2598013-21kworker/u8:2+events_unbound09:25:200
2474106991613,2cyclictest2489533-21kworker/u8:1+events_unbound07:30:200
2474106991612,3cyclictest2541435-21kworker/u8:3+flush-259:008:15:210
2474120991511,3cyclictest2628870-21kworker/u8:3+events_unbound09:53:513
247411699151,2cyclictest2790111-21munin-run12:15:002
2474116991512,2cyclictest2577100-21kworker/u8:1+events_unbound08:50:182
2474116991512,2cyclictest2525823-21kworker/u8:0+events_unbound08:05:152
2474116991512,2cyclictest2510135-21kworker/u8:2+flush-259:008:30:012
2474116991512,2cyclictest2473609-21kworker/u8:0+flush-259:007:20:232
2474116991511,3cyclictest2733560-21kworker/u8:3+flush-259:012:20:202
2474116991511,3cyclictest2628870-21kworker/u8:3+events_unbound09:45:142
2474116991511,3cyclictest2608529-21kworker/u8:4+flush-259:009:30:302
2474116991511,3cyclictest2608529-21kworker/u8:4+flush-259:009:30:302
247411199158,5cyclictest2698871-21systemd-journal10:35:001
2474111991512,2cyclictest2780535-21irqcore12:05:181
2474111991511,3cyclictest2577100-21kworker/u8:1+flush-259:009:10:171
2474106991513,1cyclictest60650irq/133-enp0s29f1:rx-010:30:220
2474106991513,1cyclictest60650irq/133-enp0s29f1:rx-010:00:160
2474106991513,1cyclictest60650irq/133-enp0s29f1:rx-009:24:190
2474106991510,3cyclictest2698871-21systemd-journal09:55:000
247412099148,5cyclictest2674673-21ntpq10:20:213
2474120991412,2cyclictest2628870-21kworker/u8:3+events_unbound10:06:443
2474120991410,3cyclictest2598013-21kworker/u8:2+flush-259:009:10:293
2474120991410,3cyclictest2510135-21kworker/u8:2+events_unbound08:23:243
2474116991412,1cyclictest2665877-21kworker/u8:2+events_unbound10:39:442
2474116991411,2cyclictest2598013-21kworker/u8:2+flush-259:009:55:182
2474116991411,2cyclictest0-21swapper/210:30:002
247411199149,3cyclictest2696176-21apt-get10:45:011
2474111991412,1cyclictest2798704-21kthreadcore12:20:171
2474106991412,2cyclictest60650irq/133-enp0s29f1:rx-009:01:110
2474106991412,2cyclictest2665877-21kworker/u8:2+flush-259:010:20:250
2474106991412,1cyclictest60650irq/133-enp0s29f1:rx-011:20:170
2474106991412,1cyclictest60650irq/133-enp0s29f1:rx-008:40:240
2474106991411,2cyclictest2441905-21kworker/u8:3+flush-259:007:40:160
2474106991410,3cyclictest2800657-21apt-get12:25:150
247412099139,3cyclictest2685923-21kworker/u8:1+events_unbound10:51:313
247412099139,3cyclictest2608529-21kworker/u8:4+events_unbound09:29:573
2474120991310,2cyclictest2587774-21kworker/u8:0+events_unbound09:20:183
2474116991310,2cyclictest2654249-21kworker/u8:0+events_unbound10:10:142
2474116991310,2cyclictest2567659-21fschecks_time08:40:152
247411199139,2cyclictest2570910-21kthreadcore08:40:191
247411199136,5cyclictest2698871-21systemd-journal12:30:001
247411199135,6cyclictest2733962-21diskstats11:20:151
2474106991311,1cyclictest60650irq/133-enp0s29f1:rx-009:47:240
2474106991310,2cyclictest60650irq/133-enp0s29f1:rx-007:45:060
247411699129,2cyclictest2733560-21kworker/u8:3+events_unbound12:00:142
247411699129,2cyclictest2441905-21kworker/u8:3+flush-259:007:40:212
247411699128,3cyclictest2685923-21kworker/u8:1+events_unbound10:47:432
247411199129,2cyclictest0-21swapper/108:30:011
247411199129,1cyclictest0-21swapper/112:15:211
247411199128,3cyclictest2738936-21kworker/u8:0+flush-259:011:25:301
247411199128,3cyclictest2623604-21kworker/u8:1+flush-259:009:35:001
247411199128,3cyclictest2623604-21kworker/u8:1+flush-259:009:35:001
247411199127,4cyclictest2733560-21kworker/u8:3+flush-259:012:00:141
247411199127,4cyclictest2723253-21kworker/u8:1+flush-259:011:40:011
247411199127,3cyclictest2544018-21kthreadcore08:15:191
247411199126,4cyclictest2575874-21seq08:45:201
247410699128,3cyclictest2582861-21apt-get08:55:130
2474106991210,1cyclictest60650irq/133-enp0s29f1:rx-011:35:480
247412099117,1cyclictest191rcu_preempt09:30:183
247412099117,1cyclictest191rcu_preempt09:30:183
247411699119,1cyclictest0-21swapper/207:15:172
247411699118,2cyclictest2587774-21kworker/u8:0+events_unbound09:20:442
247411699118,2cyclictest0-21swapper/207:35:242
247411699117,3cyclictest2738936-21kworker/u8:0+events_unbound11:25:282
247411199117,3cyclictest2701760-21apt10:50:001
247411699105,4cyclictest2744128-21seq11:30:132
247411199109,1cyclictest2733333-21kworker/1:1+events_power_efficient11:50:241
247411199107,2cyclictest32-21ksoftirqd/112:15:001
247411199107,2cyclictest2670773-21cpuspeed_turbos10:20:141
24741119995,3cyclictest2763789-21find11:45:271
24741119981,5cyclictest2594029-21irqcore09:05:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional