You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 17:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Tue Dec 30, 2025 12:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2920392994239,2cyclictest3122136-21kworker/u8:1+flush-259:010:40:183
2920386993931,5cyclictest3001995-21ntpq08:25:211
2920382993631,4cyclictest3091131-21latency_hist09:55:000
2920386993426,5cyclictest3184534-21latency_hist11:25:001
2920389993330,2cyclictest3153515-21kworker/u8:0+flush-259:010:55:012
2920389993228,3cyclictest3065418-21kworker/u8:0+flush-259:010:30:182
2920382993226,4cyclictest2930707-21latency_hist07:20:000
2920392993027,2cyclictest2987468-21kworker/u8:0+flush-259:008:30:163
2920392993025,3cyclictest3205319-21kworker/u8:2+events_unbound12:05:213
2920392992926,2cyclictest3215639-21kworker/u8:1+flush-259:012:15:173
2920386992823,3cyclictest3153515-21kworker/u8:0+flush-259:011:30:171
2920386992822,4cyclictest252-21systemd-journal08:43:341
2920392992621,3cyclictest3220573-21latency_hist12:00:013
2920389992622,3cyclictest3241837-21/usr/sbin/munin12:20:222
2920386992619,5cyclictest2924695-21ntpq07:10:221
2920389992519,4cyclictest3039730-21latency_hist09:05:012
2920386992520,3cyclictest3252183-21apt-get12:30:111
2920392992420,3cyclictest3153515-21kworker/u8:0+events_unbound11:15:263
2920392992420,3cyclictest3065418-21kworker/u8:0+events_unbound10:30:243
2920392992418,4cyclictest2962209-21diskstats07:50:153
2920386992421,2cyclictest3148354-21sh10:50:011
2920386992418,4cyclictest252-21systemd-journal08:55:001
2920382992420,3cyclictest2962015-21kworker/u8:2+events_unbound08:00:140
2920382992419,4cyclictest2961790-21latency_hist07:50:000
292038299240,23cyclictest1-21systemd12:14:590
2920392992316,5cyclictest3251924-21latency_hist12:30:013
2920389992316,5cyclictest252-21systemd-journal10:25:012
2920382992319,3cyclictest3013192-21kworker/u8:1+events_unbound08:55:130
2920382992318,3cyclictest252-21systemd-journal10:46:330
2920392992219,2cyclictest3096255-21kworker/u8:2+flush-259:010:00:143
2920392992219,2cyclictest2997777-21kworker/u8:3+flush-259:008:35:193
2920392992217,4cyclictest2920313-21latency_hist07:10:003
2920389992218,3cyclictest2956222-21kworker/u8:1+flush-259:008:20:012
2920389992217,3cyclictest252-21systemd-journal11:02:322
2920389992216,4cyclictest252-21systemd-journal10:15:122
2920386992218,3cyclictest3045158-21apt-get09:10:131
2920386992217,3cyclictest3050269-21/usr/sbin/munin09:15:231
2920386992216,5cyclictest3204236-21ntpq11:40:231
2920386992212,9cyclictest3065418-21kworker/u8:0+flush-259:010:30:001
2920382992218,3cyclictest2997777-21kworker/u8:3+flush-259:010:15:000
2920392992117,3cyclictest3143146-21kworker/u8:5+flush-259:010:45:143
2920389992117,3cyclictest509-21rs:main2
2920389992115,4cyclictest3225732-21latency_hist12:05:002
2920389992115,4cyclictest252-21systemd-journal08:40:002
2920389992114,5cyclictest252-21systemd-journal11:38:322
2920386992116,4cyclictest509-21rs:main1
2920386992115,4cyclictest3018820-21cat08:45:011
292038699211,18cyclictest3070541-21journal-offline09:34:591
2920382992118,2cyclictest3169024-21kworker/u8:1+flush-259:011:25:160
2920382992118,2cyclictest3169024-21kworker/u8:1+events_unbound11:20:230
2920382992118,2cyclictest2987468-21kworker/u8:0+events_unbound08:35:250
2920382992118,2cyclictest2962015-21kworker/u8:2+flush-259:008:10:210
2920382992118,2cyclictest2956222-21kworker/u8:1+events_unbound08:20:180
2920392992017,2cyclictest2956222-21kworker/u8:1+events_unbound08:00:233
2920389992015,4cyclictest3023013-21ntpq08:45:222
2920389992015,4cyclictest2925541-21latency_hist07:15:002
2920386992016,3cyclictest2941368-21apt-get07:30:131
2920386992014,4cyclictest3236034-21latency_hist12:15:011
2920382992017,2cyclictest3215639-21kworker/u8:1+flush-259:011:55:190
2920382992017,2cyclictest2956222-21kworker/u8:1+flush-259:008:05:260
2920382992016,3cyclictest3143146-21kworker/u8:5+events_unbound11:10:160
2920382992015,4cyclictest3168999-21latency_hist11:10:000
2920392991916,2cyclictest3013192-21kworker/u8:1+flush-259:009:05:003
2920392991915,3cyclictest3205319-21kworker/u8:2+flush-259:011:50:203
2920392991915,3cyclictest3153515-21kworker/u8:0+flush-259:011:30:213
2920392991915,3cyclictest2997777-21kworker/u8:3+flush-259:008:40:153
2920392991915,3cyclictest2962015-21kworker/u8:2+events_unbound07:55:203
2920389991916,2cyclictest3153515-21kworker/u8:0+flush-259:011:25:002
2920389991915,3cyclictest3205319-21kworker/u8:2+flush-259:012:35:222
2920389991915,3cyclictest3153515-21kworker/u8:0+flush-259:010:55:172
2920389991915,3cyclictest3050055-21kworker/u8:2+events_unbound09:30:232
2920386991916,2cyclictest0-21swapper/109:55:001
2920382991917,1cyclictest2997777-21kworker/u8:3+events_unbound08:40:220
2920382991917,1cyclictest0-21swapper/010:20:010
2920382991916,2cyclictest3215639-21kworker/u8:1+events_unbound12:15:180
2920382991916,2cyclictest3169024-21kworker/u8:1+events_unbound11:35:170
2920382991916,2cyclictest3065418-21kworker/u8:0+events_unbound10:10:000
2920382991915,3cyclictest3143146-21kworker/u8:5+events_unbound11:03:330
2920392991816,1cyclictest3143146-21kworker/u8:5+events_unbound10:55:223
2920392991815,2cyclictest3065418-21kworker/u8:0+flush-259:010:25:013
2920392991815,2cyclictest3065418-21kworker/u8:0+events_unbound10:25:233
2920392991815,2cyclictest2956222-21kworker/u8:1+events_unbound08:05:193
2920392991814,3cyclictest2871829-21kworker/u8:2+events_unbound07:15:213
2920392991812,5cyclictest252-21systemd-journal12:40:003
2920389991815,2cyclictest3153515-21kworker/u8:0+flush-259:011:45:172
2920389991812,4cyclictest3132858-21diskstats10:35:152
2920386991813,3cyclictest3164104-21apt-get11:05:121
2920386991813,3cyclictest2946543-21apt-get07:35:121
2920386991812,4cyclictest3091374-21/usr/sbin/munin09:55:141
2920382991816,2cyclictest3065418-21kworker/u8:0+events_unbound10:35:140
2920382991815,2cyclictest3050055-21kworker/u8:2+events_unbound09:30:250
2920382991815,2cyclictest2997777-21kworker/u8:3+flush-259:009:15:000
2920382991815,2cyclictest2962015-21kworker/u8:2+events_unbound07:50:180
2920382991814,3cyclictest2997777-21kworker/u8:3+flush-259:009:15:210
2920382991814,3cyclictest2997777-21kworker/u8:3+events_unbound08:45:020
2920382991814,3cyclictest2987468-21kworker/u8:0+flush-259:008:25:200
2920392991715,1cyclictest500-21systemd-logind08:50:013
2920392991714,2cyclictest3013192-21kworker/u8:1+events_unbound08:55:153
2920392991714,2cyclictest2997777-21kworker/u8:3+flush-259:009:15:163
2920392991714,2cyclictest2987468-21kworker/u8:0+flush-259:008:25:163
2920392991714,2cyclictest2871829-21kworker/u8:2+events_unbound07:25:243
2920392991714,2cyclictest0-21swapper/308:25:003
2920392991713,3cyclictest3205319-21kworker/u8:2+flush-259:012:20:193
2920389991713,3cyclictest3205319-21kworker/u8:2+events_unbound12:15:002
2920389991712,4cyclictest3060287-21latency_hist09:25:002
292038699171,7cyclictest3262226-21munin-run12:40:001
2920386991713,3cyclictest3143146-21kworker/u8:5+flush-259:011:10:211
2920386991712,3cyclictest3132698-21apt-get10:35:111
2920382991714,3cyclictest3013192-21kworker/u8:1+events_unbound09:05:240
2920382991714,2cyclictest3149025-21grep10:50:170
2920382991713,3cyclictest2962015-21kworker/u8:2+events_unbound08:15:140
2920382991713,3cyclictest2920585-21/usr/sbin/munin07:10:240
2920382991711,4cyclictest3075700-21turbostat09:40:000
2920392991613,2cyclictest3024990-21irqcore08:50:183
2920392991613,2cyclictest2997777-21kworker/u8:3+events_unbound09:35:243
2920392991613,2cyclictest2871829-21kworker/u8:2+events_unbound07:35:153
2920392991613,2cyclictest0-21swapper/311:50:003
2920392991612,3cyclictest3096255-21kworker/u8:2+events_unbound10:10:203
2920389991613,2cyclictest3111760-21sh10:14:592
2920389991612,3cyclictest3007990-21kworker/u8:2+flush-259:008:35:012
2920389991612,3cyclictest2871829-21kworker/u8:2+events_unbound07:25:222
2920386991614,1cyclictest2920372-21cyclictest11:05:011
2920386991613,2cyclictest3237139-21irqcore12:15:161
2920386991613,2cyclictest3032017-21kthreadcore08:55:171
2920386991612,3cyclictest3085993-21kworker/u8:1+flush-259:010:05:011
2920386991611,3cyclictest3205513-21diskstats11:45:131
2920386991611,3cyclictest252-21systemd-journal07:18:321
2920386991610,4cyclictest3132433-21latency_hist10:35:011
2920382991614,2cyclictest3050055-21kworker/u8:2+flush-259:009:25:220
2920382991614,2cyclictest2997777-21kworker/u8:3+events_unbound08:50:200
2920382991613,3cyclictest3122136-21kworker/u8:1+events_unbound10:40:250
2920382991613,2cyclictest61150irq/133-enp0s29f1:tx-011:45:240
2920382991613,2cyclictest61150irq/133-enp0s29f1:tx-011:30:190
2920382991613,2cyclictest60350irq/132-enp0s29f1:rx-012:09:210
2920382991613,2cyclictest60350irq/132-enp0s29f1:rx-007:55:110
2920382991613,2cyclictest2935950-21kworker/u8:3+flush-259:007:44:280
2920382991612,3cyclictest3205319-21kworker/u8:2+events_unbound12:35:140
2920382991612,3cyclictest2930247-21kworker/u8:0+flush-259:007:36:330
2920382991612,2cyclictest3205319-21kworker/u8:2+events_unbound12:20:210
2920382991612,2cyclictest3008038-21turbostat08:35:000
2920392991513,1cyclictest0-21swapper/309:49:593
2920392991512,2cyclictest3148618-21apt-get10:50:133
2920392991512,2cyclictest3085993-21kworker/u8:1+flush-259:009:50:263
2920392991512,2cyclictest3013192-21kworker/u8:1+flush-259:009:10:173
2920392991512,2cyclictest2997777-21kworker/u8:3+events_unbound09:40:233
2920392991511,3cyclictest2997777-21kworker/u8:3+flush-259:009:20:263
2920389991512,2cyclictest3065418-21kworker/u8:0+events_unbound10:25:172
2920389991512,2cyclictest2993278-21irqcore08:20:172
2920389991511,3cyclictest2962015-21kworker/u8:2+flush-259:008:10:052
2920389991511,3cyclictest2956222-21kworker/u8:1+events_unbound07:50:222
2920389991510,4cyclictest252-21systemd-journal09:00:012
2920389991510,3cyclictest3153515-21kworker/u8:0+events_unbound11:15:242
292038699159,4cyclictest252-21systemd-journal08:06:331
2920386991510,4cyclictest3050055-21kworker/u8:2+flush-259:009:21:051
2920386991510,3cyclictest252-21systemd-journal12:00:321
2920382991513,2cyclictest60350irq/132-enp0s29f1:rx-011:40:140
2920382991513,1cyclictest60350irq/132-enp0s29f1:rx-011:15:210
2920382991513,1cyclictest60350irq/132-enp0s29f1:rx-010:32:330
2920382991513,1cyclictest60350irq/132-enp0s29f1:rx-009:40:150
2920382991512,2cyclictest61150irq/133-enp0s29f1:tx-012:30:400
2920382991512,2cyclictest60350irq/132-enp0s29f1:rx-010:00:210
2920382991512,2cyclictest3085969-21sh09:50:000
2920382991512,2cyclictest2930247-21kworker/u8:0+flush-259:007:30:190
2920382991511,3cyclictest3205319-21kworker/u8:2+flush-259:011:55:000
2920382991511,3cyclictest2930247-21kworker/u8:0+flush-259:007:20:170
2920392991412,1cyclictest3252603-21irqcore12:30:163
2920392991411,2cyclictest3153515-21kworker/u8:0+events_unbound11:05:173
2920392991410,3cyclictest2956222-21kworker/u8:1+flush-259:007:43:373
292039299140,13cyclictest3163866-21munin-run11:05:013
2920389991411,2cyclictest3215639-21kworker/u8:1+events_unbound12:15:212
2920389991411,2cyclictest3205319-21kworker/u8:2+flush-259:012:05:202
292038699149,3cyclictest2930966-21apt-get07:20:101
2920386991411,2cyclictest3010586-21kthreadcore08:35:201
2920382991412,1cyclictest60350irq/132-enp0s29f1:rx-012:25:250
2920382991412,1cyclictest60350irq/132-enp0s29f1:rx-007:25:210
2920382991411,2cyclictest3106785-21kworker/u8:4+flush-259:010:20:200
292039299139,3cyclictest3169024-21kworker/u8:1+flush-259:011:44:373
292039299139,3cyclictest3013192-21kworker/u8:1+events_unbound09:05:213
2920392991310,2cyclictest3194808-21kworker/u8:3+flush-259:011:35:233
2920392991310,2cyclictest3065418-21kworker/u8:0+flush-259:010:35:163
2920392991310,2cyclictest0-21swapper/307:15:003
292038999139,3cyclictest2987468-21kworker/u8:0+flush-259:008:25:182
292038999138,4cyclictest2972299-21/usr/sbin/munin08:00:212
2920389991311,1cyclictest0-21swapper/207:40:002
2920389991310,2cyclictest3164301-21fschecks_count11:05:142
2920389991310,2cyclictest2997777-21kworker/u8:3+events_unbound09:40:012
2920389991310,1cyclictest0-21swapper/212:34:592
292038699139,3cyclictest3064502-21ntp_kernel_pll_09:25:221
292038699136,5cyclictest2987442-21awk08:15:001
2920382991312,1cyclictest60350irq/132-enp0s29f1:rx-009:56:550
2920382991311,2cyclictest60350irq/132-enp0s29f1:rx-012:02:320
2920382991311,2cyclictest60350irq/132-enp0s29f1:rx-010:56:160
2920382991311,1cyclictest60350irq/132-enp0s29f1:rx-010:25:380
2920382991311,1cyclictest0-21swapper/009:25:000
292039299129,2cyclictest3169024-21kworker/u8:1+flush-259:011:20:213
292039299129,2cyclictest3050055-21kworker/u8:2+flush-259:009:30:053
292039299129,2cyclictest0-21swapper/309:30:003
292039299128,3cyclictest3205319-21kworker/u8:2+events_unbound12:10:163
292039299127,4cyclictest2987455-21latency_hist08:15:003
292038999129,2cyclictest3085993-21kworker/u8:1+events_unbound09:50:262
292038999129,1cyclictest0-21swapper/208:55:002
292038999128,3cyclictest509-21rs:main2
292038999128,3cyclictest252-21systemd-journal10:04:332
292038999128,2cyclictest3080347-21unixbench_multi09:40:252
292038999120,11cyclictest3106573-21true10:10:012
292038699129,2cyclictest3251916-21idleruntime-cro12:29:591
292038699129,2cyclictest3149026-21sed10:50:171
292038699129,2cyclictest3003220-21cpuspeed_turbos08:30:121
292038699128,3cyclictest3081322-21fschecks_time09:45:141
292038699128,3cyclictest3076163-21fschecks_count09:40:151
292038699126,4cyclictest2997777-21kworker/u8:3+flush-259:009:00:251
292039299118,2cyclictest3169024-21kworker/u8:1+events_unbound11:25:183
292039299117,3cyclictest3143146-21kworker/u8:5+events_unbound11:10:233
292039299117,2cyclictest0-21swapper/310:20:003
292039299116,3cyclictest3225364-21sshd12:00:363
292038999119,1cyclictest0-21swapper/208:00:002
292038999118,2cyclictest3194305-21unixbench_multi11:30:262
292038999118,2cyclictest3153515-21kworker/u8:0+flush-259:011:40:192
292038999118,2cyclictest2997777-21kworker/u8:3+events_unbound09:15:182
292038999117,3cyclictest3065418-21kworker/u8:0+events_unbound09:45:202
292038999116,4cyclictest2930247-21kworker/u8:0+flush-259:007:17:362
292038999111,8cyclictest3215434-21munin-run11:54:592
292038999111,1cyclictest3018300-21sh08:45:002
292038699119,1cyclictest0-21swapper/110:20:011
292038699117,3cyclictest2962015-21kworker/u8:2+flush-259:008:00:171
292038699117,3cyclictest2956222-21kworker/u8:1+flush-259:007:46:371
292038699116,3cyclictest3075695-21latency_hist09:40:011
292038699115,4cyclictest252-21systemd-journal12:10:001
292039299107,2cyclictest0-21swapper/310:10:003
292038999107,2cyclictest3169024-21kworker/u8:1+events_unbound11:25:252
292038999107,2cyclictest0-21swapper/211:15:002
292038699107,1cyclictest0-21swapper/111:35:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional