You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-22 - 07:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Wed Jul 22, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2939988994236,4cyclictest3112668-21latency_hist21:55:012
2939985994136,3cyclictest2992250-21cat20:00:011
2939983994134,5cyclictest2992073-21latency_hist20:00:010
2939993993630,4cyclictest3279090-21latency_hist00:35:003
2939985993532,2cyclictest3143663-21kworker/u8:2+flush-259:022:45:151
2939983993531,3cyclictest3159173-21kworker/u8:4+events_unbound23:10:260
2939983993531,3cyclictest3159173-21kworker/u8:4+events_unbound23:10:250
2939983993531,3cyclictest3143663-21kworker/u8:2+events_unbound00:30:240
2939993993330,2cyclictest3143663-21kworker/u8:2+flush-259:022:45:173
2939993993329,3cyclictest3242659-21kworker/u8:0+flush-259:000:05:013
2939988993330,2cyclictest2992108-21kworker/u8:0+flush-259:020:05:012
2939985993330,2cyclictest3118054-21kworker/u8:1+events_unbound22:20:171
2939985993330,2cyclictest2950077-21kworker/u8:1+events_unbound20:15:181
2939985993229,2cyclictest859802-21kworker/u8:4+events_unbound19:10:181
2939985993227,3cyclictest2955484-21kworker/u8:3+events_unbound20:45:221
2939983993229,2cyclictest3143663-21kworker/u8:2+flush-259:023:35:230
2939983993229,2cyclictest3143663-21kworker/u8:2+events_unbound23:40:380
2939983993228,3cyclictest3019037-21kworker/u8:0+flush-259:020:30:230
2939993993128,2cyclictest3143663-21kworker/u8:2+flush-259:022:55:193
2939993993128,2cyclictest2955484-21kworker/u8:3+flush-259:020:00:163
2939993993128,2cyclictest2950077-21kworker/u8:1+flush-259:019:45:003
2939985993128,2cyclictest3143663-21kworker/u8:2+flush-259:000:10:011
2939985993127,3cyclictest3206538-21apt-get23:25:121
2939985993127,3cyclictest3174882-21kworker/u8:3+flush-259:023:01:531
2939988993027,2cyclictest2955484-21kworker/u8:3+events_unbound20:20:242
2939988993024,4cyclictest3190723-21cat23:10:012
2939988993024,4cyclictest3190723-21cat23:10:002
2939985993026,3cyclictest3216485-21kworker/u8:1+flush-259:023:55:001
2939985993026,3cyclictest2955484-21kworker/u8:3+flush-259:020:40:251
2939985993025,4cyclictest3081688-21latency_hist21:25:001
2939985993025,4cyclictest2955464-21latency_hist19:25:011
2939983993026,3cyclictest3092007-21kworker/u8:3+flush-259:021:50:220
2939983993025,3cyclictest3185269-21apt-get23:05:120
2939993992925,3cyclictest3013434-21kworker/u8:2+flush-259:021:20:203
2939988992923,5cyclictest3061067-21cat21:05:002
2939985992926,2cyclictest3013434-21kworker/u8:2+events_unbound21:00:221
2939985992925,3cyclictest3143663-21kworker/u8:2+events_unbound23:40:181
2939985992924,4cyclictest3045330-21apt-get20:50:121
2939983992923,4cyclictest3002008-21ntpq20:05:220
2939993992824,3cyclictest2886195-21kworker/u8:2+flush-259:019:25:263
2939985992825,2cyclictest3143663-21kworker/u8:2+events_unbound22:40:181
2939985992825,2cyclictest3055388-21kworker/u8:0+events_unbound21:45:171
2939985992824,3cyclictest2955484-21kworker/u8:3+events_unbound20:36:361
2939985992822,4cyclictest3148813-21latency_hist22:30:011
2939983992824,3cyclictest3152989-21kworker/u8:0+flush-259:022:55:000
2939983992823,4cyclictest3237973-21apt-get23:55:120
2939983992822,4cyclictest3179850-21latency_hist23:00:000
2939993992724,2cyclictest3060576-21gpgv21:05:003
2939988992723,3cyclictest3092007-21kworker/u8:3+flush-259:022:05:262
2939988992722,3cyclictest3268987-21apt-get00:25:142
2939985992723,3cyclictest3003113-21kworker/u8:0+events_unbound20:10:141
2939983992723,3cyclictest2886195-21kworker/u8:2+flush-259:019:25:220
2939988992619,5cyclictest3153043-21ntpq22:30:222
2939985992623,2cyclictest3185236-21kworker/u8:0+flush-259:023:05:231
2939985992623,2cyclictest3092007-21kworker/u8:3+flush-259:022:05:211
2939985992623,2cyclictest3055388-21kworker/u8:0+events_unbound21:15:131
2939985992622,3cyclictest3152989-21kworker/u8:0+events_unbound22:32:171
2939985992622,3cyclictest3143663-21kworker/u8:2+flush-259:022:50:211
2939985992622,3cyclictest3013434-21kworker/u8:2+events_unbound20:25:151
2939985992622,3cyclictest2886195-21kworker/u8:2+flush-259:019:25:161
2939983992623,2cyclictest2955484-21kworker/u8:3+events_unbound19:50:200
2939983992623,2cyclictest2886195-21kworker/u8:2+flush-259:019:15:000
2939983992622,3cyclictest2955484-21kworker/u8:3+flush-259:020:45:000
2939988992521,3cyclictest3268744-21kworker/u8:0+flush-259:000:30:172
2939988992518,5cyclictest3045276-21cp20:50:022
2939985992522,2cyclictest3216485-21kworker/u8:1+flush-259:000:00:251
2939985992522,2cyclictest3055388-21kworker/u8:0+events_unbound21:06:181
2939985992522,2cyclictest2950077-21kworker/u8:1+flush-259:019:45:261
2939985992521,3cyclictest3279319-21kworker/u8:3+flush-259:000:35:211
2939985992521,3cyclictest3242659-21kworker/u8:0+events_unbound00:12:071
2939985992521,3cyclictest3185236-21kworker/u8:0+events_unbound23:30:181
293998599250,22cyclictest1-21systemd22:39:591
2939983992521,3cyclictest3112693-21kworker/u8:2+events_unbound21:55:240
2939983992521,3cyclictest3066406-21kworker/u8:4+flush-259:021:35:000
2939983992521,3cyclictest2955484-21kworker/u8:3+events_unbound20:15:000
2939993992421,2cyclictest3216485-21kworker/u8:1+events_unbound00:25:003
2939988992420,3cyclictest3216485-21kworker/u8:1+flush-259:000:15:172
2939988992420,3cyclictest2886195-21kworker/u8:2+flush-259:019:25:172
2939988992417,4cyclictest3284242-21latency_hist00:40:012
2939985992422,2cyclictest3092007-21kworker/u8:3+flush-259:021:40:251
2939985992420,3cyclictest3242659-21kworker/u8:0+events_unbound23:59:191
2939985992420,3cyclictest3185236-21kworker/u8:0+flush-259:023:45:151
2939985992420,3cyclictest3143663-21kworker/u8:2+events_unbound22:55:161
2939985992420,3cyclictest2970991-21gpgv19:40:001
2939985992416,5cyclictest2996787-21ntpq20:00:211
2939993992318,3cyclictest3258650-21/usr/sbin/munin00:15:163
2939985992322,1cyclictest3092007-21kworker/u8:3+flush-259:021:50:241
2939985992319,3cyclictest2966018-21kworker/u8:0+events_unbound19:43:171
2939985992319,3cyclictest2886195-21kworker/u8:2+events_unbound19:31:271
2939985992316,5cyclictest335557-21systemd-journal21:14:171
2939983992320,2cyclictest2955484-21kworker/u8:3+flush-259:020:55:200
2939983992319,3cyclictest3152989-21kworker/u8:0+events_unbound22:48:190
2939983992319,3cyclictest3092007-21kworker/u8:3+events_unbound21:35:240
2939983992319,3cyclictest3055388-21kworker/u8:0+flush-259:021:05:150
2939983992318,3cyclictest3143905-21apt-get22:25:140
2939988992217,4cyclictest3143663-21kworker/u8:2+flush-259:022:40:252
2939985992219,2cyclictest2955484-21kworker/u8:3+flush-259:019:50:181
2939985992217,3cyclictest3092007-21kworker/u8:3+events_unbound22:10:301
2939983992217,4cyclictest3019037-21kworker/u8:0+events_unbound20:26:350
2939983992216,4cyclictest3075638-21ntpq21:15:210
2939993992115,4cyclictest3023144-21ntpq20:25:253
2939988992117,3cyclictest2955484-21kworker/u8:3+flush-259:020:40:192
2939988992115,4cyclictest335557-21systemd-journal20:30:002
2939985992117,3cyclictest3190234-21kworker/u8:1+events_unbound23:20:281
2939985992117,3cyclictest3013434-21kworker/u8:2+flush-259:021:25:201
2939985992116,4cyclictest3143663-21kworker/u8:2+events_unbound00:33:581
2939985992115,4cyclictest335557-21systemd-journal23:10:021
2939985992115,4cyclictest335557-21systemd-journal23:10:021
2939983992118,2cyclictest3232741-21kworker/u8:4+events_unbound23:50:230
2939983992118,2cyclictest3143663-21kworker/u8:2+events_unbound00:20:150
2939983992118,2cyclictest3071358-21cron21:15:000
2939983992118,2cyclictest2950077-21kworker/u8:1+events_unbound20:20:140
2939983992117,3cyclictest2955484-21kworker/u8:3+events_unbound20:03:180
2939983992115,5cyclictest3092007-21kworker/u8:3+events_unbound22:15:230
2939993992016,3cyclictest3152989-21kworker/u8:0+flush-259:022:33:283
2939993992016,3cyclictest3092007-21kworker/u8:3+flush-259:022:20:013
2939993992016,3cyclictest2987183-21apt-get19:55:133
2939993992016,3cyclictest2886195-21kworker/u8:2+flush-259:019:15:193
2939988992017,2cyclictest3216656-21gpgv23:35:012
2939988992017,2cyclictest3055388-21kworker/u8:0+flush-259:021:10:182
2939988992016,3cyclictest3092007-21kworker/u8:3+flush-259:022:10:172
2939988992014,4cyclictest3101430-21ntpq21:40:212
2939983992017,2cyclictest3152989-21kworker/u8:0+events_unbound22:31:160
2939983992016,3cyclictest859802-21kworker/u8:4+events_unbound19:16:180
2939983992016,2cyclictest2950077-21kworker/u8:1+events_unbound20:15:200
2939993991916,2cyclictest3233202-21irqcore23:50:173
2939993991915,3cyclictest3185236-21kworker/u8:0+flush-259:023:45:223
2939993991915,3cyclictest3143663-21kworker/u8:2+flush-259:022:40:163
2939993991914,4cyclictest335557-21systemd-journal00:30:013
2939988991917,1cyclictest3143663-21kworker/u8:2+events_unbound22:50:172
2939985991916,2cyclictest3066406-21kworker/u8:4+events_unbound21:30:181
2939985991915,3cyclictest3143663-21kworker/u8:2+events_unbound00:26:171
2939985991915,2cyclictest3199843-21kthreadcore23:15:211
2939985991915,2cyclictest3199843-21kthreadcore23:15:201
2939983991917,1cyclictest3092007-21kworker/u8:3+events_unbound22:10:150
2939983991916,2cyclictest3092007-21kworker/u8:3+events_unbound21:40:230
2939983991915,3cyclictest3174882-21kworker/u8:3+events_unbound23:02:170
2939983991915,3cyclictest3013434-21kworker/u8:2+flush-259:021:25:180
2939983991913,4cyclictest3055797-21diskstats21:00:140
2939993991815,2cyclictest3174882-21kworker/u8:3+flush-259:023:01:573
2939993991815,2cyclictest2971-21cron23:45:003
2939985991814,3cyclictest3143663-21kworker/u8:2+events_unbound00:24:191
2939985991814,3cyclictest2955484-21kworker/u8:3+flush-259:020:55:301
2939983991815,3cyclictest3055388-21kworker/u8:0+events_unbound21:45:250
2939983991814,3cyclictest3185236-21kworker/u8:0+events_unbound23:30:220
2939993991715,1cyclictest509-21rs:main3
2939993991714,2cyclictest3268744-21kworker/u8:0+flush-259:000:40:013
2939993991714,2cyclictest2950077-21kworker/u8:1+flush-259:019:30:223
2939993991711,4cyclictest2980850-21ntpq19:45:233
293998899171,7cyclictest3232495-21cron23:50:012
2939988991714,1cyclictest0-21swapper/219:55:182
2939988991713,3cyclictest3243182-21apt-get00:00:122
2939988991712,3cyclictest3092230-21/usr/sbin/munin21:35:252
2939985991714,2cyclictest2886195-21kworker/u8:2+flush-259:019:15:211
2939983991714,2cyclictest60350irq/132-enp0s29f1:rx-022:20:290
2939983991714,2cyclictest3019037-21kworker/u8:0+events_unbound20:35:170
293998399171,15cyclictest1-21systemd19:41:180
2939993991612,3cyclictest3112693-21kworker/u8:2+flush-259:021:55:223
2939993991612,2cyclictest0-21swapper/320:20:003
293998899169,5cyclictest3054467-21ntpq20:55:222
2939988991613,2cyclictest3211434-21sh23:30:002
2939988991611,4cyclictest3143663-21kworker/u8:2+flush-259:022:55:222
2939988991611,3cyclictest3196069-21/usr/sbin/munin23:15:232
2939988991611,3cyclictest3196069-21/usr/sbin/munin23:15:222
2939985991612,3cyclictest3092007-21kworker/u8:3+flush-259:022:00:271
2939985991612,3cyclictest3092007-21kworker/u8:3+flush-259:021:35:211
2939985991612,3cyclictest3019037-21kworker/u8:0+events_unbound20:30:131
2939983991613,2cyclictest0-21swapper/022:44:590
2939983991612,3cyclictest3118089-21apt-get22:00:120
2939983991612,3cyclictest3092007-21kworker/u8:3+flush-259:022:05:150
293999399159,5cyclictest3091134-21ntpq21:30:223
293999399159,4cyclictest335557-21systemd-journal20:36:383
2939993991512,2cyclictest0-21swapper/320:50:003
293998899159,5cyclictest3123005-21latency_hist22:05:022
2939988991512,2cyclictest0-21swapper/221:20:012
2939988991511,3cyclictest3092007-21kworker/u8:3+flush-259:022:20:222
2939988991511,3cyclictest2955484-21kworker/u8:3+flush-259:019:50:152
2939988991510,3cyclictest2971171-21store19:40:012
2939983991513,1cyclictest60350irq/132-enp0s29f1:rx-023:27:590
2939983991513,1cyclictest60350irq/132-enp0s29f1:rx-021:20:250
2939983991513,1cyclictest60350irq/132-enp0s29f1:rx-020:45:260
2939983991513,1cyclictest60350irq/132-enp0s29f1:rx-019:35:140
2939983991513,1cyclictest60350irq/132-enp0s29f1:rx-000:19:100
2939983991512,2cyclictest60350irq/132-enp0s29f1:rx-023:15:150
2939983991512,2cyclictest60350irq/132-enp0s29f1:rx-023:15:140
2939983991512,2cyclictest3279319-21kworker/u8:3+events_unbound00:35:220
2939983991511,3cyclictest60350irq/132-enp0s29f1:rx-000:10:110
2939983991511,3cyclictest500-21systemd-logind00:05:000
2939983991511,3cyclictest3185236-21kworker/u8:0+flush-259:023:20:210
2939983991511,3cyclictest2886195-21kworker/u8:2+events_unbound19:33:180
2939983991510,3cyclictest3045206-21gpgv20:50:000
293999399149,3cyclictest490-21dbus-daemon23:30:003
2939993991411,2cyclictest3258004-21unixbench_singl00:10:253
2939993991411,2cyclictest3143663-21kworker/u8:2+flush-259:022:35:223
2939993991411,2cyclictest0-21swapper/321:00:003
2939993991410,3cyclictest509-21rs:main3
293998899149,4cyclictest3013434-21kworker/u8:2+flush-259:021:30:232
293998899149,3cyclictest3190384-21cat23:10:002
293998899147,5cyclictest3034286-21latency_hist20:40:012
2939988991411,2cyclictest3205299-21needreboot23:20:192
293998899140,12cyclictest1-21systemd22:36:172
2939985991411,2cyclictest3055388-21kworker/u8:0+events_unbound21:55:121
2939985991410,3cyclictest3185236-21kworker/u8:0+events_unbound23:35:011
2939985991410,3cyclictest2955484-21kworker/u8:3+flush-259:020:20:201
2939983991412,1cyclictest60350irq/132-enp0s29f1:rx-019:46:400
2939983991412,1cyclictest60350irq/132-enp0s29f1:rx-019:23:200
2939983991412,1cyclictest60350irq/132-enp0s29f1:rx-000:05:250
2939983991412,1cyclictest3143663-21kworker/u8:2+events_unbound22:35:150
2939983991411,2cyclictest60350irq/132-enp0s29f1:rx-023:45:160
2939993991312,0cyclictest4020778-21kworker/3:0+events_power_efficient21:45:123
2939993991310,2cyclictest3050243-21systemd-run20:55:003
2939993991310,1cyclictest0-21swapper/322:00:183
293998899139,3cyclictest3019037-21kworker/u8:0+flush-259:020:30:252
293998899138,3cyclictest3174882-21kworker/u8:3+flush-259:023:00:102
293998899138,3cyclictest3012907-21journal-offline20:16:312
2939985991310,2cyclictest3000258-21sort20:05:191
293999399128,3cyclictest3190234-21kworker/u8:1+flush-259:023:15:163
293999399128,3cyclictest3190234-21kworker/u8:1+flush-259:023:15:153
293999399127,4cyclictest2986031-21ntpq19:50:193
293999399126,4cyclictest335557-21systemd-journal22:24:173
293998899129,2cyclictest3267711-21needreboot00:20:192
293998899129,2cyclictest3226946-21unixbench_singl23:40:392
293998899129,2cyclictest3168513-21grep22:45:202
293999399119,1cyclictest0-21swapper/322:05:213
293999399118,2cyclictest2966468-21irqcore19:35:163
293999399117,3cyclictest2902095-21kworker/u8:0+flush-259:019:20:153
293999399115,4cyclictest3195814-21cron23:15:003
293999399115,4cyclictest3195814-21cron23:14:593
293999399111,1cyclictest31002702kthreadcore21:40:193
293998899119,1cyclictest0-21swapper/223:55:242
293998899111,2cyclictest2965812-21systemd-run19:35:002
293998899110,9cyclictest3258502-21cat00:15:012
293998399119,1cyclictest60350irq/132-enp0s29f1:rx-000:25:150
293999399108,1cyclictest0-21swapper/323:55:183
293999399108,1cyclictest0-21swapper/321:35:223
293998899106,2cyclictest3007187-21ntpq20:10:222
293998899105,4cyclictest3133820-21tune2fs22:15:152
29399939997,1cyclictest0-21swapper/320:40:203
29399859996,2cyclictest3258742-21cpuspeed_turbos00:15:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional