Use efficient attention memory estimate when --use-flash-attention is set

memory_required() only checks xformers_enabled() and
pytorch_attention_flash_attention() when picking between the efficient
(area * dtype_size * 0.01 * factor) and conservative (area * 0.15 * factor)
estimate formulas. With --use-flash-attention the conservative formula is
used even though flash attention's memory footprint is comparable to SDPA's,
inflating the estimate 7.5x at bf16.

On AMD this bites whenever ENABLE_PYTORCH_ATTENTION is not auto-enabled
(e.g. torch ROCm wheels that ship without aotriton.images, like
2.12.0+rocm7.14.0): a 12.5 GB fp8 model at 2560x1440 on a 16 GB card gets a
20.5 GB estimate, falls onto the MIN_WEIGHT_MEMORY_RATIO floor, and streams
7 GB of weights over PCIe every step. With this change the same workflow
fully loads and runs 2.7x faster. Verified no behavior change for
large-latent workloads (Wan 2.2 I2V) that legitimately need the floor.

Fixes #15585
This commit is contained in:
Dan Sedano 2026-08-13 11:56:40 -07:00
parent 2f35f4a081
commit 1183f60828
1 changed files with 1 additions and 1 deletions

View File

@ -423,7 +423,7 @@ class BaseModel(torch.nn.Module):
if len(shape) > 0:
input_shapes += shape
if comfy.model_management.xformers_enabled() or comfy.model_management.pytorch_attention_flash_attention():
if comfy.model_management.xformers_enabled() or comfy.model_management.pytorch_attention_flash_attention() or comfy.model_management.flash_attention_enabled():
dtype = self.get_dtype_inference()
#TODO: this needs to be tweaked
area = sum(map(lambda input_shape: input_shape[0] * math.prod(input_shape[2:]), input_shapes))