memory_required() only checks xformers_enabled() and
pytorch_attention_flash_attention() when picking between the efficient
(area * dtype_size * 0.01 * factor) and conservative (area * 0.15 * factor)
estimate formulas. With --use-flash-attention the conservative formula is
used even though flash attention's memory footprint is comparable to SDPA's,
inflating the estimate 7.5x at bf16.
On AMD this bites whenever ENABLE_PYTORCH_ATTENTION is not auto-enabled
(e.g. torch ROCm wheels that ship without aotriton.images, like
2.12.0+rocm7.14.0): a 12.5 GB fp8 model at 2560x1440 on a 16 GB card gets a
20.5 GB estimate, falls onto the MIN_WEIGHT_MEMORY_RATIO floor, and streams
7 GB of weights over PCIe every step. With this change the same workflow
fully loads and runs 2.7x faster. Verified no behavior change for
large-latent workloads (Wan 2.2 I2V) that legitimately need the floor.
Fixes#15585
Add a ModelAttentionBackend node to manually select the attention for models in the workflows. Currently supports pytorch attention or comfy kitchen attention.
Add --use-ck-attention to enable comfy kitchen attention as the default attention backend for all models (might break some).
This avoids name collision (circular imports) for external custom nodes,
for which the comfy path is pushed into sys.path so Python's own logging module
is shadowed otherwise.
fixes: #15229
This priority scheme was broken in the case where you have pin
registration exhaustion while loading a VBAR that gets a big evicition.
The weight would stay in the loaded set but inherit the MRU priority
against other workflow models WRT pin registration which leads to async
offload without pinning.
Fix by universally promiting active pin registration above workflow
pins without concern for the weights/weights-loaded split. This diverges
from the actual budgeting where the split still makes sense.
Changes:
Remove sequential scan hint
Prefer NVML pressure on windows
Add async malloc clamp option (unused by comfy so far)
Workaround AMD windows GPU virtual address space leak
The largest change is the NVML pressure, which works around a cuMemGetInfo
drift from actual VRAM in some circumstances.
Windows has proven this logic works for a long time and there are
corner cases where this materialization actual consumes real RAM
on linux.
Its not as bad as the original windows commit charge surge, but
its still a detectable transient leak. So simplify and unify.
Dan Sedano
rattus
Alexander Brown
Alex Harper
comfyanonymous
Jukka Seppänen
Alexis Rolland
blepping
endman100
rivadart
Oliver Freyermuth
Barish Ozbay