The dynamic VBAR cast path applied want_requant to both weights and biases.
For an FP8 checkpoint with a low-VRAM LoRA patch, this returned an FP8 bias
to the optimized linear operation even though the activation dtype was FP16.
Restrict the optional requantized return value to weights while preserving
resident-storage updates for both parameters. Make the legacy FP8 linear path
release its VBAR/offload resources in a finally block so backend failures do
not leave pages pinned before the normal fallback runs.
Add regressions for the weight/bias dtype contract and exception cleanup.
Fixes#14952
Signed-off-by: liminfei <liminfei@amd.com>
When a prompt is submitted without client_id and its output nodes are
served from cache, _send_cached_ui returned early before recording the
cached UI outputs, so /api/jobs/{job_id} (and /history) reported success
with empty outputs. Record the outputs before the client_id check.
#14862 auto-enables the comfy-kitchen Triton backend whenever torch.version.hip
is set and Triton >= 3.7. The INT8 matmul kernels compile tl.dot to matrix-core
instructions (WMMA on RDNA3+/gfx11xx-gfx12xx, MFMA on CDNA/gfx9xx); RDNA1/RDNA2
(gfx10xx) have neither, so the auto-enabled INT8 path hangs the GPU there
(reported on RDNA2 + triton-windows 3.7.1: native and custom-node INT8 freeze
until reset).
Gate the automatic ROCm default on GPU architecture as well as Triton version so
RDNA1/RDNA2 stay on the working eager fallback. Add --disable-triton-backend as
an explicit override; --enable-triton-backend still force-enables on any arch.
On AMD/ROCm the CUDA backend is unavailable, so Triton is the only accelerated
comfy-kitchen backend. It was disabled by default (opt-in --enable-triton-backend),
leaving AMD on the slow eager path. Enable it by default when torch.version.hip is
set AND Triton is >= 3.7 -- older Triton lacks libdevice.rint on the HIP backend and
hard-crashes the INT8 path, so on Triton < 3.7 it stays disabled with a log line.
NVIDIA behavior is unchanged; the explicit --enable-triton-backend flag still works
as an override.
Fixes#14861
mixed_precision_ops.Linear.forward only quantized activations that were 2D, or
3D (reshaped to 2D). Inputs with rank >= 4 (e.g. Anima's MLP activations, which
are not reshaped to 3D the way the attention path is) fell through the
`input_reshaped.ndim == 2` guard and reached scaled_mm as bf16, silently
dispatching a bf16 kernel instead of FP8. Since MLP is roughly half the compute,
the FP8 speedup was far below expectation.
Generalize the existing 3D->2D reshape to any rank >= 3 (flatten the leading
dims, keep the contraction dim) and reshape the output back to the original
leading dims. 2D and 3D inputs are handled exactly as before; only rank >= 4
inputs change (now quantized instead of skipped). This matches the rank-agnostic
handling already used by the training path (flatten(0, -2) / unflatten).
Fixes#14595.
liminfei-amd
comfyanonymous
Alexander Piskun
Daxiong (Lin)
Comfy Org PR Bot
liminfei
彼彼
Alexis Rolland
comfyanonymous
Terry Jia
Gustavo Schneiter
Yousef R. Gamaleldin
John Pollock