When a prompt is submitted without client_id and its output nodes are
served from cache, _send_cached_ui returned early before recording the
cached UI outputs, so /api/jobs/{job_id} (and /history) reported success
with empty outputs. Record the outputs before the client_id check.
TrainLoraNode crashes for any model whose VAE produces 5D latents
(Qwen Image / Krea 2 and other video-style [B, C, T, H, W] VAEs) when
bucket_mode is enabled or the dataset has mixed resolutions:
RuntimeError: Number of dimensions of repeat dims can not be
smaller than number of dimensions of tensor
Both dummy-latent constructions in _run_training_loop hardcode a
4D .repeat(num_images, 1, 1, 1). Repeat along the batch dim only,
keeping every remaining dim, so the guider dummy works for latents
of any rank. The train steps themselves (standard/bucket/multi-res)
already index only the batch dim, so no other change is needed.
Verified on Krea 2 RAW fp8 (Qwen Image VAE, 5D latents) with a
14-image mixed-resolution dataset: both ResolutionBucket+bucket_mode
and the multi-res fallback path now train to completion; 4D SD/SDXL
behavior is unchanged (repeat expansion is identical for rank 4).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011ruwJZq4Tv7rVZa7pz7cqS
#14862 auto-enables the comfy-kitchen Triton backend whenever torch.version.hip
is set and Triton >= 3.7. The INT8 matmul kernels compile tl.dot to matrix-core
instructions (WMMA on RDNA3+/gfx11xx-gfx12xx, MFMA on CDNA/gfx9xx); RDNA1/RDNA2
(gfx10xx) have neither, so the auto-enabled INT8 path hangs the GPU there
(reported on RDNA2 + triton-windows 3.7.1: native and custom-node INT8 freeze
until reset).
Gate the automatic ROCm default on GPU architecture as well as Triton version so
RDNA1/RDNA2 stay on the working eager fallback. Add --disable-triton-backend as
an explicit override; --enable-triton-backend still force-enables on any arch.
On AMD/ROCm the CUDA backend is unavailable, so Triton is the only accelerated
comfy-kitchen backend. It was disabled by default (opt-in --enable-triton-backend),
leaving AMD on the slow eager path. Enable it by default when torch.version.hip is
set AND Triton is >= 3.7 -- older Triton lacks libdevice.rint on the HIP backend and
hard-crashes the INT8 path, so on Triton < 3.7 it stays disabled with a log line.
NVIDIA behavior is unchanged; the explicit --enable-triton-backend flag still works
as an override.
Fixes#14861
mannydelrio2
comfyanonymous
Alexander Piskun
Daxiong (Lin)
Comfy Org PR Bot
彼彼
Alexis Rolland
comfyanonymous
Terry Jia
Gustavo Schneiter
Manny Del Rio
Yousef R. Gamaleldin
liminfei-amd
John Pollock