The new-format branch of convert_old_quants() writes "{layer_key}.comfy_quant"
markers straight from the checkpoint's _quantization_metadata JSON, ignoring
model_prefix entirely. Checkpoint metadata stores layer keys either with the
full diffusion-model prefix or already stripped of it, and convert_old_quants()
is invoked before and/or after that prefix is stripped from state_dict
(comfy/sd.py calls it up to twice around the strip). A fixed assumption about
which convention is in play silently mismatches the other, so affected layers
never get wrapped in a QuantizedTensor and fall back to plain-dtype storage
(observed as VRAM blowup / manual-cast warnings on NVFP4 checkpoints such as
LTX-Video 2.3).
PR #13328 tried to fix this by reordering comfy/sd.py's calls around the
prefix strip, but that only swaps which convention works and was closed by
the maintainer for risking regressions on checkpoints using the other
convention. This instead makes convert_old_quants() match each layer key
against the state_dict's real key first (zero behavior change when metadata
is already aligned with state_dict), then try adding/stripping model_prefix,
and only fall back to today's blind write when neither matches. No call
sites in comfy/sd.py are touched or reordered. The write is now also
idempotent, guarding against the double-call case rewriting a resolved key.
Adds tests-unit/comfy_quant/test_convert_old_quants_prefix.py covering both
metadata conventions, the legacy scaled_fp8 branch (unaffected), the exact
two-call dance from load_diffusion_model_state_dict, and a documented (xfail)
residual gap: when a checkpoint's real weight keys carry no prefix at all but
its metadata keys do, comfy/sd.py's model_prefix="" call-site argument still
lets model_detection.unet_prefix_from_state_dict get poisoned by the
resulting spurious marker keys. That is pre-existing (reproduces identically
without this patch) and out of scope here since fixing it needs a
comfy/sd.py or model_detection.py change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
linear_dtype in comfy_quant metadata can be used to set if the int4 op does
the matrix multiplication in int8 or int4, the default is int4 on GPUs that
support it with fallback to int8 for GPUs that don't.
* Implement mixed precision operations with a registry design and metadate for quant spec in checkpoint.
* Updated design using Tensor Subclasses
* Fix FP8 MM
* An actually functional POC
* Remove CK reference and ensure correct compute dtype
* Update unit tests
* ruff lint
* Implement mixed precision operations with a registry design and metadate for quant spec in checkpoint.
* Updated design using Tensor Subclasses
* Fix FP8 MM
* An actually functional POC
* Remove CK reference and ensure correct compute dtype
* Update unit tests
* ruff lint
* Fix missing keys
* Rename quant dtype parameter
* Rename quant dtype parameter
* Fix unittests for CPU build
Alan Li
comfyanonymous
contentis