Address CodeRabbit review on #15410: clear quant_format/layout_type
when a module reloads an unquantized weight (previously stale state
made Linear.forward take the quantized path against a plain
Parameter), and raise instead of silently loading raw bytes when an
Embedding's inferred format is NVFP4, which the embedding path can't
dequantize.
The all-NUL-marker fix only covered layers with no comfy_quant config.
Layers with valid, non-empty comfy_quant JSON that carry a weight_scale
but no "format" key (e.g. a q_proj-style layer in the MiniMax H3 nvfp4
AWQ checkpoint) still hit "Unknown quantization format for layer ...".
Fall back to inferring the format from the on-disk weight dtype
(int8 -> int8_tensorwise, float8_e4m3fn -> float8_e4m3fn, uint8 ->
nvfp4) in both the shared _load_quantized_module path and the
mixed-precision Embedding's own inline handling, matching the
workaround documented in the issue.
Some quantizers mark unquantized layers in a mixed-precision checkpoint
with an all-NUL comfy_quant placeholder tensor instead of omitting the
key. json.loads() misdetects the leading NUL bytes as UTF-32 and raises
UnicodeDecodeError; once that decode is patched around, the resulting
empty {} config still fails with "Unknown quantization format" because
the loader always requires an explicit format for any present marker.
Treat an all-NUL marker as if comfy_quant were absent so the layer loads
as a plain unquantized weight, matching the official MiniMax H3 nvfp4
AWQ text encoder.
Fixes#15400
linear_dtype in comfy_quant metadata can be used to set if the int4 op does
the matrix multiplication in int8 or int4, the default is int4 on GPUs that
support it with fallback to int8 for GPUs that don't.
* Implement mixed precision operations with a registry design and metadate for quant spec in checkpoint.
* Updated design using Tensor Subclasses
* Fix FP8 MM
* An actually functional POC
* Remove CK reference and ensure correct compute dtype
* Update unit tests
* ruff lint
* Implement mixed precision operations with a registry design and metadate for quant spec in checkpoint.
* Updated design using Tensor Subclasses
* Fix FP8 MM
* An actually functional POC
* Remove CK reference and ensure correct compute dtype
* Update unit tests
* ruff lint
* Fix missing keys
* Rename quant dtype parameter
* Rename quant dtype parameter
* Fix unittests for CPU build
chelsealong
comfyanonymous
contentis