Widen w4a8 group scales to fp32 on MPS

torch's MPS backend has no fp8 casts, so the eager W4A8 dequant path
raises "Undefined type Float8_e4m3fn" on s_rel.float() and no
asym_w4a8_int8 model can load. Convert the group scale at load time;
fp32 is the only other dtype the eager backend declares for s_rel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KcrQ3SfeW47jxhsyt6KXuQ
This commit is contained in:
scriptease 2026-08-12 14:49:27 +02:00
parent bd34f338ac
commit 2346dadfe3
1 changed files with 5 additions and 0 deletions

View File

@ -1203,6 +1203,11 @@ def _load_quantized_module(module, super_load, state_dict, prefix, local_metadat
raise ValueError(f"Missing W4A8 group scale (weight_s_rel) for layer {layer_name}")
if scale.dtype == torch.uint8:
scale = scale.view(torch.float8_e4m3fn)
if comfy.model_management.get_torch_device().type == "mps":
# MPS has no fp8 casts at all, so the eager dequant path dies on
# s_rel.float(). fp32 is the only other dtype the eager backend
# declares for s_rel, so fp16 is not an option here.
scale = scale.float()
params_conf = layer_conf.get("params", {})
if not isinstance(params_conf, dict):
params_conf = {}