Widen w4a8 group scales to fp32 on MPS
torch's MPS backend has no fp8 casts, so the eager W4A8 dequant path raises "Undefined type Float8_e4m3fn" on s_rel.float() and no asym_w4a8_int8 model can load. Convert the group scale at load time; fp32 is the only other dtype the eager backend declares for s_rel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KcrQ3SfeW47jxhsyt6KXuQ
This commit is contained in:
parent
bd34f338ac
commit
2346dadfe3
|
|
@ -1203,6 +1203,11 @@ def _load_quantized_module(module, super_load, state_dict, prefix, local_metadat
|
|||
raise ValueError(f"Missing W4A8 group scale (weight_s_rel) for layer {layer_name}")
|
||||
if scale.dtype == torch.uint8:
|
||||
scale = scale.view(torch.float8_e4m3fn)
|
||||
if comfy.model_management.get_torch_device().type == "mps":
|
||||
# MPS has no fp8 casts at all, so the eager dequant path dies on
|
||||
# s_rel.float(). fp32 is the only other dtype the eager backend
|
||||
# declares for s_rel, so fp16 is not an option here.
|
||||
scale = scale.float()
|
||||
params_conf = layer_conf.get("params", {})
|
||||
if not isinstance(params_conf, dict):
|
||||
params_conf = {}
|
||||
|
|
|
|||
Loading…
Reference in New Issue