MPS does not support float8_e4m3fn/float8_e5m2 dtypes. When FP8-quantized
models (FLUX, SD3.5, Wan 2.2, LTX-Video) are loaded on Apple Silicon, the
quantization step crashes with:
TypeError: Trying to convert Float8_e4m3fn to the MPS backend but it does
not have support for that dtype.
This adds device-aware fallbacks that move tensors to CPU for the FP8
quantization step only. The rest of inference remains on MPS.
Three code paths are patched:
- comfy/float.py: stochastic_rounding() — also fixes the secondary
"Placeholder storage has not been allocated on MPS device!" error
caused by torch.Generator being bound to MPS.
- comfy/float.py: stochastic_round_quantize_nvfp4*() — these create
float8_e4m3fn block scales internally.
- comfy/quant_ops.py: _TensorCoreFP8LayoutBase.quantize() — the
ck.quantize_per_tensor_fp8 path also fails on MPS.
Fixes: #6995, #9255, #11626, #11817
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Torch has alignment enforcement when viewing with data type changes
but only relative to itself. Do all tensor constructions straight
off the memory-view individually so pytorch doesnt see an alignment
problem.
The is needed for handling misaligned safetensors weights, which are
reasonably common in third party models.
This limits usage of this safetensors loader to GPU compute only
as CPUs kernnel are very likely to bus error. But it works for
dynamic_vram, where we really dont want to take a deep copy and we
always use GPU copy_ which disentangles the misalignment.
* feat(comfy_api): add basic 3D Model file types
* update Tripo nodes to use File3DGLB
* update Rodin3D nodes to use File3DGLB
* address PR review feedback:
- Rename File3D parameter 'path' to 'source'
- Convert File3D.data property to get_data()
- Make .glb extension check case-insensitive in nodes_rodin.py
- Restrict SaveGLB node to only accept File3DGLB
* Fixed a bug in the Meshy Rig and Animation nodes
* Fix backward compatability
This is using a different layers weight with .to(). Change it to use
the ops caster if the original layer is a comfy weight so that it picks
up dynamic_vram and async_offload functionality in full.
Co-authored-by: Rattus <rattus128@gmail.com>
* mp: fix full dynamic unloading
This was not unloading dynamic models when requesting a full unload via
the unpatch() code path.
This was ok, i your workflow was all dynamic models but fails with big
VRAM leaks if you need to fully unload something for a regular ModelPatcher
It also fices the "unload models" button.
* mm: load models outside of Aimdo Mempool
In dynamic_vram mode, escape the Aimdo mempool and load into the regular
mempool. Use a dummy thread to do it.
Tashdid Khan
comfyanonymous
ComfyUI Wiki
chaObserv
Jukka Seppänen
tdrussell
asagi4
AustinMroz
Comfy Org PR Bot
comfyanonymous
blepping
Alexander Piskun
rattus