On Apple Silicon, SHARED vram state is treated like HIGH_VRAM in
several code paths, which keeps all models loaded on the GPU device.
Since MPS uses unified memory (GPU memory IS system memory), this
starves the system of memory for inference activations when running
large models (e.g. Wan 2.2 14B I2V loads 13.6GB model + 21.6GB text
encoder = 35GB, leaving only 13GB for activations on a 48GB system).
This caused GPU hangs and kernel panics on macOS when generating video
with 14B parameter models.
Changes:
- Set MPS vram_state to NORMAL_VRAM instead of SHARED, enabling smart
memory management to offload unused models during inference
- Remove SHARED from unet_inital_load_device HIGH_VRAM fast path, so
model loading respects available memory on MPS
- Remove MPS shortcut in text_encoder_initial_device that forced the
text encoder to stay on GPU regardless of memory pressure
On unified memory, "offloading to CPU" is nearly free (same physical
memory pool, just a device pointer change), so NORMAL_VRAM behavior
is strictly better — it lets ComfyUI intelligently manage what stays
on the GPU device based on available memory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Torch has alignment enforcement when viewing with data type changes
but only relative to itself. Do all tensor constructions straight
off the memory-view individually so pytorch doesnt see an alignment
problem.
The is needed for handling misaligned safetensors weights, which are
reasonably common in third party models.
This limits usage of this safetensors loader to GPU compute only
as CPUs kernnel are very likely to bus error. But it works for
dynamic_vram, where we really dont want to take a deep copy and we
always use GPU copy_ which disentangles the misalignment.
* feat(comfy_api): add basic 3D Model file types
* update Tripo nodes to use File3DGLB
* update Rodin3D nodes to use File3DGLB
* address PR review feedback:
- Rename File3D parameter 'path' to 'source'
- Convert File3D.data property to get_data()
- Make .glb extension check case-insensitive in nodes_rodin.py
- Restrict SaveGLB node to only accept File3DGLB
* Fixed a bug in the Meshy Rig and Animation nodes
* Fix backward compatability
This is using a different layers weight with .to(). Change it to use
the ops caster if the original layer is a comfy weight so that it picks
up dynamic_vram and async_offload functionality in full.
Co-authored-by: Rattus <rattus128@gmail.com>
* mp: fix full dynamic unloading
This was not unloading dynamic models when requesting a full unload via
the unpatch() code path.
This was ok, i your workflow was all dynamic models but fails with big
VRAM leaks if you need to fully unload something for a regular ModelPatcher
It also fices the "unload models" button.
* mm: load models outside of Aimdo Mempool
In dynamic_vram mode, escape the Aimdo mempool and load into the regular
mempool. Use a dummy thread to do it.
Tashdid Khan
comfyanonymous
ComfyUI Wiki
chaObserv
Jukka Seppänen
tdrussell
asagi4
AustinMroz
Comfy Org PR Bot
comfyanonymous
blepping
Alexander Piskun
rattus