* model_management: disable non-dynamic smart memory
Disable smart memory outright for non dynamic models.
This is a minor step towards deprecation of --disable-dynamic-vram
and the legacy ModelPatcher.
This is needed for estimate-free model development, where new models
can opt-out of supplying a memory estimate and not have to worry
about hard VRAM allocations due to legacy non-dynamic model patchers
This is also a general stability increase for a lot of stray use cases
where estimates may still be off and going forward we are not going
to accurately maintain such estimates.
* pinned_memory: implement with aimdo growable buffer
Use a single growable buffer so we can do threaded pre-warming on
pinned memory.
* mm: use aimdo to do transfer from disk to pin
Aimdo implements a faster threaded loader.
* Add stream host pin buffer for AIMDO casts
Introduce per-offload-stream HostBuffer reuse for pinned staging,
include it in cast buffer reset synchronization.
Defer actual casts that go via this pin path to a separate pass
such that the buffer can be allocated monolithically (to avoid
cudaHostRegister thrash).
* remove old pin path
* Implement JIT pinned memory pressure
Replace the predictive pin pressure mechanism with JIT PIN memory
pressure.
* LowVRAMPatch: change to two-phase visit
* lora: re-implement as inplace swiss-army-knife operation
* prepare for multiple pin sets
* implement pinned loras
* requirements: comfy-aimdo 0.4.0
* ops: remove unused arg
This was defeatured in aimdo iteration
* ops: sync the CPU with only the offload stream activity
This was syncing with the offload stream which itself is synced with the
compute stream, so this was syncing CPU with compute transitively. Define
the event to sync it more gently.
* pins: implement freeing intermediate for pinned memory
Pinning is more important than inactive intermediates and the stream
pin buffer is more important than even active intermediates.
* execution: implement pin eviction on RAM presure
Add back proper pin freeing on RAM pressure
* implement pin registration swaps
Uncap the windows pins from 50% by extending the pool and have a pressure
mechanism to move the pin reservations om demand.
This unfortunately implies a GPU sync to do the freeing so significant
hysterisis needs to be added to consolidate these pressure events.
* cli_args/execution: Implement lower background cache-ram threshold
Limit the amount of RAM background intermediates can use, so that
switching workflows doesn't degrade performance too much.
* make default
* bump aimdo
* model-patcher: force-cast tiny weights
Flux 2 gets crazy stalls due to a mix of tiny and giant weights
creating lopsided steam buffer rotations which creates stalls.
* ops: refactor in prep for chunking
* mm: delegate pin-on-the-way to aimdo
Aimdo is able to chunk and slice this on the way for better CPU->GPU
overlap. The main advantage is the ability to shorten the bus contention
window between previous weight transfer and the next weights vbar
fault.
* bump aimdo
* pinning updates
* specify hostbuf max allocation size
There a signs of virtual memory exhaustion on some linux systems when
throwing 128GB for every little piece. Pass the actual to save aimdo
from over-estimates
* tests: update execution tests for caching
The default caching changed to ram-cache so update these tests
accordingly.
Remove the LRU 0 test as this also falls through to RAM cache.
* chore: update workflow templates to v0.9.69
* Update comfyui-workflow-templates to version 0.9.70
* Downgrade comfyui-workflow-templates to 0.9.69
---------
Co-authored-by: Alexander Piskun <13381981+bigcat88@users.noreply.github.com>
Comfy-aimdo 0.3.0 contains several major new features.
multi-GPU support
ARM support
AMD support
Refactorings include:
Linkless architecture - linkage is now performed purely at runtime
to stop host library lookups completely and only interact with the
torch-loaded Nvidia stack.
Elimination of cudart integration on linux. Its no consistent with
windows.
Misc bugfixes and minor features.
This was doing an over-estimate of VRAM used by the async allocator when lots
of little small tensors were in play.
Also change the versioning scheme to == so we can roll forward aimdo without
worrying about stable regressions downstream in comfyUI core.
* Implement seek and read for pins
Source pins from an mmap is pad because its its a CPU->CPU copy that
attempts to fully buffer the same data twice. Instead, use seek and
read which avoids the mmap buffering while usually being a faster
read in the first place (avoiding mmap faulting etc).
* pinned_memory: Use Aimdo pinner
The aimdo pinner bypasses pytorches CPU allocator which can leak
windows commit charge.
* ops: bypass init() of weight for embedding layer
This similarly consumes large commit charge especially for TEs. It can
cause a permanement leaked commit charge which can destabilize on
systems close to the commit ceiling and generally confuses the RAM
stats.
* model_patcher: implement pinned memory counter
Implement a pinned memory counter for better accounting of what volume
of memory pins have.
* implement touch accounting
Implement accounting of touching mmapped tensors.
* mm+mp: add residency mmap getter
* utils: use the aimdo mmap to load sft files
* model_management: Implement tigher RAM pressure semantics
Implement a pressure release on entire MMAPs as windows does perform
faster when mmaps are unloaded and model loads free ramp into fully
unallocated RAM.
Make the concept of freeing for pins a completely separate concept.
Now that pins are loadable directly from original file and don' touch
the mmap, tighten the freeing budget to just the current loaded model
- what you have left over. This still over-frees pins, but its a lot
better than before.
So after the pins are freed with that algorithm, bounce entire MMAPs
to free RAM based on what the model needs, deducting off any known
resident-in-mmap tensors to the free quota to keep it as tight as
possible.
* comfy-aimdo 0.2.11
Comfy aimdo 0.2.11
* mm: Implement file_slice path for QT
* ruff
* ops: put meta-tensors in place to allow custom nodes to check geo
Comfy Aimdo 0.2.10 fixes the aimdo allocator hook for legacy cudaMalloc
consumers. Some consumers of cudaMalloc assume implicit synchronization
built in closed source logic inside cuda. This is preserved by passing
through to cuda as-is and accouting after the fact as opposed to
integrating these hooks with Aimdos VMA based allocator.
Comfy-aimdo 0.2.9 fixes a context issue where if a non-main thread does
a spurious garbage collection, cudaFrees are attempted with bad
context.
Some new APIs for displaying aimdo stats in UI widgets are also added.
These are purely additive getters that dont touch cuda APIs.
* feat: add EagerEval dataclass for frontend-side node evaluation
Add EagerEval to the V3 API schema, enabling nodes to declare
frontend-evaluated JSONata expressions. The frontend uses this to
display computation results as badges without a backend round-trip.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Math Expression node with JSONata evaluation
Add ComfyMathExpression node that evaluates JSONata expressions against
dynamically-grown numeric inputs using Autogrow + MatchType. Sends
input context via ui output so the frontend can re-evaluate when
the expression changes without a backend round-trip.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: register nodes_math.py in extras_files loader list
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address CodeRabbit review feedback
- Harden EagerEval.validate with type checks and strip() for empty strings
- Add _positional_alias for spreadsheet-style names beyond z (aa, ab...)
- Validate JSONata result is numeric before returning
- Add jsonata to requirements.txt
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: remove EagerEval, scope PR to math node only
Remove EagerEval dataclass from _io.py and eager_eval usage from
nodes_math.py. Eager execution will be designed as a general-purpose
system in a separate effort.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use TemplateNames, cap inputs at 26, improve error message
Address Kosinkadink review feedback:
- Switch from Autogrow.TemplatePrefix to Autogrow.TemplateNames so input
slots are named a-z, matching expression variables directly
- Cap max inputs at 26 (a-z) instead of 100
- Simplify execute() by removing dual-mapping hack
- Include expression and result value in error message
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add unit tests for Math Expression node
Add tests for _positional_alias (a-z mapping) and execute() covering
arithmetic operations, float inputs, $sum(values), and error cases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: replace jsonata with simpleeval for math evaluation
jsonata PyPI package has critical issues: no Python 3.12/3.13 wheels,
no ARM/Apple Silicon wheels, abandoned (last commit 2023), C extension.
Replace with simpleeval (pure Python, 3.4M downloads/month, MIT,
AST-based security). Add math module functions (sqrt, ceil, floor,
log, sin, cos, tan) and variadic sum() supporting both sum(values)
and sum(a, b, c). Pin version to >=1.0,<2.0.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: update tests for simpleeval migration
Update JSONata syntax to Python syntax ($sum -> sum, $string -> str),
add tests for math functions (sqrt, ceil, floor, sin, log10) and
variadic sum(a, b, c).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: replace MatchType with MultiType inputs and dual FLOAT/INT outputs
Allow mixing INT and FLOAT connections on the same node by switching
from MatchType (which forces all inputs to the same type) to MultiType.
Output both FLOAT and INT so users can pick the type they need.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: update tests for mixed INT/FLOAT inputs and dual outputs
Add assertions for both FLOAT (result[0]) and INT (result[1]) outputs.
Add test_mixed_int_float_inputs and test_mixed_resolution_scale to
verify the primary use case of multiplying resolutions by a float factor.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: make expression input multiline and validate empty expression
- Add multiline=True to expression input for better UX with longer expressions
- Add empty expression validation with clear "Expression cannot be empty." message
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add tests for empty expression validation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review feedback — safe pow, isfinite guard, test coverage
- Wrap pow() with _safe_pow to prevent DoS via huge exponents
(pow() bypasses simpleeval's safe_power guard on **)
- Add math.isfinite() check to catch inf/nan before int() conversion
- Add int/float converters to MATH_FUNCTIONS for explicit casting
- Add "calculator" search alias
- Replace _positional_alias helper with string.ascii_lowercase
- Narrow test assertions and add error path + function coverage tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update requirements.txt
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Jedrzej Kosinski <kosinkadink1@gmail.com>
Co-authored-by: Christian Byrne <abolkonsky.rem@gmail.com>
Comfy-aimdo 0.2.7 fixes a crash when a spurious cudaAsyncFree comes in
and would cause an infinite stack overflow (via detours hooks).
A lock is also introduced on the link list holding the free sections
to avoid any possibility of threaded miscellaneous cuda allocations
being the root cause.
* ops: dont unpin nothing
This was calling into aimdo in the none case (offloaded weight). Whats worse,
is aimdo syncs for unpinning an offloaded weight, as that is the corner case of
a weight getting evicted by its own use which does require a sync. But this
was heppening every offloaded weight causing slowdown.
* mp: fix get_free_memory policy
The ModelPatcherDynamic get_free_memory was deducting the model from
to try and estimate the conceptual free memory with doing any
offloading. This is kind of what the old memory_memory_required
was estimating in ModelPatcher load logic, however in practical
reality, between over-estimates and padding, the loader usually
underloaded models enough such that sampling could send CFG +/-
through together even when partially loaded.
So don't regress from the status quo and instead go all in on the
idea that offloading is less of an issue than debatching. Tell the
sampler it can use everything.
Comfy Aimdo 0.2.4 fixes a VRAM buffer alignment issue that happens in
someworkflows where action is able to bypass the pytorch allocator
and go straight to the cuda hook.
Comfy Aimdo 0.2.2 moves the cuda allocator hook from the cudart API to
the cuda driver API on windows. This is needed to handle Windows+cu13
where cudart is statically linked.
Integrate comfy-aimdo 0.2 which takes a different approach to
installing the memory allocator hook. Instead of using the complicated
and buggy pytorch MemPool+CudaPluggableAlloctor, cuda is directly hooked
making the process much more transparent to both comfy and pytorch. As
far as pytorch knows, aimdo doesnt exist anymore, and just operates
behind the scenes.
Remove all the mempool setup stuff for dynamic_vram and bump the
comfy-aimdo version. Remove the allocator object from memory_management
and demote its use as an enablment check to a boolean flag.
Comfy-aimdo 0.2 also support the pytorch cuda async allocator, so
remove the dynamic_vram based force disablement of cuda_malloc and
just go back to the old settings of allocators based on command line
input.
* revert threaded model loader change
This change was only needed to get around the pytorch 2.7 mempool bugs,
and should have been reverted along with #12260. This fixes a different
memory leak where pytorch gets confused about cache emptying.
* load non comfy weights
* MPDynamic: Pre-generate the tensors for vbars
Apparently this is an expensive operation that slows down things.
* bump to aimdo 1.8
New features:
watermark limit feature
logging enhancements
-O2 build on linux
Daxiong (Lin)
rattus
Comfy Org PR Bot
Jedrzej Kosinski
Octopus
comfyanonymous
Alexander Brown
Christian Byrne
Luke Mino-Altherr
Dante
pythongosssss