68f0d35296
Add support for VIDEO as a built-in type ( #7844 )
...
* Add basic support for videos as types
This PR adds support for VIDEO as first-class types. In order to avoid
unnecessary costs, VIDEO outputs must implement the `VideoInput` ABC,
but their implementation details can vary. Included are two
implementations of this type which can be returned by other nodes:
* `VideoFromFile` - Created with either a path on disk (as a string) or
a `io.BytesIO` containing the contents of a file in a supported format
(like .mp4). This implementation won't actually load the video unless
necessary. It will also avoid re-encoding when saving if possible.
* `VideoFromComponents` - Created from an image tensor and an optional
audio tensor.
Currently, only h264 encoded videos in .mp4 containers are supported for
saving, but the plan is to add additional encodings/containers in the
near future (particularly .webm).
* Add optimization to avoid parsing entire video
* Improve type declarations to reduce warnings
* Make sure bytesIO objects can be read many times
* Fix a potential issue when saving long videos
* Fix incorrect type annotation
* Add a `LoadVideo` node to make testing easier
* Refactor new types out of the base comfy folder
I've created a new `comfy_api` top-level module. The intention is that
anything within this folder would be covered by semver-style versioning
that would allow custom nodes to rely on them not introducing breaking
changes.
* Fix linting issue
2025-04-29 05:58:00 -04:00
83d04717b6
Support HiDream E1 model. ( #7857 )
2025-04-28 15:01:15 -04:00
c15909bb62
CFG++ for gradient estimation sampler ( #7809 )
2025-04-28 13:51:35 -04:00
5a50c3c7e5
Fix stream priority to support older pytorch. ( #7856 )
2025-04-28 13:07:21 -04:00
30159a7fe6
Save v pred zsnr metadata ( #7840 )
2025-04-28 13:03:21 -04:00
c8cd7ad795
Use stream for casting if enabled. ( #7833 )
2025-04-27 05:38:11 -04:00
ac10a0d69e
Make loras work with --async-offload ( #7824 )
2025-04-26 19:56:22 -04:00
0dcc75ca54
Add experimental --async-offload lowvram weight offloading. ( #7820 )
...
This should speed up the lowvram mode a bit. It currently is only enabled when --async-offload is used but it will be enabled by default in the future if there are no problems.
2025-04-26 16:11:21 -04:00
23e39f2ba7
Add a T5TokenizerOptions node to set options for the T5 tokenizer. ( #7803 )
2025-04-25 19:36:00 -04:00
78992c4b25
[NodeDef] Add documentation on widgetType ( #7768 )
...
* [NodeDef] Add documentation on widgetType
* Document required version for widgetType
2025-04-25 13:35:07 -04:00
f935d42d8e
Support SimpleTuner lycoris lora format for HiDream.
2025-04-25 03:11:14 -04:00
e2eed9eb9b
throw away alpha channel in clip vision preprocessor ( #7769 )
...
saves users having to explicitly discard the channel
2025-04-23 21:28:36 -04:00
552615235d
Fix for dino lowvram. ( #7748 )
2025-04-23 04:12:52 -04:00
0738e4ea5d
[API nodes] Add backbone for supporting api nodes in ComfyUI ( #7745 )
...
* Add Ideogram generate node.
* Add staging api.
* COMFY_API_NODE_NAME node property
* switch to boolean flag and use original node name for id
* add optional to type
* Add API_NODE and common error for missing auth token (#5 )
* Add Minimax Video Generation + Async Task queue polling example (#6 )
* [Minimax] Show video preview and embed workflow in ouput (#7 )
* [API Nodes] Send empty request body instead of empty dictionary. (#8 )
* Fixed: removed function from rebase.
* Add pydantic.
* Remove uv.lock
* Remove polling operations.
* Update stubs workflow.
* Remove polling comments.
* Update stubs.
* Use pydantic v2.
* Use pydantic v2.
* Add basic OpenAITextToImage node
* Add.
* convert image to tensor.
* Improve types.
* Ruff.
* Push tests.
* Handle multi-form data.
- Don't set content-type for multi-part/form
- Use data field instead of JSON
* Change to api.comfy.org
* Handle error code 409.
* Remove nodes.
---------
Co-authored-by: bymyself <cbyrne@comfy.org>
Co-authored-by: Yoland Y <4950057+yoland68@users.noreply.github.com>
2025-04-23 02:18:08 -04:00
2d6805ce57
Add option for using fp8_e8m0fnu for model weights. ( #7733 )
...
Seems to break every model I have tried but worth testing?
2025-04-22 06:17:38 -04:00
a8f63c0d5b
Support dora_scale on both axis ( #7727 )
2025-04-22 05:01:27 -04:00
966c43ce26
Add OFT/BOFT algorithm in weight adapter ( #7725 )
2025-04-22 04:59:47 -04:00
3ab231f01f
Fix issue with WAN VACE implementation. ( #7724 )
2025-04-21 23:36:12 -04:00
1f3fba2af5
Unified Weight Adapter system for better maintainability and future feature of Lora system ( #7540 )
2025-04-21 20:15:32 -04:00
5d0d4ee98a
Add strength control for vace. ( #7717 )
2025-04-21 19:36:20 -04:00
5d51794607
Add node type hint for socketless option ( #7714 )
...
* Add node type hint for socketless option
* nit - Doc
2025-04-21 16:13:00 -04:00
ce22f687cc
Support for WAN VACE preview model. ( #7711 )
...
* Support for WAN VACE preview model.
* Remove print.
2025-04-21 14:40:29 -04:00
2c735c13b4
Slightly better fix for #7687
2025-04-20 11:33:27 -04:00
fd27494441
Use empty t5 of size 128 for hidream, seems to give closer results.
2025-04-19 19:49:40 -04:00
f43e1d7f41
Hidream: Allow loading hidream text encoders in CLIPLoader and DualCLIPLoader ( #7676 )
...
* Hidream: Allow partial loading text encoders
* reformat code for ruff check.
2025-04-19 19:47:30 -04:00
636d4bfb89
Fix hard crash when the spiece tokenizer path is bad.
2025-04-19 15:55:43 -04:00
dbcfd092a2
Set default context_img_len to 257
2025-04-17 12:42:34 -04:00
c14429940f
Support loading WAN FLF model.
2025-04-17 12:04:48 -04:00
0d720e4367
Don't hardcode length of context_img in wan code.
2025-04-17 06:25:39 -04:00
1fc00ba4b6
Make hidream work with any latent resolution.
2025-04-16 18:34:14 -04:00
9899d187b1
Limit T5 to 128 tokens for HiDream: #7620
2025-04-16 18:07:55 -04:00
f00f340a56
Reuse code from flux model.
2025-04-16 17:43:55 -04:00
cce1d9145e
[Type] Mark input options NotRequired ( #7614 )
2025-04-16 15:41:00 -04:00
b4dc03ad76
Fix issue on old torch.
2025-04-16 04:53:56 -04:00
9ad792f927
Basic support for hidream i1 model.
2025-04-15 17:35:05 -04:00
6fc5dbd52a
Cleanup.
2025-04-15 12:13:28 -04:00
3e8155f7a3
More flexible long clip support.
...
Add clip g long clip support.
Text encoder refactor.
Support llama models with different vocab sizes.
2025-04-15 10:32:21 -04:00
8a438115fb
add RMSNorm to comfy.ops
2025-04-14 18:00:33 -04:00
e51d9ba5fc
Add SEEDS (stage 2 & 3 DP) sampler ( #7580 )
...
* Add seeds stage 2 & 3 (DP) sampler
* Change the name to SEEDS in comment
2025-04-12 18:36:08 -04:00
1714a4c158
Add CublasOps support ( #7574 )
...
* CublasOps support
* Guard CublasOps behind --fast arg
2025-04-12 18:29:15 -04:00
ed945a1790
Dependency Aware Node Caching for low RAM/VRAM machines ( #7509 )
...
* add dependency aware cache that removed a cached node as soon as all of its decendents have executed. This allows users with lower RAM to run workflows they would otherwise not be able to run. The downside is that every workflow will fully run each time even if no nodes have changed.
* remove test code
* tidy code
2025-04-11 06:55:51 -04:00
98bdca4cb2
Deprecate InputTypeOptions.defaultInput ( #7551 )
...
* Deprecate InputTypeOptions.defaultInput
* nit
* nit
2025-04-10 06:57:06 -04:00
e346d8584e
Add prepare_sampling wrapper allowing custom nodes to more accurately report noise_shape ( #7500 )
2025-04-09 09:43:35 -04:00
70d7242e57
Support the wan fun reward loras.
2025-04-07 05:01:47 -04:00
3bfe4e5276
Support 512 siglip model.
2025-04-05 07:01:01 -04:00
89e4ea0175
Add activations_shape info in UNet models ( #7482 )
...
* Add activations_shape info in UNet models
* activations_shape should be a list
2025-04-04 21:27:54 -04:00
3a100b9a55
Disable partial offloading of audio VAE.
2025-04-04 21:24:56 -04:00
2222cf67fd
MLU memory optimization ( #7470 )
...
Co-authored-by: huzhan <huzhan@cambricon.com>
2025-04-02 19:24:04 -04:00
301e26b131
Add option to store TE in bf16 ( #7461 )
2025-04-01 13:48:53 -04:00
a3100c8452
Remove useless code.
2025-03-29 20:12:56 -04:00
2d17d8910c
Don't error if wan concat image has extra channels.
2025-03-28 08:49:29 -04:00
0a1f8869c9
Add WanFunInpaintToVideo node for the Wan fun inpaint models.
2025-03-27 11:13:27 -04:00
3661c833bc
Support the WAN 2.1 fun control models.
...
Use the new WanFunControlToVideo node.
2025-03-26 19:54:54 -04:00
8edc1f44c1
Support more float8 types.
2025-03-25 05:23:49 -04:00
e471c726e5
Fallback to pytorch attention if sage attention fails.
2025-03-22 15:45:56 -04:00
d9fa9d307f
Automatically set the right sampling type for lotus.
2025-03-21 14:19:37 -04:00
83e839a89b
Native LotusD Implementation ( #7125 )
...
* draft pass at a native comfy implementation of Lotus-D depth and normal est
* fix model_sampling kludges
* fix ruff
---------
Co-authored-by: comfyanonymous <121283862+comfyanonymous@users.noreply.github.com>
2025-03-21 14:04:15 -04:00
3872b43d4b
A few fixes for the hunyuan3d models.
2025-03-20 04:52:31 -04:00
32ca0805b7
Fix orientation of hunyuan 3d model.
2025-03-19 19:55:24 -04:00
11f1b41bab
Initial Hunyuan3Dv2 implementation.
...
Supports the multiview, mini, turbo models and VAEs.
2025-03-19 16:52:58 -04:00
3b19fc76e3
Allow disabling pe in flux code for some other models.
2025-03-18 05:09:25 -04:00
50614f1b79
Fix regression with clip vision.
2025-03-17 13:56:11 -04:00
6dc7b0bfe3
Add support for giant dinov2 image encoder.
2025-03-17 05:53:54 -04:00
e8e990d6b8
Cleanup code.
2025-03-16 06:29:12 -04:00
2e24a15905
Call unpatch_hooks at the start of ModelPatcher.partially_unload ( #7253 )
...
* Call unpatch_hooks at the start of ModelPatcher.partially_unload
* Only call unpatch_hooks in partially_unload if lowvram is possible
2025-03-16 06:02:45 -04:00
fd5297131f
Guard the edge cases of noise term in er_sde ( #7265 )
2025-03-16 06:02:25 -04:00
55a1b09ddc
Allow loading diffusion model files with the "Load Checkpoint" node.
2025-03-15 08:27:49 -04:00
3c3988df45
Show a better error message if the VAE is invalid.
2025-03-15 08:26:36 -04:00
a2448fc527
Remove useless code.
2025-03-14 18:10:37 -04:00
6a0daa79b6
Make the SkipLayerGuidanceDIT node work on WAN.
2025-03-14 10:55:19 -04:00
9c98c6358b
Tolerate missing `@torch.library.custom_op` ( #7234 )
...
This can happen on Pytorch versions older than 2.4.
2025-03-14 09:51:26 -04:00
7aceb9f91c
Add --use-flash-attention flag. ( #7223 )
...
* Add --use-flash-attention flag.
This is useful on AMD systems, as FA builds are still 10% faster than Pytorch cross-attention.
2025-03-14 03:22:41 -04:00
35504e2f93
Fix.
2025-03-13 15:03:18 -04:00
299436cfed
Print mac version.
2025-03-13 10:05:40 -04:00
9b6cd9b874
[NodeDef] Add documentation on multi_select input option ( #7212 )
2025-03-12 17:29:39 -04:00
3fc688aebd
Ensure the extra_args in dpmpp sde series ( #7204 )
2025-03-12 17:28:59 -04:00
01015bff16
Add er_sde sampler ( #7187 )
2025-03-12 02:42:37 -04:00
ca8efab79f
Support control loras on Wan.
2025-03-10 17:23:13 -04:00
9aac21f894
Fix issues with new hunyuan img2vid model and bumb version to v0.3.26
2025-03-09 05:07:22 -04:00
528d1b3563
When cached_hook_patches contain weights for hooks, only use hook_backup for unused keys ( #7067 )
2025-03-09 04:26:31 -04:00
7395b0c0d1
Support new hunyuan video i2v model.
...
Use the new "v2 (replace)" guidance type in HunyuanImageToVideo and set
image_interleave to 4 on the "Text Encode Hunyuan Video" node.
2025-03-08 20:34:47 -05:00
0952569493
Fix stable cascade VAE on some lowvram machines.
2025-03-08 20:24:04 -05:00
be4e760648
Add an image_interleave option to the Hunyuan image to video encode node.
...
See the tooltip for what it does.
2025-03-07 19:56:26 -05:00
11b1f27cb1
Set WAN default compute dtype to fp16.
2025-03-07 04:52:36 -05:00
70e15fd743
No need for scale_input when fp8 matrix mult is disabled.
2025-03-07 04:49:20 -05:00
e1474150de
Support fp8_scaled diffusion models that don't use fp8 matrix mult.
2025-03-07 04:39:21 -05:00
e62d72e8ca
Typo in node_typing.py ( #7092 )
2025-03-06 15:24:04 -05:00
dfa36e6855
Fix some things breaking when embeddings fail to apply.
2025-03-06 13:31:55 -05:00
29a70ca101
Support HunyuanVideo image to video model.
2025-03-06 03:07:15 -05:00
0bef826a98
Support llava clip vision model.
2025-03-06 00:24:43 -05:00
85ef295069
Make applying embeddings more efficient.
...
Adding new tokens no longer makes a whole copy of the embeddings weight
which can be massive on certain models.
2025-03-05 17:34:38 -05:00
5d84607bf3
Add type hint for FileLocator ( #6968 )
...
* Add type hint for FileLocator
* nit
2025-03-05 15:35:26 -05:00
c1909f350f
Better argument handling of front-end-root ( #7043 )
...
* Better argument handling of front-end-root
Improves handling of front-end-root launch argument. Several instances where users have set it and ComfyUI launches as normal and completely disregards the launch arg which doesn't make sense. Better to indicate to user that something is incorrect.
* Removed unused import
There was no real reason to use "Optional" typing in ther front-end-root argument.
2025-03-05 15:34:22 -05:00
52b3469606
[NodeDef] Explicitly add control_after_generate to seed/noise_seed ( #7059 )
...
* [NodeDef] Explicitly add control_after_generate to seed/noise_seed
* Update comfy/comfy_types/node_typing.py
Co-authored-by: filtered <176114999+webfiltered@users.noreply.github.com>
---------
Co-authored-by: filtered <176114999+webfiltered@users.noreply.github.com>
2025-03-05 15:33:23 -05:00
369b079ff6
Fix lowvram issue with ltxv vae.
2025-03-05 05:26:08 -05:00
9c9a7f012a
Adjust ltxv memory factor.
2025-03-05 05:16:05 -05:00
93fedd92fe
Support LTXV 0.9.5.
...
Credits: Lightricks team.
2025-03-05 00:13:49 -05:00
65042f7d39
Make it easier to set a custom template for hunyuan video.
2025-03-04 09:26:05 -05:00
7c7c70c400
Refactor skyreels i2v code.
2025-03-04 00:15:45 -05:00
f86c724ef2
Temporal area composition.
...
New ConditioningSetAreaPercentageVideo node.
2025-03-03 06:50:31 -05:00
guill
comfyanonymous
chaObserv
Pam
AustinMroz
comfyanonymous
thot experiment
Robin Huang
Kohaku-Blueleaf
filtered
power88
Chenlei Hu
catboxanon
Chargeuk
Jedrzej Kosinski
Raphael Walker
BiologicalExplosion
BVH
FeepingCreature
JettHu
Chenlei Hu
Silver