2d17d8910c
Don't error if wan concat image has extra channels.
2025-03-28 08:49:29 -04:00
0a1f8869c9
Add WanFunInpaintToVideo node for the Wan fun inpaint models.
2025-03-27 11:13:27 -04:00
3661c833bc
Support the WAN 2.1 fun control models.
...
Use the new WanFunControlToVideo node.
2025-03-26 19:54:54 -04:00
8edc1f44c1
Support more float8 types.
2025-03-25 05:23:49 -04:00
e471c726e5
Fallback to pytorch attention if sage attention fails.
2025-03-22 15:45:56 -04:00
d9fa9d307f
Automatically set the right sampling type for lotus.
2025-03-21 14:19:37 -04:00
83e839a89b
Native LotusD Implementation ( #7125 )
...
* draft pass at a native comfy implementation of Lotus-D depth and normal est
* fix model_sampling kludges
* fix ruff
---------
Co-authored-by: comfyanonymous <121283862+comfyanonymous@users.noreply.github.com>
2025-03-21 14:04:15 -04:00
3872b43d4b
A few fixes for the hunyuan3d models.
2025-03-20 04:52:31 -04:00
32ca0805b7
Fix orientation of hunyuan 3d model.
2025-03-19 19:55:24 -04:00
11f1b41bab
Initial Hunyuan3Dv2 implementation.
...
Supports the multiview, mini, turbo models and VAEs.
2025-03-19 16:52:58 -04:00
3b19fc76e3
Allow disabling pe in flux code for some other models.
2025-03-18 05:09:25 -04:00
50614f1b79
Fix regression with clip vision.
2025-03-17 13:56:11 -04:00
6dc7b0bfe3
Add support for giant dinov2 image encoder.
2025-03-17 05:53:54 -04:00
e8e990d6b8
Cleanup code.
2025-03-16 06:29:12 -04:00
2e24a15905
Call unpatch_hooks at the start of ModelPatcher.partially_unload ( #7253 )
...
* Call unpatch_hooks at the start of ModelPatcher.partially_unload
* Only call unpatch_hooks in partially_unload if lowvram is possible
2025-03-16 06:02:45 -04:00
fd5297131f
Guard the edge cases of noise term in er_sde ( #7265 )
2025-03-16 06:02:25 -04:00
55a1b09ddc
Allow loading diffusion model files with the "Load Checkpoint" node.
2025-03-15 08:27:49 -04:00
3c3988df45
Show a better error message if the VAE is invalid.
2025-03-15 08:26:36 -04:00
a2448fc527
Remove useless code.
2025-03-14 18:10:37 -04:00
6a0daa79b6
Make the SkipLayerGuidanceDIT node work on WAN.
2025-03-14 10:55:19 -04:00
9c98c6358b
Tolerate missing `@torch.library.custom_op` ( #7234 )
...
This can happen on Pytorch versions older than 2.4.
2025-03-14 09:51:26 -04:00
7aceb9f91c
Add --use-flash-attention flag. ( #7223 )
...
* Add --use-flash-attention flag.
This is useful on AMD systems, as FA builds are still 10% faster than Pytorch cross-attention.
2025-03-14 03:22:41 -04:00
35504e2f93
Fix.
2025-03-13 15:03:18 -04:00
299436cfed
Print mac version.
2025-03-13 10:05:40 -04:00
9b6cd9b874
[NodeDef] Add documentation on multi_select input option ( #7212 )
2025-03-12 17:29:39 -04:00
3fc688aebd
Ensure the extra_args in dpmpp sde series ( #7204 )
2025-03-12 17:28:59 -04:00
01015bff16
Add er_sde sampler ( #7187 )
2025-03-12 02:42:37 -04:00
ca8efab79f
Support control loras on Wan.
2025-03-10 17:23:13 -04:00
9aac21f894
Fix issues with new hunyuan img2vid model and bumb version to v0.3.26
2025-03-09 05:07:22 -04:00
528d1b3563
When cached_hook_patches contain weights for hooks, only use hook_backup for unused keys ( #7067 )
2025-03-09 04:26:31 -04:00
7395b0c0d1
Support new hunyuan video i2v model.
...
Use the new "v2 (replace)" guidance type in HunyuanImageToVideo and set
image_interleave to 4 on the "Text Encode Hunyuan Video" node.
2025-03-08 20:34:47 -05:00
0952569493
Fix stable cascade VAE on some lowvram machines.
2025-03-08 20:24:04 -05:00
be4e760648
Add an image_interleave option to the Hunyuan image to video encode node.
...
See the tooltip for what it does.
2025-03-07 19:56:26 -05:00
11b1f27cb1
Set WAN default compute dtype to fp16.
2025-03-07 04:52:36 -05:00
70e15fd743
No need for scale_input when fp8 matrix mult is disabled.
2025-03-07 04:49:20 -05:00
e1474150de
Support fp8_scaled diffusion models that don't use fp8 matrix mult.
2025-03-07 04:39:21 -05:00
e62d72e8ca
Typo in node_typing.py ( #7092 )
2025-03-06 15:24:04 -05:00
dfa36e6855
Fix some things breaking when embeddings fail to apply.
2025-03-06 13:31:55 -05:00
29a70ca101
Support HunyuanVideo image to video model.
2025-03-06 03:07:15 -05:00
0bef826a98
Support llava clip vision model.
2025-03-06 00:24:43 -05:00
85ef295069
Make applying embeddings more efficient.
...
Adding new tokens no longer makes a whole copy of the embeddings weight
which can be massive on certain models.
2025-03-05 17:34:38 -05:00
5d84607bf3
Add type hint for FileLocator ( #6968 )
...
* Add type hint for FileLocator
* nit
2025-03-05 15:35:26 -05:00
c1909f350f
Better argument handling of front-end-root ( #7043 )
...
* Better argument handling of front-end-root
Improves handling of front-end-root launch argument. Several instances where users have set it and ComfyUI launches as normal and completely disregards the launch arg which doesn't make sense. Better to indicate to user that something is incorrect.
* Removed unused import
There was no real reason to use "Optional" typing in ther front-end-root argument.
2025-03-05 15:34:22 -05:00
52b3469606
[NodeDef] Explicitly add control_after_generate to seed/noise_seed ( #7059 )
...
* [NodeDef] Explicitly add control_after_generate to seed/noise_seed
* Update comfy/comfy_types/node_typing.py
Co-authored-by: filtered <176114999+webfiltered@users.noreply.github.com>
---------
Co-authored-by: filtered <176114999+webfiltered@users.noreply.github.com>
2025-03-05 15:33:23 -05:00
369b079ff6
Fix lowvram issue with ltxv vae.
2025-03-05 05:26:08 -05:00
9c9a7f012a
Adjust ltxv memory factor.
2025-03-05 05:16:05 -05:00
93fedd92fe
Support LTXV 0.9.5.
...
Credits: Lightricks team.
2025-03-05 00:13:49 -05:00
65042f7d39
Make it easier to set a custom template for hunyuan video.
2025-03-04 09:26:05 -05:00
7c7c70c400
Refactor skyreels i2v code.
2025-03-04 00:15:45 -05:00
f86c724ef2
Temporal area composition.
...
New ConditioningSetAreaPercentageVideo node.
2025-03-03 06:50:31 -05:00
9af6320ec9
Make 2d area composition nodes work on video models.
2025-03-02 08:19:16 -05:00
4dc6709307
Rename argument in last commit and document the options.
2025-03-01 02:43:49 -05:00
4d55f16ae8
Use enum list for --fast options ( #7024 )
2025-03-01 02:37:35 -05:00
cf0b549d48
--fast now takes a number as argument to indicate how fast you want it.
...
The idea is that you can indicate how much quality vs speed you want.
At the moment:
--fast 2 enables fp16 accumulation if your pytorch supports it.
--fast 5 enables fp8 matrix mult on fp8 models and the optimization above.
--fast without a number enables all optimizations.
2025-02-28 02:48:20 -05:00
eb4543474b
Use fp16 for intermediate for fp8 weights with --fast if supported.
2025-02-28 02:17:50 -05:00
1804397952
Use fp16 if checkpoint weights are fp16 and the model supports it.
2025-02-27 16:39:57 -05:00
f4dac8ab6f
Wan code small cleanup.
2025-02-27 07:22:42 -05:00
89253e9fe5
Support Cambricon MLU ( #6964 )
...
Co-authored-by: huzhan <huzhan@cambricon.com>
2025-02-26 20:45:13 -05:00
3ea3bc8546
Fix wan issues when prompt length is long.
2025-02-26 20:34:02 -05:00
0270a0b41c
Reduce artifacts on Wan by doing the patch embedding in fp32.
2025-02-26 16:59:26 -05:00
c37f15f98e
Add fast preview support for Wan models.
2025-02-26 08:56:23 -05:00
4bca7367f3
Don't try to use clip_fea on t2v model.
2025-02-26 08:38:09 -05:00
b6fefe686b
Better wan memory estimation.
2025-02-26 07:51:22 -05:00
fa62287f1f
More code reuse in wan.
...
Fix bug when changing the compute dtype on wan.
2025-02-26 05:22:29 -05:00
0844998db3
Slightly better wan i2v mask implementation.
2025-02-26 03:49:50 -05:00
4ced06b879
WIP support for Wan I2V model.
2025-02-26 01:49:43 -05:00
cb06e9669b
Wan seems to work with fp16.
2025-02-25 21:37:12 -05:00
9a66bb972d
Make wan work with all latent resolutions.
...
Cleanup some code.
2025-02-25 19:56:04 -05:00
ea0f939df3
Fix issue with wan and other attention implementations.
2025-02-25 19:13:39 -05:00
f37551c1d2
Change wan rope implementation to the flux one.
...
Should be more compatible.
2025-02-25 19:11:14 -05:00
63023011b9
WIP support for Wan t2v model.
2025-02-25 17:20:35 -05:00
f40076096e
Cleanup some lumina te code.
2025-02-25 04:10:26 -05:00
96d891cb94
Speedup on some models by not upcasting bfloat16 to float32 on mac.
2025-02-24 05:41:32 -05:00
ace899e71a
Prioritize fp16 compute when using allow_fp16_accumulation
2025-02-23 04:45:54 -05:00
aff16532d4
Remove some useless code.
2025-02-22 04:45:14 -05:00
072db3bea6
Assume the mac black image bug won't be fixed before v16.
2025-02-21 20:24:07 -05:00
a6deca6d9a
Latest mac still has the black image bug.
2025-02-21 20:14:30 -05:00
41c30e92e7
Let all model memory be offloaded on nvidia.
2025-02-21 06:32:21 -05:00
12da6ef581
Apparently directml supports fp16.
2025-02-20 09:30:24 -05:00
c5be423d6b
Fix link pointing to non-exisiting docs ( #6891 )
...
* Fix link pointing to non-exisiting docs
The current link is pointing to a path that does not exist any longer.
I changed it to point to the currect correct path for custom nodes datatypes.
* Update node_typing.py
2025-02-20 07:07:07 -05:00
5715be2ca9
Fix Hunyuan unet config detection for some models. ( #6877 )
...
The change to support 32 channel hunyuan models is missing the `key_prefix` on the key.
This addresses a complain in the comments of acc152b674
2025-02-19 07:14:45 -05:00
afc85cdeb6
Add Load Image Output node ( #6790 )
...
* add LoadImageOutput node
* add route for input/output/temp files
* update node_typing.py
* use literal type for image_folder field
* mark node as beta
2025-02-18 17:53:01 -05:00
acc152b674
Support loading and using SkyReels-V1-Hunyuan-I2V ( #6862 )
...
* Support SkyReels-V1-Hunyuan-I2V
* VAE scaling
* Fix T2V
oops
* Proper latent scaling
2025-02-18 17:06:54 -05:00
b07258cef2
Fix typo.
...
Let me know if this slows things down on 2000 series and below.
2025-02-18 07:28:33 -05:00
31e54b7052
Improve AMD arch detection.
2025-02-17 04:53:40 -05:00
8c0bae50c3
bf16 manual cast works on old AMD.
2025-02-17 04:42:40 -05:00
530412cb9d
Refactor torch version checks to be more future proof.
2025-02-17 04:36:45 -05:00
e2919d38b4
Disable bf16 on AMD GPUs that don't support it.
2025-02-16 05:46:10 -05:00
1cd6cd6080
Disable pytorch attention in VAE for AMD.
2025-02-14 05:42:14 -05:00
d7b4bf21a2
Auto enable mem efficient attention on gfx1100 on pytorch nightly 2.7
...
I'm not not sure which arches are supported yet. If you see improvements in
memory usage while using --use-pytorch-cross-attention on your AMD GPU let
me know and I will add it to the list.
2025-02-14 04:18:14 -05:00
019c7029ea
Add a way to set a different compute dtype for the model at runtime.
...
Currently only works for diffusion models.
2025-02-13 20:34:03 -05:00
8773ccf74d
Better memory estimation for ROCm that support mem efficient attention.
...
There is no way to check if the card actually supports it so it assumes
that it does if you use --use-pytorch-cross-attention with yours.
2025-02-13 08:32:36 -05:00
1d5d6586f3
Fix ruff.
2025-02-12 06:49:16 -05:00
35740259de
mix_ascend_bf16_infer_err ( #6794 )
2025-02-12 06:48:11 -05:00
ab888e1e0b
Add add_weight_wrapper function to model patcher.
...
Functions can now easily be added to wrap/modify model weights.
2025-02-12 05:55:35 -05:00
d9f0fcdb0c
Cleanup.
2025-02-11 17:17:03 -05:00
b124256817
Fix for running via DirectML ( #6542 )
...
* Fix for running via DirectML
Fix DirectML empty image generation issue with Flux1. add CPU fallback for unsupported path. Verified the model works on AMD GPUs
* fix formating
* update casual mask calculation
2025-02-11 17:11:32 -05:00
af4b7c91be
Make --force-fp16 actually force the diffusion model to be fp16.
2025-02-11 08:33:09 -05:00
4027466c80
Make lumina model work with any latent resolution.
2025-02-10 00:24:20 -05:00
095d867147
Remove useless function.
2025-02-09 07:02:57 -05:00
caeb27c3a5
res_multistep: Fix cfgpp and add ancestral samplers ( #6731 )
2025-02-08 19:39:58 -05:00
3d06e1c555
Make error more clear to user.
2025-02-08 18:57:24 -05:00
43a74c0de1
Allow FP16 accumulation with `--fast` ( #6453 )
...
Currently only applies to PyTorch nightly releases. (>=20250208)
2025-02-08 17:00:56 -05:00
079eccc92a
Don't compress http response by default.
...
Remove argument to disable it.
Add new --enable-compress-response-body argument to enable it.
2025-02-07 03:29:21 -05:00
14880e6dba
Remove some useless code.
2025-02-06 05:00:37 -05:00
37cd448529
Set the shift for Lumina back to 6.
2025-02-05 14:49:52 -05:00
94f21f9301
Upcasting rope to fp32 seems to make no difference in this model.
2025-02-05 04:32:47 -05:00
60653004e5
Use regular numbers for rope in lumina model.
2025-02-05 04:17:25 -05:00
a57d635c5f
Fix lumina 2 batches.
2025-02-04 21:48:11 -05:00
8ac2dddeed
Lower the default shift of lumina to reduce artifacts.
2025-02-04 06:50:37 -05:00
3e880ac709
Fix on python 3.9
2025-02-04 04:20:56 -05:00
e5ea112a90
Support Lumina 2 model.
2025-02-04 04:16:30 -05:00
44e19a28d3
Use maximum negative value instead of -inf for masks in text encoders.
...
This is probably more correct.
2025-02-02 09:46:00 -05:00
0a0df5f136
better guide message for sageattention ( #6634 )
2025-02-02 09:26:47 -05:00
24d6871e47
add disable-compres-response-body cli args; add compress middleware; ( #6672 )
2025-02-02 09:24:55 -05:00
9e1d301129
Only use stable cascade lora format with cascade model.
2025-02-01 06:35:22 -05:00
8d8dc9a262
Allow batch of different sigmas when noise scaling.
2025-01-30 06:49:52 -05:00
222f48c0f2
Allow changing folder_paths.base_path via command line argument. ( #6600 )
...
* Reimpl. CLI arg directly inside folder_paths.
* Update tests to use CLI arg mocking.
* Revert last-minute refactor.
* Fix test state polution.
2025-01-29 08:06:28 -05:00
13fd4d6e45
More friendly error messages for corrupted safetensors files.
2025-01-28 09:41:09 -05:00
255edf2246
Lower minimum ratio of loaded weights on Nvidia.
2025-01-27 05:26:51 -05:00
67feb05299
Remove redundant code.
2025-01-25 19:04:53 -05:00
14ca5f5a10
Remove useless code.
2025-01-24 06:15:54 -05:00
96e2a45193
Remove useless code.
2025-01-23 05:56:23 -05:00
dfa2b6d129
Remove unused function lcm in conds.py ( #6572 )
2025-01-23 05:54:09 -05:00
d6bbe8c40f
Remove support for python 3.8.
2025-01-22 17:04:30 -05:00
e857dd48b8
Add gradient estimation sampler ( #6554 )
2025-01-22 05:29:40 -05:00
fb2ad645a3
Add FluxDisableGuidance node to disable using the guidance embed.
2025-01-20 14:50:24 -05:00
d8a7a32779
Cleanup old TODO.
2025-01-20 03:44:13 -05:00
ebf038d4fa
Use `torch.special.expm1` ( #6388 )
...
* Use `torch.special.expm1`
This function provides greater precision than `exp(x) - 1` for small values of `x`.
Found with TorchFix https://github.com/pytorch-labs/torchfix/
* Use non-alias
2025-01-19 04:54:32 -05:00
b1a02131c9
Remove comfy.samplers self-import ( #6506 )
2025-01-18 17:49:51 -05:00
507199d9a8
Uni pc sampler now works with audio and video models.
2025-01-18 05:27:58 -05:00
2f3ab40b62
Add warning when using old pytorch versions.
2025-01-17 18:47:27 -05:00
0aa2368e46
Fix some cosmos fp8 issues.
2025-01-16 17:45:37 -05:00
cca96a85ae
Fix cosmos VAE failing with videos longer than 121 frames.
2025-01-16 16:30:06 -05:00
31831e6ef1
Code refactor.
2025-01-16 07:23:54 -05:00
88ceb28e20
Tweak hunyuan memory usage factor.
2025-01-16 06:31:03 -05:00
23289a6a5c
Clean up some debug lines.
2025-01-16 04:24:39 -05:00
9d8b6c1f46
More accurate memory estimation for cosmos and hunyuan video.
2025-01-16 03:48:40 -05:00
6320d05696
Slightly lower hunyuan video memory usage.
2025-01-16 00:23:01 -05:00
25683b5b02
Lower cosmos diffusion model memory usage.
2025-01-15 23:46:42 -05:00
4758fb64b9
Lower cosmos VAE memory usage by a bit.
2025-01-15 22:57:52 -05:00
008761166f
Optimize first attention block in cosmos VAE.
2025-01-15 21:48:46 -05:00
cba58fff0b
Remove unsafe embedding load for very old pytorch.
2025-01-15 04:32:23 -05:00
2feb8d0b77
Force safe loading of files in torch format on pytorch 2.4+
...
If this breaks something for you make an issue.
2025-01-15 03:50:27 -05:00
c78a45685d
Rewrite res_multistep sampler and implement res_multistep_cfg_pp sampler. ( #6462 )
2025-01-14 18:20:06 -05:00
3aaabb12d4
Implement Cosmos Image/Video to World (Video) diffusion models.
...
Use CosmosImageToVideoLatent to set the input image/video.
2025-01-14 05:14:10 -05:00
1f1c7b7b56
Remove useless code.
2025-01-13 03:52:37 -05:00
90f349f93d
Add res_multistep sampler from the cosmos code.
...
This sampler should work with all models.
2025-01-12 03:10:07 -05:00
6c9bd11fa3
Hooks Part 2 - TransformerOptionsHook and AdditionalModelsHook ( #6377 )
...
* Add 'sigmas' to transformer_options so that downstream code can know about the full scope of current sampling run, fix Hook Keyframes' guarantee_steps=1 inconsistent behavior with sampling split across different Sampling nodes/sampling runs by referencing 'sigmas'
* Cleaned up hooks.py, refactored Hook.should_register and add_hook_patches to use target_dict instead of target so that more information can be provided about the current execution environment if needed
* Refactor WrapperHook into TransformerOptionsHook, as there is no need to separate out Wrappers/Callbacks/Patches into different hook types (all affect transformer_options)
* Refactored HookGroup to also store a dictionary of hooks separated by hook_type, modified necessary code to no longer need to manually separate out hooks by hook_type
* In inner_sample, change "sigmas" to "sampler_sigmas" in transformer_options to not conflict with the "sigmas" that will overwrite "sigmas" in _calc_cond_batch
* Refactored 'registered' to be HookGroup instead of a list of Hooks, made AddModelsHook operational and compliant with should_register result, moved TransformerOptionsHook handling out of ModelPatcher.register_all_hook_patches, support patches in TransformerOptionsHook properly by casting any patches/wrappers/hooks to proper device at sample time
* Made hook clone code sane, made clear ObjectPatchHook and SetInjectionsHook are not yet operational
* Fix performance of hooks when hooks are appended via Cond Pair Set Props nodes by properly caching between positive and negative conds, make hook_patches_backup behave as intended (in the case that something pre-registers WeightHooks on the ModelPatcher instead of registering it at sample time)
* Filter only registered hooks on self.conds in CFGGuider.sample
* Make hook_scope functional for TransformerOptionsHook
* removed 4 whitespace lines to satisfy Ruff,
* Add a get_injections function to ModelPatcher
* Made TransformerOptionsHook contribute to registered hooks properly, added some doc strings and removed a so-far unused variable
* Rename AddModelsHooks to AdditionalModelsHook, rename SetInjectionsHook to InjectionsHook (not yet implemented, but at least getting the naming figured out)
* Clean up a typehint
2025-01-11 12:20:23 -05:00
ee8a7ab69d
Fast latent preview for Cosmos.
2025-01-11 04:41:24 -05:00
comfyanonymous
thot experiment
Jedrzej Kosinski
chaObserv
FeepingCreature
Chenlei Hu
JettHu
Chenlei Hu
Silver
BiologicalExplosion
maedtb
bymyself
Jukka Seppänen
zhoufan2956
HishamC
Pam
catboxanon
Dr.Lt.Data
KarryCharon
filtered
Sergii Dymchenko