fc247150fe
Implement EasyCache and Invent LazyCache ( #9496 )
...
* Attempting a universal implementation of EasyCache, starting with flux as test; I screwed up the math a bit, but when I set it just right it works.
* Fixed math to make threshold work as expected, refactored code to use EasyCacheHolder instead of a dict wrapped by object
* Use sigmas from transformer_options instead of timesteps to be compatible with a greater amount of models, make end_percent work
* Make log statement when not skipping useful, preparing for per-cond caching
* Added DIFFUSION_MODEL wrapper around forward function for wan model
* Add subsampling for heuristic inputs
* Add subsampling to output_prev (output_prev_subsampled now)
* Properly consider conds in EasyCache logic
* Created SuperEasyCache to test what happens if caching and reuse is moved outside the scope of conds, added PREDICT_NOISE wrapper to facilitate this test
* Change max reuse_threshold to 3.0
* Mark EasyCache/SuperEasyCache as experimental (beta)
* Make Lumina2 compatible with EasyCache
* Add EasyCache support for Qwen Image
* Fix missing comma, curse you Cursor
* Add EasyCache support to AceStep
* Add EasyCache support to Chroma
* Added EasyCache support to Cosmos Predict t2i
* Make EasyCache not crash with Cosmos Predict ImagToVideo latents, but does not work well at all
* Add EasyCache support to hidream
* Added EasyCache support to hunyuan video
* Added EasyCache support to hunyuan3d
* Added EasyCache support to LTXV (not very good, but does not crash)
* Implemented EasyCache for aura_flow
* Renamed SuperEasyCache to LazyCache, hardcoded subsample_factor to 8 on nodes
* Eatra logging when verbose is true for EasyCache
2025-08-22 22:41:08 -04:00
fe31ad0276
Add elementwise fusions ( #9495 )
...
* Add elementwise fusions
* Add addcmul pattern to Qwen
2025-08-22 19:39:15 -04:00
ff57793659
Support InstantX Qwen controlnet. ( #9488 )
2025-08-22 00:53:11 -04:00
f7bd5e58dd
Make it easier to implement future qwen controlnets. ( #9485 )
2025-08-21 23:18:04 -04:00
0963493a9c
Support for Qwen Diffsynth Controlnets canny and depth. ( #9465 )
...
These are not real controlnets but actually a patch on the model so they
will be treated as such.
Put them in the models/model_patches/ folder.
Use the new ModelPatchLoader and QwenImageDiffsynthControlnet nodes.
2025-08-20 22:26:37 -04:00
8d38ea3bbf
Fix bf16 precision issue with qwen image embeddings. ( #9441 )
2025-08-20 02:58:54 -04:00
7cd2c4bd6a
Qwen rotary embeddings should now match reference code. ( #9437 )
2025-08-20 00:45:27 -04:00
ed43784b0d
WIP Qwen edit model: The diffusion model part. ( #9383 )
2025-08-17 16:45:39 -04:00
0f2b8525bc
Qwen image model refactor. ( #9375 )
2025-08-16 17:51:28 -04:00
1702e6df16
Implement wan2.2 camera model. ( #9357 )
...
Use the old WanCameraImageToVideo node.
2025-08-15 17:29:58 -04:00
c308a8840a
Add FluxKontextMultiReferenceLatentMethod node. ( #9356 )
...
This node is only useful if someone trains the kontext model to properly
use multiple reference images via the index method.
The default is the offset method which feeds the multiple images like if
they were stitched together as one. This method works with the current
flux kontext model.
2025-08-15 15:50:39 -04:00
ad19a069f6
Make SLG nodes work on Qwen Image model. ( #9345 )
2025-08-14 23:16:01 -04:00
9df8792d4b
Make last PR not crash comfy on old pytorch. ( #9324 )
2025-08-13 15:12:41 -04:00
3da5a07510
SDPA backend priority ( #9299 )
2025-08-13 14:53:27 -04:00
560d38f34c
Wan2.2 fun control support. ( #9292 )
2025-08-12 23:26:33 -04:00
d044a24398
Fix default shift and any latent size for qwen image model. ( #9186 )
2025-08-05 06:12:27 -04:00
c012400240
Initial support for qwen image model. ( #9179 )
2025-08-04 22:53:25 -04:00
1e638a140b
Tiny wan vae optimizations. ( #9136 )
2025-08-01 05:25:38 -04:00
61b08d4ba6
Replace manual x * sigmoid(x) with torch silu in VAE nonlinearity ( #9057 )
2025-07-30 19:25:56 -04:00
da9dab7edd
Small wan camera memory optimization. ( #9111 )
2025-07-30 05:55:26 -04:00
dca6bdd4fa
Make wan2.2 5B i2v take a lot less memory. ( #9102 )
2025-07-29 19:44:18 -04:00
c60dc4177c
Remove unecessary clones in the wan2.2 VAE. ( #9083 )
2025-07-28 14:48:19 -04:00
a88788dce6
Wan 2.2 support. ( #9080 )
2025-07-28 08:00:23 -04:00
0621d73a9c
Remove useless code. ( #9059 )
2025-07-26 04:44:19 -04:00
e6e5d33b35
Remove useless code. ( #9041 )
...
This is only needed on old pytorch 2.0 and older.
2025-07-25 04:58:28 -04:00
9bc2798f72
LTXV VAE decoder: switch default padding mode ( #8930 )
2025-07-16 13:54:38 -04:00
938d3e8216
Remove windows line endings. ( #8866 )
2025-07-11 02:37:51 -04:00
974254218a
Un-hardcode chroma patch_size ( #8840 )
2025-07-08 15:56:59 -04:00
9093301a49
Don't add tiny bit of random noise when VAE encoding. ( #8705 )
...
Shouldn't change outputs but might make things a tiny bit more
deterministic.
2025-06-27 14:14:56 -04:00
ef5266b1c1
Support Flux Kontext Dev model. ( #8679 )
2025-06-26 11:28:41 -04:00
ec70ed6aea
Omnigen2 model implementation. ( #8669 )
2025-06-25 19:35:57 -04:00
f7fb193712
Small flux optimization. ( #8611 )
2025-06-20 05:37:32 -04:00
7e9267fa77
Make flux controlnet work with sd3 text enc. ( #8599 )
2025-06-19 18:50:05 -04:00
91d40086db
Fix pytorch warning. ( #8593 )
2025-06-19 11:04:52 -04:00
7ea79ebb9d
Add correct eps to ltxv rmsnorm. ( #8542 )
2025-06-15 12:21:25 -04:00
29596bd53f
Small cosmos attention code refactor. ( #8530 )
2025-06-14 05:02:05 -04:00
520eb77b72
LoRA Trainer: LoRA training node in weight adapter scheme ( #8446 )
2025-06-13 19:25:59 -04:00
c69af655aa
Uncap cosmos predict2 res and fix mem estimation. ( #8518 )
2025-06-13 07:30:18 -04:00
251f54a2ad
Basic initial support for cosmos predict2 text to image 2B and 14B models. ( #8517 )
2025-06-13 07:05:23 -04:00
8a4ff747bd
Fix mistake in last commit. ( #8496 )
...
* Move to right place.
2025-06-11 15:13:29 -04:00
af1eb58be8
Fix black images on some flux models in fp16. ( #8495 )
2025-06-11 15:09:11 -04:00
4248b1618f
Let chroma TE work on regular flux. ( #8429 )
2025-06-05 10:07:17 -04:00
08b7cc7506
use fused multiply-add pointwise ops in chroma ( #8279 )
2025-05-30 18:09:54 -04:00
5e5e46d40c
Not really tested WAN Phantom Support. ( #8321 )
2025-05-28 23:46:15 -04:00
5a87757ef9
Better error if sageattention is installed but a dependency is missing. ( #8264 )
2025-05-24 06:43:12 -04:00
30b2eb8a93
create arange on-device ( #8255 )
2025-05-23 16:15:06 -04:00
f85c08df06
Make VACE conditionings stackable. ( #8240 )
2025-05-22 19:22:26 -04:00
87f9130778
Revert "This doesn't seem to be needed on chroma. ( #8209 )" ( #8210 )
...
This reverts commit 7e84bf5373
2025-05-20 05:39:55 -04:00
7e84bf5373
This doesn't seem to be needed on chroma. ( #8209 )
2025-05-20 05:29:23 -04:00
c820ef950d
Add Wan-FUN Camera Control models and Add WanCameraImageToVideo node ( #8013 )
...
* support wan camera models
* fix by ruff check
* change camera_condition type; make camera_condition optional
* support camera trajectory nodes
* fix camera direction
---------
Co-authored-by: Qirui Sun <sunqr0667@126.com>
2025-05-15 19:00:43 -04:00
4a9014e201
Hunyuan Custom initial untested implementation. ( #8101 )
2025-05-13 15:53:47 -04:00
640c47e7de
Fix torch warning about deprecated function. ( #8075 )
...
Drop support for torch versions below 2.2 on the audio VAEs.
2025-05-12 14:32:01 -04:00
42da274717
Use normal ComfyUI attention in ACE-Steps model ( #8023 )
...
* Use normal ComfyUI attention in ACE-Steps model
* Let optimized_attention handle output reshape for ACE
2025-05-09 13:51:02 -04:00
fd08e39588
Make torchaudio not a hard requirement. ( #7987 )
...
Some platforms can't install it apparently so if it's not there it should
only break models that actually use it.
2025-05-07 21:37:12 -04:00
cc33cd3422
Experimental lyrics strength for ACE. ( #7984 )
2025-05-07 19:22:07 -04:00
16417b40d9
Initial ACE-Step model implementation. ( #7972 )
2025-05-07 08:33:34 -04:00
80a44b97f5
Change lumina to native RMSNorm. ( #7935 )
2025-05-04 06:39:23 -04:00
9187a09483
Change cosmos and hydit models to use the native RMSNorm. ( #7934 )
2025-05-04 06:26:20 -04:00
3041e5c354
Switch mochi and wan modes to use pytorch RMSNorm. ( #7925 )
...
* Switch genmo model to native RMSNorm.
* Switch WAN to native RMSNorm.
2025-05-03 19:07:55 -04:00
aa9d759df3
Switch ltxv to use the pytorch RMSNorm. ( #7897 )
2025-05-01 06:33:42 -04:00
08ff5fa08a
Cleanup chroma PR.
2025-04-30 20:57:30 -04:00
4ca3d84277
Support for Chroma - Flux1 Schnell distilled with CFG ( #7355 )
...
* Upload files for Chroma Implementation
* Remove trailing whitespace
* trim more trailing whitespace..oops
* remove unused imports
* Add supported_inference_dtypes
* Set min_length to 0 and remove attention_mask=True
* Set min_length to 1
* get_mdulations added from blepping and minor changes
* Add lora conversion if statement in lora.py
* Update supported_models.py
* update model_base.py
* add uptream commits
* set modelType.FLOW, will cause beta scheduler to work properly
* Adjust memory usage factor and remove unnecessary code
* fix mistake
* reduce code duplication
* remove unused imports
* refactor for upstream sync
* sync chroma-support with upstream via syncbranch patch
* Update sd.py
* Add Chroma as option for the OptimalStepsScheduler node
2025-04-30 20:57:00 -04:00
dbc726f80c
Better vace memory estimation. ( #7875 )
2025-04-29 20:42:00 -04:00
83d04717b6
Support HiDream E1 model. ( #7857 )
2025-04-28 15:01:15 -04:00
3ab231f01f
Fix issue with WAN VACE implementation. ( #7724 )
2025-04-21 23:36:12 -04:00
5d0d4ee98a
Add strength control for vace. ( #7717 )
2025-04-21 19:36:20 -04:00
ce22f687cc
Support for WAN VACE preview model. ( #7711 )
...
* Support for WAN VACE preview model.
* Remove print.
2025-04-21 14:40:29 -04:00
dbcfd092a2
Set default context_img_len to 257
2025-04-17 12:42:34 -04:00
c14429940f
Support loading WAN FLF model.
2025-04-17 12:04:48 -04:00
0d720e4367
Don't hardcode length of context_img in wan code.
2025-04-17 06:25:39 -04:00
1fc00ba4b6
Make hidream work with any latent resolution.
2025-04-16 18:34:14 -04:00
f00f340a56
Reuse code from flux model.
2025-04-16 17:43:55 -04:00
9ad792f927
Basic support for hidream i1 model.
2025-04-15 17:35:05 -04:00
8a438115fb
add RMSNorm to comfy.ops
2025-04-14 18:00:33 -04:00
89e4ea0175
Add activations_shape info in UNet models ( #7482 )
...
* Add activations_shape info in UNet models
* activations_shape should be a list
2025-04-04 21:27:54 -04:00
e471c726e5
Fallback to pytorch attention if sage attention fails.
2025-03-22 15:45:56 -04:00
32ca0805b7
Fix orientation of hunyuan 3d model.
2025-03-19 19:55:24 -04:00
11f1b41bab
Initial Hunyuan3Dv2 implementation.
...
Supports the multiview, mini, turbo models and VAEs.
2025-03-19 16:52:58 -04:00
3b19fc76e3
Allow disabling pe in flux code for some other models.
2025-03-18 05:09:25 -04:00
e8e990d6b8
Cleanup code.
2025-03-16 06:29:12 -04:00
a2448fc527
Remove useless code.
2025-03-14 18:10:37 -04:00
6a0daa79b6
Make the SkipLayerGuidanceDIT node work on WAN.
2025-03-14 10:55:19 -04:00
9c98c6358b
Tolerate missing `@torch.library.custom_op` ( #7234 )
...
This can happen on Pytorch versions older than 2.4.
2025-03-14 09:51:26 -04:00
7aceb9f91c
Add --use-flash-attention flag. ( #7223 )
...
* Add --use-flash-attention flag.
This is useful on AMD systems, as FA builds are still 10% faster than Pytorch cross-attention.
2025-03-14 03:22:41 -04:00
9aac21f894
Fix issues with new hunyuan img2vid model and bumb version to v0.3.26
2025-03-09 05:07:22 -04:00
7395b0c0d1
Support new hunyuan video i2v model.
...
Use the new "v2 (replace)" guidance type in HunyuanImageToVideo and set
image_interleave to 4 on the "Text Encode Hunyuan Video" node.
2025-03-08 20:34:47 -05:00
0952569493
Fix stable cascade VAE on some lowvram machines.
2025-03-08 20:24:04 -05:00
369b079ff6
Fix lowvram issue with ltxv vae.
2025-03-05 05:26:08 -05:00
93fedd92fe
Support LTXV 0.9.5.
...
Credits: Lightricks team.
2025-03-05 00:13:49 -05:00
f4dac8ab6f
Wan code small cleanup.
2025-02-27 07:22:42 -05:00
3ea3bc8546
Fix wan issues when prompt length is long.
2025-02-26 20:34:02 -05:00
0270a0b41c
Reduce artifacts on Wan by doing the patch embedding in fp32.
2025-02-26 16:59:26 -05:00
4bca7367f3
Don't try to use clip_fea on t2v model.
2025-02-26 08:38:09 -05:00
fa62287f1f
More code reuse in wan.
...
Fix bug when changing the compute dtype on wan.
2025-02-26 05:22:29 -05:00
4ced06b879
WIP support for Wan I2V model.
2025-02-26 01:49:43 -05:00
9a66bb972d
Make wan work with all latent resolutions.
...
Cleanup some code.
2025-02-25 19:56:04 -05:00
ea0f939df3
Fix issue with wan and other attention implementations.
2025-02-25 19:13:39 -05:00
f37551c1d2
Change wan rope implementation to the flux one.
...
Should be more compatible.
2025-02-25 19:11:14 -05:00
63023011b9
WIP support for Wan t2v model.
2025-02-25 17:20:35 -05:00
96d891cb94
Speedup on some models by not upcasting bfloat16 to float32 on mac.
2025-02-24 05:41:32 -05:00
Jedrzej Kosinski
contentis
comfyanonymous
chaObserv
Harel Cain
josephrocca
Kohaku-Blueleaf
drhead
George0726
blepping
comfyanonymous
Silver
Raphael Walker
FeepingCreature