60c1ac6a50
Add support for Boogu Image and Boogu Image Edit
2026-06-18 15:05:49 -06:00
178eb5fbbe
Add unconditional lora support so Ideogram 4 inference will more closely resemble the full pipeline results. I pushed a finetuned unconditional lora to the hub as an adapter.
2026-06-16 13:27:43 -06:00
86b19589a0
Update the Ideogram 4 prompt generation/parsing/ui to handle the updated format notes better.
2026-06-16 09:44:38 -06:00
c730d64478
Added a flag to keep loading the image when latents are cached. Useful for DFE and other methods that target pixelspace losses.
2026-06-15 05:31:48 -06:00
c8cd78b1a4
Allow nested transformer block names for quantization, lora targeting, quantizing
2026-06-13 14:33:16 -06:00
2f91db8363
Defauly to compiling full graph to false
2026-06-13 07:28:51 -06:00
e908d85f5e
Allow quantized unet offload compile and force fullgraph false ( #881 )
2026-06-13 07:27:28 -06:00
d83f7dd4d9
Fix a few issues with compile. Changed defaults. Future proofed block layer compile.
2026-06-12 11:43:43 -06:00
6b1f89f30b
Enhanced torch.compile System with Block-Level Compilation and Unified Whole-Model Fallback ( #866 )
...
* Add block-level compile and qcompile torch.compile whole model modes
* Update torch compile system
2026-06-12 10:35:12 -06:00
55ce6570f2
Automagic 3 rework. Stable in my testing.
2026-06-12 07:52:44 -06:00
01b6a9806b
Another complete rework of automagic3. Added a decay to the LR spread to the mean to prevent LRs fighting with eachother
2026-06-09 12:08:45 -06:00
acb1548722
Updated the comments and doc for Automagic v3
2026-06-09 07:05:48 -06:00
a1ac6e8b01
Reworked automagic v3 again. Seems more stable. Still testing.
2026-06-08 22:03:54 -06:00
5d6887fd98
Major updates to automagic3 optimizer. Seems to be functioning more ideally and naturally decays, as it should.
2026-06-08 14:09:02 -06:00
e281df70dd
Allow automagic3 to run unfused. Add some clipping.
2026-06-08 08:59:06 -06:00
c78b1404e3
Deepen offload prefetch pipeline with per-slot events
...
Replace the 2-slot ping-pong + single global "compute-started" event
with a depth-N ring buffer where each transfer waits only on the slot
it's reusing (D layers back) instead of the most-recent compute. Applies
to forward and backward, Linear and Conv. Depth is tunable via
AI_TOOLKIT_OFFLOAD_DEPTH (default 4).
Bit-exact vs non-offload (output, grad_input, weight grads). No speedup
on a bandwidth-bound PCIe link (already saturated at depth 2), but the
cleaner per-slot design removes the fragile shared-event serialization
and lets deeper prefetch help on faster buses.
2026-06-07 16:07:13 -06:00
cdff6e36aa
Pin inner stores of torachao to speed up layer offloading for quantized models around 25%
2026-06-07 15:52:01 -06:00
75781fb5a5
Fix float8 weights not offloading to CPU in layer offloading
2026-06-07 15:36:17 -06:00
7c1a76f336
Fix text encoder offload bug when caching embeddings ( #868 )
2026-06-07 14:09:32 -06:00
1dc9a797cf
Added Automagic v3
2026-06-07 12:06:42 -06:00
41157b460c
Added ability to set the caption extention in dataset viewer, captioner, and trainer so one dataset can have multiple caption styles in different files with different extensions. Added dataset caption template for a blank ideogram 4 formatted template.
2026-06-06 08:32:24 -06:00
6a8d9333b6
Improved the prompt handeling of ideogram4 model. Now used advanced prompts class to store them smaller and allow longer prompts
2026-06-04 13:00:23 -06:00
d9f26c2f87
Add gradient checkpointing to tipsv2 heads
2026-06-01 05:00:15 -06:00
e5439509b5
Added pure lpips dfe
2026-05-31 11:52:03 -06:00
87bac27513
Fixed issue with new bucket scaler
2026-05-28 11:34:16 -06:00
3e86d81fc6
Adjust bucket sizes to achieve maximum pixels without going over.
2026-05-28 09:36:17 -06:00
15082cfb8a
Round buckets for divisibility instead of always rounding down.
2026-05-28 09:08:21 -06:00
c9264bdd0b
Reworked the bucketing system to precisly match model specific divisibility. The old SDXL bucket sizes needed to go.
2026-05-28 08:35:58 -06:00
2aa60e4ca5
Update default agreement threshold for automagic v2 to be 0.5
2026-05-26 07:57:54 -06:00
12304e170f
Added some experimental loss targets
2026-05-24 14:13:23 -06:00
aecd554128
Add sapiens2 matting as a mask generator. Begin transition to model paths and model folders.
2026-05-20 08:56:16 -06:00
b42acb988f
Remove future steps from loss log if resuming from an earlier step
2026-05-18 09:23:55 -06:00
4bfe944792
Scale dfe 7 with velocity_equiv_weight
2026-05-13 09:06:59 -06:00
fc4d6ebf39
Add support for fine-tuning Hidream O1 ( #831 )
...
* Initial support for hidream. Lora keys likely need work
* Fix saving for hidream-o1
* Remove dependence on flash attention for hidream o1
* Fix gradient checkpointing for hidream o1
* A lot of fixes for hidream. Handle loading and saving as comfy model.
* Omit layers not used in comfy. Fix issue with lora loading keys in comfy
* Version bumpo
2026-05-12 11:15:16 -06:00
f38de2a2fe
Add tipsv2 locally and fix gradient checkpointing for it
2026-05-10 14:47:44 -06:00
a12ddd72a1
Change the velocity weight cap on dfe 9
2026-05-07 07:37:05 -06:00
6bb8acbffc
Add agreement_threshold default of 0.6 to automagic 2
2026-05-05 19:13:00 -06:00
963a9f42b2
Add decode latent to wan 2.1 models. Add gradinet checkpointing to wan vae.
2026-05-05 11:30:16 -06:00
4260a3c5b6
Add optimizer test suite and make minor speed adjustments to Automagicv2
2026-05-05 10:02:30 -06:00
aeca7fe404
Add Automagic v2 optimizer. It uses significantly less vram and is much more efficient.
2026-05-05 09:09:07 -06:00
af6458d1b5
Enable caching of ACE step latents.
2026-04-28 13:39:20 -06:00
acc6a36214
Scale DFE 9 to a velocity equiv weight to match flow matching gradient strength. Probably need to rework all DFEs to do this as the math checks out.
2026-04-28 09:10:02 -06:00
1fc4ad3979
Add sapiens2 as a diffusion feature extractor
2026-04-27 15:59:03 -06:00
f4445cd78c
Added psuedo_huber loss
2026-04-19 09:51:46 -06:00
488878f354
Use hidden layers in the loss for DFE 7 and 8
2026-04-18 13:07:38 -06:00
beb40ae29b
Add DFE8 with partial step
2026-04-17 17:40:16 -06:00
22ea3dd620
Fixed issue on some systems where Logger didnt have atty
2026-04-16 21:09:52 +00:00
ab1ee4df34
Hotfix some issues with Wan models caused by diffusers and transformers updates
2026-04-16 20:53:50 +00:00
2faba22b46
Fix issue when saving advanced prompt embeds. No such file or directory error
2026-04-16 12:22:56 -06:00
e74bc9ac7b
Fix issue with concatinating advanced prompt embeds.
2026-04-14 16:04:34 -06:00
Jaret Burkett
PlagueKind