dd08579eda
Add ability to pull control images from same folder group
2026-07-08 05:21:31 -06:00
b1e1a834d4
Added support to train Krea2 as an edit model
2026-07-04 09:12:39 -06:00
ad87aacec0
Add ability to set certain layers to full for loras
2026-06-26 13:01:48 -06:00
5f04ae7ad5
Rework merge_network_on_save to handle dequantization on merging and saving for much more efficient full finetuning.
2026-06-25 13:19:38 -06:00
6ecff36f26
Add a way to do full rank lora modules of non lora layers such as embeddings, norm, etc
2026-06-25 11:55:07 -06:00
4eb0707639
Add a control generation script.
2026-06-25 10:14:24 -06:00
d14f6e567a
Allow individual models to scale the loss after it is calculated.
2026-06-25 10:13:58 -06:00
089e41dd1c
Compile improvements - auto cache size, fix fullgraph setting, fix triton detection ( #899 )
...
* Compile improvements - auto cache size, fix fullgraph setting, fix triton detection
* remove forced torchao no longer needed
2026-06-24 07:59:41 -06:00
d586125b40
Force HF downloads to show progress bars when output is not a TTY ( #909 )
2026-06-24 07:55:38 -06:00
af594061ab
Add ability to do hidden states with tipsv2
2026-06-22 11:00:04 -06:00
820d534d6e
Add features for models that may need a non masked loss such as inpainiting.
2026-06-22 10:59:17 -06:00
c133c55cf5
Fix issue with mask generator revision
2026-06-22 08:19:27 -06:00
d51463ca52
Add ability to recover from a truncated image file in the dataset
2026-06-22 05:07:30 -06:00
ba0b3dbb65
Force batch size when bucket is too small by duplicating items in the batch
2026-06-21 20:01:29 -06:00
99f8fd44e3
Add a fallback bbox adjustment when json parsing fails on ideogram4 captioner
2026-06-19 07:25:54 -06:00
60c1ac6a50
Add support for Boogu Image and Boogu Image Edit
2026-06-18 15:05:49 -06:00
178eb5fbbe
Add unconditional lora support so Ideogram 4 inference will more closely resemble the full pipeline results. I pushed a finetuned unconditional lora to the hub as an adapter.
2026-06-16 13:27:43 -06:00
86b19589a0
Update the Ideogram 4 prompt generation/parsing/ui to handle the updated format notes better.
2026-06-16 09:44:38 -06:00
c730d64478
Added a flag to keep loading the image when latents are cached. Useful for DFE and other methods that target pixelspace losses.
2026-06-15 05:31:48 -06:00
c8cd78b1a4
Allow nested transformer block names for quantization, lora targeting, quantizing
2026-06-13 14:33:16 -06:00
2f91db8363
Defauly to compiling full graph to false
2026-06-13 07:28:51 -06:00
e908d85f5e
Allow quantized unet offload compile and force fullgraph false ( #881 )
2026-06-13 07:27:28 -06:00
d83f7dd4d9
Fix a few issues with compile. Changed defaults. Future proofed block layer compile.
2026-06-12 11:43:43 -06:00
6b1f89f30b
Enhanced torch.compile System with Block-Level Compilation and Unified Whole-Model Fallback ( #866 )
...
* Add block-level compile and qcompile torch.compile whole model modes
* Update torch compile system
2026-06-12 10:35:12 -06:00
55ce6570f2
Automagic 3 rework. Stable in my testing.
2026-06-12 07:52:44 -06:00
01b6a9806b
Another complete rework of automagic3. Added a decay to the LR spread to the mean to prevent LRs fighting with eachother
2026-06-09 12:08:45 -06:00
acb1548722
Updated the comments and doc for Automagic v3
2026-06-09 07:05:48 -06:00
a1ac6e8b01
Reworked automagic v3 again. Seems more stable. Still testing.
2026-06-08 22:03:54 -06:00
5d6887fd98
Major updates to automagic3 optimizer. Seems to be functioning more ideally and naturally decays, as it should.
2026-06-08 14:09:02 -06:00
e281df70dd
Allow automagic3 to run unfused. Add some clipping.
2026-06-08 08:59:06 -06:00
c78b1404e3
Deepen offload prefetch pipeline with per-slot events
...
Replace the 2-slot ping-pong + single global "compute-started" event
with a depth-N ring buffer where each transfer waits only on the slot
it's reusing (D layers back) instead of the most-recent compute. Applies
to forward and backward, Linear and Conv. Depth is tunable via
AI_TOOLKIT_OFFLOAD_DEPTH (default 4).
Bit-exact vs non-offload (output, grad_input, weight grads). No speedup
on a bandwidth-bound PCIe link (already saturated at depth 2), but the
cleaner per-slot design removes the fragile shared-event serialization
and lets deeper prefetch help on faster buses.
2026-06-07 16:07:13 -06:00
cdff6e36aa
Pin inner stores of torachao to speed up layer offloading for quantized models around 25%
2026-06-07 15:52:01 -06:00
75781fb5a5
Fix float8 weights not offloading to CPU in layer offloading
2026-06-07 15:36:17 -06:00
7c1a76f336
Fix text encoder offload bug when caching embeddings ( #868 )
2026-06-07 14:09:32 -06:00
1dc9a797cf
Added Automagic v3
2026-06-07 12:06:42 -06:00
41157b460c
Added ability to set the caption extention in dataset viewer, captioner, and trainer so one dataset can have multiple caption styles in different files with different extensions. Added dataset caption template for a blank ideogram 4 formatted template.
2026-06-06 08:32:24 -06:00
6a8d9333b6
Improved the prompt handeling of ideogram4 model. Now used advanced prompts class to store them smaller and allow longer prompts
2026-06-04 13:00:23 -06:00
d9f26c2f87
Add gradient checkpointing to tipsv2 heads
2026-06-01 05:00:15 -06:00
e5439509b5
Added pure lpips dfe
2026-05-31 11:52:03 -06:00
87bac27513
Fixed issue with new bucket scaler
2026-05-28 11:34:16 -06:00
3e86d81fc6
Adjust bucket sizes to achieve maximum pixels without going over.
2026-05-28 09:36:17 -06:00
15082cfb8a
Round buckets for divisibility instead of always rounding down.
2026-05-28 09:08:21 -06:00
c9264bdd0b
Reworked the bucketing system to precisly match model specific divisibility. The old SDXL bucket sizes needed to go.
2026-05-28 08:35:58 -06:00
2aa60e4ca5
Update default agreement threshold for automagic v2 to be 0.5
2026-05-26 07:57:54 -06:00
12304e170f
Added some experimental loss targets
2026-05-24 14:13:23 -06:00
aecd554128
Add sapiens2 matting as a mask generator. Begin transition to model paths and model folders.
2026-05-20 08:56:16 -06:00
b42acb988f
Remove future steps from loss log if resuming from an earlier step
2026-05-18 09:23:55 -06:00
4bfe944792
Scale dfe 7 with velocity_equiv_weight
2026-05-13 09:06:59 -06:00
fc4d6ebf39
Add support for fine-tuning Hidream O1 ( #831 )
...
* Initial support for hidream. Lora keys likely need work
* Fix saving for hidream-o1
* Remove dependence on flash attention for hidream o1
* Fix gradient checkpointing for hidream o1
* A lot of fixes for hidream. Handle loading and saving as comfy model.
* Omit layers not used in comfy. Fix issue with lora loading keys in comfy
* Version bumpo
2026-05-12 11:15:16 -06:00
f38de2a2fe
Add tipsv2 locally and fix gradient checkpointing for it
2026-05-10 14:47:44 -06:00
Jaret Burkett
PlagueKind
fatalis
PlagueKind