ai-toolkit/toolkit/models
Jaret Burkett 78cf049c29
Add support for ACE-Step 1.5 and ACE-Step 1.5 XL. Also added dataset captioning through the UI. (#785)
* Base ace step 1.5 xl added. Generating, still wip on training and ui

* Base training code done

* Fix some issues with caching text embeddings. Update sample cards to show audio

* Fix issue with quantizing ace step

* Add album artwork to samples with waveform.

* Cleanup logs

* Add album art endpoint to speed up album art loading

* Made an make video with artwork script

* Make ui handle basic audio models. Make multi line adjustments to the editor and better syntax hilighting.

* Add prompt tagging system for special tagged models.

* prompt tagging processing for ui working.

* Moved default samples to a special file so we can add more when needed and they can be adjusted for a specific model

* Add a captioner job with music captioner that is prepped for use with the ui

* Add basit ui setup for captioning modal and handeling captioning jobs

* Starting captioning job from ui working. Still better management for it.

* Better filtering of job options in the job view for captioning jobs

* Added qwen3 vl as a captioner for images

* Have an indicator when a dataset is being captioned.

* Adjust the way caption jobs look in the queue

* Fix a few issues. Adjust defaults.

* Version bump

* Added ace step to the readme.
2026-04-09 15:02:03 -06:00
..
loaders Switch to unified text encoder for wan models. Pred for 2.2 14b 2025-08-14 10:07:18 -06:00
wan21 Fixed error with wan models when caching text embeddings 2026-02-06 14:26:53 -07:00
DoRA.py WIP - adding support for flux DoRA and ip adapter training 2024-08-22 04:36:39 -06:00
FakeVAE.py DEF for fake vae and adjust scaling 2025-09-12 18:09:08 -06:00
LoRAFormer.py Remove ip adapter submodule 2025-04-18 09:59:42 -06:00
RRDB.py Fixed numerous issues with traing ESRGAN 2023-08-08 20:03:19 -06:00
auraflow.py Partial implementation for training auraflow. 2024-07-12 12:11:38 -06:00
autoencoder_tiny_with_pooled_exits.py Improvements to VAE trainer. Allow CLIP loss. 2025-07-24 06:50:56 -06:00
base_model.py Add support for ACE-Step 1.5 and ACE-Step 1.5 XL. Also added dataset captioning through the UI. (#785) 2026-04-09 15:02:03 -06:00
block.py Added training for a custom version of ERSGAN arcitecture. Testing training now 2023-08-07 18:04:23 -06:00
clip_fusion.py Work on additional image embedding methods. Finalized zipper resampler. It works amazing 2024-02-10 09:00:05 -07:00
clip_pre_processor.py Adjustments to the clip preprocessor. Allow merging in new weights for ip adapters so you can change the arcitecture while maintaining as much data as possible 2024-01-06 11:56:53 -07:00
cogview4.py Added ability to add models to finetune as plugins. Also added flux2 new arch via that method. 2025-03-27 16:07:00 -06:00
control_lora_adapter.py More work on mean flow loss. Moved it to an adapter. Still not functioning properly though. 2025-06-16 07:17:35 -06:00
decorator.py Added training for an experimental decoratgor embedding. Allow for turning off guidance embedding on flux (for unreleased model). Various bug fixes and modifications 2024-12-15 08:59:27 -07:00
diffusion_feature_extraction.py Use pooler embeddings for DFE v6 with dino v3 2026-03-27 07:02:07 -06:00
flux.py More work on mean flow loss. Moved it to an adapter. Still not functioning properly though. 2025-06-16 07:17:35 -06:00
flux_sage_attn.py Added support for full finetuning flux with randomized param activation. Examples coming soon 2024-11-21 13:05:32 -07:00
i2v_adapter.py Cleanup 2025-04-18 11:44:49 -06:00
ilora.py Remove ip adapter submodule 2025-04-18 09:59:42 -06:00
ilora2.py Various experiments and minor bug fixes for edge cases 2025-04-25 13:44:38 -06:00
llm_adapter.py Various experiments and minor bug fixes for edge cases 2025-04-25 13:44:38 -06:00
lokr.py Add LTX-2 Support (#644) 2026-01-13 04:55:30 -07:00
mean_flow_adapter.py Work on mean flow. Minor bug fixes. Omnigen improvements 2025-06-26 13:46:20 -06:00
pixtral_vision.py Quick patch to scope xformer imports until a better solution 2024-09-28 15:36:42 -06:00
redux.py Added support for training flux redux adapters 2024-11-21 20:01:52 -07:00
single_value_adapter.py Removed all submodules. Submodule free now, yay. 2025-04-18 10:39:15 -06:00
size_agnostic_feature_encoder.py WIP on SAFE encoder. Work on fp16 training improvements. Various other tweaks and improvements 2024-05-27 10:50:24 -06:00
sref.py Avoid loading state dict for automagic for now until I can sort out some issues 2025-02-26 17:03:14 -07:00
subpixel_adapter.py small bug fixes 2025-03-30 20:09:40 -06:00
te_adapter.py Remove ip adapter submodule 2025-04-18 09:59:42 -06:00
te_aug_adapter.py Remove ip adapter submodule 2025-04-18 09:59:42 -06:00
vd_adapter.py Remove ip adapter submodule 2025-04-18 09:59:42 -06:00
zipper_resampler.py Work on additional image embedding methods. Finalized zipper resampler. It works amazing 2024-02-10 09:00:05 -07:00