RomanNum3ral
/
ai-toolkit
mirror of https://github.com/ostris/ai-toolkit.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,253 Commits 9 Branches 0 Tags 95 MiB
Commit Graph

211 Commits

Author SHA1 Message Date
Jaret Burkett adcf884c0f Built out the ui trainer plugin with db comminication 2025-02-21 05:53:35 -07:00
Jaret Burkett 4af6c5cf30 Work on supporting flex.2 potential arch 2025-02-17 14:10:25 -07:00
Jaret Burkett 7679105d52 Added llm text encoder adapter 2025-02-13 08:28:32 -07:00
Jaret Burkett 2622de1e01 DFE tweaks. Adding support for more llms as text encoders 2025-02-13 04:31:49 -07:00
Jaret Burkett 9a7266275d Wokr on lumina2 2025-02-08 14:52:39 -07:00
Jaret Burkett d138f07365 Imitial lumina3 support 2025-02-08 10:59:53 -07:00
Jaret Burkett 216ab164ce Experimental features and bug fixes 2025-02-04 13:36:34 -07:00
Jaret Burkett e6180d1e1d Bug fixes 2025-01-31 13:23:01 -07:00
Jaret Burkett 15a57bc89f Add new version of DFE. Kitchen sink 2025-01-31 11:42:27 -07:00
Jaret Burkett 34a1c6947a Added flux_shift as timestep type 2025-01-27 07:35:00 -07:00
Jaret Burkett 5e663746b8 Working multi gpu training. Still need a lot of tweaks and testing. 2025-01-25 16:46:20 -07:00
Jaret Burkett fadb2f3a76 Allow quantizing the te independently on flux. added lognorm_blend timestep schedule 2025-01-18 18:02:31 -07:00
Jaret Burkett 4723f23c0d Added ability to split up flux across gpus (experimental). Changed the way timestep scheduling works to prep for more specific schedules. 2024-12-31 07:06:55 -07:00
Jaret Burkett 8ef07a9c36 Added training for an experimental decoratgor embedding. Allow for turning off guidance embedding on flux (for unreleased model). Various bug fixes and modifications 2024-12-15 08:59:27 -07:00
Jaret Burkett 6509ba4484 Fix seed generation to make it deterministic so it is consistant from gpu to gpu 2024-11-15 12:11:13 -07:00
Jaret Burkett 4aa19b5c1d Only quantize flux T5 is also quantizing model. Load TE from original name and path if fine tuning. 2024-10-29 14:25:31 -06:00
Jaret Burkett 22cd40d7b9 Improvements for full tuning flux. Added debugging launch config for vscode 2024-10-29 04:54:08 -06:00
Jaret Burkett 3400882a80 Added preliminary support for SD3.5-large lora training 2024-10-22 12:21:36 -06:00
Jaret Burkett 9452929300 Apply a mask to the embeds for SD if using T5 encoder 2024-10-04 10:55:20 -06:00
Jaret Burkett a800c9d19e Add a method to have an inference only lora 2024-10-04 10:06:53 -06:00
Jaret Burkett 58537fc92b Added initial direct vision pixtral support 2024-09-28 10:47:51 -06:00
Jaret Burkett 40a8ff5731 Load local hugging face packages for assistant adapter 2024-09-23 10:37:12 -06:00
Jaret Burkett 2776221497 Added option to cache empty prompt or trigger and unload text encoders while training 2024-09-21 20:54:09 -06:00
Plat 79b4e04b80
Feat: Wandb logging (#95) 
* wandb logging

* fix: start logging before train loop

* chore: add wandb dir to gitignore

* fix: wrap wandb functions

* fix: forget to send last samples

* chore: use valid type

* chore: use None when not type-checking

* chore: resolved complicated logic

* fix: follow log_every

---------

Co-authored-by: Plat <github@p1at.dev>
Co-authored-by: Jaret Burkett <jaretburkett@gmail.com>
 2024-09-19 20:01:01 -06:00
Jaret Burkett fc34a69bec Ignore guidance embed when full tuning flux. adjust block scaler to decat to 1.0. Add MLP resampler for reducing vision adapter tokens 2024-09-09 16:24:46 -06:00
Jaret Burkett e5fadddd45 Added ability to do prompt attn masking for flux 2024-09-02 17:29:36 -06:00
Jaret Burkett 60232def91 Made peleminary arch for flux ip adapter training 2024-08-28 08:55:39 -06:00
Jaret Burkett 338c77d677 Fixed breaking change with diffusers. Allow flowmatch on normal stable diffusion models. 2024-08-22 14:36:22 -06:00
Jaret Burkett c45887192a Unload interum weights when doing multi lora fuse 2024-08-18 09:35:10 -06:00
Jaret Burkett 13a965a26c Fixed bad key naming on lora fuse I just pushed 2024-08-18 09:33:31 -06:00
Jaret Burkett f944eeaa4d Fuse flux schnell assistant adapter in pieces when doing lowvram to drastically speed ip up from minutes to seconds. 2024-08-18 09:09:11 -06:00
Jaret Burkett 81899310f8 Added support for training on flux schnell. Added example config and instructions for training on flux schnell 2024-08-17 06:58:39 -06:00
Jaret Burkett f9179540d2 Flush after sampling 2024-08-16 17:29:42 -06:00
Jaret Burkett 452e0e286d For lora assisted training, merge in before quantizing then sample with schnell at -1 weight. Almost doubles training speed with lora adapter. 2024-08-16 17:28:44 -06:00
Jaret Burkett 7fed4ea761 fixed huge flux training bug. Added ability to use an assistatn lora 2024-08-14 10:14:13 -06:00
Jaret Burkett 599fafe01f Allow user to have the full flux checkpoint local 2024-08-12 09:57:16 -06:00
Jaret Burkett 6490a326e5 Fixed issue for vaes without a shift 2024-08-11 10:30:55 -06:00
Jaret Burkett ec1ea7aa0e Added support for training on primary gpu with low_vram flag. Updated example script to remove creepy horse sample at that seed 2024-08-11 09:54:30 -06:00
Jaret Burkett b3e03295ad Reworked flux pred. Again 2024-08-08 13:06:34 -06:00
Jaret Burkett acafe9984f Adjustments to loading of flux. Added a feedback to ema 2024-08-07 13:17:26 -06:00
Jaret Burkett c2424087d6 8 bit training working on flux 2024-08-06 11:53:27 -06:00
Jaret Burkett 272c8608c2 Make a CFG version of flux pipeline 2024-08-05 16:35:53 -06:00
Jaret Burkett 187663ab55 Use peft format for flux loras so they are compatible with diffusers. allow loading an assistant lora 2024-08-05 14:34:37 -06:00
Jaret Burkett edb7e827ee Adjusted flow matching so target noise multiplier works properly with it. 2024-08-05 11:40:05 -06:00
Jaret Burkett 0ea27011d5 Bug fix 2024-08-04 11:07:19 -06:00
Jaret Burkett f321de7bdb Setup to retrain guidance embedding for flux. Use defualt timestep distribution for flux 2024-08-04 10:37:23 -06:00
Jaret Burkett 9beea1c268 Flux training should work now... maybe 2024-08-03 09:17:34 -06:00
Jaret Burkett 369aa143bc Only train a few blocks on flux (for now) 2024-08-03 07:02:27 -06:00
Jaret Burkett 87ba867fdc Added flux training. Still a WIP. Wont train right without rectified flow working right 2024-08-02 15:00:30 -06:00
Jaret Burkett e81e19fd0f Added target_norm_std which is a game changer 2024-07-28 16:08:33 -06:00
Jaret Burkett 0bc4d555c7 A lot of pixart sigma training tweaks 2024-07-28 11:23:18 -06:00
Jaret Burkett 11e426fdf1 Various features and fixes. Too much brain fog to do a proper description 2024-07-18 07:34:14 -06:00
Jaret Burkett e4558dff4b Partial implementation for training auraflow. 2024-07-12 12:11:38 -06:00
Jaret Burkett 045e4a6e15 Save entire pixart model again 2024-07-07 07:56:48 -06:00
Jaret Burkett cab8a1c7b8 WIP to add the caption_proj weight to pixart sigma TE adapter 2024-07-06 13:00:21 -06:00
Jaret Burkett 657fd09f25 Added more control over Sigma sizes 2024-06-26 08:57:53 -06:00
Jaret Burkett 7165f2d25a Work to omprove pixart training 2024-06-23 20:46:48 +00:00
Jaret Burkett 5d47244c57 Added support for pixart sigma loras 2024-06-16 11:56:30 -06:00
Jaret Burkett 696f73c30d Removed variant 2024-06-14 17:09:47 -06:00
Jaret Burkett 37cebd9458 WIP Ilora 2024-06-14 09:31:01 -06:00
Jaret Burkett bd10d2d668 Some work on sd3 training. Not working 2024-06-13 12:19:16 -06:00
Jaret Burkett 3f3636b788 Bug fixes and little improvements here and there. 2024-06-08 06:24:20 -06:00
Jaret Burkett 68b7e159bc Bug Fixes 2024-05-17 08:41:20 -06:00
Jaret Burkett 10e1ecf1e8 Added single value adapter training 2024-04-28 06:04:47 -06:00
Jaret Burkett b96913d73c Improvements to dataloader 2024-04-27 09:28:28 -06:00
Jaret Burkett 5da3613e0b Bug fixes and minor features 2024-04-25 06:14:31 -06:00
Jaret Burkett 5a70b7f38d Added pixart sigma support, but it wont work until i address breaking changes with lora code in diffusers so it can be upgraded. 2024-04-20 10:46:56 -06:00
Jaret Burkett 7284aab7c0 Added specialized scaler training to ip adapters 2024-04-05 08:17:09 -06:00
Jaret Burkett 427847ac4c Small tweaks and fixes for specialized ip adapter training 2024-03-26 11:35:26 -06:00
Jaret Burkett 016687bda1 Adapter work. Bug fixes. Auto adjust LR when resuming optimizer. 2024-03-17 10:21:47 -06:00
Jaret Burkett 72de68d8aa WIP on clip vision encoder 2024-03-13 07:24:08 -06:00
Jaret Burkett 337945de9a Added this not that guidance. Added ability to replace prompts. 2024-02-28 20:10:14 -07:00
Jaret Burkett 561914d8e6 Removed old code for fixing multistep sampler that is no longer needed 2024-02-25 11:53:35 -07:00
Jaret Burkett b68c3ef734 Added te aug adapter 2024-02-21 21:30:26 -07:00
Jaret Burkett 49c41e6a5f Bug fixes. allow for random negative prompts 2024-02-21 04:51:52 -07:00
Jaret Burkett 2478554c95 Bug fixes. Added IP adapter training for Pixart 2024-02-17 10:06:57 -07:00
Jaret Burkett 93b52932c1 Added training for pixart-a 2024-02-13 16:00:04 -07:00
Jaret Burkett 4ec4025cbb Added adapter modules for text encoders and direct vision 2024-02-12 08:46:18 -07:00
Jaret Burkett e18e0cb5f8 Added comparitive loss when training clip encoder. Allow selecting clip layer. on ip adapter. Improvements to prior prediction 2024-02-05 07:40:03 -07:00
Jaret Burkett 177c7130ec improved correction of pred norm by targeting the prior 2024-02-01 06:31:04 -07:00
Jaret Burkett 1ae1017748 Bug fixes. added ability to use l1 loss. varous other tests and improvements 2024-01-31 06:30:54 -07:00
Jaret Burkett f17ad8d794 various bug fixes. Created an contextual alpha mask module to calculate alpha mask 2024-01-18 16:34:27 -07:00
Jaret Burkett eebd3c8212 Initial training script for photomaker training. Needs a little more work. 2024-01-15 18:46:26 -07:00
Jaret Burkett 5276975fb0 Added additional config options for custom plugins I needed 2024-01-15 08:31:09 -07:00
Jaret Burkett afc231efc1 Added reference adapters, many bug fixes, more ip adapter work and customizability 2024-01-01 17:15:53 -07:00
Jaret Burkett bafacf3b65 Initial commit 2023-12-29 13:07:35 -07:00
Jaret Burkett eeee4a1620 Created a size agnostic feature encoder (SAFE) model to be trained in replace of CLIP for ip adapters. It is mostly conv layers so will hopefully be able to handle facial features better than clip can. Also bug fixes 2023-12-28 12:20:27 -07:00
Jaret Burkett 05ae95ca89 Added a clip vision adapter trainer. Only works for sd15 for now 2023-12-24 13:26:04 -07:00
Jaret Burkett b653906715 Fixed ip adapter training. Works now 2023-12-17 08:22:59 -07:00
Jaret Burkett 39870411d8 More guidance work. Improved LoRA module resolver for unet. Added vega mappings and LoRA training for it. Various other bigfixes and changes 2023-12-15 06:02:10 -07:00
Jaret Burkett eaa0fb6253 Tons of bug fixes and improvements to special training. Fixed slider training. 2023-12-09 16:38:10 -07:00
Jaret Burkett bd2bce9b92 Switched to trailing timestep spacing to make timesteps for consistant across schedulers. Honed in on targeted guidance. It is finally perfect. (I think) 2023-11-29 14:32:48 -07:00
Jaret Burkett 7624241032 More fixes for noise schedules and fixed targeted guidance inverted masked prior 2023-11-29 10:13:31 -07:00
Jaret Burkett bef10a639c Merge remote-tracking branch 'origin/development' 
# Conflicts:
#	toolkit/stable_diffusion_model.py
 2023-11-28 10:40:05 -07:00
Jaret Burkett c2a4b8e058 More timestep fixes 2023-11-28 10:34:08 -07:00
Jaret Burkett 792a5e37e2 Numerous fixes for time sampling. Still not perfect 2023-11-28 07:34:43 -07:00
Jaret Burkett fbec68681d Added timestep modifications to lcm scheduler for more evenly spaced timesteps 2023-11-17 23:26:52 -07:00
Jaret Burkett ad50921c41 Sampling tests and added fixes for cleanups 2023-11-16 08:33:23 -07:00
Jaret Burkett e47006ed70 Added some features for an LCM condenser plugin 2023-11-15 08:56:45 -07:00
Jaret Burkett 4f9cdd916a added prompt dropout to happen indempendently on each TE 2023-11-14 05:26:51 -07:00