RomanNum3ral
/
ai-toolkit
mirror of https://github.com/ostris/ai-toolkit.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
162 Commits 9 Branches 0 Tags 95 MiB
Commit Graph

58 Commits

Author SHA1 Message Date
Jaret Burkett 19255cdc7c Bugfixes. Added small augmentations to dataloader. Will switch to abluminations soon though. Added ability to adjust step count on start to override what is in the file 2023-09-20 05:30:10 -06:00
Jaret Burkett 0f105690cc Added some further extendability for plugins 2023-09-19 05:41:44 -06:00
Jaret Burkett 61badf85a7 t2i training working from what I can tell at least 2023-09-17 15:56:43 -06:00
Jaret Burkett 27f343fc08 Added base setup for training t2i adapters. Currently untested, saw something else shiny i wanted to finish sirst. Added content_or_style to the training config. It defaults to balanced, which is standard uniform time step sampling. If style or content is passed, it will use cubic sampling for timesteps to favor timesteps that are beneficial for training them. for style, favor later timesteps. For content, favor earlier timesteps. 2023-09-16 08:30:38 -06:00
Jaret Burkett 17e4fe40d7 Prevent lycoris network moduels if not training that part of network. Skew timesteps to favor later steps. It performs better 2023-09-14 15:13:24 -06:00
Jaret Burkett 569d7464d5 implemented device placement preset system more places. Vastly improved speed on setting network multiplier and activating network. Fixed timing issues on progress bar 2023-09-14 08:31:54 -06:00
Jaret Burkett 4e945917df added dropout to LoRA networks 2023-09-13 15:23:07 -06:00
Jaret Burkett ae70200d3c Bug fixes, speed improvements, compatability adjustments withdiffusers updates 2023-09-13 07:03:53 -06:00
Jaret Burkett d74dd636ee Memory optimizations. Default to using cudamalloc when torch 2.0 for mem allocation 2023-09-12 04:30:23 -06:00
Jaret Burkett 34bfeba229 Massive speed increase. Added latent caching both to disk and to memory 2023-09-10 08:54:49 -06:00
Jaret Burkett 626ed2939a bug fixes 2023-09-09 15:04:44 -06:00
Jaret Burkett 2128ac1e08 fixed issue with embed name, save whole config to dir instead of just process so it can be easily shared. Only make one config, no timesteps 2023-09-09 12:24:08 -06:00
Jaret Burkett be804c9cf5 Save embeddings as their trigger to match auto and comfy style loading. Also, FINALLY found why gradients were wonkey and fixed it. The root problem is dropping out of network state before backward pass. 2023-09-09 12:02:07 -06:00
Jaret Burkett 408c50ead1 actually got gradient checkpointing working, again, again, maybe 2023-09-09 11:27:42 -06:00
Jaret Burkett b01ab5d375 FINALLY fixed gradient checkpointing issue. Big batches baby. 2023-09-08 15:21:46 -06:00
Jaret Burkett ce4f9fe02a Bug fixes and improvements to token injection 2023-09-08 06:10:59 -06:00
Jaret Burkett 92a086d5a5 Fixed issue with token replacements 2023-09-07 13:42:39 -06:00
Jaret Burkett 3feb663a51 Switched to new bucket system that matched sdxl trained buckets. Fixed requirements. Updated embeddings to work with sdxl. Added method to train lora with an embedding at the trigger. Still testing but works amazingly well from what I can see 2023-09-07 13:06:18 -06:00
Jaret Burkett 64a5441832 Fully tested and now supporting locon on sdxl. If you have the ram 2023-09-04 14:05:10 -06:00
Jaret Burkett a4c3507a62 Added LoCON from LyCORIS 2023-09-04 08:48:07 -06:00
Jaret Burkett fa8fc32c0a Corrected key saving and loading to better match kohya 2023-09-04 00:22:34 -06:00
Jaret Burkett 22ed539321 Allow special args for schedulers 2023-09-03 20:38:44 -06:00
Jaret Burkett 2a40937b4f reworked samplers. Trying to find what is wrong with diffusers sampling is sdxl 2023-09-03 07:56:09 -06:00
Jaret Burkett 836fee47a6 Fixed some mismatched weights by adjusting tolerance. The mismatch ironically made the models better lol 2023-08-29 15:20:03 -06:00
Jaret Burkett 14ff51ceb4 fixed issues with converting and saving models. Cleaned keys. Improved testing for cycle load saving. 2023-08-29 12:31:19 -06:00
Jaret Burkett 714854ee86 Hude rework to move the batch to a DTO to make it far more modular to the future ui 2023-08-29 10:22:19 -06:00
Jaret Burkett bd758ff203 Cleanup and small bug fixes 2023-08-29 05:45:49 -06:00
Jaret Burkett a008d9e63b Fixed issue with loadin models after resume function added. Added additional flush if not training text encoder to clear out vram before grad accum 2023-08-28 17:56:30 -06:00
Jaret Burkett e866c75638 Built base interfaces for a DTO to handle batch infomation transports for the dataloader 2023-08-28 12:43:31 -06:00
Jaret Burkett c446f768ea Huge memory optimizations, many big fixes 2023-08-27 17:48:02 -06:00
Jaret Burkett 9b164a8688 Fixed issue with bucket dataloader corpping in too much. Added normalization capabilities to LoRA modules. Testing effects, but should prevent them from burning and also make them more compatable with stacking many LoRAs 2023-08-27 09:40:01 -06:00
Jaret Burkett 6bd3851058 Fixed issue with prompt token replace adding more than one replacement 2023-08-26 18:52:23 -06:00
Jaret Burkett 8105c05c12 Added bucketting capabilities to dataloader. Finally have full planned capability. noice 2023-08-26 16:36:32 -06:00
Jaret Burkett 3367ab6b2c Moved SD batch processing to a shared method and added it for use in slider training. Still testing if it affects quality over sampling 2023-08-26 08:55:00 -06:00
Jaret Burkett 7157c316af Added support for training lora, dreambooth, and fine tuning. Still need testing and docs 2023-08-23 15:37:00 -06:00
Jaret Burkett d298240cec Tied in ant tested TI script 2023-08-23 13:26:28 -06:00
Jaret Burkett 2e6c55c720 WIP creating textual inversion training script 2023-08-22 21:02:38 -06:00
Jaret Burkett 1a7e346b41 Added inbuild plugins and made one for image referenced. WIP 2023-08-10 16:02:44 -06:00
Jaret Burkett df48f0a843 Moved some of the job config into base process so it will be easier to extend extensions 2023-08-10 12:14:05 -06:00
Jaret Burkett fbc8a87a05 Reworked the sd rescaler script 2023-08-09 08:57:27 -06:00
Jaret Burkett 8c90fa86c6 Complete reqork of how slider training works and optimized it to hell. Can run entire algorythm in 1 batch now with less VRAM consumption than a quarter of it used to take 2023-08-05 18:46:08 -06:00
Jaret Burkett 66c6f0f6f7 Big refactor of SD runner and added image generator 2023-08-03 14:51:25 -06:00
Jaret Burkett 1a25b275c8 Did some work on SD rescaler. Need to run a long test on it eventually. 2023-08-02 07:59:27 -06:00
Jaret Burkett 8b8d53888d Added Model rescale and prepared a release upgrade 2023-08-01 13:49:54 -06:00
Jaret Burkett c01673f1b5 Added random weight adjuster to prevent overfitting 2023-07-29 19:30:14 -06:00
Jaret Burkett 8ba1b11557 Merge branch 'sdxl' into WIP 
# Conflicts:
#	jobs/process/BaseSDTrainProcess.py
#	jobs/process/TrainSliderProcess.py
 2023-07-29 14:29:18 -06:00
Jaret Burkett 9cdf2dd6e4 Bug fixes 2023-07-29 13:39:57 -06:00
Jaret Burkett 1e50b39442 Work on slider rework 2023-07-28 18:11:10 -06:00
Jaret Burkett 5fc2bb5d9c Information trainer 2023-07-28 08:16:29 -06:00
Jaret Burkett b2e2e4bf47 Added sd1.5 and 2.1 do the diffusers pipeline flow 2023-07-27 12:34:48 -06:00