RomanNum3ral
/
ai-toolkit
mirror of https://github.com/ostris/ai-toolkit.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
408 Commits 9 Branches 0 Tags 95 MiB
Commit Graph

330 Commits

Author SHA1 Message Date
Jaret Burkett 561914d8e6 Removed old code for fixing multistep sampler that is no longer needed 2024-02-25 11:53:35 -07:00
Jaret Burkett b0a0f28191 Bug fixes 2024-02-25 08:28:29 -07:00
Jaret Burkett f965a1299f Fixed Dora implementation. Still highly experimental 2024-02-24 10:26:01 -07:00
Jaret Burkett 1bd94f0f01 Added early DoRA support, but will change shortly. Dont use right now. 2024-02-23 05:55:41 -07:00
Jaret Burkett b68c3ef734 Added te aug adapter 2024-02-21 21:30:26 -07:00
Jaret Burkett 49c41e6a5f Bug fixes. allow for random negative prompts 2024-02-21 04:51:52 -07:00
Jaret Burkett 2478554c95 Bug fixes. Added IP adapter training for Pixart 2024-02-17 10:06:57 -07:00
Jaret Burkett 93b52932c1 Added training for pixart-a 2024-02-13 16:00:04 -07:00
Jaret Burkett 4ec4025cbb Added adapter modules for text encoders and direct vision 2024-02-12 08:46:18 -07:00
Jaret Burkett e074058faa Work on additional image embedding methods. Finalized zipper resampler. It works amazing 2024-02-10 09:00:05 -07:00
Jaret Burkett a8481c1670 randomly adjust scale of unconditional noise on ip adapters if training with cfg 2024-02-06 03:44:54 -07:00
Jaret Burkett e18e0cb5f8 Added comparitive loss when training clip encoder. Allow selecting clip layer. on ip adapter. Improvements to prior prediction 2024-02-05 07:40:03 -07:00
Jaret Burkett 177c7130ec improved correction of pred norm by targeting the prior 2024-02-01 06:31:04 -07:00
Jaret Burkett 1ae1017748 Bug fixes. added ability to use l1 loss. varous other tests and improvements 2024-01-31 06:30:54 -07:00
Jaret Burkett 92b9c71d44 Many bug fixes. Ip adapter bug fixes. Added noise to unconditional, it works better. added an ilora adapter for 1 shotting LoRAs 2024-01-28 08:20:03 -07:00
Jaret Burkett f17ad8d794 various bug fixes. Created an contextual alpha mask module to calculate alpha mask 2024-01-18 16:34:27 -07:00
Jaret Burkett 86c70a2a1f Added an experimental clip fusion model that is showing promise for embedding concepts 2024-01-17 13:13:04 -07:00
Jaret Burkett 655533d4c7 More work on custom adapter 2024-01-16 17:41:26 -07:00
Jaret Burkett eebd3c8212 Initial training script for photomaker training. Needs a little more work. 2024-01-15 18:46:26 -07:00
Jaret Burkett 5276975fb0 Added additional config options for custom plugins I needed 2024-01-15 08:31:09 -07:00
Jaret Burkett e190fbaeb8 Prepwork for ilora 2024-01-12 06:41:15 -07:00
Jaret Burkett 290393f7ae Imporvements to ip weight adaptation. Bug fixes. Added masking to direct guidance loss. Allow importing a file for random triggers. Handle bas meta images with improper sizing. 2024-01-11 12:22:16 -07:00
Jaret Burkett b2a54c8f36 Added siglip support 2024-01-09 20:52:21 -07:00
Jaret Burkett b767d29b3c Adjustments to the clip preprocessor. Allow merging in new weights for ip adapters so you can change the arcitecture while maintaining as much data as possible 2024-01-06 11:56:53 -07:00
Jaret Burkett 645b27f97a Bug fixes with ip adapter training. Made a clip pre processor that can be trained with ip adapter to help augment the clip input to squeeze in more detail from a larget input. moved clip processing to the dataloader for speed. 2024-01-04 12:59:38 -07:00
Jaret Burkett 65c08b09c3 Added ability to do cfg during training. Various bug fixes 2024-01-02 11:29:57 -07:00
Jaret Burkett afc231efc1 Added reference adapters, many bug fixes, more ip adapter work and customizability 2024-01-01 17:15:53 -07:00
Jaret Burkett bafacf3b65 Initial commit 2023-12-29 13:07:35 -07:00
Jaret Burkett 0892dec4a5 Fixed some new bugs i added. woops 2023-12-28 14:03:42 -07:00
Jaret Burkett eeee4a1620 Created a size agnostic feature encoder (SAFE) model to be trained in replace of CLIP for ip adapters. It is mostly conv layers so will hopefully be able to handle facial features better than clip can. Also bug fixes 2023-12-28 12:20:27 -07:00
Jaret Burkett d11ed7f66c Big fixes and added method to standardize values in both latent and pixel space before feeding into the network. Target values were determined over huge generated regularization sets. 2023-12-26 06:19:48 -07:00
Jaret Burkett 27ad79053e Added SDXL support for clip vision embedder trainer 2023-12-24 14:31:29 -07:00
Jaret Burkett 05ae95ca89 Added a clip vision adapter trainer. Only works for sd15 for now 2023-12-24 13:26:04 -07:00
Jaret Burkett 0f8daa5612 Bug fixes, work on maing IP adapters more customizable. 2023-12-24 08:32:39 -07:00
Jaret Burkett 7703e3a15e Fixes for sdxl ip adapter training. Bug fixes 2023-12-21 11:15:58 -07:00
Jaret Burkett 0f597f453e Switched ip adapter dataloader to clip_image paths so the control paths can be used for training assistant adapters while training ip adapters 2023-12-20 10:32:24 -07:00
Jaret Burkett dfb64b5957 Allow ip adapters to be much more variable in their creation 2023-12-20 06:18:33 -07:00
Jaret Burkett 82098e5d6e Added more functionality for ip adapters 2023-12-19 09:54:56 -07:00
Jaret Burkett b653906715 Fixed ip adapter training. Works now 2023-12-17 08:22:59 -07:00
Jaret Burkett 13d32423f6 Added a polarity balancer to guidance 2023-12-15 15:19:14 -07:00
Jaret Burkett 39870411d8 More guidance work. Improved LoRA module resolver for unet. Added vega mappings and LoRA training for it. Various other bigfixes and changes 2023-12-15 06:02:10 -07:00
Jaret Burkett e5177833b2 Targeted guidance work 2023-12-09 19:06:18 -07:00
Jaret Burkett eaa0fb6253 Tons of bug fixes and improvements to special training. Fixed slider training. 2023-12-09 16:38:10 -07:00
Jaret Burkett eaec2f5a52 Added guidance mentiods. WIP 2023-12-08 08:39:21 -07:00
Jaret Burkett bd2bce9b92 Switched to trailing timestep spacing to make timesteps for consistant across schedulers. Honed in on targeted guidance. It is finally perfect. (I think) 2023-11-29 14:32:48 -07:00
Jaret Burkett 7624241032 More fixes for noise schedules and fixed targeted guidance inverted masked prior 2023-11-29 10:13:31 -07:00
Jaret Burkett bef10a639c Merge remote-tracking branch 'origin/development' 
# Conflicts:
#	toolkit/stable_diffusion_model.py
 2023-11-28 10:40:05 -07:00
Jaret Burkett c2a4b8e058 More timestep fixes 2023-11-28 10:34:08 -07:00
Jaret Burkett 792a5e37e2 Numerous fixes for time sampling. Still not perfect 2023-11-28 07:34:43 -07:00
Jaret Burkett d7e55b6ad4 Bug fixes, negative prompting during training, hardened catching 2023-11-24 07:25:11 -07:00
Jaret Burkett fbec68681d Added timestep modifications to lcm scheduler for more evenly spaced timesteps 2023-11-17 23:26:52 -07:00
Jaret Burkett 6280284d8b Fixed cleanup of emebddings. 2023-11-16 20:26:11 -07:00
Jaret Burkett ad50921c41 Sampling tests and added fixes for cleanups 2023-11-16 08:33:23 -07:00
Jaret Burkett e47006ed70 Added some features for an LCM condenser plugin 2023-11-15 08:56:45 -07:00
Jaret Burkett 4f9cdd916a added prompt dropout to happen indempendently on each TE 2023-11-14 05:26:51 -07:00
Jaret Burkett 7782caa468 Varous bug fixes. Finalized targeted guidance algo 2023-11-10 12:18:08 -07:00
Jaret Burkett fa6d91ba76 Diffirential guidance working, but I may have a better way 2023-11-09 06:09:36 -07:00
Jaret Burkett 1ee62562a4 diffirential guidance is WORKING (from what I can tell) 2023-11-07 19:24:12 -07:00
Jaret Burkett dc8448d958 Added a way pass refiner ratio to sample config 2023-11-06 09:22:58 -07:00
Jaret Burkett a8b3b8b8da Fixed weight mapping for refiner 2023-11-06 07:37:47 -07:00
Jaret Burkett 93ea955d7c Added refiner fine tuning. Works, but needs some polish. 2023-11-05 17:15:03 -07:00
Jaret Burkett 8a9e8f708f Added base for using guidance during training. Still not working right. 2023-11-05 04:03:32 -07:00
Jaret Burkett d35733ac06 Added support for training ssd-1B. Added support for saving models into diffusers format. We can currently save in safetensors format for ssd-1b, but diffusers cannot load it yet. 2023-11-03 05:01:16 -06:00
Jaret Burkett ceaf1d9454 Various bug fixes, wip stuff, and tweaks 2023-11-02 18:19:20 -06:00
Jaret Burkett 7d707b2fe6 Added masking to slider training. Something is still weird though 2023-11-01 14:51:29 -06:00
Jaret Burkett a899ec91c8 Added some split prompting started code, adamw8bit, replacements improving, learnable snr gos. A lot of good stuff. 2023-11-01 06:52:21 -06:00
Jaret Burkett 436a09430e Added flat snr gamma vs min. Fixes timestep timing 2023-10-29 15:41:55 -06:00
Jaret Burkett 48a9bac22d Added doffusers schedulers 2023-10-29 12:39:50 -06:00
Jaret Burkett 298001439a Added gradient accumulation finally 2023-10-28 13:14:29 -06:00
Jaret Burkett 6f3e0d5af2 Improved lorm extraction and training 2023-10-28 08:21:59 -06:00
Jaret Burkett 0a79ac9604 Added lorm. WIP 2023-10-26 18:23:51 -06:00
Jaret Burkett 9636194c09 Added fuyu captioning 2023-10-25 14:14:53 -06:00
Jaret Burkett d742792ee4 Fixed issue with not loading short prompt 2023-10-24 16:19:32 -06:00
Jaret Burkett 002279cec3 Allow short and long caption combinations like form the new captioning system. Merge the network into the model before inference and reextract when done. Doubles inference speed on locon models during inference. allow splitting a batch into individual components and run them through alone. Basicallt gradient accumulation with single batch size. 2023-10-24 16:02:07 -06:00
Jaret Burkett 73c8b50975 Added ability to use adagrad from transformers 2023-10-24 11:16:01 -06:00
Jaret Burkett 34eb563d55 Added dataset tagging and management tools using llava 2023-10-24 10:11:47 -06:00
Jaret Burkett 9905a1e205 Fixes and longer prompts 2023-10-22 08:57:37 -06:00
Jaret Burkett 0e9fc42816 Allow for inverted masked prior 2023-10-21 06:50:17 -06:00
Jaret Burkett d46112a354 Fixes for aug pipeline 2023-10-20 13:24:07 -06:00
Jaret Burkett 07bf7bd7de Allow augmentations and targeting different loss types fron the config file 2023-10-18 03:04:57 -06:00
Jaret Burkett da6302ada8 added a method to apply multipliers to noise and latents prior to combining 2023-10-17 06:09:16 -06:00
Jaret Burkett a05459afaf Fixed issue with adapters that only had 1 input channel. Added ability to set the percentage chance of adapter matching 2023-10-15 15:13:35 -06:00
Jaret Burkett b1a22d0b3e hardened reading prompts from json 2023-10-15 07:20:33 -06:00
Jaret Burkett 7909b50d24 Added adapter assistance to SD training 2023-10-14 08:44:53 -06:00
Jaret Burkett 38e441a29c allow flipping for point of interesting autocropping. allow num repeats. Fixed some bugs with new free u 2023-10-12 21:02:47 -06:00
Jaret Burkett 4e3b2c2569 Allow for alpha to be used as a mask 2023-10-10 17:08:59 -06:00
Jaret Burkett 239addba51 Fixed memory leak when cachine latents to disk 2023-10-10 13:31:47 -06:00
Jaret Burkett 63ceffae24 Massive speed increases and ram optimizations 2023-10-10 06:07:55 -06:00
Jaret Burkett f4c90bb589 Added config to set the min value of a mask 2023-10-09 15:47:54 -06:00
Jaret Burkett bb1d3793e3 Added ability to add masks to dataloader and sd trainer to adjust weight of image 2023-10-09 11:21:00 -06:00
Jaret Burkett b1cfafa0c6 always prompt both even when training only one encoder 2023-10-07 11:23:09 -06:00
Jaret Burkett cac8754399 Allow for training loras on onle one text encoder for sdxl 2023-10-06 08:11:56 -06:00
Jaret Burkett f73402473b Bug fixes. Added some functionality to help with private extensions 2023-10-05 07:09:34 -06:00
Jaret Burkett 579650eaf8 Fixed big issue with bucketing dataloader and added random cripping to a point of interest 2023-10-02 18:31:08 -06:00
Jaret Burkett 320e109c5f Allow loading from a json detail file for captions 2023-10-02 13:25:09 -06:00
Jaret Burkett 560251a24f fixed issue with down block residuals when doing slider cfg on sdxl with t2i adapter assisted training 2023-10-01 07:32:48 -06:00
Jaret Burkett 085787b799 Allow loading auxillery images from dataloader 2023-09-30 07:28:23 -06:00
Jaret Burkett 8509da60cb Added a way to add a t2i adapter guided slider training for more consitant images 2023-09-28 14:08:56 -06:00
Jaret Burkett 76c764af49 Fixed issues with adapter and disabled telementry for now 2023-09-24 12:51:29 -06:00
Jaret Burkett abf7cd221d allow setting adapter weight in prompts 2023-09-24 06:51:54 -06:00
Jaret Burkett e5153d87c9 Fixed issues with dataloader bucketing. Allow using standard base image for t2i adapters. 2023-09-24 05:19:57 -06:00
Jaret Burkett 830e87cb87 Added IP adapter training. Not functioning correctly yet 2023-09-24 02:39:43 -06:00
Jaret Burkett 19255cdc7c Bugfixes. Added small augmentations to dataloader. Will switch to abluminations soon though. Added ability to adjust step count on start to override what is in the file 2023-09-20 05:30:10 -06:00
Jaret Burkett 0f105690cc Added some further extendability for plugins 2023-09-19 05:41:44 -06:00
Jaret Burkett 61badf85a7 t2i training working from what I can tell at least 2023-09-17 15:56:43 -06:00
Jaret Burkett 181f237a7b added flipping x and y for dataset loader 2023-09-17 08:42:54 -06:00
Jaret Burkett c698837241 Fixes to esrgan trainer. Moved logic for sd prompt embeddings out of diffusers pipeline so I can manipulate it 2023-09-16 17:41:07 -06:00
Jaret Burkett 27f343fc08 Added base setup for training t2i adapters. Currently untested, saw something else shiny i wanted to finish sirst. Added content_or_style to the training config. It defaults to balanced, which is standard uniform time step sampling. If style or content is passed, it will use cubic sampling for timesteps to favor timesteps that are beneficial for training them. for style, favor later timesteps. For content, favor earlier timesteps. 2023-09-16 08:30:38 -06:00
Jaret Burkett 17e4fe40d7 Prevent lycoris network moduels if not training that part of network. Skew timesteps to favor later steps. It performs better 2023-09-14 15:13:24 -06:00
Jaret Burkett 569d7464d5 implemented device placement preset system more places. Vastly improved speed on setting network multiplier and activating network. Fixed timing issues on progress bar 2023-09-14 08:31:54 -06:00
Jaret Burkett 4e945917df added dropout to LoRA networks 2023-09-13 15:23:07 -06:00
Jaret Burkett ae70200d3c Bug fixes, speed improvements, compatability adjustments withdiffusers updates 2023-09-13 07:03:53 -06:00
Jaret Burkett d8d1e6fd1e big fixes 2023-09-12 18:48:39 -06:00
Jaret Burkett 257da9493d Dont load image if we are cachine latents 2023-09-12 18:39:41 -06:00
Jaret Burkett b5a2669b74 Fixed memory leak 2023-09-12 07:03:10 -06:00
Jaret Burkett d74dd636ee Memory optimizations. Default to using cudamalloc when torch 2.0 for mem allocation 2023-09-12 04:30:23 -06:00
Jaret Burkett e8583860ad Upgraded to dev for t2i on diffusers. Minor migrations to make it work. 2023-09-11 14:46:06 -06:00
bendeguzvaradi 3d387103cd safety checker to None 2023-09-11 17:05:55 +02:00
Jaret Burkett 083cefa78c Bugfixes for slider reference 2023-09-10 18:36:23 -06:00
Jaret Burkett b5ec8e4eb1 Improve reference slider memory and speed 2023-09-10 18:26:44 -06:00
Jaret Burkett 708b07adb7 Fixed issue with interleaving when doing cfg 2023-09-10 10:26:58 -06:00
Jaret Burkett a437aed45f bug fix 2023-09-10 09:52:14 -06:00
Jaret Burkett 34bfeba229 Massive speed increase. Added latent caching both to disk and to memory 2023-09-10 08:54:49 -06:00
Jaret Burkett 41a3f63b72 allow smaller images in buckets and bucket them 2023-09-10 03:43:02 -06:00
Jaret Burkett be804c9cf5 Save embeddings as their trigger to match auto and comfy style loading. Also, FINALLY found why gradients were wonkey and fixed it. The root problem is dropping out of network state before backward pass. 2023-09-09 12:02:07 -06:00
Jaret Burkett 408c50ead1 actually got gradient checkpointing working, again, again, maybe 2023-09-09 11:27:42 -06:00
Jaret Burkett 4ed03a8d92 Fixed issue with buckets scaling. again 2023-09-08 16:32:14 -06:00
Jaret Burkett b01ab5d375 FINALLY fixed gradient checkpointing issue. Big batches baby. 2023-09-08 15:21:46 -06:00
Jaret Burkett cb91b0d6da Changed model download from HF to fp16 2023-09-08 07:57:19 -06:00
Jaret Burkett ce4f9fe02a Bug fixes and improvements to token injection 2023-09-08 06:10:59 -06:00
Jaret Burkett 92a086d5a5 Fixed issue with token replacements 2023-09-07 13:42:39 -06:00
Jaret Burkett 3feb663a51 Switched to new bucket system that matched sdxl trained buckets. Fixed requirements. Updated embeddings to work with sdxl. Added method to train lora with an embedding at the trigger. Still testing but works amazingly well from what I can see 2023-09-07 13:06:18 -06:00
Jaret Burkett 436bf0c6a3 Added experimental concept replacer, replicate converter, bucket maker, and other goodies 2023-09-06 18:50:32 -06:00
Jaret Burkett f84500159c Fixed issue with lora layer check 2023-09-04 14:27:37 -06:00
Jaret Burkett 64a5441832 Fully tested and now supporting locon on sdxl. If you have the ram 2023-09-04 14:05:10 -06:00
Jaret Burkett a4c3507a62 Added LoCON from LyCORIS 2023-09-04 08:48:07 -06:00
Jaret Burkett fa8fc32c0a Corrected key saving and loading to better match kohya 2023-09-04 00:22:34 -06:00
Jaret Burkett 22ed539321 Allow special args for schedulers 2023-09-03 20:38:44 -06:00
Jaret Burkett 2a40937b4f reworked samplers. Trying to find what is wrong with diffusers sampling is sdxl 2023-09-03 07:56:09 -06:00
Jaret Burkett 4ca819a05e Fixes for dataloader 2023-08-31 04:54:10 -06:00
Jaret Burkett addf024630 Fixed issue with omitting square pictures 2023-08-30 15:00:22 -06:00
Jaret Burkett 33267e117c Reworked bucket loader to scale buckets to pixels amounts not just minimum size. Makes the network more consistant 2023-08-30 14:52:12 -06:00
Jaret Burkett d401348c2e Make data loader resiliant to bad headers in meta 2023-08-29 18:56:06 -06:00
Jaret Burkett 836fee47a6 Fixed some mismatched weights by adjusting tolerance. The mismatch ironically made the models better lol 2023-08-29 15:20:03 -06:00
Jaret Burkett 14ff51ceb4 fixed issues with converting and saving models. Cleaned keys. Improved testing for cycle load saving. 2023-08-29 12:31:19 -06:00
Jaret Burkett 714854ee86 Hude rework to move the batch to a DTO to make it far more modular to the future ui 2023-08-29 10:22:19 -06:00
Jaret Burkett bd758ff203 Cleanup and small bug fixes 2023-08-29 05:45:49 -06:00
Jaret Burkett a008d9e63b Fixed issue with loadin models after resume function added. Added additional flush if not training text encoder to clear out vram before grad accum 2023-08-28 17:56:30 -06:00
Jaret Burkett b79ced3e10 Merge branch 'main' into development 2023-08-28 16:21:51 -06:00
Jaret Burkett bee0b6a235 Added converters for all stable diffusion models to convert back to ldm format from diffusers. 2023-08-28 16:12:32 -06:00