Jaret Burkett
|
e5153d87c9
|
Fixed issues with dataloader bucketing. Allow using standard base image for t2i adapters.
|
2023-09-24 05:19:57 -06:00 |
|
Jaret Burkett
|
830e87cb87
|
Added IP adapter training. Not functioning correctly yet
|
2023-09-24 02:39:43 -06:00 |
|
Jaret Burkett
|
19255cdc7c
|
Bugfixes. Added small augmentations to dataloader. Will switch to abluminations soon though. Added ability to adjust step count on start to override what is in the file
|
2023-09-20 05:30:10 -06:00 |
|
Jaret Burkett
|
0f105690cc
|
Added some further extendability for plugins
|
2023-09-19 05:41:44 -06:00 |
|
Jaret Burkett
|
61badf85a7
|
t2i training working from what I can tell at least
|
2023-09-17 15:56:43 -06:00 |
|
Jaret Burkett
|
181f237a7b
|
added flipping x and y for dataset loader
|
2023-09-17 08:42:54 -06:00 |
|
Jaret Burkett
|
c698837241
|
Fixes to esrgan trainer. Moved logic for sd prompt embeddings out of diffusers pipeline so I can manipulate it
|
2023-09-16 17:41:07 -06:00 |
|
Jaret Burkett
|
27f343fc08
|
Added base setup for training t2i adapters. Currently untested, saw something else shiny i wanted to finish sirst. Added content_or_style to the training config. It defaults to balanced, which is standard uniform time step sampling. If style or content is passed, it will use cubic sampling for timesteps to favor timesteps that are beneficial for training them. for style, favor later timesteps. For content, favor earlier timesteps.
|
2023-09-16 08:30:38 -06:00 |
|
Jaret Burkett
|
17e4fe40d7
|
Prevent lycoris network moduels if not training that part of network. Skew timesteps to favor later steps. It performs better
|
2023-09-14 15:13:24 -06:00 |
|
Jaret Burkett
|
569d7464d5
|
implemented device placement preset system more places. Vastly improved speed on setting network multiplier and activating network. Fixed timing issues on progress bar
|
2023-09-14 08:31:54 -06:00 |
|
Jaret Burkett
|
4e945917df
|
added dropout to LoRA networks
|
2023-09-13 15:23:07 -06:00 |
|
Jaret Burkett
|
ae70200d3c
|
Bug fixes, speed improvements, compatability adjustments withdiffusers updates
|
2023-09-13 07:03:53 -06:00 |
|
Jaret Burkett
|
d8d1e6fd1e
|
big fixes
|
2023-09-12 18:48:39 -06:00 |
|
Jaret Burkett
|
257da9493d
|
Dont load image if we are cachine latents
|
2023-09-12 18:39:41 -06:00 |
|
Jaret Burkett
|
b5a2669b74
|
Fixed memory leak
|
2023-09-12 07:03:10 -06:00 |
|
Jaret Burkett
|
d74dd636ee
|
Memory optimizations. Default to using cudamalloc when torch 2.0 for mem allocation
|
2023-09-12 04:30:23 -06:00 |
|
Jaret Burkett
|
e8583860ad
|
Upgraded to dev for t2i on diffusers. Minor migrations to make it work.
|
2023-09-11 14:46:06 -06:00 |
bendeguzvaradi
|
3d387103cd
|
safety checker to None
|
2023-09-11 17:05:55 +02:00 |
|
Jaret Burkett
|
083cefa78c
|
Bugfixes for slider reference
|
2023-09-10 18:36:23 -06:00 |
|
Jaret Burkett
|
b5ec8e4eb1
|
Improve reference slider memory and speed
|
2023-09-10 18:26:44 -06:00 |
|
Jaret Burkett
|
708b07adb7
|
Fixed issue with interleaving when doing cfg
|
2023-09-10 10:26:58 -06:00 |
|
Jaret Burkett
|
a437aed45f
|
bug fix
|
2023-09-10 09:52:14 -06:00 |
|
Jaret Burkett
|
34bfeba229
|
Massive speed increase. Added latent caching both to disk and to memory
|
2023-09-10 08:54:49 -06:00 |
|
Jaret Burkett
|
41a3f63b72
|
allow smaller images in buckets and bucket them
|
2023-09-10 03:43:02 -06:00 |
|
Jaret Burkett
|
be804c9cf5
|
Save embeddings as their trigger to match auto and comfy style loading. Also, FINALLY found why gradients were wonkey and fixed it. The root problem is dropping out of network state before backward pass.
|
2023-09-09 12:02:07 -06:00 |
|
Jaret Burkett
|
408c50ead1
|
actually got gradient checkpointing working, again, again, maybe
|
2023-09-09 11:27:42 -06:00 |
|
Jaret Burkett
|
4ed03a8d92
|
Fixed issue with buckets scaling. again
|
2023-09-08 16:32:14 -06:00 |
|
Jaret Burkett
|
b01ab5d375
|
FINALLY fixed gradient checkpointing issue. Big batches baby.
|
2023-09-08 15:21:46 -06:00 |
|
Jaret Burkett
|
cb91b0d6da
|
Changed model download from HF to fp16
|
2023-09-08 07:57:19 -06:00 |
|
Jaret Burkett
|
ce4f9fe02a
|
Bug fixes and improvements to token injection
|
2023-09-08 06:10:59 -06:00 |
|
Jaret Burkett
|
92a086d5a5
|
Fixed issue with token replacements
|
2023-09-07 13:42:39 -06:00 |
|
Jaret Burkett
|
3feb663a51
|
Switched to new bucket system that matched sdxl trained buckets. Fixed requirements. Updated embeddings to work with sdxl. Added method to train lora with an embedding at the trigger. Still testing but works amazingly well from what I can see
|
2023-09-07 13:06:18 -06:00 |
|
Jaret Burkett
|
436bf0c6a3
|
Added experimental concept replacer, replicate converter, bucket maker, and other goodies
|
2023-09-06 18:50:32 -06:00 |
|
Jaret Burkett
|
f84500159c
|
Fixed issue with lora layer check
|
2023-09-04 14:27:37 -06:00 |
|
Jaret Burkett
|
64a5441832
|
Fully tested and now supporting locon on sdxl. If you have the ram
|
2023-09-04 14:05:10 -06:00 |
|
Jaret Burkett
|
a4c3507a62
|
Added LoCON from LyCORIS
|
2023-09-04 08:48:07 -06:00 |
|
Jaret Burkett
|
fa8fc32c0a
|
Corrected key saving and loading to better match kohya
|
2023-09-04 00:22:34 -06:00 |
|
Jaret Burkett
|
22ed539321
|
Allow special args for schedulers
|
2023-09-03 20:38:44 -06:00 |
|
Jaret Burkett
|
2a40937b4f
|
reworked samplers. Trying to find what is wrong with diffusers sampling is sdxl
|
2023-09-03 07:56:09 -06:00 |
|
Jaret Burkett
|
4ca819a05e
|
Fixes for dataloader
|
2023-08-31 04:54:10 -06:00 |
|
Jaret Burkett
|
addf024630
|
Fixed issue with omitting square pictures
|
2023-08-30 15:00:22 -06:00 |
|
Jaret Burkett
|
33267e117c
|
Reworked bucket loader to scale buckets to pixels amounts not just minimum size. Makes the network more consistant
|
2023-08-30 14:52:12 -06:00 |
|
Jaret Burkett
|
d401348c2e
|
Make data loader resiliant to bad headers in meta
|
2023-08-29 18:56:06 -06:00 |
|
Jaret Burkett
|
836fee47a6
|
Fixed some mismatched weights by adjusting tolerance. The mismatch ironically made the models better lol
|
2023-08-29 15:20:03 -06:00 |
|
Jaret Burkett
|
14ff51ceb4
|
fixed issues with converting and saving models. Cleaned keys. Improved testing for cycle load saving.
|
2023-08-29 12:31:19 -06:00 |
|
Jaret Burkett
|
714854ee86
|
Hude rework to move the batch to a DTO to make it far more modular to the future ui
|
2023-08-29 10:22:19 -06:00 |
|
Jaret Burkett
|
bd758ff203
|
Cleanup and small bug fixes
|
2023-08-29 05:45:49 -06:00 |
|
Jaret Burkett
|
a008d9e63b
|
Fixed issue with loadin models after resume function added. Added additional flush if not training text encoder to clear out vram before grad accum
|
2023-08-28 17:56:30 -06:00 |
|
Jaret Burkett
|
b79ced3e10
|
Merge branch 'main' into development
|
2023-08-28 16:21:51 -06:00 |
|
Jaret Burkett
|
bee0b6a235
|
Added converters for all stable diffusion models to convert back to ldm format from diffusers.
|
2023-08-28 16:12:32 -06:00 |
|
Jaret Burkett
|
2ecb5cf024
|
Merge branch 'main' into development
|
2023-08-28 14:01:44 -06:00 |
|
Jaret Burkett
|
fab7c2b04a
|
Fixed issue with key mapping from diffusers back to ldm
|
2023-08-28 14:01:26 -06:00 |
|
Jaret Burkett
|
e866c75638
|
Built base interfaces for a DTO to handle batch infomation transports for the dataloader
|
2023-08-28 12:43:31 -06:00 |
|
Jaret Burkett
|
71da78c8af
|
improved normalization for a network with varrying batch network weights
|
2023-08-28 12:42:57 -06:00 |
|
Jaret Burkett
|
c446f768ea
|
Huge memory optimizations, many big fixes
|
2023-08-27 17:48:02 -06:00 |
|
Jaret Burkett
|
cc49786ee9
|
Dataloader bug fixes
|
2023-08-27 14:36:38 -06:00 |
|
Jaret Burkett
|
9b164a8688
|
Fixed issue with bucket dataloader corpping in too much. Added normalization capabilities to LoRA modules. Testing effects, but should prevent them from burning and also make them more compatable with stacking many LoRAs
|
2023-08-27 09:40:01 -06:00 |
|
Jaret Burkett
|
6bd3851058
|
Fixed issue with prompt token replace adding more than one replacement
|
2023-08-26 18:52:23 -06:00 |
|
Jaret Burkett
|
fd338e67bb
|
Fixed bug with dataloader not seperating mulitple datasets
|
2023-08-26 18:07:24 -06:00 |
|
Jaret Burkett
|
8105c05c12
|
Added bucketting capabilities to dataloader. Finally have full planned capability. noice
|
2023-08-26 16:36:32 -06:00 |
|
Jaret Burkett
|
3367ab6b2c
|
Moved SD batch processing to a shared method and added it for use in slider training. Still testing if it affects quality over sampling
|
2023-08-26 08:55:00 -06:00 |
|
Jaret Burkett
|
aeaca13d69
|
Fixed issue with shuffeling permutations
|
2023-08-23 22:02:00 -06:00 |
|
Jaret Burkett
|
b408f9f3eb
|
Fixed issue with timestep I broke for sliders
|
2023-08-23 16:15:30 -06:00 |
|
Jaret Burkett
|
7157c316af
|
Added support for training lora, dreambooth, and fine tuning. Still need testing and docs
|
2023-08-23 15:37:00 -06:00 |
|
Jaret Burkett
|
d298240cec
|
Tied in ant tested TI script
|
2023-08-23 13:26:28 -06:00 |
|
Jaret Burkett
|
2e6c55c720
|
WIP creating textual inversion training script
|
2023-08-22 21:02:38 -06:00 |
|
Jaret Burkett
|
36ba08d3fa
|
Added a converter back to ldm from diffusers for sdxl. Can finally get to training it properly
|
2023-08-21 16:22:01 -06:00 |
|
Jaret Burkett
|
e8667f856f
|
Fix issue with there being an extra . on gene
|
2023-08-20 15:54:38 -06:00 |
|
Jaret Burkett
|
bef5551ea5
|
Ultimate slider training built, still needs tuning
|
2023-08-19 18:54:34 -06:00 |
|
Jaret Burkett
|
b77b9acc0b
|
Added base for ultimate slider. WIP
|
2023-08-19 15:35:24 -06:00 |
|
Jaret Burkett
|
c6675e2801
|
Added shuffeling to prompts
|
2023-08-19 07:57:30 -06:00 |
|
Jaret Burkett
|
90eedb78bf
|
Added multiplier jitter, min_snr, ability to choose sdxl encoders to use, shuffle generator, and other fun
|
2023-08-19 05:54:22 -06:00 |
|
Jaret Burkett
|
d51c4ca704
|
Added ability to use two seperate folders for datasets when doing image reference sliders
|
2023-08-18 11:44:33 -06:00 |
|
Jaret Burkett
|
8d09eb44ec
|
Fixed an issue with CFG time embeds on SDXL
|
2023-08-15 18:02:14 -06:00 |
|
Jaret Burkett
|
55a5fcc7d9
|
Added method to get specific keys from model
|
2023-08-15 14:51:04 -06:00 |
|
Jaret Burkett
|
355c80df07
|
Added ability to use civit ai url ar model name and built a model downloader and cache manager for it
|
2023-08-13 13:09:51 -06:00 |
|
Jaret Burkett
|
1487d13191
|
Moved the run job command
|
2023-08-13 10:25:56 -06:00 |
|
Jaret Burkett
|
383bad958d
|
Added a way to run as a library by passing job dict
|
2023-08-13 09:54:39 -06:00 |
|
Jaret Burkett
|
196b693cf0
|
Worked on reference slider script. It is working well currently. Still going to tune it a bit before a writeup though
|
2023-08-12 17:59:24 -06:00 |
|
Jaret Burkett
|
fd95e7b60c
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2023-08-12 05:59:58 -06:00 |
|
Jaret Burkett
|
379992d89e
|
Various bug fixes and improvements
|
2023-08-12 05:59:50 -06:00 |
|
Jaret Burkett
|
c7054d714f
|
Finally muted the annoying safety checker notification
|
2023-08-12 01:38:18 -06:00 |
|
Jaret Burkett
|
1a7e346b41
|
Added inbuild plugins and made one for image referenced. WIP
|
2023-08-10 16:02:44 -06:00 |
|
Jaret Burkett
|
df48f0a843
|
Moved some of the job config into base process so it will be easier to extend extensions
|
2023-08-10 12:14:05 -06:00 |
|
Jaret Burkett
|
bf90740b59
|
Fixed numerous issues with traing ESRGAN
|
2023-08-08 20:03:19 -06:00 |
|
Jaret Burkett
|
ff2c9f3d04
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2023-08-07 18:05:03 -06:00 |
|
Jaret Burkett
|
8bd536df7e
|
Added training for a custom version of ERSGAN arcitecture. Testing training now
|
2023-08-07 18:04:23 -06:00 |
|
Jaret Burkett
|
64fbd4c92a
|
Fix some windows dependency issues
|
2023-08-05 20:28:49 -06:00 |
|
Jaret Burkett
|
8c90fa86c6
|
Complete reqork of how slider training works and optimized it to hell. Can run entire algorythm in 1 batch now with less VRAM consumption than a quarter of it used to take
|
2023-08-05 18:46:08 -06:00 |
|
Jaret Burkett
|
7e4e660663
|
Added extensions and an example extension that merges models
|
2023-08-04 09:37:24 -06:00 |
|
Jaret Burkett
|
b865ac8b24
|
Various windows bug fixes
|
2023-08-04 05:51:58 -06:00 |
|
Jaret Burkett
|
66c6f0f6f7
|
Big refactor of SD runner and added image generator
|
2023-08-03 14:51:25 -06:00 |
|
Jaret Burkett
|
75ec5d9292
|
Hotfix to handle latest transformers clip model missing key suddenly
|
2023-08-02 12:26:18 -06:00 |
|
Jaret Burkett
|
1a25b275c8
|
Did some work on SD rescaler. Need to run a long test on it eventually.
|
2023-08-02 07:59:27 -06:00 |
|
Jaret Burkett
|
2bf3e529ce
|
Set gradient checkpointing on unet enabled by default. Help out immensly with sdxl backprop spikes
|
2023-08-01 15:43:27 -06:00 |
|
Jaret Burkett
|
8b8d53888d
|
Added Model rescale and prepared a release upgrade
|
2023-08-01 13:49:54 -06:00 |
|
Jaret Burkett
|
63cacf4362
|
Merge remote-tracking branch 'origin/main' into WIP
|
2023-07-31 15:14:28 -06:00 |
|
Jaret Burkett
|
c1b1e800df
|
Updated module names to be compatable with ney koyha unet
|
2023-07-31 13:16:29 -06:00 |
|
Jaret Burkett
|
7726911562
|
Allow name to be passed through command line
|
2023-07-31 11:56:57 -06:00 |
|
Jaret Burkett
|
1e50b39442
|
Work on slider rework
|
2023-07-28 18:11:10 -06:00 |
|
Jaret Burkett
|
5fc2bb5d9c
|
Information trainer
|
2023-07-28 08:16:29 -06:00 |
|
Jaret Burkett
|
c7640b0865
|
WIP diffusers pipeline is weird. Starting to hate sdxl
|
2023-07-27 17:35:24 -06:00 |
|
Jaret Burkett
|
b2e2e4bf47
|
Added sd1.5 and 2.1 do the diffusers pipeline flow
|
2023-07-27 12:34:48 -06:00 |
|
Jaret Burkett
|
596e57a6a6
|
Pipelines working on SDXL for noise prediction
|
2023-07-27 11:24:33 -06:00 |
|
Jaret Burkett
|
6ab8b8b0f1
|
WIP. just need to put it here
|
2023-07-27 01:46:30 -06:00 |
|
Jaret Burkett
|
d3ad195b51
|
Added rescaling, locon, sdxl, all kinds of stuff. sdxl is still weird
|
2023-07-26 16:19:50 -06:00 |
|
Jaret Burkett
|
cb70c03273
|
SDXL should be working, but I broke something where it is not converging.
|
2023-07-25 13:50:59 -06:00 |
|
Jaret Burkett
|
7032717294
|
Added ability to just erase or enhance concepts from a model
|
2023-07-24 17:33:45 -06:00 |
|
Jaret Burkett
|
61dd818608
|
Added anchors to regulate the lora
|
2023-07-24 14:59:16 -06:00 |
|
Jaret Burkett
|
390192c6a1
|
Moved the base SD training process to a parent class so we can do some more creative stuff without copy pasta
|
2023-07-24 11:29:39 -06:00 |
|
Jaret Burkett
|
e6fb0229bf
|
Added better optimizer chooised and param support
|
2023-07-24 09:21:58 -06:00 |
|
Jaret Burkett
|
9367089d48
|
Added example for slider training that will run as is
|
2023-07-23 11:24:12 -06:00 |
|
Jaret Burkett
|
434fb22458
|
Adde dnoise offset
|
2023-07-22 15:01:01 -06:00 |
|
Jaret Burkett
|
ddcd9069e1
|
Base for loopback lora training setup, still working on proper sliders
|
2023-07-21 18:26:02 -06:00 |
|
Jaret Burkett
|
0761656a90
|
Added my good ole pattern loss. God I love that thing, conv transpose pattern instantly wiped from vae
|
2023-07-20 15:44:16 -06:00 |
|
Jaret Burkett
|
982e0be7a9
|
Removed train config, updating it, and added my llvae as pytorch model
|
2023-07-20 14:12:28 -06:00 |
|
Jaret Burkett
|
557732e7ff
|
Added Critic support to VAE training. Still tweaking and working on it. Many other fixes
|
2023-07-19 15:57:32 -06:00 |
|
Jaret Burkett
|
6ada328d8d
|
Fixed issues with vgg19 preprocessing. Added yaml support on config file
|
2023-07-19 06:12:57 -06:00 |
|
Jaret Burkett
|
8d8229dfc0
|
Handle conversions back to ldm for saving
|
2023-07-18 19:34:35 -06:00 |
|
Jaret Burkett
|
17c13eef88
|
Default to only training mse. Did a lot of cleanup with script. Added logging via tensorboard.
|
2023-07-18 09:40:51 -06:00 |
|
Jaret Burkett
|
94d52572d4
|
Style and content loss working
|
2023-07-18 07:47:01 -06:00 |
|
Jaret Burkett
|
439310e4dc
|
Made a very basic vae trainer.
|
2023-07-17 19:03:50 -06:00 |
|
Jaret Burkett
|
9df7af1720
|
Adjusted some names and allow to ignore text encoder on extract
|
2023-07-16 12:52:39 -06:00 |
|
Jaret Burkett
|
8d6edae9fd
|
Added support for traditional LoRa extract using LoCon script
|
2023-07-12 19:51:40 -06:00 |
|
Jaret Burkett
|
57f14e5ef2
|
WIP implementing training
|
2023-07-12 08:23:46 -06:00 |
|
Jaret Burkett
|
47d094e528
|
Setup base for training jobs. Added sd-scripts as a submodule
|
2023-07-08 13:50:59 -06:00 |
|
Jaret Burkett
|
37354b006e
|
Reworked so everything is in classes for easy expansion. Single entry point for all config files now.
|
2023-07-08 09:51:42 -06:00 |
|
Jaret Burkett
|
27df03a486
|
Actually fixed meta data
|
2023-07-05 22:00:58 -06:00 |
|
Jaret Burkett
|
18077ba265
|
Fixed issue with meta.
|
2023-07-05 16:49:59 -06:00 |
|
Jaret Burkett
|
e4de8983c9
|
Initial commit
|
2023-07-05 16:44:58 -06:00 |