Jaret Burkett
|
6f3e0d5af2
|
Improved lorm extraction and training
|
2023-10-28 08:21:59 -06:00 |
|
Jaret Burkett
|
002279cec3
|
Allow short and long caption combinations like form the new captioning system. Merge the network into the model before inference and reextract when done. Doubles inference speed on locon models during inference. allow splitting a batch into individual components and run them through alone. Basicallt gradient accumulation with single batch size.
|
2023-10-24 16:02:07 -06:00 |
|
Jaret Burkett
|
cac8754399
|
Allow for training loras on onle one text encoder for sdxl
|
2023-10-06 08:11:56 -06:00 |
|
Jaret Burkett
|
569d7464d5
|
implemented device placement preset system more places. Vastly improved speed on setting network multiplier and activating network. Fixed timing issues on progress bar
|
2023-09-14 08:31:54 -06:00 |
|
Jaret Burkett
|
be804c9cf5
|
Save embeddings as their trigger to match auto and comfy style loading. Also, FINALLY found why gradients were wonkey and fixed it. The root problem is dropping out of network state before backward pass.
|
2023-09-09 12:02:07 -06:00 |
|
Jaret Burkett
|
408c50ead1
|
actually got gradient checkpointing working, again, again, maybe
|
2023-09-09 11:27:42 -06:00 |
|
Jaret Burkett
|
436bf0c6a3
|
Added experimental concept replacer, replicate converter, bucket maker, and other goodies
|
2023-09-06 18:50:32 -06:00 |
|
Jaret Burkett
|
f84500159c
|
Fixed issue with lora layer check
|
2023-09-04 14:27:37 -06:00 |
|
Jaret Burkett
|
64a5441832
|
Fully tested and now supporting locon on sdxl. If you have the ram
|
2023-09-04 14:05:10 -06:00 |
|
Jaret Burkett
|
a4c3507a62
|
Added LoCON from LyCORIS
|
2023-09-04 08:48:07 -06:00 |
|
Jaret Burkett
|
fa8fc32c0a
|
Corrected key saving and loading to better match kohya
|
2023-09-04 00:22:34 -06:00 |
|
Jaret Burkett
|
bd758ff203
|
Cleanup and small bug fixes
|
2023-08-29 05:45:49 -06:00 |
|
Jaret Burkett
|
71da78c8af
|
improved normalization for a network with varrying batch network weights
|
2023-08-28 12:42:57 -06:00 |
|
Jaret Burkett
|
c446f768ea
|
Huge memory optimizations, many big fixes
|
2023-08-27 17:48:02 -06:00 |
|
Jaret Burkett
|
9b164a8688
|
Fixed issue with bucket dataloader corpping in too much. Added normalization capabilities to LoRA modules. Testing effects, but should prevent them from burning and also make them more compatable with stacking many LoRAs
|
2023-08-27 09:40:01 -06:00 |
|
Jaret Burkett
|
379992d89e
|
Various bug fixes and improvements
|
2023-08-12 05:59:50 -06:00 |
|
Jaret Burkett
|
8c90fa86c6
|
Complete reqork of how slider training works and optimized it to hell. Can run entire algorythm in 1 batch now with less VRAM consumption than a quarter of it used to take
|
2023-08-05 18:46:08 -06:00 |
|
Jaret Burkett
|
1a25b275c8
|
Did some work on SD rescaler. Need to run a long test on it eventually.
|
2023-08-02 07:59:27 -06:00 |
|
Jaret Burkett
|
cb70c03273
|
SDXL should be working, but I broke something where it is not converging.
|
2023-07-25 13:50:59 -06:00 |
|
Jaret Burkett
|
ddcd9069e1
|
Base for loopback lora training setup, still working on proper sliders
|
2023-07-21 18:26:02 -06:00 |