a0224793ce
Restore device runtime scales for compiled adapters ( #963 )
...
Co-authored-by: Rydén Johan <johan.ryden@bostad.uppsala.se>
2026-07-18 08:19:57 -06:00
4ece17b71f
Fix adapter scalar handling under torch.compile ( #946 )
...
* Fix adapter scalar handling under torch.compile
* Fix instance where error could happen when merging in a lora to the base model
---------
Co-authored-by: Rydén Johan <johan.ryden@bostad.uppsala.se>
Co-authored-by: Jaret Burkett <jaretburkett@gmail.com>
2026-07-15 11:07:55 -06:00
4625406093
WIP working on convrot offloading
2026-07-11 15:37:28 -06:00
1d1e21177a
Add ostris linear layer to linear layer searches.
2026-07-11 14:21:16 -06:00
6f3e0d5af2
Improved lorm extraction and training
2023-10-28 08:21:59 -06:00
002279cec3
Allow short and long caption combinations like form the new captioning system. Merge the network into the model before inference and reextract when done. Doubles inference speed on locon models during inference. allow splitting a batch into individual components and run them through alone. Basicallt gradient accumulation with single batch size.
2023-10-24 16:02:07 -06:00
cac8754399
Allow for training loras on onle one text encoder for sdxl
2023-10-06 08:11:56 -06:00
0f105690cc
Added some further extendability for plugins
2023-09-19 05:41:44 -06:00
17e4fe40d7
Prevent lycoris network moduels if not training that part of network. Skew timesteps to favor later steps. It performs better
2023-09-14 15:13:24 -06:00
569d7464d5
implemented device placement preset system more places. Vastly improved speed on setting network multiplier and activating network. Fixed timing issues on progress bar
2023-09-14 08:31:54 -06:00
be804c9cf5
Save embeddings as their trigger to match auto and comfy style loading. Also, FINALLY found why gradients were wonkey and fixed it. The root problem is dropping out of network state before backward pass.
2023-09-09 12:02:07 -06:00
408c50ead1
actually got gradient checkpointing working, again, again, maybe
2023-09-09 11:27:42 -06:00
64a5441832
Fully tested and now supporting locon on sdxl. If you have the ram
2023-09-04 14:05:10 -06:00
a4c3507a62
Added LoCON from LyCORIS
2023-09-04 08:48:07 -06:00
Zironic
Jaret Burkett