8 Commits

Author	SHA1	Message	Date
Jaret Burkett	5e84bf0d0b	Fixed issue with hidream-01 that could cause a weird nan state. Took forever to track down as it was 1 in 10 starts.	2026-05-28 12:48:24 -06:00
Jaret Burkett	e03c6e4dc9	Fix potential inconsistency with different attention mentods in hidream01	2026-05-13 09:08:11 -06:00
Jaret Burkett (Ostris)	fc4d6ebf39	Add support for fine-tuning Hidream O1 (#831) ... * Initial support for hidream. Lora keys likely need work * Fix saving for hidream-o1 * Remove dependence on flash attention for hidream o1 * Fix gradient checkpointing for hidream o1 * A lot of fixes for hidream. Handle loading and saving as comfy model. * Omit layers not used in comfy. Fix issue with lora loading keys in comfy * Version bumpo	2026-05-12 11:15:16 -06:00
Jaret Burkett	cefa2ca5fe	Added initial support for Hidream E1 training	2025-07-27 15:12:56 -06:00
Jaret Burkett	524bd2edfc	Make flash attn optional. Handle larger batch sizes.	2025-04-14 14:34:46 +00:00
Jaret Burkett	3a5ea2c742	Remove some moe stuff for finetuning. Drastically reduces vram usage	2025-04-14 00:57:34 +00:00
Jaret Burkett	f80cf99f40	Hidream is training, but has a memory leak	2025-04-13 23:28:18 +00:00
Jaret Burkett	594e166ca3	Initial support for hidream. Still a WIP	2025-04-13 13:50:11 -06:00