mirror of https://github.com/razor-ai/soup.git
66be481ade
Seven of the eight configs in examples/configs/ did not parse. They were
written against a pre-nesting schema and stayed that way through several
schema changes, so `soup train --config examples/configs/sft_basic.yaml` --
the first command examples/README.md tells you to run -- failed validation
with "base: Field required; data -> train: Field required".
model: -> base:
data.path: -> data.train:
max_seq_length: -> data.max_length
quantization: -> training.quantization ("null" -> "none", int8 -> 8bit)
lora_r/alpha/... -> training.lora.{r,alpha,dropout,target_modules}
num_epochs -> training.epochs
learning_rate -> training.lr
lr_scheduler_type -> training.scheduler
output_dir: -> output:
Dropped keys with no schema equivalent: seed, eval_steps, eval_strategy,
save_strategy, load_best_model_at_end.
Three correctness fixes beyond the mechanical migration:
- dpo_chat / rlhf_step2_reward declared `format: sharegpt`, but
chat_preferences.jsonl is prompt/chosen/rejected -- detect_format() calls
it dpo, and a reward model needs the pair shape. Now `format: dpo`.
- target_modules listed `out_proj`, which no Llama has (it is `o_proj`), so
PEFT would have raised on the two configs using it.
- vision_llama pointed at examples/data/vision_dataset.jsonl and
examples/data/images/, neither of which has ever existed in this repo. It
is now marked a TEMPLATE in its first line, with placeholder paths and the
reason there is no fixture: we do not commit image files. Its base model id
was `llama-vision-13b`, which is not a real repo id; now
meta-llama/Llama-3.2-11B-Vision-Instruct, matching the vision template.
batch_size and max_length on the five TinyLlama configs were leftovers that
made them unrunnable rather than tuned choices: batch_size 16 against a
10-row fixture never forms a single batch, and max_length 2048 against a
longest-row of ~233 tokens only inflated the activation budget. The VRAM
pre-flight refused all five on a 4 GB card. Now batch_size 4 / max_length
512, with the reason written above the line so the number does not drift
back anonymously. Verified: all five clear `soup train --dry-run` on a 4 GB
card and report their data as valid. dpo_example is unchanged (its values
are pinned by tests/test_dpo_example.py).
All 8 configs now pass load_config_from_string.
|
||
|---|---|---|
| .. | ||
| dpo_chat.yaml | ||
| dpo_example.yaml | ||
| grpo_reasoning.yaml | ||
| rlhf_step1_sft.yaml | ||
| rlhf_step2_reward.yaml | ||
| rlhf_step3_ppo.yaml | ||
| sft_basic.yaml | ||
| vision_llama.yaml | ||