Release v0.10.0: Phase 6.1 - Community (CONTRIBUTING, CODE_OF_CONDUCT, SECURITY, examples, FUNDING)

This commit is contained in:
Alpamys 2026-03-23 23:10:45 +05:00
parent cc030df391
commit e0f8e921bd
18 changed files with 1071 additions and 2 deletions

2
.github/FUNDING.yml vendored Normal file
View File

@ -0,0 +1,2 @@
github: MakazhanAlpamys
buy_me_a_coffee: makazhanalpamys

View File

@ -89,6 +89,8 @@ soup train --config soup.yaml
**Web UI:** `commands/ui.py` launches a local web interface via `soup ui`. `ui/app.py` creates a FastAPI app with REST API endpoints for experiment management (`/api/runs`, `/api/runs/{id}/metrics`), config validation (`/api/config/validate`), training control (`/api/train/start`, `/api/train/status`, `/api/train/stop`), data inspection (`/api/data/inspect`), system info (`/api/system`), and templates (`/api/templates`). `ui/static/` contains a self-contained SPA (HTML/CSS/JS) with four pages: Dashboard (experiments list, loss charts via Chart.js), New Training (config editor with templates), Data Explorer (browse datasets), and Model Chat (chat with a `soup serve` instance). Config validation uses `config/loader.py`'s `load_config_from_string()`. Requires `pip install 'soup-cli[ui]'`. Auto-opens browser on launch (disable with `--no-browser`).
**Community & Examples:** `CONTRIBUTING.md` provides full contributor guide (fork, test, lint, code style, PR process). `CODE_OF_CONDUCT.md` uses Contributor Covenant v2.1. `SECURITY.md` documents responsive disclosure policy (48h response, severity levels). `.github/FUNDING.yml` adds GitHub Sponsors + Buy Me a Coffee buttons. `examples/` folder includes 7 real YAML configs (SFT, DPO, GRPO, Vision, RLHF 3-stage), 3 sample JSONL datasets, and comprehensive README with quick-start examples.
**Confirmation prompts:** `commands/train.py` and `commands/sweep.py` ask for confirmation before starting. Skip with `--yes` / `-y`.
**Version:** `cli.py` `version()` command supports `--full` flag that shows version, Python version, GPU backend, and installed optional extras in one line.

77
CODE_OF_CONDUCT.md Normal file
View File

@ -0,0 +1,77 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our community include:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
- Focusing on what is best not just for us as individuals, but for the overall community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email address, without their explicit permission
- Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at **conduct@soup-cli.dev**. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leadership, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of actions.
**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 2.1, available at https://www.contributor-covenant.org/version/2_1/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder](https://github.com/mozilla/diversity).
For answers to common questions about this code of conduct, see the FAQ at https://www.contributor-covenant.org/faq. Translations are available at https://www.contributor-covenant.org/translations.

319
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,319 @@
# Contributing to Soup
Thank you for your interest in contributing to Soup! We welcome bug reports, feature requests, and pull requests from the community.
## Getting Started
### 1. Fork & Clone
```bash
git clone https://github.com/YOUR-USERNAME/Soup.git
cd Soup
```
### 2. Set Up Development Environment
Install the project in editable mode with dev dependencies:
```bash
pip install -e ".[dev]"
```
This installs:
- `pytest` for testing
- `ruff` for linting
- `pytest-cov` for coverage
- `httpx` for HTTP testing
### 3. Verify Setup
Run the test suite to confirm everything works:
```bash
pytest tests/ -v --tb=short
```
Run the linter:
```bash
ruff check soup_cli/ tests/
```
## Code Style
We use **ruff** for all code style and linting. Before committing, run:
```bash
# Check for issues
ruff check soup_cli/ tests/
# Auto-fix issues
ruff check --fix soup_cli/ tests/
```
### Style Guidelines
- **Line length:** 100 characters (enforced by ruff)
- **Imports:** Sorted and organized (ruff I rule)
- **Naming:** No single-letter variable names (ruff E741) — use `entry`, `part`, `length` instead of `l`, `p`, etc.
- **Lazy imports:** Heavy dependencies (torch, transformers, peft, trl, etc.) should be imported inside functions, not at module level, to keep the CLI responsive
- **Config validation:** Always use Pydantic v2 with `BaseModel` and `Field`
- **Output:** Use `rich.console.Console` for all output — never bare `print()`
- **Type hints:** Always include type hints for function parameters and return values
Example:
```python
# ❌ WRONG
from torch import cuda
import transformers
def train():
print("Starting training")
model = transformers.AutoModel.from_pretrained("llama-7b")
# ✅ CORRECT
def train():
from torch import cuda
import transformers
console = Console()
console.print("Starting training")
model = transformers.AutoModel.from_pretrained("llama-7b")
```
## Project Structure
Key directories:
```
soup_cli/
cli.py - Main entry point, command routing
commands/ - Command implementations (train, chat, eval, etc.)
config/ - Config schema (schema.py) and loader (loader.py)
data/ - Data loading and format conversion
trainer/ - Training wrappers (SFT, DPO, GRPO, PPO, reward_model)
monitoring/ - Callbacks and live dashboard
experiment/ - SQLite experiment tracking
utils/ - GPU detection, batch size estimation, error handling
ui/ - Web UI (FastAPI + HTML/JS)
templates/ - YAML config templates (chat, code, medical, vision, rlhf, reasoning)
tests/ - Test suite (40+ files, 600+ tests)
examples/ - Real-world config examples and datasets
```
## Running Tests
### All Tests
```bash
pytest tests/ -v --tb=short
```
### Single Test File
```bash
pytest tests/test_config.py -v
```
### Single Test
```bash
pytest tests/test_data.py::test_detect_alpaca_format -v
```
### With Coverage
```bash
pytest tests/ --cov=soup_cli --cov-report=html
```
### Test Categories
- `test_config.py` — Config loading and validation
- `test_data.py` — Data format detection and conversion
- `test_trainer_*.py` — Individual trainer tests
- `test_smoke_train.py` — Full pipeline tests (GPU required)
- `test_cli.py` — Command-line interface tests
- `test_errors.py` — Error message handling
- `test_*_command.py` — Specific command tests (chat, push, eval, etc.)
## Making Changes
### 1. Create a Branch
```bash
git checkout -b feature/your-feature-name
# or
git checkout -b fix/your-bug-fix
```
### 2. Make Your Changes
- Write code following the style guidelines above
- Add tests for new functionality
- Update docstrings and comments
- Keep commits focused and logical
### 3. Run Tests & Lint
Before pushing, ensure everything passes:
```bash
# Lint first
ruff check --fix soup_cli/ tests/
# Then run tests
pytest tests/ -v --tb=short
```
If you've added new test files, increase the test count in `plan.md`.
### 4. Commit
Write clear, descriptive commit messages:
```bash
git add .
git commit -m "Add feature: descriptive message"
```
### 5. Push & Open a PR
```bash
git push origin feature/your-feature-name
```
Then open a pull request on GitHub with:
- Clear title describing the change
- Description of what and why
- Reference any related issues (e.g., "Closes #123")
- Test results
## Submitting a Pull Request
### PR Template
Please use the following structure:
```markdown
## What's this PR about?
Brief description of the change.
## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Documentation
- [ ] Performance improvement
## Testing
Describe how you tested this (e.g., `pytest tests/test_X.py -v`).
## Checklist
- [ ] Linting passes: `ruff check soup_cli/ tests/`
- [ ] Tests pass: `pytest tests/ -v`
- [ ] New tests added for new functionality
- [ ] Docstrings and comments added
- [ ] No breaking changes (or documented)
## Related Issues
Closes #123 (if applicable)
```
## Architecture & Design Decisions
### Lazy Imports for Speed
Heavy ML imports (torch, transformers, trl) are imported inside command handlers so the CLI stays fast. Users can run `soup version` or `soup --help` instantly without waiting for PyTorch to load.
### Pydantic for Config Validation
All YAML configs are validated using Pydantic v2 models. These models are the single source of truth for valid fields and defaults. See `config/schema.py`.
### Trainers as Wrappers
`trainer/sft.py`, `trainer/dpo.py`, `trainer/grpo.py`, `trainer/ppo.py` wrap HuggingFace TRL trainers with:
- Auto quantization (BitsAndBytes, torchao QAT)
- Auto LoRA setup (PEFT)
- Auto batch size estimation
- Progress bar integration
### Experiment Tracking is SQLite
No external dependencies required. All runs, metrics, and eval results go to `~/.soup/experiments.db`.
### Data Format Normalization
Multiple formats (Alpaca, ShareGPT, ChatML, LLaVA, ShareGPT4V) are normalized to a unified `{"messages": [...]}` structure in `data/formats.py`.
## Adding a New Feature
### 1. New Training Task Type
If adding a new training algorithm (e.g., DPO, GRPO):
1. Create `trainer/your_trainer.py` with a class inheriting from `BaseTrainer`
2. Add Pydantic config class to `config/schema.py`
3. Add template to `templates/your.yaml` and `config/schema.py`
4. Update `commands/train.py` to route to your trainer
5. Add 30+ tests in `tests/test_your_trainer.py`
6. Update `CLAUDE.md` and `README.md`
### 2. New Data Format
1. Add detection and conversion logic to `data/formats.py`
2. Add tests in `tests/test_formats.py`
3. Update `data/loader.py` if needed
4. Document in `CLAUDE.md`
### 3. New Command
1. Create `commands/your_command.py` with a handler function
2. Register in `soup_cli/cli.py` with `@app.command()`
3. Add tests in `tests/test_your_command.py`
4. Update help text and README
## CI/CD
GitHub Actions runs on every push:
- **ruff** linting (must pass)
- **pytest** on Python 3.9, 3.11, 3.12 (must pass)
See `.github/workflows/ci.yml`.
## Releases
The project follows semantic versioning: `MAJOR.MINOR.PATCH`
### Version Bump Process
1. Update version in `pyproject.toml` and `soup_cli/__init__.py`
2. Run full test suite and linting
3. Update `CLAUDE.md`, `README.md`, `plan.md`
4. Commit with message: `Release v0.X.0`
5. Tag: `git tag v0.X.0 && git push --tags`
6. GitHub Actions auto-publishes to PyPI
See `CLAUDE.md` for the complete release checklist.
## Community
- **Issues:** Report bugs and request features on [GitHub Issues](https://github.com/MakazhanAlpamys/Soup/issues)
- **Discussions:** Ask questions on [GitHub Discussions](https://github.com/MakazhanAlpamys/Soup/discussions)
- **Code of Conduct:** Please read [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)
- **Security:** Report security issues to [SECURITY.md](SECURITY.md)
## Questions?
- Check the [README](README.md) for quick start and features
- Check [CLAUDE.md](CLAUDE.md) for architecture details
- Open a GitHub Discussion for questions
- Join the community on Reddit ([r/LocalLLaMA](https://www.reddit.com/r/LocalLLaMA/))
Thank you for contributing! 🍲

150
SECURITY.md Normal file
View File

@ -0,0 +1,150 @@
# Security Policy
## Supported Versions
We provide security updates for the following versions:
- **Latest minor version:** Active support (e.g., v0.9.x)
- **Previous minor versions:** Bug-fix support only
- **Versions older than 3 minor versions:** No support
Example:
- v0.9.0-0.9.x → Full support (latest)
- v0.8.0-0.8.x → Bug-fix support only
- v0.7.x and below → No support
## Reporting a Vulnerability
**Do not** open a public issue or pull request for security vulnerabilities.
Instead, email your findings to **security@soup-cli.dev** with:
1. **Description**: A clear explanation of the vulnerability
2. **Steps to Reproduce**: How to trigger or demonstrate the issue
3. **Affected Versions**: Which Soup versions are impacted
4. **Suggested Fix** (optional): Any proposed solutions
5. **Contact Info**: Your email for follow-up (optional)
### What to Include
```
To: security@soup-cli.dev
Subject: Security Vulnerability Report: [Brief Title]
Description:
[Explain the vulnerability in detail]
Affected Component:
[e.g., data/loader.py, trainer/sft.py, etc.]
Steps to Reproduce:
1. [Step 1]
2. [Step 2]
3. ...
Impact:
[What could go wrong? Data exposure? RCE? DoS?]
Suggested Fix (optional):
[Your proposed solution, if any]
```
## Response Timeline
- **Initial Response**: Within 48 hours
- **Assessment**: 1-3 business days
- **Fix Development**: Varies by severity
- **Patch Release**: As soon as possible after fix verification
- **Public Disclosure**: Coordinated with reporter (typically 90 days after patch release)
## Severity Levels
- **Critical**: Remote code execution, data exposure, complete compromise (patch within 24-48 hours)
- **High**: Authentication bypass, privilege escalation, denial of service (patch within 1 week)
- **Medium**: Information disclosure, partial compromise (patch within 2 weeks)
- **Low**: Minor issues with limited impact (patch in next regular release)
## Security Best Practices
When using Soup, follow these practices to stay secure:
### 1. Keep Soup Updated
```bash
pip install --upgrade soup-cli
```
### 2. Protect API Keys
Never commit API keys or secrets to version control. Use environment variables:
```bash
export HUGGINGFACE_TOKEN=your_token_here
export WANDB_API_KEY=your_key_here
soup train
```
### 3. Validate Data
- Only use trusted datasets
- Verify checksums for large datasets
- Inspect data for malicious content before training
### 4. Model Permissions
- Be cautious when downloading models from untrusted sources
- Use model hub providers with verified publishers (HuggingFace, Meta, etc.)
- Keep track of which models you've fine-tuned and their base model sources
### 5. GPU/Compute Safety
- Run on isolated machines if training on sensitive data
- Clear cache and temporary files after training
- Don't share fine-tuned models containing sensitive information
## Known Vulnerabilities
We maintain a log of known security issues and their fixes. This will be updated as issues are discovered and resolved.
### Current Status
No known critical vulnerabilities in current releases.
## Security Scanning
- All code is scanned with `ruff` for style and common issues
- Dependencies are regularly updated to patch known CVEs
- GitHub's dependency scanning alerts us to vulnerable dependencies
- We use GitHub Actions CI/CD for continuous integration
## Dependency Updates
We actively monitor and update dependencies:
- Major dependency updates: Tested in PR before merging
- Security patches: Applied immediately and released as patch versions
- Deprecated dependencies: Replaced proactively
## Coming Soon
- [x] Automated dependency scanning
- [ ] SBOM (Software Bill of Materials) for each release
- [ ] Third-party security audit (after 1.0.0 release)
## Questions?
If you have security questions (not vulnerability reports) or need clarification:
- Open a GitHub Discussion tagged `security`
- Email us at support@soup-cli.dev (non-vulnerability inquiries)
- Check our [CONTRIBUTING.md](CONTRIBUTING.md) for general support
## License
This Security Policy is provided under the MIT license, same as the Soup project.
---
**Last Updated**: March 2026
For the latest version of this policy, visit: https://github.com/MakazhanAlpamys/Soup/blob/main/SECURITY.md

278
examples/README.md Normal file
View File

@ -0,0 +1,278 @@
# Soup Examples
Real-world configuration examples and sample datasets to get you running quickly.
## Quick Start with Examples
### 1. Basic SFT (Supervised Fine-Tuning)
Fine-tune TinyLlama on a small instruction-following dataset:
```bash
soup train --config examples/configs/sft_basic.yaml
```
**What it does:**
- Trains TinyLlama-1.1B for 1 epoch
- Uses LoRA for efficient memory usage
- Outputs to `./output_sft_basic/`
- Takes ~2-3 minutes on a consumer GPU
### 2. Chat Assistant (DPO)
Train a chat model with preference learning:
```bash
soup train --config examples/configs/dpo_chat.yaml
```
**What it does:**
- Uses Llama 2 7B base model
- Trains with DPO (Direct Preference Optimization) on chat preferences
- Better alignment than SFT alone
- Outputs to `./output_dpo_chat/`
### 3. Reasoning Model (GRPO)
Fine-tune a reasoning model with step-by-step answer verification:
```bash
soup train --config examples/configs/grpo_reasoning.yaml
```
**What it does:**
- Trains on reasoning tasks (math, logic)
- Uses GRPO (Group Relative Policy Optimization) to optimize for correctness
- Generates multiple outputs per prompt and selects the best
- Outputs to `./output_reasoning/`
### 4. Vision Model
Fine-tune LLaMA-Vision on image-caption pairs:
```bash
soup train --config examples/configs/vision_llama.yaml
```
**What it does:**
- Trains LLaMA-3.2-Vision-90B on image-text data
- Uses LLaVA format for images + text
- Outputs to `./output_vision/`
### 5. Full RLHF Pipeline
Complete reinforcement learning from human feedback:
```bash
# Step 1: Pre-train with SFT
soup train --config examples/configs/rlhf_step1_sft.yaml
# Step 2: Train a reward model
soup train --config examples/configs/rlhf_step2_reward.yaml
# Step 3: PPO with reward model
soup train --config examples/configs/rlhf_step3_ppo.yaml
```
## Dataset Formats
Datasets are included in JSONL format. Soup auto-detects and normalizes:
- **Alpaca**: `instruction`, `input`, `output` fields
- **ShareGPT**: `conversations` with `from`/`value` fields
- **ChatML**: OpenAI-style `messages` with `role`/`content`
- **LLaVA**: Vision format with `image` + `conversations`
### Example: Inspect a Dataset
```bash
soup data inspect examples/data/alpaca_tiny.jsonl
```
Output:
```
📊 Dataset Statistics
Format detected: alpaca
Total entries: 50
Sample 1:
instruction: "Identify the odd one out"
input: "twitter, instagram, skype"
output: "skype"
```
### Example: Convert Between Formats
```bash
# Convert Alpaca to ChatML
soup data convert examples/data/alpaca_tiny.jsonl \
--from alpaca --to chatml \
--output alpaca_as_chatml.jsonl
```
## Directory Structure
```
examples/
configs/ # YAML configuration files
sft_basic.yaml
dpo_chat.yaml
grpo_reasoning.yaml
vision_llama.yaml
rlhf_step1_sft.yaml
rlhf_step2_reward.yaml
rlhf_step3_ppo.yaml
data/ # Sample datasets (JSONL)
alpaca_tiny.jsonl
chat_multichat.jsonl
reasoning_math.jsonl
vision_images.tar.gz (with image files)
```
## Using Your Own Data
1. **Prepare data** in one of the supported formats
2. **Update the config** with your data path:
```yaml
data:
path: /path/to/your/data.jsonl
format: alpaca # or sharegpt, chatml, llava
```
3. **Run training**:
```bash
soup train --config your_config.yaml
```
## Tips & Tricks
### Save Space: Use Quantization
Add quantization to reduce model size:
```yaml
quantization: int8 # Reduces memory by 4x
```
### Speed Up Training: Use Unsloth Backend
Unsloth is 2-5x faster training:
```bash
pip install 'soup-cli[fast]'
```
Then in your config:
```yaml
backend: unsloth
```
### Monitor Training: Use Weights & Biases
Enable W&B logging:
```bash
pip install wandb
soup train --config your_config.yaml --wandb
```
### Export for Inference: Convert to GGUF
After training, convert for Ollama/llama.cpp:
```bash
soup export output_sft_basic/ --output model.gguf --quant q8_0
```
Then use with Ollama:
```bash
ollama create my-model -f Ollama.modelfile
```
### Merge LoRA Adapter
Merge your LoRA adapter into a standalone model:
```bash
soup merge output_sft_basic/ --output merged_model/
```
## Common Issues
### "CUDA out of memory"
- Reduce `batch_size` in config
- Enable quantization: `quantization: int8`
- Use smaller model: Mistral-7B instead of Llama-70B
### "Dataset not found"
- Check file path in config (use absolute path if unsure)
- Verify format is correct: `soup data inspect your_data.jsonl`
### "Model not found on Hugging Face"
- Check model ID spelling
- Ensure you have HuggingFace token: `huggingface-cli login`
- Or use a different model that's publicly available
## Creating Your Own Configs
### Minimal Config Template
```yaml
model: tinyllama-1.1b
data:
path: ./your_data.jsonl
format: alpaca
task: sft
lora_r: 16
lora_alpha: 32
batch_size: 32
num_epochs: 3
learning_rate: 5e-4
output_dir: ./output/
```
### Advanced Config Template
```yaml
model: llama-2-7b
data:
path: ./dataset.jsonl
format: sharegpt
task: dpo
backend: unsloth
quantization: int8
lora_r: 64
lora_alpha: 128
lora_dropout: 0.05
batch_size: 16
gradient_accumulation_steps: 4
num_epochs: 2
learning_rate: 1e-4
warmup_ratio: 0.1
max_seq_length: 2048
output_dir: ./output_advanced/
```
See [config schema documentation](../CLAUDE.md#config-system) for all available options.
## Learn More
- **README**: [Main documentation](../README.md)
- **CONTRIBUTING**: [How to contribute](../CONTRIBUTING.md)
- **CLAUDE.md**: [Architecture and detailed docs](../CLAUDE.md)
## Questions?
- Check the [GitHub Discussions](https://github.com/MakazhanAlpamys/Soup/discussions)
- Open an [Issue](https://github.com/MakazhanAlpamys/Soup/issues)
- Read [SECURITY.md](../SECURITY.md) for security questions
Happy training! 🍲

View File

@ -0,0 +1,35 @@
# DPO Chat Example
# Train a chat model with Direct Preference Optimization
# Uses Llama2-7B and preference data (chosen vs rejected)
model: meta-llama/Llama-2-7b-chat-hf
data:
path: examples/data/chat_preferences.jsonl
format: sharegpt
task: dpo
backend: transformers
quantization: int8
lora_r: 64
lora_alpha: 128
lora_dropout: 0.05
lora_target_modules:
- q_proj
- v_proj
- k_proj
- out_proj
batch_size: 8
gradient_accumulation_steps: 2
num_epochs: 2
learning_rate: 5e-4
lr_scheduler_type: cosine
warmup_ratio: 0.1
weight_decay: 0.01
max_seq_length: 2048
output_dir: ./output_dpo_chat/
seed: 42
logging_steps: 10
save_steps: 100
eval_steps: 100
eval_strategy: steps
load_best_model_at_end: true
dpo_beta: 0.1

View File

@ -0,0 +1,34 @@
# GRPO Reasoning Example
# Fine-tune a reasoning model (math, logic, step-by-step)
# Uses Group Relative Policy Optimization to select best outputs
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
data:
path: examples/data/reasoning_math.jsonl
format: alpaca
task: grpo
backend: transformers
quantization: null
lora_r: 64
lora_alpha: 128
lora_dropout: 0.05
lora_target_modules:
- q_proj
- v_proj
batch_size: 8
gradient_accumulation_steps: 2
num_epochs: 2
learning_rate: 1e-4
lr_scheduler_type: cosine
warmup_ratio: 0.1
weight_decay: 0.01
max_seq_length: 2048
output_dir: ./output_reasoning/
seed: 42
grpo_beta: 0.05
num_generations: 4
reward_fn: accuracy # built-in reward function
logging_steps: 10
save_steps: 50
eval_steps: 50
eval_strategy: steps

View File

@ -0,0 +1,30 @@
# RLHF Step 1: Supervised Fine-Tuning (SFT)
# Pre-train the model before reward model training
# Use high-quality examples
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
data:
path: examples/data/alpaca_tiny.jsonl
format: alpaca
task: sft
backend: transformers
quantization: null
lora_r: 64
lora_alpha: 128
lora_dropout: 0.05
lora_target_modules:
- q_proj
- v_proj
batch_size: 16
gradient_accumulation_steps: 1
num_epochs: 1
learning_rate: 5e-4
lr_scheduler_type: linear
warmup_ratio: 0.05
weight_decay: 0.0
max_seq_length: 2048
output_dir: ./output_rlhf_sft/
seed: 42
logging_steps: 10
save_steps: 50
save_strategy: steps

View File

@ -0,0 +1,24 @@
# RLHF Step 2: Train Reward Model
# Learn to score responses: is this output good or bad?
# Uses preference pairs (chosen vs rejected)
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
data:
path: examples/data/chat_preferences.jsonl # must have chosen/rejected
format: sharegpt
task: reward_model
backend: transformers
quantization: null
batch_size: 8
gradient_accumulation_steps: 2
num_epochs: 2
learning_rate: 1e-4
lr_scheduler_type: cosine
warmup_ratio: 0.1
weight_decay: 0.01
max_seq_length: 2048
output_dir: ./output_reward_model/
seed: 42
logging_steps: 10
save_steps: 50
save_strategy: steps

View File

@ -0,0 +1,33 @@
# RLHF Step 3: PPO Training
# Optimize with reinforcement learning using the reward model
# Generates outputs and ranks them with the reward model
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
data:
path: examples/data/alpaca_tiny.jsonl
format: alpaca
task: ppo
backend: transformers
quantization: null
lora_r: 64
lora_alpha: 128
lora_dropout: 0.05
lora_target_modules:
- q_proj
- v_proj
batch_size: 8
gradient_accumulation_steps: 2
num_epochs: 2
learning_rate: 1e-5
lr_scheduler_type: cosine
warmup_ratio: 0.1
weight_decay: 0.01
max_seq_length: 2048
output_dir: ./output_ppo/
seed: 42
ppo_epochs: 4
ppo_clip_ratio: 0.2
ppo_kl_penalty: 0.05
reward_model: ./output_reward_model/
logging_steps: 10
save_steps: 50

View File

@ -0,0 +1,32 @@
# SFT Basic Example
# Fine-tune TinyLlama-1.1B on instruction-following data
# Quick to train (2-3 minutes on consumer GPU) — perfect for testing
model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
data:
path: examples/data/alpaca_tiny.jsonl
format: alpaca
task: sft
backend: transformers
quantization: null
lora_r: 16
lora_alpha: 32
lora_dropout: 0.05
lora_target_modules:
- q_proj
- v_proj
batch_size: 16
gradient_accumulation_steps: 1
num_epochs: 1
learning_rate: 5e-4
lr_scheduler_type: linear
warmup_ratio: 0.05
weight_decay: 0.0
max_seq_length: 512
output_dir: ./output_sft_basic/
seed: 42
logging_steps: 10
save_steps: 50
eval_steps: 50
eval_strategy: steps
load_best_model_at_end: false

View File

@ -0,0 +1,33 @@
# Vision Fine-tuning Example
# Train LLaMA-Vision on image-caption pairs
# Uses LLaVA format (image + conversation)
model: llama-vision-13b # LLaMA-3.2-Vision-90B or similar
data:
path: examples/data/vision_dataset.jsonl
format: llava
image_dir: examples/data/images/
task: sft
modality: vision
backend: transformers
quantization: int8
lora_r: 128
lora_alpha: 256
lora_dropout: 0.1
lora_target_modules:
- q_proj
- v_proj
- k_proj
- out_proj
batch_size: 4
gradient_accumulation_steps: 4
num_epochs: 2
learning_rate: 1e-4
lr_scheduler_type: cosine
warmup_ratio: 0.1
weight_decay: 0.01
max_seq_length: 4096
output_dir: ./output_vision/
seed: 42
logging_steps: 10
save_steps: 50

View File

@ -0,0 +1,10 @@
{"instruction": "Classify the sentiment of this sentence.", "input": "I love this product, it's amazing!", "output": "Positive"}
{"instruction": "Translate to Spanish.", "input": "Hello, how are you?", "output": "Hola, ¿cómo estás?"}
{"instruction": "Write a haiku about nature.", "input": "", "output": "Green leaves dance in wind\nWhispering ancient forest\nPeace in silence dwells"}
{"instruction": "Explain quantum computing in simple terms.", "input": "", "output": "Quantum computers use quantum bits (qubits) instead of normal bits. Unlike regular bits which are 0 or 1, qubits can be both at once (superposition). This allows quantum computers to solve certain problems much faster than regular computers."}
{"instruction": "Fix the grammar in this sentence.", "input": "She go to the store yesterday.", "output": "She went to the store yesterday."}
{"instruction": "List 3 benefits of exercise.", "input": "", "output": "1. Improves cardiovascular health and strengthens the heart.\n2. Helps maintain a healthy weight and builds muscle.\n3. Reduces stress and improves mental health and mood."}
{"instruction": "What is the capital of France?", "input": "", "output": "The capital of France is Paris."}
{"instruction": "Summarize this text in one sentence.", "input": "Machine learning is a subset of artificial intelligence that enables computers to learn from data without being explicitly programmed. It uses algorithms to identify patterns in data and improve through experience.", "output": "Machine learning allows computers to learn patterns from data without explicit programming."}
{"instruction": "Create a Python function to add two numbers.", "input": "", "output": "def add_numbers(a, b):\n \"\"\"Add two numbers and return the result.\"\"\"\n return a + b\n\n# Example usage\nresult = add_numbers(5, 3)\nprint(result) # Output: 8"}
{"instruction": "Identify the main idea of this paragraph.", "input": "Climate change is caused by increased greenhouse gases in the atmosphere, primarily from human activities like burning fossil fuels. This leads to global warming, which affects weather patterns, sea levels, and ecosystems.", "output": "Greenhouse gases from human activities cause climate change and have significant environmental impacts."}

View File

@ -0,0 +1,5 @@
{"conversations": [{"from": "user", "value": "What is machine learning?"}, {"from": "assistant", "value": "Machine learning is a field of artificial intelligence where systems learn from data automatically. It uses algorithms to identify patterns and improve performance through experience, without being explicitly programmed for every task."}], "chosen": true}
{"conversations": [{"from": "user", "value": "How do I learn Python?"}, {"from": "assistant", "value": "Here are effective ways to learn Python:\n1. Start with fundamentals: variables, loops, functions\n2. Practice with small projects\n3. Use resources like Python.org docs, tutorials\n4. Build projects to apply what you learn\n5. Join communities for support"}], "chosen": true}
{"conversations": [{"from": "user", "value": "What's the difference between AI and ML?"}, {"from": "assistant", "value": "AI (Artificial Intelligence) is the broader field of creating intelligent systems. ML (Machine Learning) is a subset of AI focused on learning from data. All machine learning is AI, but not all AI uses machine learning."}], "chosen": true}
{"conversations": [{"from": "user", "value": "Explain neural networks simply."}, {"from": "assistant", "value": "Neural networks are inspired by the human brain. They consist of layers of connected nodes (neurons) that process information. Each connection has a weight that adjusts during training. This allows the network to learn complex patterns in data."}], "chosen": true}
{"conversations": [{"from": "user", "value": "How should I structure a machine learning project?"}, {"from": "assistant", "value": "Follow this structure:\n1. Define the problem and gather data\n2. Explore and clean the data\n3. Choose and train a model\n4. Evaluate performance\n5. Iterate and improve\n6. Deploy to production\n7. Monitor and maintain"}], "chosen": true}

View File

@ -0,0 +1,5 @@
{"instruction": "Solve this math problem: 7 + 5 * 2 =", "input": "", "output": "To solve 7 + 5 * 2, I need to follow the order of operations (PEMDAS).\n\nStep 1: Multiply first (5 * 2)\n5 * 2 = 10\n\nStep 2: Add (7 + 10)\n7 + 10 = 17\n\nAnswer: 17"}
{"instruction": "Explain the Pythagorean theorem and solve an example.", "input": "", "output": "The Pythagorean theorem states that in a right triangle, a² + b² = c², where c is the hypotenuse (longest side).\n\nExample: If a = 3 and b = 4, find c\n\nStep 1: Substitute into formula\n3² + 4² = c²\n\nStep 2: Calculate squares\n9 + 16 = c²\n\nStep 3: Add\n25 = c²\n\nStep 4: Take square root\nc = √25 = 5\n\nAnswer: c = 5"}
{"instruction": "Find the GCD of 48 and 18.", "input": "", "output": "To find the GCD (Greatest Common Divisor) of 48 and 18:\n\nMethod: Euclidean algorithm\n\nStep 1: 48 = 18 × 2 + 12\nStep 2: 18 = 12 × 1 + 6\nStep 3: 12 = 6 × 2 + 0\n\nWhen remainder is 0, the GCD is the last non-zero remainder.\n\nGCD(48, 18) = 6\n\nAnswer: 6"}
{"instruction": "Solve: x + 5 = 12", "input": "", "output": "This is a simple linear equation.\n\nGiven: x + 5 = 12\n\nStep 1: Subtract 5 from both sides\nx + 5 - 5 = 12 - 5\n\nStep 2: Simplify\nx = 7\n\nVerification: 7 + 5 = 12 ✓\n\nAnswer: x = 7"}
{"instruction": "What is 15% of 200?", "input": "", "output": "To find 15% of 200:\n\nMethod: Multiply by the percentage as a decimal\n15% = 15/100 = 0.15\n\n0.15 × 200 = 30\n\nAnswer: 30"}

View File

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "soup-cli"
version = "0.9.0"
version = "0.10.0"
description = "Fine-tune LLMs in one command. No SSH, no config hell."
readme = "README.md"
license = "MIT"

View File

@ -1,3 +1,3 @@
"""Soup CLI — Fine-tune LLMs in one command."""
__version__ = "0.9.0"
__version__ = "0.10.0"