RomanNum3ral
/
soup
mirror of https://github.com/razor-ai/soup.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
soup/tests
History
Alpamys 3ab36e2aad feat(security): trust_remote_code opt-in across non-SFT trainers + 5 commands (v0.40.4 Part A) 
Closes the v0.36.0 #63 known gap. Every non-SFT trainer wrapper (DPO /
GRPO / KTO / ORPO / SimPO / IPO / PPO / RewardModel / Pretrain /
Embedding / BCO + the unified Preference dispatcher) now accepts
trust_remote_code: bool = False on __init__, resolves once via the
v0.36.0 helper (model_requires_trust_remote_code +
resolve_trust_remote_code), and stores the resolved value on
self._trust_remote_code. Every from_pretrained call site reads from
the resolved attribute — no remaining trust_remote_code=True literal
in any trainer file (asserted by tests/test_v0404_part_a.py).

Five standalone commands gain a --trust-remote-code Typer flag with
the same default-deny + KNOWN_SAFE_PREFIXES allowlist behaviour as
soup train: soup diff, soup export, soup merge, soup infer,
soup data generate.

commands/train.py removes the v0.36.0 sft_kwargs split — every trainer
receives trust_remote_code from the same trainer_kwargs dict.

PreferenceTrainerWrapper forwards the raw bool to the inner DPO /
SimPO / ORPO / IPO / BCO wrapper kwargs at both _build_inner and
_build_multi_objective sites; the resolver fires inside the inner
wrapper at construction time.

_load_reward_model (module-level helper in ppo.py) accepts a
trust_remote_code: bool parameter and resolves internally — design
intent is that the helper is independently safe to call outside
PPOTrainerWrapper.

_export_onnx / _export_tensorrt / _export_awq / _export_gptq and
_merge_adapter helpers all gain a trust_remote_code: bool = False
parameter threaded from the Typer flag.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 		2026-05-09 13:20:13 +05:00
..
__init__.py Initial project setup: CLI skeleton + config + trainer + data pipeline 2026-02-20 16:14:56 +05:00
conftest.py Fix all ruff lint errors and failing test 2026-02-20 16:25:46 +05:00
test_adapters.py feat: v0.22.0 — Training Profiler, Multi-Adapter Serving, Data Sampling, Adapter Management 2026-04-03 12:54:24 +05:00
test_advanced_peft.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_assistant_mask.py feat(correctness): v0.36.0 — Correctness First (4 Parts: A/B/C/D) 2026-04-30 11:52:51 +05:00
test_audio.py v0.17.0: data quality filters, audio modality, SGLang backend, server provider 2026-03-26 13:46:17 +05:00
test_auto_tuning.py test(auto-tuning): strip ANSI from Typer --help output (v0.32.0 follow-up) 2026-04-26 15:41:38 +05:00
test_autopilot.py fix(autopilot): Windows py3.9 path traversal false-positive 2026-04-13 13:31:43 +05:00
test_awq_gptq_export.py fix: cross-platform output path validation tests for CI 2026-04-04 20:39:17 +05:00
test_batch_probe.py feat(correctness): v0.36.0 — Correctness First (4 Parts: A/B/C/D) 2026-04-30 11:52:51 +05:00
test_bco.py feat(preference): v0.40.0 — Preference Variety (4 Parts: BCO + dispatcher + DPO variants + multi-objective) 2026-05-01 22:26:30 +05:00
test_bench.py refactor(bench): strengthen test assertions from PR #31 2026-04-19 22:13:35 +05:00
test_bugfixes.py fix(ci): Windows encoding failure in TestGRPOCPUMinNewTokens 2026-04-13 13:12:44 +05:00
test_callback.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_cans.py feat(cans): soup can run + publish (v0.33.0 Part A wave 2) 2026-04-27 18:19:39 +05:00
test_chat.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_chat_template.py feat(correctness): v0.36.0 — Correctness First (4 Parts: A/B/C/D) 2026-04-30 11:52:51 +05:00
test_cli.py add --json flag to version command for machine-readable output in CI/… (#6) 2026-04-04 20:42:46 +05:00
test_cli_subprocess.py feat: add eval platform with custom evals, LLM judge, human eval, leaderboard (v0.19.0) 2026-04-01 14:47:08 +05:00
test_config.py Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests 2026-02-23 21:18:19 +05:00
test_cost.py refactor(cost): polish soup cost from PR #42 2026-04-22 23:13:42 +05:00
test_crash_reporter.py feat(observability): v0.34.0 — Observability & Dev UX (7 Parts) 2026-04-28 13:14:49 +05:00
test_curriculum.py fix: v0.23.1 — CI fix, security warnings, expanded test coverage 2026-04-03 14:20:21 +05:00
test_data.py Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests 2026-02-23 21:18:19 +05:00
test_data_augment.py feat(v0.25.0): Beyond the Wrapper — 8 major features 2026-04-13 12:58:11 +05:00
test_data_sample.py fix(v0.40.1): QA Hardening — UTF-8 bootstrap, schema strictness, multi-objective preference runtime, CLI UX 2026-05-08 12:03:16 +05:00
test_data_split.py feat: v0.23.0 — AWQ/GPTQ Export, Sample Packing, Data Split, Curriculum Learning 2026-04-03 13:55:01 +05:00
test_data_tools.py Phase 2: experiment tracking, data tools, model evaluation 2026-02-23 23:34:28 +05:00
test_dataset_hub.py feat: v0.24.0 — Dataset Hub, Freeze Training, Loss Watchdog, Dataset Registry 2026-04-03 16:35:23 +05:00
test_dataset_registry.py feat: v0.24.0 — Dataset Hub, Freeze Training, Loss Watchdog, Dataset Registry 2026-04-03 16:35:23 +05:00
test_deepspeed.py Fix ANSI escape code issue in deepspeed help output test 2026-03-05 17:22:21 +05:00
test_deploy_ollama.py fix: use ANSI-safe assertions in deploy help tests (macOS CI fix) 2026-04-01 14:01:39 +05:00
test_diff.py Add Phase 3: serve, data generate, sweep, diff, DeepSpeed (v0.3.0) 2026-03-05 17:14:08 +05:00
test_display.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_doctor.py feat(doctor): add RAM and disk space checks to soup doctor command wi… (#7) 2026-04-05 17:28:05 +05:00
test_dpo_example.py refactor(tests): polish DPO example tests from PR #48 2026-04-23 12:22:52 +05:00
test_dpo_variants.py feat(preference): v0.40.0 — Preference Variety (4 Parts: BCO + dispatcher + DPO variants + multi-objective) 2026-05-01 22:26:30 +05:00
test_embedding.py v0.16.0: embedding models, ONNX/TensorRT export, speculative decoding 2026-03-26 12:41:39 +05:00
test_errors.py Fix ANSI escape code issue in verbose help output test 2026-03-05 19:22:53 +05:00
test_eval.py feat: add eval platform with custom evals, LLM judge, human eval, leaderboard (v0.19.0) 2026-04-01 14:47:08 +05:00
test_eval_gate.py test(eval_gate): strip ANSI escapes in train --help CI assertion 2026-04-20 21:51:23 +05:00
test_eval_platform.py fix: use ANSI-safe assertions in eval human help test (macOS CI fix) 2026-04-01 14:51:38 +05:00
test_export.py v0.16.0: embedding models, ONNX/TensorRT export, speculative decoding 2026-03-26 12:41:39 +05:00
test_formats.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_fp8_recipe.py fix(fp8): wire fp8_recipe through v028_features (covers 10 trainers) 2026-04-28 20:19:58 +05:00
test_freeze_training.py feat: v0.24.0 — Dataset Hub, Freeze Training, Loss Watchdog, Dataset Registry 2026-04-03 16:35:23 +05:00
test_generate.py Add Phase 3: serve, data generate, sweep, diff, DeepSpeed (v0.3.0) 2026-03-05 17:14:08 +05:00
test_gpu.py Initial project setup: CLI skeleton + config + trainer + data pipeline 2026-02-20 16:14:56 +05:00
test_grpo.py Add GRPO reasoning training (Phase 4) — v0.4.2 2026-03-23 16:29:25 +05:00
test_hf_integration.py fix(tests): strip ANSI from Typer --help output + patch os.environ for null-byte test (v0.29.0) 2026-04-24 11:06:45 +05:00
test_infer.py v0.13.2: add missing test coverage for infer + tensorboard 2026-03-25 18:57:15 +05:00
test_inference_advanced.py feat(serve): structured-output + auto-quant live (v0.33.0 Part D) 2026-04-27 18:43:07 +05:00
test_init.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_ipo.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_jinja_analyzer.py feat(multipack): v0.37.0 — Multipack (5 Parts A/B/C/D/E) 2026-04-30 13:48:16 +05:00
test_kto.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_loader.py feat: v0.14.0 — pre-training + MoE support 2026-03-25 22:26:01 +05:00
test_log_level.py fix(tests): strip ANSI in --log-level help-visible assertion 2026-04-28 13:25:54 +05:00
test_loss_watchdog.py feat: v0.24.0 — Dataset Hub, Freeze Training, Loss Watchdog, Dataset Registry 2026-04-03 16:35:23 +05:00
test_merge.py Phase 2.5: add export GGUF, merge LoRA, resume training, W&B integration (v0.2.0) 2026-03-03 22:29:44 +05:00
test_migrate.py feat: v0.21.0 — migrate, recipes, NEFTune, rsLoRA 2026-04-02 14:08:36 +05:00
test_mlx_backend.py feat(v0.25.0): Beyond the Wrapper — 8 major features 2026-04-13 12:58:11 +05:00
test_moe.py test: add MoE integration, DeepSeek naming, and model type coverage 2026-03-25 22:34:55 +05:00
test_multi_adapter.py feat(inference): v0.30.0 — Inference Excellence 2026-04-24 23:39:04 +05:00
test_multi_gpu.py feat(v0.27.0): Multi-GPU Mastery — topology, ZeRO++, FSDP2+compile, MII, PP, recipes 2026-04-21 14:54:34 +05:00
test_multipack_config.py feat(multipack): v0.37.0 — Multipack (5 Parts A/B/C/D/E) 2026-04-30 13:48:16 +05:00
test_multipack_invariants.py feat(multipack): v0.37.0 — Multipack (5 Parts A/B/C/D/E) 2026-04-30 13:48:16 +05:00
test_multipack_sampler.py feat(multipack): v0.37.0 — Multipack (5 Parts A/B/C/D/E) 2026-04-30 13:48:16 +05:00
test_neat_packing.py feat(multipack): v0.37.0 — Multipack (5 Parts A/B/C/D/E) 2026-04-30 13:48:16 +05:00
test_neftune_rslora.py feat: v0.21.0 — migrate, recipes, NEFTune, rsLoRA 2026-04-02 14:08:36 +05:00
test_onnx_tensorrt_export.py feat: v0.23.0 — AWQ/GPTQ Export, Sample Packing, Data Split, Curriculum Learning 2026-04-03 13:55:01 +05:00
test_orpo.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_packing.py fix: v0.23.1 — CI fix, security warnings, expanded test coverage 2026-04-03 14:20:21 +05:00
test_part_a_wave1.py fix(v0.33.0): review-wave findings (CRITICAL + HIGH + MEDIUM + LOW) 2026-04-27 19:57:57 +05:00
test_part_a_wave2.py fix(v0.33.0): review-wave findings (CRITICAL + HIGH + MEDIUM + LOW) 2026-04-27 19:57:57 +05:00
test_part_b.py fix(v0.33.0): review-wave findings (CRITICAL + HIGH + MEDIUM + LOW) 2026-04-27 19:57:57 +05:00
test_part_c.py feat(trainers): v0.35.0 — Trainer Coverage (closes #60, #61, #45) 2026-04-28 15:01:42 +05:00
test_part_d.py fix(v0.33.0): review-wave findings (CRITICAL + HIGH + MEDIUM + LOW) 2026-04-27 19:57:57 +05:00
test_part_e.py fix(v0.33.0): review-wave findings (CRITICAL + HIGH + MEDIUM + LOW) 2026-04-27 19:57:57 +05:00
test_part_f_hardening.py fix(tests): macOS CI failure on isolation_strategy_linux_with_unshare 2026-04-27 21:56:49 +05:00
test_peft_methods.py feat(v0.25.0): Beyond the Wrapper — 8 major features 2026-04-13 12:58:11 +05:00
test_peft_patches.py feat(lora): v0.39.0 — LoRA Quality (PiSSA + ReLoRA + per-pattern rank + surgical patches + templates registry) 2026-05-01 16:18:07 +05:00
test_performance.py feat(preference): v0.40.0 — Preference Variety (4 Parts: BCO + dispatcher + DPO variants + multi-objective) 2026-05-01 22:26:30 +05:00
test_pissa_init.py fix(v0.40.1): QA Hardening — UTF-8 bootstrap, schema strictness, multi-objective preference runtime, CLI UX 2026-05-08 12:03:16 +05:00
test_ppo.py Add PPO / Full RLHF pipeline (Phase 10) — v0.9.0 2026-03-23 22:17:55 +05:00
test_preference_dispatcher.py feat(preference): v0.40.0 — Preference Variety (4 Parts: BCO + dispatcher + DPO variants + multi-objective) 2026-05-01 22:26:30 +05:00
test_preference_multi.py fix(v0.40.1): QA Hardening — UTF-8 bootstrap, schema strictness, multi-objective preference runtime, CLI UX 2026-05-08 12:03:16 +05:00
test_preference_multi_runtime.py fix(v0.40.1): QA Hardening — UTF-8 bootstrap, schema strictness, multi-objective preference runtime, CLI UX 2026-05-08 12:03:16 +05:00
test_pretrain.py test: add MoE integration, DeepSeek naming, and model type coverage 2026-03-25 22:34:55 +05:00
test_profile.py feat: v0.22.0 — Training Profiler, Multi-Adapter Serving, Data Sampling, Adapter Management 2026-04-03 12:54:24 +05:00
test_profiling.py feat(observability): v0.34.0 — Observability & Dev UX (7 Parts) 2026-04-28 13:14:49 +05:00
test_progress.py Add sweep early stopping, Rich download progress bars, fix click compat — v0.4.0 2026-03-23 15:59:27 +05:00
test_push.py feat(hf): v0.29.0 — HuggingFace Hub Deep Integration 2026-04-23 16:17:28 +05:00
test_qat.py Add Quantization-Aware Training support (Phase 7) — v0.6.0 2026-03-23 20:35:35 +05:00
test_quality_filter.py fix: rename APIs to match test plan, fix RoPE factor detection 2026-03-26 15:14:24 +05:00
test_quant_check.py feat(v0.26.0): Parts B-E — Eval Gate, Trace-to-Pref, Quant-Check, Soup Cans 2026-04-20 21:37:05 +05:00
test_quant_menu.py feat(quant): v0.38.0 — Quant Menu (8 Parts: A-H) 2026-05-01 13:31:49 +05:00
test_quickstart.py Add Phase 3.1: friendly errors, soup doctor, soup quickstart, UX polish (v0.3.1) 2026-03-05 19:10:36 +05:00
test_rank_pattern.py feat(lora): v0.39.0 — LoRA Quality (PiSSA + ReLoRA + per-pattern rank + surgical patches + templates registry) 2026-05-01 16:18:07 +05:00
test_recipes.py feat(recipes): v0.31.0 — Model & Recipe Breadth 2026-04-25 19:03:23 +05:00
test_recipes_v031.py feat(recipes): v0.31.0 — Model & Recipe Breadth 2026-04-25 19:03:23 +05:00
test_registry.py feat(registry): add Local Model Registry / Provenance Vault (v0.26.0 Part A) 2026-04-20 20:07:54 +05:00
test_relora.py feat(lora): v0.39.0 — LoRA Quality (PiSSA + ReLoRA + per-pattern rank + surgical patches + templates registry) 2026-05-01 16:18:07 +05:00
test_replay.py feat(observability): v0.34.0 — Observability & Dev UX (7 Parts) 2026-04-28 13:14:49 +05:00
test_resume.py Fix ANSI escape code issue in help output tests 2026-03-03 22:34:09 +05:00
test_rlvr.py feat(v0.25.0): Beyond the Wrapper — 8 major features 2026-04-13 12:58:11 +05:00
test_run_cost.py feat(observability): v0.34.0 — Observability & Dev UX (7 Parts) 2026-04-28 13:14:49 +05:00
test_runs.py Introduce 'soup runs clean' for smart checkpoint space management (#9) 2026-04-06 22:35:16 +05:00
test_serve.py Add Phase 3: serve, data generate, sweep, diff, DeepSpeed (v0.3.0) 2026-03-05 17:14:08 +05:00
test_server_generate.py test: add missing coverage for _parse_json_array, _validate_example, SSRF guards 2026-03-26 13:57:03 +05:00
test_sglang_serve.py fix: rename APIs to match test plan, fix RoPE factor detection 2026-03-26 15:14:24 +05:00
test_simpo.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_smoke_train.py Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests 2026-02-23 21:18:19 +05:00
test_speculative_decoding.py fix: strip ANSI escape codes in CLI help flag tests 2026-03-26 12:54:38 +05:00
test_sweep.py Add sweep early stopping, Rich download progress bars, fix click compat — v0.4.0 2026-03-23 15:59:27 +05:00
test_synth_data_pro.py fix: use ANSI-safe assertions in synth data pro help tests (macOS CI fix) 2026-04-01 18:16:09 +05:00
test_templates_yaml.py feat(lora): v0.39.0 — LoRA Quality (PiSSA + ReLoRA + per-pattern rank + surgical patches + templates registry) 2026-05-01 16:18:07 +05:00
test_tensorboard.py v0.13.2: add missing test coverage for infer + tensorboard 2026-03-25 18:57:15 +05:00
test_tool_calling.py feat(v0.25.0): Beyond the Wrapper — 8 major features 2026-04-13 12:58:11 +05:00
test_trace_to_pref.py feat(v0.26.0): Parts B-E — Eval Gate, Trace-to-Pref, Quant-Check, Soup Cans 2026-04-20 21:37:05 +05:00
test_tracker.py Fix run_id collision in CI: increase suffix from 4 to 8 hex chars 2026-02-23 23:48:12 +05:00
test_trainer_coverage_v035.py feat(trainers): v0.35.0 — Trainer Coverage (closes #60, #61, #45) 2026-04-28 15:01:42 +05:00
test_trainer_init.py chore: add trainer init tests, fix coverage threshold for CI 2026-03-25 12:36:05 +05:00
test_training_intelligence.py feat(v0.25.0): Beyond the Wrapper — 8 major features 2026-04-13 12:58:11 +05:00
test_training_speed.py feat(trainers): v0.35.0 — Trainer Coverage (closes #60, #61, #45) 2026-04-28 15:01:42 +05:00
test_trust_remote_code.py fix(tests): strip ANSI in --trust-remote-code help-visible assertions (v0.36.0 follow-up) 2026-04-30 12:01:01 +05:00
test_tui.py feat(observability): v0.34.0 — Observability & Dev UX (7 Parts) 2026-04-28 13:14:49 +05:00
test_ui.py v0.10.10: Security hardening — Web UI auth, CORS, SSRF, path traversal protection 2026-03-25 12:14:10 +05:00
test_ui_chat.py fix(security): harden chat proxy SSRF with ipaddress.is_loopback validation 2026-04-07 19:28:08 +05:00
test_ui_config_builder.py feat(ui): add Web UI Enhancement with live training monitor, enhanced metrics, chat upgrade, and config builder (v0.24.2) 2026-04-07 19:13:55 +05:00
test_ui_live_monitor.py feat(ui): add Web UI Enhancement with live training monitor, enhanced metrics, chat upgrade, and config builder (v0.24.2) 2026-04-07 19:13:55 +05:00
test_ui_metrics.py feat(ui): add Web UI Enhancement with live training monitor, enhanced metrics, chat upgrade, and config builder (v0.24.2) 2026-04-07 19:13:55 +05:00
test_unsloth.py Add Unsloth backend for 2-5x faster training (Phase 5) — v0.4.3 2026-03-23 16:55:44 +05:00
test_v0401_part_c.py fix(v0.40.1): QA Hardening — UTF-8 bootstrap, schema strictness, multi-objective preference runtime, CLI UX 2026-05-08 12:03:16 +05:00
test_v0401_part_d.py fix(v0.40.1): QA Hardening — UTF-8 bootstrap, schema strictness, multi-objective preference runtime, CLI UX 2026-05-08 12:03:16 +05:00
test_v0401_part_e.py fix(v0.40.1): QA Hardening — UTF-8 bootstrap, schema strictness, multi-objective preference runtime, CLI UX 2026-05-08 12:03:16 +05:00
test_v0402_part_a.py fix(v0.40.2): make help-text assertions width-independent 2026-05-08 13:26:44 +05:00
test_v0402_part_b.py fix(v0.40.2): make help-text assertions width-independent 2026-05-08 13:26:44 +05:00
test_v0403_part_a.py feat(v0.40.3): Stub-to-live wave 1 (#33, #64; #65 still deferred) 2026-05-08 16:02:45 +05:00
test_v0403_part_b.py feat(v0.40.3): Stub-to-live wave 1 (#33, #64; #65 still deferred) 2026-05-08 16:02:45 +05:00
test_v0403_part_c.py fix(v0.40.3): strip ANSI escapes in help-text assertions 2026-05-08 20:50:10 +05:00
test_v0404_part_a.py feat(security): trust_remote_code opt-in across non-SFT trainers + 5 commands (v0.40.4 Part A) 2026-05-09 13:20:13 +05:00
test_validator.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_vision.py v0.17.0: data quality filters, audio modality, SGLang backend, server provider 2026-03-26 13:46:17 +05:00
test_vllm_serve.py feat(trainers): v0.35.0 — Trainer Coverage (closes #60, #61, #45) 2026-04-28 15:01:42 +05:00
test_why.py feat(observability): v0.34.0 — Observability & Dev UX (7 Parts) 2026-04-28 13:14:49 +05:00
test_windows_encoding.py fix(v0.40.1): QA Hardening — UTF-8 bootstrap, schema strictness, multi-objective preference runtime, CLI UX 2026-05-08 12:03:16 +05:00