Alpamys
|
03ddc05573
|
feat(hf): v0.29.0 — HuggingFace Hub Deep Integration
Auto-push checkpoints, HF Collections, self-hosted endpoint, HF datasets
push, HF Spaces auto-deploy, model card v2.
- utils/hf.py: single source of truth for HF token resolution (env >
cached login), HF_ENDPOINT validation, repo_id / collection_slug
validation, HfApi factory, add_to_collection. HF_ENDPOINT SSRF-hardened:
scheme allowlist, 0.0.0.0 rejected, plain HTTP limited to loopback,
RFC1918 / link-local / cloud-metadata (169.254.x) IPs rejected via
ipaddress.ip_address.
- monitoring/hf_push.py: HFPushCallback pushes each save_steps checkpoint
as 'checkpoint-<N>' branch. Sticky _repo_failed flag short-circuits
retries after hard failure. prepare_hf_resume enforces cwd containment
and passes local_dir_use_symlinks=False. allow_patterns whitelist
(safetensors/bin/pt/json/tokenizer*/trainer_state.json) keeps .env
and source files out of auto-pushed branches.
- commands/push.py: --collection flag, generate_model_card_v2 (task /
base / lr / optimizer from training_config.yaml; optional eval
scorecard; markdown-active chars neutralised on task names and
non-numeric scores; data_lineage HTML-escaped). --model cwd
containment, repo_id validation, deprecated --token warning, commit
message stripped to first 200 chars.
- commands/data.py: soup data push --input --hf-dataset uploads local
JSONL as HF dataset. Cwd containment on input, repo_id validation.
- commands/deploy.py: soup deploy hf-space --model --space --template
[gradio-chat|streamlit-chat]. render_space_template validates model
repo id before substitution into rendered app.py (defeats Python
injection from a crafted repo id).
- commands/train.py: --push-as <repo> attaches HFPushCallback to
trainer_wrapper.trainer after setup. --hf-resume pulls latest
checkpoint branch into output_dir before training.
Tests: +100 tests in test_hf_integration.py (65 initial + 35 review-
driven) covering all parts plus validate_collection_slug negatives,
build_push_callback factory paths, on_train_begin lifecycle, repo-failed
short-circuit, private-IP SSRF (10.x/172.16.x/192.168.x/169.254.x/
0.0.0.0), resolve_token edge cases. Full suite: 2801 tests pass.
Reviews: python-review, code-review, security-review, tdd-guide,
verification-loop — every HIGH / MEDIUM / LOW finding addressed.
Docs: README '## HuggingFace Hub Deep Integration' section added;
What's New replaced. CLAUDE.md / SECURITY.md / CONTRIBUTING.md updated
with new test count (93/2677 -> 94/2801) and v0.29.0 security entries.
License migration (MIT -> Apache-2.0) known-limitation note surfaced
in What's New per plan.md deferral from v0.27.0.
|
2026-04-23 16:17:28 +05:00 |
|
Alpamys
|
5f728c04cc
|
Expand test suite from ~70 to 147 tests, fix flaky ordering bug
Add 8 new test files covering previously untested modules:
- test_chat.py: _detect_base_model, adapter validation
- test_push.py: _format_size, _generate_model_card, token checks
- test_init.py: all templates, overwrite confirm/deny, YAML validation
- test_callback.py: SoupTrainerCallback with mocks
- test_display.py: TrainingDisplay rendering + edge cases
- test_loader.py: JSON/CSV/JSONL loading, empty lines, bad JSON
- test_validator.py: validate_and_stats, extended_stats, _percentile
- test_formats.py: reverse conversion, round-trips, edge cases
Fix flaky test_list_runs_ordering by adding rowid DESC as tiebreaker
in list_runs SQL query (runs created in same second had nondeterministic order).
Update CLAUDE.md with test file inventory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-03-02 20:52:33 +05:00 |