2 Commits

Author	SHA1	Message	Date
Alpamys	8dc2e1ebc7	test: add MoE integration, DeepSeek naming, and model type coverage ... Address TDD reviewer findings: - MoE integration tests for _setup_transformers (moe_lora, aux_loss, non-MoE fallback) - DeepSeek w1/w2/w3 expert naming discovery tests - Parametrized tests for jetmoe, arctic, grok, qwen2_moe, deepseek_v2 types - Plaintext line-level chunking and Unicode loading tests 1014 total tests, 58.31% coverage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-25 22:34:55 +05:00
Alpamys	15a6daf342	feat: v0.14.0 — pre-training + MoE support ... Add continued pre-training task and Mixture of Experts model support: - `task: pretrain` for continued pre-training on raw text data - `plaintext` data format ({"text": "..."} JSONL or .txt files) - MoE model detection (Mixtral, Qwen3 MoE, DeepSeek V3, DBRX, OLMoE) - ScatterMoE LoRA (`moe_lora: true`) targets expert FFN + attention layers - `moe_aux_loss_coeff` for router load-balancing loss - Templates: `soup init --template pretrain` and `--template moe` - 85 new tests across test_pretrain.py and test_moe.py (1002 total) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-25 22:26:01 +05:00