Two tests in tests/test_bugfixes.py::TestGRPOCPUMinNewTokens fail on
windows-latest / Python 3.11 when importing trl.trainer.grpo_trainer:
RuntimeError: Failed to import trl.trainer.grpo_trainer because of
the following error:
'charmap' codec can't decode byte 0x90 in position 6555: character
maps to <undefined>
Root cause: upstream trl reads an auxiliary file without an explicit
encoding, so Python uses the system default. On Windows that is cp1252
('charmap'), which chokes on non-ASCII bytes present in the file. This
is an upstream issue but Soup needs a green CI.
Two-layer fix:
1. .github/workflows/ci.yml — set PYTHONUTF8=1 and PYTHONIOENCODING=utf-8
as job-level env. Python's UTF-8 mode makes all file I/O default to
UTF-8 regardless of locale, which is the correct global fix for this
class of bug.
2. tests/test_bugfixes.py — add a _trl_grpo_importable() helper that
returns False on UnicodeDecodeError / ImportError / RuntimeError, and
use it as a belt-and-braces skip in both TestGRPOCPUMinNewTokens
tests. Ensures the tests skip cleanly instead of erroring out if a
future CI change accidentally drops PYTHONUTF8.
Local verification: both tests pass with 'pytest tests/test_bugfixes.py::
TestGRPOCPUMinNewTokens -v' (Python 3.10, Windows).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Check stdout encoding instead of type to detect non-UTF-8 consoles
- Redirect stdout to UTF-8 TextIOWrapper before plotext renders
- Add unit test that simulates cp1251 stdout with plotext
- 1022 tests total
- soup data validate: default --format changed from 'alpaca' to 'auto',
uses detect_format() to auto-detect dataset format
- soup data stats: force UTF-8 stdout on Windows for plotext histograms
- soup ui: add --show-token flag, document auth token in --help
- 7 new tests (BUG-013/014/015), 1021 tests total
GRPO:
- Set default chat_template on tokenizer when missing (fixes ValueError
from trl's apply_chat_template on models without chat support)
- Ensure batch_size >= num_generations (trl 0.28 requirement)
- Verified end-to-end GRPO training on CPU succeeds
PPO:
- Tokenize dataset via .map() before passing to PPOTrainer (adds input_ids
and attention_mask columns required by trl experimental API)
Tests: 666 passed, 5 new tests for chat_template/tokenization fixes,
3 existing mock tests updated for new tokenization step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- PPO: Skip resume_from_checkpoint when experimental PPOTrainer.train()
doesn't accept it (inspect signature at runtime, warn and proceed).
- CPU: Use device_map="cpu" instead of "auto" on CPU across all trainers
(SFT, DPO, GRPO, PPO, RewardModel) to prevent meta tensor errors.
- Add 12 new tests for both fixes (661 total passing).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- PPO: Support trl >=0.28 experimental API (ref_model, reward_model,
train_dataset, value_model positional args). Auto-import from
trl.experimental.ppo with fallback. Create reward/value models when needed.
- GRPO: Fix CPU empty generation tensor mismatch by passing
generation_kwargs={"min_new_tokens": 1} on CPU devices.
- Add 6 new tests for both fixes (649 total passing).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use create=True in mock.patch so tests work when trl has moved
PPOTrainer to trl.experimental and it's not in the trl namespace.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PPOTrainer.__init__() no longer accepts dataset= in newer trl versions.
Now checks via inspect.signature whether train_dataset or dataset is
accepted; if neither, sets dataset on trainer before .train() call.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- PPO: detect trl API via inspect — args= (>=0.28) vs config= (<0.28)
- PPO: split train into _train_builtin (trl >=0.28) and _train_manual
- GRPO: update error message to mention GRPO/PPO CPU limitation
- 2 new tests for PPO API detection (639 total, all passing)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- PPO: add use_cpu=True to PPOConfig when running on CPU
- GRPO: add CPU warning + use_cpu flag via inspect (trl bug workaround)
- Add use_cpu error pattern to friendly error map
- 7 new tests for CPU fixes (637 total, all passing)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ASCII progress bar in display.py (replace Unicode blocks with #/-)
- Catch UnicodeEncodeError from plotext in data stats (Windows fallback)
- Auto-disable 4bit/8bit quantization on CPU (bitsandbytes doesn't support it)
- Friendly error messages for CPU tensor/dtype/bf16/torchvision issues
- Fix diff.py: torch_dtype= -> dtype= (deprecation warning)
- Add torchvision version compatibility check to soup doctor
7 new tests (631 total), ruff clean.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- BUG-001: Replace Unicode arrows/dashes with ASCII in all console output
to fix UnicodeEncodeError on Windows cp1252 (~10 commands affected)
- BUG-002: PPO trainer uses inspect.signature to detect trl parameter names
(ppo_epochs vs num_ppo_epochs) for trl 0.28.0 compatibility
- BUG-003: Add get_compute_dtype() - uses float32 on CPU, bfloat16/float16
on CUDA. Fixes dtype mismatch in reward model and all trainers
- BUG-004: Add warning when using quantization on CPU
- BUG-005: Fix dtype -> torch_dtype in diff.py model loading
- BUG-006: Pin wandb<0.18.0 to avoid trl import conflict, add runtime guard
13 new tests (624 total), ruff clean.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Alpamys