mirror of https://github.com/razor-ai/soup.git
e44e0bd663
Two tests in tests/test_bugfixes.py::TestGRPOCPUMinNewTokens fail on
windows-latest / Python 3.11 when importing trl.trainer.grpo_trainer:
RuntimeError: Failed to import trl.trainer.grpo_trainer because of
the following error:
'charmap' codec can't decode byte 0x90 in position 6555: character
maps to <undefined>
Root cause: upstream trl reads an auxiliary file without an explicit
encoding, so Python uses the system default. On Windows that is cp1252
('charmap'), which chokes on non-ASCII bytes present in the file. This
is an upstream issue but Soup needs a green CI.
Two-layer fix:
1. .github/workflows/ci.yml — set PYTHONUTF8=1 and PYTHONIOENCODING=utf-8
as job-level env. Python's UTF-8 mode makes all file I/O default to
UTF-8 regardless of locale, which is the correct global fix for this
class of bug.
2. tests/test_bugfixes.py — add a _trl_grpo_importable() helper that
returns False on UnicodeDecodeError / ImportError / RuntimeError, and
use it as a belt-and-braces skip in both TestGRPOCPUMinNewTokens
tests. Ensures the tests skip cleanly instead of erroring out if a
future CI change accidentally drops PYTHONUTF8.
Local verification: both tests pass with 'pytest tests/test_bugfixes.py::
TestGRPOCPUMinNewTokens -v' (Python 3.10, Windows).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| ci.yml | ||
| docker.yml | ||
| publish.yml | ||