From cb44d9e64ef52f89ae24fd59dce84965c4c8041d Mon Sep 17 00:00:00 2001 From: Alpamys Date: Mon, 3 Aug 2026 20:08:58 +0500 Subject: [PATCH] fix(tests): three environment-specific failures in the v0.72.4 suite MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Down from 34 CI failures to 0 after the trl cap; these three are the remainder, all in the new tests and all device- or version-specific rather than product bugs: - test_trl_itself_still_refuses_batch_one built a KTOConfig without bf16=False. Newer TRL configs default bf16 on, so a CPU-only runner raised "Your setup doesn't support bf16/gpu" before ever reaching the batch-size check the test exists to pin. Reproduced locally with CUDA_VISIBLE_DEVICES=-1. - test_kto_streams_at_batch_two runs a real trainer.train(). That works on CUDA (dev box and CI) but fails CPU-only under torch 2.13/trl 0.24 with "Tensor on device cpu is not on the expected device meta!". Streaming exists to bound VRAM, so a streamed model on CPU is a test convenience rather than a configuration — the stance v0.72.3 already took for PEFT re-dispatch — so the full-step test is gated to the production device. KTO's schema gate, setup, reference behaviour and layer-read accounting are all still checked on CPU. - test_a_reference_using_loss_reads_more_layers_than_sft[kto] hit MPS on the macOS runners; its sibling tests already carry the MPS skip and this one had been missed. Verified: 9 passed with CUDA hidden, 67 passed on CUDA. The published benchmarks record notes the CPU limitation, and that it was invisible on a CUDA dev box — a locally-green suite is a weak signal whenever a path forks on device. --- benchmarks/gate-v0.72.4-preference-losses.md | 15 +++++++++++++++ tests/test_v07204.py | 4 ++++ 2 files changed, 19 insertions(+) diff --git a/benchmarks/gate-v0.72.4-preference-losses.md b/benchmarks/gate-v0.72.4-preference-losses.md index f265463..64667d9 100644 --- a/benchmarks/gate-v0.72.4-preference-losses.md +++ b/benchmarks/gate-v0.72.4-preference-losses.md @@ -276,6 +276,21 @@ All four saved adapters: **120 tensors, 0 keys carrying the streaming wrapper segment, 60/60 non-zero `lora_B`** — i.e. ordinary LoRA adapters that load into any non-streaming model. +### One CPU-only limitation, found by CI rather than locally + +A full KTO **training step** over a streamed model runs on CUDA (verified here and +on the dev box) but fails on a CPU-only runner under newer torch/TRL with +`Tensor on device cpu is not on the expected device meta!`. Streaming exists to +bound VRAM, so a streamed model on CPU is a test convenience rather than a real +configuration — the same stance v0.72.3 took on PEFT's re-dispatch — and that test +is therefore gated to the production device. Everything else about KTO (the schema +gate, `setup()`, the reference behaviour, the layer-read accounting) is still +exercised on CPU. + +Worth stating plainly: this was invisible on the dev box, which has CUDA. The +locally-green suite is a weaker signal than it looks whenever a code path forks on +device. + Ten rejected configurations each named its own reason (rollout tasks, unsupported task, KTO at batch 1, unsloth backend, 8-bit, no adapter, DoRA, `batch_size: auto`, packing). diff --git a/tests/test_v07204.py b/tests/test_v07204.py index ec6b8bd..fc054b4 100644 --- a/tests/test_v07204.py +++ b/tests/test_v07204.py @@ -962,6 +962,10 @@ class TestKtoBatchIsRefusedEarly: report_to=[], max_length=32, max_prompt_length=16, + # Newer TRL configs enable bf16 by default, which a CPU-only runner + # rejects with "Your setup doesn't support bf16/gpu" BEFORE reaching + # the batch-size check this test exists to pin. + bf16=False, ) with pytest.raises(ValueError, match="batch size"): KTOTrainer(