simple_batch_embed raised ValueError when any input exceeded the per-input token cap, which failed the entire deriver save when a single observation was over-length. Add on_oversize="truncate": oversize inputs are embedded from a token-capped prefix (re-encoded until it fits, with a warning), preserving one vector per input. Default stays "raise" so existing callers are unchanged. RepresentationManager opts into truncate. Also add a live embedding test that fails on main (raise / missing kwarg) and passes once a mixed short+oversize batch survives. Refs #569 |
||
|---|---|---|
| .. | ||
| test_document.py | ||
| test_peer_card.py | ||
| test_representation_manager.py | ||
| test_session.py | ||
| test_workspace.py | ||