fix: resolve tiktoken encoding without constructing the embedding client (#955)
EmbeddingClient.encoding forced full client construction, which raises 'OpenAI API key is required' even though tiktoken needs no credentials. The document dedup tie-break (src/crud/document.py) only needs .encoding for token counting, so any test hitting that path fails in environments without embedding keys — notably CI for pull requests from forks, where repo secrets are unavailable (e.g. #908's test-python job failing on tests/crud/test_document.py::test_duplicate_rejection_reinforces_existing). Resolve the encoding from the configured model directly, falling back to cl100k_base, and only reuse the underlying client's encoding when it has already been constructed. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
e6d4d78ba1
commit
ca32f50797
|
|
@ -677,8 +677,19 @@ class EmbeddingClient:
|
|||
|
||||
@property
|
||||
def encoding(self) -> tiktoken.Encoding:
|
||||
"""Get the tiktoken encoding."""
|
||||
return self._get_client().encoding
|
||||
"""Get the tiktoken encoding.
|
||||
|
||||
Resolved without constructing the underlying client: tiktoken needs no
|
||||
API key, and token-counting callers (e.g. the document dedup tie-break)
|
||||
must work in environments with no embedding credentials, such as CI for
|
||||
pull requests from forks.
|
||||
"""
|
||||
if self._instance is not None:
|
||||
return self._instance.encoding
|
||||
try:
|
||||
return tiktoken.encoding_for_model(self._resolve_runtime_config().model)
|
||||
except KeyError:
|
||||
return tiktoken.get_encoding("cl100k_base")
|
||||
|
||||
|
||||
# Shared singleton embedding client instance
|
||||
|
|
|
|||
Loading…
Reference in New Issue