RomanNum3ral
/
cai
mirror of https://github.com/aliasrobotics/cai.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix duplicate API calls causing hang with Anthropic streaming 

Remove duplicate litellm.acompletion() calls that were causing every
streaming request to make two identical API calls, with the first
result being discarded. This doubled latency and caused apparent
hanging behavior, especially noticeable with Anthropic API.
This commit is contained in:
0xhsn 2026-01-27 16:52:24 +01:00
parent e38ac2b031
commit df882783ca
No known key found for this signature in database
1 changed files with 0 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/cai/sdk/agents/models/openai_chatcompletions.py
View File

@ -3305,7 +3305,6 @@ class OpenAIChatCompletionsModel(Model):
try:
if stream:
# Standard LiteLLM handling for streaming
ret = await litellm.acompletion(**kwargs)
stream_obj = await litellm.acompletion(**kwargs)
response = Response(
@ -3359,7 +3358,6 @@ class OpenAIChatCompletionsModel(Model):
kwargs["messages"] = messages
# Retry once, silently
if stream:
ret = await litellm.acompletion(**kwargs)
stream_obj = await litellm.acompletion(**kwargs)
response = Response(
id=FAKE_RESPONSES_ID,