Commit Graph

2 Commits

Author SHA1 Message Date
Teknium a0b29343b4 fix(gateway): offload preflight-compression warning enrichment; behavioral offload tests
Follow-ups to the previous commit (#74155 by @Drexuxux):

- enrich_model_switch_warnings_for_gateway() -> merge_preflight_compression_warning()
  still called the sync resolve_display_context_length() provider probe ladder
  inline in both async /model call sites; dispatch it via asyncio.to_thread.
- Replace the inspect.getsource() test (source-reading tests are banned by
  AGENTS.md) with behavioral tests that drive the real _handle_model_command:
  assert the resolver runs off the loop thread and that the warning enrichment
  is dispatched through asyncio.to_thread.
2026-07-31 22:39:34 -07:00
Drexuxux 95eae03883 fix(gateway): offload /model context-length resolution off the event loop
resolve_display_context_length() runs two blocking chains: the route
comparison in should_clear_context_pin() and the provider probe ladder in
get_model_context_length() (blocking requests calls to Anthropic /v1/models,
Copilot, Nous, Codex, GMI, Ollama, models.dev and OpenRouter).

The gateway message path already offloads both via
get_model_context_length_async() and should_clear_context_pin_async(), but
the /model slash-command handlers (_handle_model_command, _finish_switch)
called the sync helper directly, freezing the whole event loop for the
duration of the probe ladder - no messages processed on any platform, and
the Discord heartbeat timeouts that get_model_context_length_async() was
introduced to prevent.

Add resolve_display_context_length_async(), a thin asyncio.to_thread wrapper
mirroring the two existing *_async helpers (no logic duplication), and await
it at both handlers.
2026-07-31 22:39:34 -07:00