Commit Graph

2 Commits

Author SHA1 Message Date
Vineeth Voruganti cf07068d95
Vineeth/dev 2418 (#1057)
* chore: update issue templates and docs

* chore: slim bug/quality forms and add integration template

Drop high-friction required fields from bug and quality issue forms.
Add an integration-request form (app stores, plugins, frameworks) labeled
integration, and point contributing docs at it.

* chore: route security mail to support@ and polish docs intake

Use support@honcho.dev for private vulnerability email. List the
documentation template in contributing guides, rename Media prove,
and add public-issue redaction/security redirects on the docs form.

* fix: address render issue in templates and add version field

* feat(docs): Initial draft of new contributing policies

* feat(ci): defer issue-gate closes to a scheduled sweeper

Addresses review feedback on #1041.

The gate now reads GitHub's resolved closing references
(closingIssuesReferences) instead of regex-parsing the pull request body,
so an issue linked through the sidebar Development panel counts, and a
bare `#123` mention no longer does.

It also no longer closes on the pull request event. It labels and
explains; pr-sweeper.yml re-checks every six hours and closes only what is
still failing 72 hours after the notice. That re-check is load-bearing:
linking an issue via the sidebar fires no webhook, so an event-only gate
could never observe a contributor complying that way. The sweeper also
closes drafts from outside the org after 30 days.

The shared check lives in .github/scripts/issue-gate.js so both workflows
run identical logic, with a dependency-free self-check wired into static
analysis. Its one regression guard: author_association CONTRIBUTOR stays
gated, since GitHub assigns it to anyone who has previously committed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(codeowners): drop the third reviewer from most areas

Discussed with @akattelu. Also reassigns SECURITY.md to @Rajat-Ahuja1997
and strips trailing whitespace from the deployment block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(v2): port the issue gate policy into the v2 contributing guide

The v2 guide is still published (v2.5.1 in docs.json) but carried no
mention of the issue gate, so a contributor reading it would not learn
that a pull request needs an approved issue until the bot labelled theirs.

Ports the policy, both linking routes, and the gate's place among the
automated checks, keeping the v2 guide's own structure and unwrapped
prose rather than importing the v3 rewrite wholesale.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ci): count only bot-authored gate notices, share the exemption list

Two review findings on the issue gate, with a common root cause.

MARKER is an invisible HTML comment, so anyone who can comment on a
public repository can paste it. findNotices accepted any comment
containing it, so a third party could post one on someone else's pull
request: runGate posts a notice only when none exists, so the author
would never be told, and runSweep would then measure the 72-hour grace
window from the stranger's timestamp and close them unwarned. Notices now
require bot authorship.

The stale-draft sweep re-listed the gate's exemptions and had lost the
bot case, so a bot's long-lived draft was closable despite checkGate
exempting bots. Both callers now share one exemptReason(pr) rather than
keeping parallel lists that drift.

Not changed: closingIssuesReferences(first: 20) truncation. It needs a
pull request with 21+ closing references where only a later one carries
the label, and the outcome would be a label plus the grace window, not a
close.

Coverage goes 11 -> 20 cases, including the stale-draft close path, which
had none. Both fixes were confirmed to fail their tests when reverted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Aakash Kattelu <aakash@plasticlabs.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 17:27:02 -04:00
Eugene Eisenstein 9e087e8771
feat(llm backend): enable combined tool calling + structured output in the LLM backend transport layer (#907)
* fix(llm): support combined tool calling and structured output across backends

- OpenAI: parse() 500s on non-strict function tools; route tool-carrying
  structured requests through create() with an explicit json_schema
  response_format (mirrors the streaming path)
- Anthropic: skip the '{' JSON prefill when tools are present so tool_use
  blocks stay reachable; make the schema instruction conditional and rely
  on parse + repair
- Gemini: native response_schema + function calling is rejected before
  Gemini 3; with tools present, inject a schema instruction into the final
  turn instead and rely on parse + repair
- All backends: tool-call turns carry no consumable content, so skip
  structured-output parsing on them

Extracted from the dialectic structured-output branch (DEV-1652) so the
transport layer can land independently.

DEV-2035

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(live_llm): exercise combined tools + structured output per provider

Two-turn live flow per backend: a forced tool-call turn (structured
parsing must be skipped) followed by a replay turn that must return a
schema-conforming answer with tools still attached. Asserts the
provider-specific request shaping: no parse() for OpenAI (500s on
non-strict tools), no '{' prefill for Anthropic, no native
response_schema for Gemini.

Verified against live OpenAI (gpt-4.1, gpt-5, gpt-5.4, gpt-5.4-mini)
and Gemini (gemini-2.5-flash).

DEV-2035

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: some needed unrelated test failures

* ci: add label-triggered live LLM test workflow

Adding the run-live-llm label to a PR (or workflow_dispatch) runs
tests/live_llm/ against real provider APIs — the only place the
--live-llm suite runs in CI. Reuses the unified-tests environment and
its Secrets Manager staging-dotenv resolution for provider keys; runs
on ubuntu-latest (no Fly runner, no Docker — the suite only touches the
LLM backends). Pins LIVE_LLM_ANTHROPIC_45_PLUS_MODELS=claude-sonnet-4-5
since the Anthropic family has no default models and would otherwise
silently collect empty.

Opt-in by design: live model behavior is variable, so this is a signal,
not a required check.

DEV-2035

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: run live LLM tests on main pushes touching the transport

Mirrors unified-tests' push trigger, scoped to paths that can affect
the live suite (src/llm/, config, the tests, deps, and the workflow
itself) so provider API calls aren't spent on unrelated changes.

DEV-2035

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: disable auth in live LLM test environment

The staging dotenv sets AUTH_USE_AUTH=true without a usable JWT secret,
and src/config.py validates the pair at import time — the same reason
unified-tests overrides it. This suite never runs the API server.

DEV-2035

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(live_llm): fix gpt-5.4 reasoning_effort and gemini replay-turn flake

- test_live_openai: gpt-5.4 dropped 'minimal' from the reasoning_effort
  vocabulary, so the gpt5 caching test 400'd — and the OpenAI backend's
  BadRequestError terminal swallowed it into an empty CompletionResult.
  Pick the effort per model generation.
- test_live_tools_structured_output: use tool_choice='auto' on the
  replay turn, matching the production dialectic loop (which never
  forces 'none') — NONE mode is what provoked gemini-2.5-flash's empty
  candidates. Drop the temperature pin so retries actually resample,
  and treat a repeat tool call as a retryable attempt.

Verified live: full suite green, gemini 4/4 consecutive passes.

DEV-2035

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: fail live LLM run when no staging secret was loaded

If the latest-tag fetch fails and no second tag exists, the fallback
step is skipped rather than failed, and the job would proceed without
provider keys — every test then skips via require_provider_key and the
run goes green. Guard on both fetch outcomes so that path fails loudly.

DEV-2035

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(live-llm-tests-GHA): remove extra comments

* ci(CODEOWNERS): introduce CODEOWNERS and gate GHA heavy test runs behind being a CODEOWNER

* ci(GHA-live-LLM-tests): consolidate common GHA steps

* test(test_live_openai): fix reasoning level adjustment for gpt-5

* test(live-llm-tests): temporary removal of gate to test the workflow

* test(live-llm-tests): revert removal of gate

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 11:47:49 -04:00