From cf07068d95cee717a84dfaacdf08863ce8ca26fc Mon Sep 17 00:00:00 2001 From: Vineeth Voruganti <13438633+VVoruganti@users.noreply.github.com> Date: Mon, 24 Aug 2026 17:27:02 -0400 Subject: [PATCH] Vineeth/dev 2418 (#1057) * chore: update issue templates and docs * chore: slim bug/quality forms and add integration template Drop high-friction required fields from bug and quality issue forms. Add an integration-request form (app stores, plugins, frameworks) labeled integration, and point contributing docs at it. * chore: route security mail to support@ and polish docs intake Use support@honcho.dev for private vulnerability email. List the documentation template in contributing guides, rename Media prove, and add public-issue redaction/security redirects on the docs form. * fix: address render issue in templates and add version field * feat(docs): Initial draft of new contributing policies * feat(ci): defer issue-gate closes to a scheduled sweeper Addresses review feedback on #1041. The gate now reads GitHub's resolved closing references (closingIssuesReferences) instead of regex-parsing the pull request body, so an issue linked through the sidebar Development panel counts, and a bare `#123` mention no longer does. It also no longer closes on the pull request event. It labels and explains; pr-sweeper.yml re-checks every six hours and closes only what is still failing 72 hours after the notice. That re-check is load-bearing: linking an issue via the sidebar fires no webhook, so an event-only gate could never observe a contributor complying that way. The sweeper also closes drafts from outside the org after 30 days. The shared check lives in .github/scripts/issue-gate.js so both workflows run identical logic, with a dependency-free self-check wired into static analysis. Its one regression guard: author_association CONTRIBUTOR stays gated, since GitHub assigns it to anyone who has previously committed. Co-Authored-By: Claude Opus 5 (1M context) * chore(codeowners): drop the third reviewer from most areas Discussed with @akattelu. Also reassigns SECURITY.md to @Rajat-Ahuja1997 and strips trailing whitespace from the deployment block. Co-Authored-By: Claude Opus 5 (1M context) * docs(v2): port the issue gate policy into the v2 contributing guide The v2 guide is still published (v2.5.1 in docs.json) but carried no mention of the issue gate, so a contributor reading it would not learn that a pull request needs an approved issue until the bot labelled theirs. Ports the policy, both linking routes, and the gate's place among the automated checks, keeping the v2 guide's own structure and unwrapped prose rather than importing the v3 rewrite wholesale. Co-Authored-By: Claude Opus 5 (1M context) * fix(ci): count only bot-authored gate notices, share the exemption list Two review findings on the issue gate, with a common root cause. MARKER is an invisible HTML comment, so anyone who can comment on a public repository can paste it. findNotices accepted any comment containing it, so a third party could post one on someone else's pull request: runGate posts a notice only when none exists, so the author would never be told, and runSweep would then measure the 72-hour grace window from the stranger's timestamp and close them unwarned. Notices now require bot authorship. The stale-draft sweep re-listed the gate's exemptions and had lost the bot case, so a bot's long-lived draft was closable despite checkGate exempting bots. Both callers now share one exemptReason(pr) rather than keeping parallel lists that drift. Not changed: closingIssuesReferences(first: 20) truncation. It needs a pull request with 21+ closing references where only a later one carries the label, and the outcome would be a label plus the grace window, not a close. Coverage goes 11 -> 20 cases, including the stale-draft close path, which had none. Both fixes were confirmed to fail their tests when reverted. Co-Authored-By: Claude Opus 5 (1M context) --------- Co-authored-by: Aakash Kattelu Co-authored-by: Claude Opus 5 (1M context) --- .github/CODEOWNERS | 44 ++- .github/scripts/issue-gate.js | 272 ++++++++++++++++ .github/scripts/issue-gate.test.js | 139 ++++++++ .github/workflows/issue-gate.yml | 37 +++ .github/workflows/pr-sweeper.yml | 41 +++ .github/workflows/staticanalysis.yml | 8 + CONTRIBUTING.md | 452 ++++++++++++++++++--------- README.md | 70 +---- SECURITY.md | 76 ++++- docs/v2/contributing/guidelines.mdx | 24 +- docs/v3/contributing/guidelines.mdx | 426 ++++++++++++++++++------- 11 files changed, 1233 insertions(+), 356 deletions(-) create mode 100644 .github/scripts/issue-gate.js create mode 100644 .github/scripts/issue-gate.test.js create mode 100644 .github/workflows/issue-gate.yml create mode 100644 .github/workflows/pr-sweeper.yml diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 039d4e6e..09ed2279 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -8,6 +8,48 @@ # # The workflow gates only understand individual @usernames (no @org/team # entries). +# +# Order matters: GitHub applies the LAST matching pattern, so narrower rules +# go further down. Paths not listed here have no automatic reviewer. + +# Telemetry, tracing, metrics. +/src/telemetry/ @akattelu @Rajat-Ahuja1997 + +# Data model, connections, configuration, LLM transport. +/src/db.py @akattelu @eisene +/src/models.py @akattelu @eisene +/src/config.py @akattelu @eisene +/src/cache/ @akattelu @eisene +/src/crud/ @akattelu @eisene +/migrations/ @akattelu @eisene +/src/llm/ @akattelu @eisene + +# Client-facing surfaces and API shape. +/sdks/ @ajspig @akattelu +/mcp/ @ajspig @akattelu +/honcho-cli/ @ajspig @akattelu +/src/routers/ @ajspig @akattelu +/src/schemas/ @ajspig @akattelu + +# The reasoning agents, their prompts, and shared agent tooling. +/src/deriver/ @eisene @akattelu +/src/dreamer/ @eisene @akattelu +/src/dialectic/ @eisene @akattelu +/src/utils/ @eisene @akattelu + +# Deployment, and swappable storage and inference backends. +# /src/llm/backends/ must stay below /src/llm/ above — last match wins. +/docker/ @eisene @Rajat-Ahuja1997 +/Dockerfile @eisene @Rajat-Ahuja1997 +/docker-compose.yml.example @eisene @Rajat-Ahuja1997 +/src/vector_store/ @eisene @Rajat-Ahuja1997 +/src/llm/backends/ @eisene @Rajat-Ahuja1997 + +# Documentation and contributor-facing policy. +/docs/ @ajspig @akattelu +/README.md @ajspig @akattelu +/CONTRIBUTING.md @akattelu @ajspig +/SECURITY.md @Rajat-Ahuja1997 @ajspig # Reviewers auto-requested on changes under .github/ (workflows, this file, # templates). @@ -16,4 +58,4 @@ # CI-trigger allowlist only: this path matches no real file, so these people # are never auto-requested for review, but the workflow gates still pick # them up. -/ci-trigger-allowlist @3un01a @adavyas @ajspig @courtlandleer @erosika @lowyelling @matthewlanders @vintrocode +/ci-trigger-allowlist @ajspig @courtlandleer @erosika @lowyelling @vintrocode diff --git a/.github/scripts/issue-gate.js b/.github/scripts/issue-gate.js new file mode 100644 index 00000000..c9bbb668 --- /dev/null +++ b/.github/scripts/issue-gate.js @@ -0,0 +1,272 @@ +'use strict'; + +/** + * Issue gate — shared logic for `.github/workflows/issue-gate.yml` (immediate + * feedback on pull request events) and `.github/workflows/pr-sweeper.yml` + * (deferred re-check, close, and stale-draft cleanup). + * + * Both workflows `require` this file through actions/github-script, so it must + * stay dependency-free: neither job runs an install step. + * + * See CONTRIBUTING.md for the policy this enforces. + */ + +const REQUIRED_LABEL = 'maintainer-approved'; +const GATE_LABEL = 'needs-approved-issue'; +const EXEMPT_LABEL = 'gate-exempt'; +const MARKER = ''; +const DISCORD = 'http://discord.gg/honcho'; + +// Hours a labelled pull request has before the sweeper closes it. Measured from +// the notice comment, so the clock starts when the author was actually told — +// not when the pull request was opened. +const GRACE_HOURS = 72; + +// Days without activity before a draft from outside the org is closed. +const DRAFT_STALE_DAYS = 30; + +const hasLabel = (pr, name) => (pr.labels || []).some((l) => l.name === name); + +const isBot = (account) => Boolean(account) && account.type === 'Bot'; + +/** + * Why this pull request is exempt from the gate, or null if it is not. + * + * Single source of truth: every caller that acts on a pull request runs this. + * The stale-draft sweep previously re-listed these checks and silently lost the + * bot case. + */ +const exemptReason = (pr) => { + if (isBot(pr.user)) return 'author is a bot'; + if (WRITE_ACCESS.includes(pr.author_association)) { + return `author_association is ${pr.author_association}`; + } + if (hasLabel(pr, EXEMPT_LABEL)) return `carries the ${EXEMPT_LABEL} label`; + return null; +}; + +// Write access to the repository. CONTRIBUTOR is deliberately absent: GitHub uses +// it for "has previously committed to the repository", which describes every +// returning outside contributor, not a maintainer. Do not add it. +const WRITE_ACCESS = ['OWNER', 'MEMBER', 'COLLABORATOR']; + +const CLOSING_ISSUES = ` + query($owner: String!, $repo: String!, $number: Int!) { + repository(owner: $owner, name: $repo) { + pullRequest(number: $number) { + closingIssuesReferences(first: 20) { + nodes { + number + state + labels(first: 50) { nodes { name } } + } + } + } + } + } +`; + +/** + * Decide whether a pull request clears the gate. + * + * Reads GitHub's own resolved issue links rather than parsing the body, so both + * `Fixes #123` and the sidebar "Development" link count. A bare `#123` mention + * deliberately does not — that is a reference, not a claim to close. + * + * @returns {Promise<{passed: boolean, skipped?: string, issue?: number, reason?: string}>} + */ +async function checkGate({ github, owner, repo, pr }) { + if (pr.state !== 'open') return { passed: true, skipped: 'pull request is not open' }; + if (pr.draft) return { passed: true, skipped: 'pull request is a draft' }; + const exempt = exemptReason(pr); + if (exempt) return { passed: true, skipped: exempt }; + + const data = await github.graphql(CLOSING_ISSUES, { owner, repo, number: pr.number }); + const issues = data.repository.pullRequest.closingIssuesReferences.nodes; + + if (issues.length === 0) { + return { passed: false, reason: 'This pull request is not linked to an issue.' }; + } + + const approved = issues.find( + (i) => i.state === 'OPEN' && i.labels.nodes.some((l) => l.name === REQUIRED_LABEL), + ); + if (approved) return { passed: true, issue: approved.number }; + + const detail = issues + .map((i) => `#${i.number} (${i.state === 'CLOSED' ? 'closed' : 'not approved'})`) + .join(', '); + return { + passed: false, + reason: + `The linked ${issues.length === 1 ? 'issue is' : 'issues are'} not open with the ` + + `\`${REQUIRED_LABEL}\` label: ${detail}.`, + }; +} + +function noticeBody({ owner, repo, reason }) { + return [ + MARKER, + 'Thanks for the contribution. This pull request does not clear our issue gate yet.', + '', + `**${reason}**`, + '', + `Every pull request to Honcho needs to be linked to an open issue carrying the \`${REQUIRED_LABEL}\` label. We do this so the review queue only holds work we have already agreed should be built — it means nobody spends time on a change we cannot merge.`, + '', + 'To get this moving:', + '', + `1. Find or open an issue describing the change. [Approved issues are here](https://github.com/${owner}/${repo}/issues?q=is%3Aissue+is%3Aopen+label%3A${REQUIRED_LABEL}).`, + `2. Make the case for it in [Discord](${DISCORD}) — maintainers are most active there, and it is by far the fastest route to a decision.`, + `3. Once the issue has the label, link it: put \`Fixes #\` in this pull request's description, or use **Development** in the sidebar.`, + '', + `**This will close automatically in ${GRACE_HOURS} hours if it is still unlinked.** Nothing is lost if that happens — link the issue, reopen, and it goes into the review queue.`, + '', + `See [CONTRIBUTING.md](https://github.com/${owner}/${repo}/blob/main/CONTRIBUTING.md) for the full process. If you think this is wrong, say so here and a maintainer will take a look.`, + ].join('\n'); +} + +/** + * Every gate notice this bot posted on a pull request, oldest first. + * + * Authorship is part of the test, not decoration. MARKER is an invisible HTML + * comment, so anyone who can comment on a public repository can paste it. If + * user comments counted, a third party could post one on someone else's pull + * request: `runGate` posts a notice only when none exists, so the author would + * never be told, and `runSweep` would then measure the grace window from the + * stranger's timestamp and close them unwarned. + */ +async function findNotices({ github, owner, repo, number }) { + const comments = await github.paginate(github.rest.issues.listComments, { + owner, repo, issue_number: number, per_page: 100, + }); + return comments.filter((c) => isBot(c.user) && (c.body || '').includes(MARKER)); +} + +/** + * Drop the gate label and delete the notice. + * + * Deleting matters: `runGate` posts a notice only when none exists, and the + * sweeper measures grace from the notice timestamp. A notice left behind after + * the gate clears would make a later re-block look weeks old and be closed with + * no warning. + */ +async function clearGate({ github, owner, repo, pr }) { + if (hasLabel(pr, GATE_LABEL)) { + await github.rest.issues + .removeLabel({ owner, repo, issue_number: pr.number, name: GATE_LABEL }) + .catch(() => {}); + } + for (const notice of await findNotices({ github, owner, repo, number: pr.number })) { + await github.rest.issues + .deleteComment({ owner, repo, comment_id: notice.id }) + .catch(() => {}); + } +} + +/** + * Entry point for `.github/workflows/issue-gate.yml`. + * Labels and explains. Never closes — that is the sweeper's job. + */ +async function runGate({ github, core, context }) { + const pr = context.payload.pull_request; + const { owner, repo } = context.repo; + const result = await checkGate({ github, owner, repo, pr }); + + if (result.passed) { + core.info( + result.skipped ? `Skipping gate: ${result.skipped}` : `Gate passed via #${result.issue}`, + ); + await clearGate({ github, owner, repo, pr }); + return; + } + + core.warning(`Gate failed: ${result.reason}`); + await github.rest.issues.addLabels({ + owner, repo, issue_number: pr.number, labels: [GATE_LABEL], + }); + + const notices = await findNotices({ github, owner, repo, number: pr.number }); + if (notices.length > 0) return; + + await github.rest.issues.createComment({ + owner, repo, issue_number: pr.number, + body: noticeBody({ owner, repo, reason: result.reason }), + }); +} + +/** Entry point for `.github/workflows/pr-sweeper.yml`. */ +async function runSweep({ github, core, context, dryRun }) { + const { owner, repo } = context.repo; + + const act = async (what, fn) => { + core.info(dryRun ? `[dry run] ${what}` : what); + if (!dryRun) await fn(); + }; + + const close = (pr, body) => async () => { + await github.rest.issues.createComment({ owner, repo, issue_number: pr.number, body }); + await github.rest.pulls.update({ owner, repo, pull_number: pr.number, state: 'closed' }); + }; + + const prs = await github.paginate(github.rest.pulls.list, { + owner, repo, state: 'open', per_page: 100, + }); + core.info(`${prs.length} open pull requests${dryRun ? ' (dry run)' : ''}`); + + // Re-check everything wearing the gate label. Never close blind: a pull request + // linked through the sidebar fires no webhook, so the gate workflow cannot have + // noticed it — this pass is the only thing that will. + for (const pr of prs.filter((p) => hasLabel(p, GATE_LABEL))) { + const result = await checkGate({ github, owner, repo, pr }); + + if (result.passed) { + const why = result.skipped || `via #${result.issue}`; + await act(`#${pr.number}: gate now clear (${why})`, async () => { + await clearGate({ github, owner, repo, pr }); + await github.rest.issues.createComment({ + owner, repo, issue_number: pr.number, + body: 'The issue link is in place — this pull request has cleared the gate and is waiting on review.', + }); + }); + continue; + } + + const [notice] = await findNotices({ github, owner, repo, number: pr.number }); + if (!notice) { + core.info(`#${pr.number}: labelled but never notified — leaving it for the gate workflow`); + continue; + } + + const hours = (Date.now() - Date.parse(notice.created_at)) / 3_600_000; + if (hours < GRACE_HOURS) { + core.info(`#${pr.number}: ${Math.round(GRACE_HOURS - hours)}h of grace left`); + continue; + } + + await act(`#${pr.number}: closing — notified ${Math.round(hours)}h ago, still failing`, close(pr, + `Closing this: ${GRACE_HOURS} hours have passed and the gate is still not clear. This is not a judgement on the code. Link an approved issue and reopen — it goes straight into the review queue.`, + )); + } + + // Stale drafts. The gate skips drafts entirely, so they never carry the label; + // this pass keys off inactivity and applies the shared exemptions itself. + for (const pr of prs.filter((p) => p.draft)) { + const exempt = exemptReason(pr); + if (exempt) { + core.info(`#${pr.number}: leaving stale draft alone — ${exempt}`); + continue; + } + + const days = (Date.now() - Date.parse(pr.updated_at)) / 86_400_000; + if (days < DRAFT_STALE_DAYS) continue; + + await act(`#${pr.number}: closing stale draft — ${Math.round(days)}d without activity`, close(pr, + `Closing this draft after ${DRAFT_STALE_DAYS} days without activity, to keep the pull request list readable. Reopen whenever you pick it back up — nothing here is lost.`, + )); + } +} + +module.exports = { + checkGate, runGate, runSweep, noticeBody, findNotices, exemptReason, + REQUIRED_LABEL, GATE_LABEL, EXEMPT_LABEL, MARKER, GRACE_HOURS, DRAFT_STALE_DAYS, +}; diff --git a/.github/scripts/issue-gate.test.js b/.github/scripts/issue-gate.test.js new file mode 100644 index 00000000..695df3a2 --- /dev/null +++ b/.github/scripts/issue-gate.test.js @@ -0,0 +1,139 @@ +'use strict'; + +// Self-check for the gate decision logic. No framework, no install: +// node .github/scripts/issue-gate.test.js +// Covers checkGate() only — the side-effecting halves (runGate/runSweep) are +// exercised against the real API via `pr-sweeper.yml`'s dry_run dispatch. + +const assert = require('node:assert'); +const { + checkGate, findNotices, runSweep, REQUIRED_LABEL, EXEMPT_LABEL, MARKER, +} = require('./issue-gate.js'); + +const pull = (over = {}) => ({ + number: 1, state: 'open', draft: false, + user: { type: 'User' }, author_association: 'NONE', labels: [], + ...over, +}); + +// `linked` is the list of issues GitHub resolves as closing references. +const stub = (linked) => ({ + graphql: async () => ({ + repository: { pullRequest: { closingIssuesReferences: { + nodes: linked.map((i) => ({ + number: i.number, state: i.state || 'OPEN', + labels: { nodes: (i.labels || []).map((name) => ({ name })) }, + })), + } } }, + }), +}); + +const run = (linked, over) => + checkGate({ github: stub(linked), owner: 'o', repo: 'r', pr: pull(over) }); + +const cases = [ + ['no linked issue fails', () => run([]), (r) => r.passed === false], + ['linked but unapproved fails', () => run([{ number: 7 }]), (r) => r.passed === false], + ['linked and approved passes', + () => run([{ number: 7, labels: [REQUIRED_LABEL] }]), + (r) => r.passed === true && r.issue === 7], + ['approved but closed fails', + () => run([{ number: 7, state: 'CLOSED', labels: [REQUIRED_LABEL] }]), + (r) => r.passed === false], + ['picks the approved one out of several', + () => run([{ number: 7 }, { number: 8, labels: [REQUIRED_LABEL] }]), + (r) => r.passed === true && r.issue === 8], + + // Exemptions. + ['maintainer skips', () => run([], { author_association: 'MEMBER' }), (r) => r.passed === true], + ['collaborator skips', () => run([], { author_association: 'COLLABORATOR' }), (r) => r.passed === true], + ['bot skips', () => run([], { user: { type: 'Bot' } }), (r) => r.passed === true], + ['draft skips', () => run([], { draft: true }), (r) => r.passed === true], + [`${EXEMPT_LABEL} skips`, () => run([], { labels: [{ name: EXEMPT_LABEL }] }), (r) => r.passed === true], + + // Regression guard: GitHub hands CONTRIBUTOR to anyone who has previously + // committed, i.e. every returning outside contributor. It must stay gated. + ['CONTRIBUTOR is still gated', + () => run([], { author_association: 'CONTRIBUTOR' }), + (r) => r.passed === false], +]; + +// --- findNotices: only the bot's own notices count ------------------------- +// A stranger pasting the invisible MARKER into a comment must not suppress the +// notice or become the grace-window clock. +const commentsStub = (comments) => ({ + paginate: async () => comments, + rest: { issues: { listComments: null } }, +}); + +const noticeCases = [ + ['a user comment carrying MARKER is not a notice', + [{ id: 1, user: { type: 'User' }, body: `sneaky ${MARKER}`, created_at: 'x' }], 0], + ['a bot comment carrying MARKER is a notice', + [{ id: 2, user: { type: 'Bot' }, body: `${MARKER}\nnotice`, created_at: 'x' }], 1], + ['a bot comment without MARKER is not a notice', + [{ id: 3, user: { type: 'Bot' }, body: 'unrelated', created_at: 'x' }], 0], + ['a user MARKER does not mask the real bot notice', + [{ id: 4, user: { type: 'User' }, body: MARKER, created_at: 'x' }, + { id: 5, user: { type: 'Bot' }, body: MARKER, created_at: 'y' }], 1], +]; + +// --- runSweep: the stale-draft pass must honour every exemption ------------ +const draft = (over) => ({ + number: 9, draft: true, state: 'open', labels: [], + user: { type: 'User' }, author_association: 'NONE', + updated_at: new Date(Date.now() - 400 * 86400_000).toISOString(), + ...over, +}); + +async function sweepClosed(pr) { + const closed = []; + const github = { + paginate: async (route) => (route === 'pulls' ? [pr] : []), + rest: { + pulls: { + list: 'pulls', + update: async ({ pull_number }) => closed.push(pull_number), + }, + issues: { listComments: 'comments', createComment: async () => {} }, + }, + }; + await runSweep({ + github, core: { info() {}, warning() {} }, + context: { repo: { owner: 'o', repo: 'r' } }, dryRun: false, + }); + return closed; +} + +const sweepCases = [ + ['stale draft from an outside author closes', draft({}), 1], + ['stale draft from a bot is left alone', draft({ user: { type: 'Bot' } }), 0], + ['stale draft from a maintainer is left alone', draft({ author_association: 'MEMBER' }), 0], + [`stale draft with ${EXEMPT_LABEL} is left alone`, draft({ labels: [{ name: EXEMPT_LABEL }] }), 0], + ['recent draft is left alone', draft({ updated_at: new Date().toISOString() }), 0], +]; + +(async () => { + let failed = 0; + for (const [name, comments, want] of noticeCases) { + const got = (await findNotices({ github: commentsStub(comments), owner: 'o', repo: 'r', number: 1 })).length; + if (got === want) console.log(` ok ${name}`); + else { failed++; console.log(` FAIL ${name} -> ${got} notices, wanted ${want}`); } + } + for (const [name, pr, want] of sweepCases) { + const got = (await sweepClosed(pr)).length; + if (got === want) console.log(` ok ${name}`); + else { failed++; console.log(` FAIL ${name} -> closed ${got}, wanted ${want}`); } + } + for (const [name, thunk, ok] of cases) { + const result = await thunk(); + if (ok(result)) { + console.log(` ok ${name}`); + } else { + failed++; + console.log(` FAIL ${name} -> ${JSON.stringify(result)}`); + } + } + assert.strictEqual(failed, 0, `${failed} case(s) failed`); + console.log(`\n${cases.length + noticeCases.length + sweepCases.length} passed`); +})(); diff --git a/.github/workflows/issue-gate.yml b/.github/workflows/issue-gate.yml new file mode 100644 index 00000000..74df40c7 --- /dev/null +++ b/.github/workflows/issue-gate.yml @@ -0,0 +1,37 @@ +name: Issue Gate + +# Labels pull requests that are not linked to an issue carrying the +# `maintainer-approved` label, and comments explaining how to fix it. +# +# This workflow never closes anything. `pr-sweeper.yml` re-checks later and closes +# only after the grace period — that gives contributors time to link an issue, and +# gives maintainers time to wave through a one-line fix. It is also the only thing +# that can notice a sidebar issue link, which fires no webhook of its own. +# +# `pull_request_target` is required so the job has write access on pull requests +# from forks. It must therefore NEVER run code from the pull request. The checkout +# below is safe because on `pull_request_target` actions/checkout defaults to the +# BASE ref, which is repo-trusted code. Never point it at `pr.head.sha`. +# +# Not triggered on `synchronize`: re-running on every push would be noise. +# Drafts are ignored until marked ready. + +on: + pull_request_target: + types: [opened, edited, reopened, ready_for_review] + +permissions: + contents: read + issues: write + pull-requests: write + +jobs: + gate: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/github-script@v7 + with: + script: | + const gate = require(`${process.env.GITHUB_WORKSPACE}/.github/scripts/issue-gate.js`); + await gate.runGate({ github, core, context }); diff --git a/.github/workflows/pr-sweeper.yml b/.github/workflows/pr-sweeper.yml new file mode 100644 index 00000000..d5bde42c --- /dev/null +++ b/.github/workflows/pr-sweeper.yml @@ -0,0 +1,41 @@ +name: PR Sweeper + +# Deferred half of the issue gate. Every six hours: +# +# 1. Re-check every pull request carrying `needs-approved-issue`. Clear the ones +# that now link an approved issue; close the ones still failing 72h after they +# were told. The re-check is the point — linking an issue through the sidebar +# fires no webhook, so `issue-gate.yml` never sees it. +# 2. Close drafts from outside the org after 30 days without activity. +# +# Runs on `schedule`, so it never touches pull request code and needs none of the +# `pull_request_target` precautions. Dispatch manually with dry_run to see what it +# would do before it does it. + +on: + schedule: + - cron: '17 */6 * * *' + workflow_dispatch: + inputs: + dry_run: + description: 'Log intended actions without closing anything' + type: boolean + default: true + +permissions: + contents: read + issues: write + pull-requests: write + +jobs: + sweep: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/github-script@v7 + env: + DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || 'false' }} + with: + script: | + const gate = require(`${process.env.GITHUB_WORKSPACE}/.github/scripts/issue-gate.js`); + await gate.runSweep({ github, core, context, dryRun: process.env.DRY_RUN === 'true' }); diff --git a/.github/workflows/staticanalysis.yml b/.github/workflows/staticanalysis.yml index 18bb37fd..9cad8b10 100644 --- a/.github/workflows/staticanalysis.yml +++ b/.github/workflows/staticanalysis.yml @@ -26,3 +26,11 @@ jobs: run: uv sync --all-extras --dev - name: run basedpyright run: uv run basedpyright + + issue-gate: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + # The gate runs from `pull_request_target`, where a crash is invisible until + # a contributor's PR is silently ungated. Check it here instead. + - run: node .github/scripts/issue-gate.test.js diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 23eb7c58..7b131f70 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,219 +1,371 @@ # Contributing to Honcho -Thank you for your interest in contributing to Honcho! This guide outlines the process for contributing to the project and our development conventions. + -## Getting Started +Thanks for your interest in contributing. This guide covers how work gets accepted, how +Honcho is put together, and what a mergeable pull request looks like. -Before you start contributing, please: +Honcho is a small team maintaining a project that gets more proposals than we can review. +The rules below exist so that the work you do has somewhere to land — not to keep you out. -1. **Set up your development environment** - Follow the [Local Development guide](./README.md#local-development) in the README to get Honcho running locally. +## Contents -2. **Join our community** - Feel free to join us in our [Discord](http://discord.gg/honcho) to discuss your changes, get help, or ask questions. +- [Before you write code](#before-you-write-code) +- [What gets prioritized](#what-gets-prioritized) +- [If you're an agent](#if-youre-an-agent) +- [How Honcho works](#how-honcho-works) +- [Where to change what](#where-to-change-what) +- [Local setup](#local-setup) +- [Making the change](#making-the-change) +- [Opening the pull request](#opening-the-pull-request) +- [Reporting bugs and requesting features](#reporting-bugs-and-requesting-features) +- [Security](#security) +- [License](#license) -3. **Review existing issues** - Check the [issues tab](https://github.com/plastic-labs/honcho/issues) to see what's already being worked on or to find something to contribute to. +## Before you write code -## Contribution Workflow +**Every pull request needs an issue, and that issue needs the `maintainer-approved` label.** -### 1. Fork and Clone +A pull request that is not linked to an approved issue gets labelled +`needs-approved-issue`, with a comment explaining why. You then have 72 hours to link one +before it is closed automatically. Reopening costs nothing once the link is in place. This +is automated. We do this because an unreviewable backlog helps nobody: a PR against an +unapproved issue is work you did that we may not be able to merge, no matter how good it +is. -1. Fork the repository on GitHub -2. Clone your fork locally: +So, in order: - ```bash - git clone https://github.com/YOUR_USERNAME/honcho.git - cd honcho - ``` +1. **Find approved work.** Browse + [issues labelled `maintainer-approved`](https://github.com/plastic-labs/honcho/issues?q=is%3Aissue+is%3Aopen+label%3Amaintainer-approved). + That label is the queue of things we have agreed should be built. Anything in it is fair + game — comment on the issue to claim it. -3. Add the upstream repository as a remote: +2. **Or open an issue and get it approved.** Use the + [issue templates](https://github.com/plastic-labs/honcho/issues/new/choose). Maintainers + triage and apply the label. - ```bash - git remote add upstream https://github.com/plastic-labs/honcho.git - ``` +3. **If you feel strongly about an issue, come to [Discord](https://discord.gg/honcho).** + This is the fastest path by a wide margin. Maintainers are more active there than in the + issue tracker, and a five-minute conversation about what you want to build usually + resolves whether it fits before either side spends real time on it. -### 2. Create a Branch +4. **Then open the PR** and link the issue — either `Fixes #123` in the description, or + **Development → link an issue** in the sidebar. Both work. -Create a new branch for your feature or bug fix: +Small exceptions we will not be pedantic about: fixing a typo, a broken link, or an +obviously wrong code sample. Open the PR, explain it in one line, and we will sort out the +issue linkage. + +## What gets prioritized + +Roughly, work on Honcho falls along these axes. Knowing which one your idea sits on tells +you a lot about how likely it is to get approved. + +| Axis | What it covers | +| --- | --- | +| **Observability** | Understanding how Honcho behaves in production — telemetry, tracing, CloudEvents, metrics. | +| **Memory quality** | Better conclusions from the same input — the deriver, dreamer, and dialectic; eval results. | +| **Developer experience** | Fitting cleanly into more application architectures — SDKs, scopes, composable peers, the CLI. | +| **Breadth of input** | Widening what Honcho can ingest and represent — multimodal and non-conversational data. | +| **Ubiquity** | Reachable wherever a developer already works — integrations, self-hosting, alternate vector-store and inference backends, local-first defaults. | +| **Reliability and cost** | Trustworthy in production — connection and concurrency hardening, queue throughput, cost per token. | + +In practice, **Ubiquity** and **Developer experience** are where outside contributions land +most easily. A new integration, a self-hosting rough edge, a vector-store or inference +backend, an SDK ergonomics fix — these are additive and rarely collide with work already in +flight. + +Changes to the reasoning pipeline itself — deriver prompts, dialectic tool design, dreamer +strategy — are the hardest to accept from outside. Not because they are unwelcome, but +because they are measured against eval results we run internally, and they frequently +conflict with in-flight work. Talk to us in Discord first, always. + +## If you're an agent + +If you are a coding agent working on this repository, read this section before writing code. +The most common failure we see is a well-formed, well-tested pull request against an issue +that was never approved. That gets closed, and the work is wasted. + +- **Check the gate first.** Before writing code: + + ```bash + gh issue view --repo plastic-labs/honcho --json number,title,state,labels + ``` + + Stop if there is no issue number, if the issue is closed, or if `maintainer-approved` is + not in the labels. Report that to the person you are working with instead of proceeding. + +- **Do not open a PR in order to establish the issue link afterwards.** The issue comes + first. + +- **Do not report checks you did not run.** If you did not execute the test command, say so. + A PR body claiming a green run that did not happen costs a maintainer more time than no + claim at all. + +- **Use the checklist.** [`skills/pre-pr/SKILL.md`](./skills/pre-pr/SKILL.md) in this repo + encodes the gate, the test-layer matrix, and the PR body format. If your harness supports + skills, invoke it rather than reimplementing the checks. + +## How Honcho works + +Enough architecture to find your way around. For the user-facing model — what a Peer is, what +`get_context` returns — see [Core Concepts in the README](./README.md#core-concepts) and the +[documentation](https://honcho.dev/docs/). + +### Two processes + +Honcho runs as two cooperating processes over a shared Postgres database and Redis cache. + +| | API server | Deriver worker | +| --- | --- | --- | +| Start | `uv run fastapi dev src/main.py` | `uv run python -m src.deriver` | +| Entry | `src/main.py` | `src/deriver/__main__.py` | +| Does | Serves HTTP, enqueues background work, returns immediately | Consumes the queue: Deriver, Summarizer, Dreamer, Reconciler | +| Hosts | The Dialectic agent, inline on the request path | Everything else | + +The split is the load-bearing design decision: **an HTTP request never blocks on LLM work**, +with the single exception of the Dialectic chat endpoint, which is synchronous by nature. +If you are adding something slow, it belongs in the worker. + +The deriver is a separate process. If messages go in and nothing ever comes out, the usual +cause is that nobody started it. + +### The path of a message + +Worth tracing once, because it crosses most of the codebase: + +1. `POST /v3/workspaces/{w}/sessions/{s}/messages` lands in `src/routers/messages.py`. +2. The row is written, then `enqueue()` in `src/deriver/enqueue.py` creates `queue_item` + rows — one set of work per observing peer. +3. `src/deriver/queue_manager.py` polls the queue, claiming work units so that messages in a + session are processed in order. +4. `process_item()` in `src/deriver/consumer.py` dispatches on task type — representation, + summary, deletion, reconciliation. +5. For a representation task, `process_representation_tasks_batch()` in + `src/deriver/deriver.py` makes **one structured-output LLM call for the whole batch** and + writes the resulting conclusions into the collection keyed by the + `(observer, observed)` peer pair. +6. Later, `src/dialectic/` reads those conclusions back at recall time to answer a chat + request. + +Embedding is deliberately *not* on this path. `MessageEmbedding` rows are written with +`sync_state='pending'` and embedded asynchronously by the Reconciler +(`src/reconciler/sync_vectors.py`), which runs on a scheduler inside the deriver process. + +### The four agents + +They share tool definitions in `src/utils/agent_tools.py` and the provider-agnostic LLM +client in `src/llm/`. Each has its own `MODEL_CONFIG` with a fallback chain in +`src/config.py`. + +| Agent | Where | Shape | +| --- | --- | --- | +| **Deriver** | `src/deriver/` | A single structured-output call per message batch. Not a tool loop — this is a deliberate cost and latency tradeoff. | +| **Dialectic** | `src/dialectic/` | The one tool-using agent on the request path. Loops over tools until it can answer. Five reasoning tiers from `minimal` to `max`, each with its own model and tool set. | +| **Dreamer** | `src/dreamer/` | Off-queue consolidation. Two specialist phases (deduction, then induction) that build reasoning trees over existing conclusions. | +| **Summarizer** | `src/utils/summarizer.py` | Direct LLM call, no tools. Two tiers — short and long summaries at different message counts. | + +Prompts live in `src/deriver/prompts.py`, `src/dialectic/prompts.py`, and +`src/dreamer/specialists.py`. + +### A note on naming + +What the public API and documentation call **conclusions** are called **observations** +throughout the code — `create_observations`, `get_observation_context`, and so on. Likewise +**collections** and **documents** are internal storage concepts that are not exposed +directly through the API. Do not rename across that boundary in a drive-by change; the +public and internal vocabularies are being reconciled deliberately. + +## Where to change what + +| I want to change... | Start here | +| --- | --- | +| An HTTP endpoint | `src/routers/` — one module per resource | +| A database query | `src/crud/` — mirrors the router layout | +| The database schema | `src/models.py`, plus a migration in `migrations/versions/` | +| A configuration value | `src/config.py`, and add it to `config.toml.example` and `.env.template` | +| A tool an agent can call | `src/utils/agent_tools.py` — definitions plus the per-agent tool lists | +| A prompt | `src/deriver/prompts.py`, `src/dialectic/prompts.py`, `src/dreamer/specialists.py` | +| LLM provider behavior | `src/llm/backends/` — `anthropic.py`, `gemini.py`, `openai.py` | +| Embeddings or vector storage | `src/embedding_client.py`, `src/vector_store/` | +| Telemetry or metrics | `src/telemetry/` — see the notes in `CLAUDE.md` before adding an event type | +| Authentication and scoping | `src/security.py`, `src/dependencies.py` | +| The Python or TypeScript SDK | `sdks/python/`, `sdks/typescript/` | +| The CLI | `honcho-cli/` | +| The MCP server | `mcp/` | +| Public documentation | `docs/v3/` — Mintlify; nav lives in `docs/docs.json` | + +Tests in `tests/` mirror `src/`. `CLAUDE.md` at the repo root has more detail on house +conventions, and is worth skimming even if you are not using an agent. + +## Local setup + +Get a stack running first — [Self-hosting in the README](./README.md#self-hosting) covers +both the Docker path and a manual Postgres setup. Then, for development: + +```bash +uv sync # create the venv and install dependencies +uv run alembic upgrade head # apply migrations +``` + +Run both processes, in separate terminals: + +```bash +uv run fastapi dev src/main.py # API server, reloads on change +uv run python -m src.deriver # background worker +``` + +Everything Python goes through `uv run`. Redis is optional for local development; without it +caching is simply disabled. + +## Making the change + +### Branches and commits ```bash git checkout -b feature/your-feature-name -# or -git checkout -b fix/your-bug-fix-name ``` -**Branch naming conventions:** +Prefixes: `feature/`, `fix/`, `docs/`, `refactor/`, `test/`. -- `feature/description` - for new features -- `fix/description` - for bug fixes -- `docs/description` - for documentation updates -- `refactor/description` - for code refactoring -- `test/description` - for adding or updating tests - -### 3. Make Your Changes - -- Write clean, readable code that follows our coding standards (see below) -- Add tests for new functionality -- Update documentation as needed -- Make sure your changes don't break existing functionality - -### 4. Commit Your Changes - -We follow conventional commit standards. Format your commit messages as: - -``` -type(scope): description - -[optional body] - -[optional footer] -``` - -**Types:** - -- `feat`: A new feature -- `fix`: A bug fix -- `docs`: Documentation only changes -- `style`: Changes that do not affect the meaning of the code -- `refactor`: A code change that neither fixes a bug nor adds a feature -- `test`: Adding missing tests or correcting existing tests -- `chore`: Changes to the build process or auxiliary tools - -**Examples:** +Commits follow [Conventional Commits](https://www.conventionalcommits.org/), enforced by a +`commit-msg` hook: ```bash git commit -m "feat(api): add new dialectic endpoint for user insights" git commit -m "fix(db): resolve connection pool timeout issue" -git commit -m "docs(readme): update installation instructions" ``` -### 5. Submit a Pull Request +Types: `feat`, `fix`, `docs`, `style`, `refactor`, `test`, `chore`. -1. Push your branch to your fork: +### Pre-commit hooks - ```bash - git push origin your-branch-name - ``` - -2. Create a pull request on GitHub from your branch to the `main` branch - -3. Fill out the pull request template with: - - A clear description of what changes you've made - - The motivation for the changes - - Any relevant issue numbers (use "Closes #123" to auto-close issues) - - Screenshots or examples if applicable - -## Pre-commit Hooks - -Honcho uses pre-commit hooks to enforce code quality and consistency. They run linting, formatting, type checking, and security scans before each commit. - -### Installation +Install them. CI runs the same checks, and it is much faster to find out locally. ```bash -uv add --dev pre-commit uv run pre-commit install \ --hook-type pre-commit \ --hook-type commit-msg \ --hook-type pre-push ``` -### What the hooks do +At **commit** time: ruff lint and format, biome for TypeScript, basedpyright, bandit, +markdownlint, and file hygiene. At **push** time: pytest, the alembic migration tests, and +the SDK builds. -- **Code Quality** — Python linting and formatting (ruff), TypeScript linting (biome) -- **Type Checking** — Static analysis with basedpyright -- **Security** — Vulnerability scanning with bandit -- **Documentation** — Markdown linting and license header checks -- **Testing** — Automated test runs for Python and TypeScript -- **File Hygiene** — Trailing whitespace, line endings, file size checks -- **Commit Standards** — Conventional commit message validation +That split matters — **a clean commit is not a clean push.** The test suite only runs at +`pre-push`, so the first time you see test failures may be well after you thought you were +done. -### Manual execution - -Run against all files without committing: +Run them by hand at any time: ```bash uv run pre-commit run --all-files +uv run pre-commit run ruff --all-files ``` -Run a specific hook: +Or the individual tools: ```bash -uv run pre-commit run ruff --all-files -uv run pre-commit run basedpyright --all-files +uv run ruff check src/ +uv run ruff format src/ +uv run basedpyright ``` -## Coding Standards +### Tests -### Python Code Style +Write tests for new functionality, in the directory under `tests/` that mirrors the code you +changed. Which layer you need depends on what you touched: -- Follow [PEP 8](https://www.python.org/dev/peps/pep-0008/) style guidelines -- Use [ruff](https://docs.astral.sh/ruff/) for linting and code formatting -- Use type hints where possible -- Write docstrings for functions and classes using Google style docstrings +| What you changed | What to run | +| --- | --- | +| Anything in `src/` | Unit tests in the matching `tests/` tree — `uv run pytest tests/...` | +| Deriver, dialectic, dreamer, or the LLM path | Unit tests, and consider `tests/live_llm/` (gated behind `--live-llm`) | +| Queue behavior, config hierarchy, multi-turn flows, SDK contracts | `uv run python -m tests.unified.run` | +| A `/v3` endpoint or deriver queue behavior | Actually run the stack and exercise it — not just pytest | +| A migration | `uv run python scripts/run_alembic_tests.py`; every revision needs a test file | -### Code Organization - -- Keep functions focused and single-purpose -- Use meaningful variable and function names -- Add comments for complex logic -- Follow existing patterns in the codebase - -### Testing - -- Write unit tests for new functionality -- Ensure existing tests pass before submitting -- Use descriptive test names that explain what is being tested -- Mock external dependencies appropriately +The TypeScript SDK tests need a running server with a database and Redis, which pytest +orchestrates. Run them with `uv run pytest tests/ -k typescript` from the repo root — +`bun test` on its own will fail. To type-check the SDK alone: +`cd sdks/typescript && bun run tsc --noEmit`. ### Documentation -- Update relevant documentation for new features -- Include examples in docstrings where helpful -- Keep README and other docs up to date with changes +Update docs in the same PR when you change a public surface: `/v3` endpoints, SDK exports, +or anything in `config.toml` / settings. Docs live in `docs/v3/`, and new pages need an entry +in `docs/docs.json` or they will not appear in the nav. -## Review Process +## Opening the pull request -1. **Automated checks** - Your PR will run through automated checks including tests and linting -2. **Project maintainer review** - A project maintainer will review your code for: - - Code quality and adherence to standards - - Functionality and correctness - - Test coverage - - Documentation completeness -3. **Discussion and iteration** - You may be asked to make changes or clarifications -4. **Approval and merge** - Once approved, your PR will be merged into `main` +### Leave "Allow edits by maintainers" checked -## Types of Contributions +This is the single most useful thing you can do to get your PR merged quickly. -We welcome various types of contributions: +Most contributor PRs arrive nearly right, needing a rename, a missing test, or a lint fix. +If we can push that commit ourselves, it merges the same day. If we cannot, it becomes a +review comment, and then we wait — sometimes for weeks — for a round trip on a two-line +change. -- **Bug fixes** - Help us squash bugs and improve stability -- **New features** - Add functionality that benefits the community -- **Documentation** - Improve or expand our documentation -- **Tests** - Increase test coverage and reliability -- **Performance improvements** - Help make Honcho faster and more efficient -- **Examples and tutorials** - Help other developers use Honcho +GitHub checks the box by default when you fork. Leave it checked. -## Issue Reporting +One caveat worth knowing: **the option does not exist on forks owned by an organization.** +If you have the choice, fork from your personal account. -When reporting bugs or requesting features: +### Fill out the template -1. Check if the issue already exists -2. Use the appropriate [issue template](https://github.com/plastic-labs/honcho/issues/new/choose) (bug, memory/recall quality, feature, integration, or documentation) -3. Provide clear reproduction steps for bugs -4. Include relevant environment information (managed vs self-hosted, server version, SDK) -5. Be specific about expected vs actual behavior -6. Redact secrets, JWTs, and production user content +`.github/pull_request_template.md` asks for a description, proofs, and the issue checkbox. -## Questions and Support +"Proofs" means evidence the change works: the command you ran and its result, a log snippet, +a screenshot, the failing case before and after. This is the section that most determines +how fast your PR gets reviewed. Do not add sections to the template. -- **General questions** - Join our [Discord](https://discord.gg/honcho) -- **Bug reports** - GitHub issues → Bug report template -- **Memory / recall quality** - GitHub issues → Memory / recall quality template -- **Feature requests** - GitHub issues → Feature request template -- **Integrations / plugins / app-store listings** - GitHub issues → Integration request template -- **Documentation issues** - GitHub issues → Documentation issue template -- **Security issues** - Report **privately** only — see [`SECURITY.md`](./SECURITY.md) (GitHub Private Vulnerability Reporting or email). Do not open a public issue. +Link the issue so the gate can see it: `Fixes #123` in the description, or the +**Development** section of the sidebar. The gate reads GitHub's own resolved issue links, so +either route works — but a bare `#123` mention is only a reference and does not count. + +### Review + +1. Automated checks run — tests, linting, static analysis, and the issue gate. +2. A maintainer reviews for correctness, test coverage, and fit with the surrounding code. + `.github/CODEOWNERS` routes the request to whoever owns the area you touched. +3. You may be asked for changes. Or we may just push them, if you left edits enabled. +4. Once approved, we merge to `main`. + +If a PR goes quiet, nudge us in [Discord](https://discord.gg/honcho). + +## Reporting bugs and requesting features + +Use the [issue templates](https://github.com/plastic-labs/honcho/issues/new/choose). There is +one per kind of report, and picking the right one is most of what gets an issue triaged +quickly: + +- **Bug report** — something is broken or behaves incorrectly +- **Memory / recall quality** — the deriver or dialectic returns poor, wrong, or missing context +- **Feature request** — a new capability or API surface +- **Integration request** — plugins, framework integrations, app-store listings +- **Documentation issue** — anything wrong or missing in the docs +- **General questions** — not an issue at all; ask in [Discord](https://discord.gg/honcho) + +Before opening one, search existing issues, including closed ones. + +A good bug report has the Honcho version or commit, whether you are self-hosted or on +`api.honcho.dev`, the steps to reproduce, and what you expected instead. If it involves the +deriver, logs from the worker process are usually the thing we ask for first. + +**Redact before you post.** Issues are public, and Honcho stores conversational data — strip +API keys, JWTs, and production user content out of any log or payload you attach. + +## Security + +Do not open a public issue for a suspected vulnerability. Report it privately through +[GitHub Private Vulnerability Reporting](https://github.com/plastic-labs/honcho/security/advisories/new), +which is the preferred channel, or by email. See [SECURITY.md](./SECURITY.md) for what to +include, and note that Honcho does not operate a bug bounty. ## License -By contributing to Honcho, you agree that your contributions will be licensed under the same [AGPL-3.0 License](./LICENSE) that covers the project. +By contributing to Honcho, you agree that your contributions will be licensed under the same +[AGPL-3.0 License](./LICENSE) that covers the project. Thank you for helping make Honcho better! 🫡 diff --git a/README.md b/README.md index bef96c61..2c3e1d72 100644 --- a/README.md +++ b/README.md @@ -458,75 +458,15 @@ Contributors: see [`CONTRIBUTING.md`](./CONTRIBUTING.md) for pre-commit setup. D Honcho uses a flexible configuration system that supports both TOML files and environment variables. Configuration values are loaded in priority order: **environment variables > `.env` file > `config.toml` > defaults**. - -
-Full configuration reference - -### Using config.toml - -Copy the example configuration file to get started: +Copy the example file to get started: ```bash cp config.toml.example config.toml ``` -Then modify the values as needed. The TOML file is organized into sections: +The file is organized by subsystem — `[app]`, `[db]`, `[auth]`, `[cache]`, `[llm]`, `[deriver]`, `[dialectic]`, `[summary]`, `[dream]`, `[peer_card]`, `[webhook]`, `[metrics]`, `[telemetry]`, `[vector_store]`, and `[sentry]`. Any value can be overridden by an environment variable named `{SECTION}_{KEY}`, using `__` for nesting (`DIALECTIC_LEVELS__low__MODEL_CONFIG__MODEL`), or just `{KEY}` for app-level settings. -- `[app]` - Application-level settings (log level, session limits, embedding settings, namespace) -- `[db]` - Database connection and pool settings -- `[auth]` - Authentication configuration -- `[cache]` - Redis cache configuration -- `[llm]` - LLM provider API keys and general settings -- `[deriver]` - Background worker settings and representation configuration -- `[peer_card]` - Peer card generation settings -- `[dialectic]` - Chat Endpoint configuration with per-level reasoning settings -- `[summary]` - Session summarization settings -- `[dream]` - Dream processing configuration (including specialist models and surprisal settings) -- `[webhook]` - Webhook configuration -- `[metrics]` - Prometheus pull-based metrics -- `[telemetry]` - CloudEvents telemetry for analytics -- `[vector_store]` - Vector store configuration (pgvector, turbopuffer, or lancedb) -- `[sentry]` - Error tracking and monitoring settings - -### Using Environment Variables - -All configuration values can be overridden using environment variables. The environment variable names follow this pattern: - -- `{SECTION}_{KEY}` for top-level section settings -- Use `__` inside `{KEY}` for nested settings -- Just `{KEY}` for app-level settings - -Examples: - -- `DB_CONNECTION_URI` - Database connection string -- `AUTH_JWT_SECRET` - JWT secret key -- `DERIVER_MODEL_CONFIG__TRANSPORT` - Transport for the background deriver -- `SUMMARY_MODEL_CONFIG__MODEL` - Summary model override -- `DIALECTIC_LEVELS__low__MODEL_CONFIG__MODEL` - Model for low reasoning level -- `LOG_LEVEL` - Application log level -- `METRICS_ENABLED` - Enable Prometheus metrics -- `TELEMETRY_ENABLED` - Enable CloudEvents telemetry - -### Example - -If you have this in `config.toml`: - -```toml -[db] -CONNECTION_URI = "postgresql+psycopg://localhost/honcho_dev" -POOL_SIZE = 10 -``` - -You can override just the connection URI in production: - -```bash -export DB_CONNECTION_URI="postgresql+psycopg://prod-server/honcho_prod" -``` - -The application will use the production connection URI while keeping the pool size from config.toml. - -
- +See the [configuration reference](https://honcho.dev/docs/v3/contributing/configuration) for every available option, and [`.env.template`](./.env.template) for an annotated list of environment variables. ## Architecture @@ -680,7 +620,9 @@ See the [SDK Reference](https://honcho.dev/docs/v3/documentation/reference/sdk) ## Contributing -We welcome contributions to Honcho! Please read our [Contributing Guide](./CONTRIBUTING.md) for details on our development process, coding conventions, and how to submit pull requests. +We welcome contributions to Honcho. One thing to know before you start: **pull requests must be linked to an issue carrying the `maintainer-approved` label**, or they are closed automatically. [Browse the approved queue](https://github.com/plastic-labs/honcho/issues?q=is%3Aissue+is%3Aopen+label%3Amaintainer-approved), or make your case in [Discord](http://discord.gg/honcho) — that is where maintainers are most active. + +See [CONTRIBUTING.md](./CONTRIBUTING.md) for the full process, an architecture walkthrough, and a map of where to change what. For vulnerabilities, see [SECURITY.md](./SECURITY.md) — note that Honcho does not operate a bug bounty. ## License diff --git a/SECURITY.md b/SECURITY.md index 172758d3..1b71cb45 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,31 +1,73 @@ # Security Policy -## Reporting a vulnerability +## Supported Versions -**Do not file a public GitHub issue for security vulnerabilities.** +The `main` branch of this repo maps to the latest canary version of Honcho. To see which versions are supported please refer to the git tags in the repo or the [compatibility guide](https://honcho.dev/docs/changelog/compatibility-guide). -Please report security issues privately using one of: +## Reporting a Vulnerability -1. **[GitHub Private Vulnerability Reporting](https://github.com/plastic-labs/honcho/security/advisories/new)** (preferred) -2. Email **** with subject line `[SECURITY] …` +Do not open a public issue for a suspected vulnerability. Report it privately through one of: -Include as much of the following as you can: +1. **[GitHub Private Vulnerability Reporting](https://github.com/plastic-labs/honcho/security/advisories/new)** — preferred; it keeps the report, our replies, and any fix coordinated in one place. +2. Email [support@honcho.dev](mailto:support@honcho.dev) with `[SECURITY]` in the subject. -- Description of the issue and its impact -- Steps to reproduce, or a proof of concept -- Affected component (API, deriver, auth/JWT, SDK, managed offering, etc.) -- Honcho version or image tag, and whether you are on managed or self-hosted +Include as much of the following as you have: -Honcho stores conversational data and peer representations. **Do not** attach production user content, API keys, JWTs, or other secrets to a report unless we explicitly ask for a redacted sample. +- **Version** — a git commit SHA, or the release tag you are running +- **Deployment** — self-hosted or the managed service at `api.honcho.dev` +- **Affected component** — API, deriver, dialectic, auth/JWT, an SDK, or the managed offering +- **Reproduction** — the exact steps, requests, or script that trigger it +- **Proof of concept** — the smallest thing that demonstrates the issue actually works +- **Impact** — what an attacker gains, and what they need to already have to get it +- **How you found it** — manual review, fuzzing, a scanner, or model-assisted analysis -## What to expect +Reports with a working proof of concept get looked at first. A report that only describes a +theoretical problem is much slower for us to act on, because we have to build the repro +ourselves before we can confirm anything. -We will acknowledge valid reports as soon as we can and will keep you updated on remediation status. Please give us a reasonable window to investigate and fix before any public disclosure. +Honcho stores conversational data and peer representations. **Do not attach production user +content, API keys, or JWTs** to a report — if we need a sample, we will ask for a redacted +one. -## Supported versions +## Testing -Security fixes are applied to the latest release on `main` and, when practical, to the most recent tagged release line. Older versions may not receive backports. +Test against an instance you operate. Do not run security testing against `api.honcho.dev` +or against any Honcho deployment that is not yours — self-hosting is a first-class path and +takes a few minutes to set up, see [Self-hosting](./README.md#self-hosting). -## Non-security bugs +## What to Expect -For ordinary bugs, memory/recall quality issues, and feature requests, use the [issue templates](https://github.com/plastic-labs/honcho/issues/new/choose). +We will acknowledge your report and tell you whether we consider it in scope. If it is, we +will let you know when a fix ships. + +We do not commit to a response SLA, we do not coordinate CVE assignment on request, and we +do not operate a disclosure timeline you can hold us to. This is a small team. + +## Out of Scope + +The following are not treated as vulnerabilities. Reports consisting only of these will be +closed without a detailed response: + +- Automated scanner output with no working proof of concept +- Model-generated findings that have not been verified by a human against a running instance +- Missing security headers or TLS configuration with no demonstrated exploit +- Rate limiting, or resource exhaustion with no demonstrated impact beyond your own instance +- Vulnerabilities in dependencies with no demonstrated exploit path through Honcho +- Configuration weaknesses that require an already-compromised host, or that come from + deliberately insecure settings (for example running with `AUTH_USE_AUTH=false`, which is + the documented local-development default and is not intended for a public deployment) +- Social engineering, phishing, and physical access + +For ordinary bugs, memory or recall quality problems, and feature requests, use the +[issue templates](https://github.com/plastic-labs/honcho/issues/new/choose) instead. + +## No Bug Bounty + +The Honcho project does not offer any rewards for reported bugs or +vulnerabilities. We do not aid security researchers to get such rewards for +Honcho problems from other sources. + +A bug bounty gives people too strong incentives to find and make up "problems" +in bad faith that cause overload and abuse. + +We still appreciate and value valid vulnerability reports. diff --git a/docs/v2/contributing/guidelines.mdx b/docs/v2/contributing/guidelines.mdx index 12f44179..2fa66f1f 100644 --- a/docs/v2/contributing/guidelines.mdx +++ b/docs/v2/contributing/guidelines.mdx @@ -5,13 +5,31 @@ icon: 'handshake' Thank you for your interest in contributing to Honcho! This guide outlines the process for contributing to the project and our development conventions. +## Before you write code + +**Every pull request needs an issue, and that issue needs the `maintainer-approved` label.** + +A pull request that is not linked to an approved issue gets labelled `needs-approved-issue`, with a comment explaining why. You then have 72 hours to link one before it is closed automatically. Reopening costs nothing once the link is in place. This is automated. We do this because an unreviewable backlog helps nobody: a PR against an unapproved issue is work you did that we may not be able to merge, no matter how good it is. + +So, in order: + +1. **Find approved work.** Browse [issues labelled `maintainer-approved`](https://github.com/plastic-labs/honcho/issues?q=is%3Aissue+is%3Aopen+label%3Amaintainer-approved). That label is the queue of things we have agreed should be built. Anything in it is fair game — comment on the issue to claim it. + +2. **Or open an issue and get it approved.** Use the [issue templates](https://github.com/plastic-labs/honcho/issues/new/choose). Maintainers triage and apply the label. + +3. **If you feel strongly about an issue, come to [Discord](https://discord.gg/honcho).** This is the fastest path by a wide margin. Maintainers are more active there than in the issue tracker, and a five-minute conversation about what you want to build usually resolves whether it fits before either side spends real time on it. + +4. **Then open the PR** and link the issue — either `Fixes #123` in the description, or **Development → link an issue** in the sidebar. Both work. + +Small exceptions we will not be pedantic about: fixing a typo, a broken link, or an obviously wrong code sample. Open the PR, explain it in one line, and we will sort out the issue linkage. + ## Getting Started Before you start contributing, please: 1. **Set up your development environment** - Follow the [Local Development guide](https://github.com/plastic-labs/honcho/blob/main/CONTRIBUTING.md#local-development) in the Honcho repository to get Honcho running locally. -2. **Join our community** - Feel free to join us in our [Discord](http://discord.gg/honcho) to discuss your changes, get help, or ask questions. +2. **Join our community** - Feel free to join us in our [Discord](https://discord.gg/honcho) to discuss your changes, get help, or ask questions. 3. **Review existing issues** - Check the [issues tab](https://github.com/plastic-labs/honcho/issues) to see what's already being worked on or to find something to contribute to. @@ -94,7 +112,7 @@ git commit -m "docs(readme): update installation instructions" 3. Fill out the pull request template with: - A clear description of what changes you've made - The motivation for the changes - - Any relevant issue numbers (use "Closes #123" to auto-close issues) + - A link to the approved issue — `Fixes #123` in the description, or **Development → link an issue** in the sidebar. This is required; see [Before you write code](#before-you-write-code). - Screenshots or examples if applicable ## Coding Standards @@ -128,7 +146,7 @@ git commit -m "docs(readme): update installation instructions" ## Review Process -1. **Automated checks** - Your PR will run through automated checks including tests and linting +1. **Automated checks** - Your PR will run through automated checks including tests, linting, and the issue gate 2. **Project maintainer review** - A project maintainer will review your code for: - Code quality and adherence to standards - Functionality and correctness diff --git a/docs/v3/contributing/guidelines.mdx b/docs/v3/contributing/guidelines.mdx index 12f44179..b98bd5c9 100644 --- a/docs/v3/contributing/guidelines.mdx +++ b/docs/v3/contributing/guidelines.mdx @@ -3,174 +3,358 @@ title: 'Contributing Guidelines' icon: 'handshake' --- -Thank you for your interest in contributing to Honcho! This guide outlines the process for contributing to the project and our development conventions. +{/* This file mirrors CONTRIBUTING.md in the repo root. Update both. */} -## Getting Started +Thanks for your interest in contributing. This guide covers how work gets accepted, how +Honcho is put together, and what a mergeable pull request looks like. -Before you start contributing, please: +Honcho is a small team maintaining a project that gets more proposals than we can review. +The rules below exist so that the work you do has somewhere to land — not to keep you out. -1. **Set up your development environment** - Follow the [Local Development guide](https://github.com/plastic-labs/honcho/blob/main/CONTRIBUTING.md#local-development) in the Honcho repository to get Honcho running locally. +## Before you write code -2. **Join our community** - Feel free to join us in our [Discord](http://discord.gg/honcho) to discuss your changes, get help, or ask questions. +**Every pull request needs an issue, and that issue needs the `maintainer-approved` label.** -3. **Review existing issues** - Check the [issues tab](https://github.com/plastic-labs/honcho/issues) to see what's already being worked on or to find something to contribute to. +A pull request that is not linked to an approved issue gets labelled +`needs-approved-issue`, with a comment explaining why. You then have 72 hours to link one +before it is closed automatically. Reopening costs nothing once the link is in place. This +is automated. We do this because an unreviewable backlog helps nobody: a PR against an +unapproved issue is work you did that we may not be able to merge, no matter how good it +is. -## Contribution Workflow +So, in order: -### 1. Fork and Clone +1. **Find approved work.** Browse + [issues labelled `maintainer-approved`](https://github.com/plastic-labs/honcho/issues?q=is%3Aissue+is%3Aopen+label%3Amaintainer-approved). + That label is the queue of things we have agreed should be built. Anything in it is fair + game — comment on the issue to claim it. -1. Fork the repository on GitHub -2. Clone your fork locally: - ```bash - git clone https://github.com/YOUR_USERNAME/honcho.git - cd honcho - ``` -3. Add the upstream repository as a remote: - ```bash - git remote add upstream https://github.com/plastic-labs/honcho.git - ``` +2. **Or open an issue and get it approved.** Use the + [issue templates](https://github.com/plastic-labs/honcho/issues/new/choose). Maintainers + triage and apply the label. -### 2. Create a Branch +3. **If you feel strongly about an issue, come to [Discord](https://discord.gg/honcho).** + This is the fastest path by a wide margin. Maintainers are more active there than in the + issue tracker, and a five-minute conversation about what you want to build usually + resolves whether it fits before either side spends real time on it. -Create a new branch for your feature or bug fix: +4. **Then open the PR** and link the issue — either `Fixes #123` in the description, or + **Development → link an issue** in the sidebar. Both work. + +Small exceptions we will not be pedantic about: fixing a typo, a broken link, or an +obviously wrong code sample. Open the PR, explain it in one line, and we will sort out the +issue linkage. + +## What gets prioritized + +Roughly, work on Honcho falls along these axes. Knowing which one your idea sits on tells +you a lot about how likely it is to get approved. + +| Axis | What it covers | +| --- | --- | +| **Observability** | Understanding how Honcho behaves in production — telemetry, tracing, CloudEvents, metrics. | +| **Memory quality** | Better conclusions from the same input — the deriver, dreamer, and dialectic; eval results. | +| **Developer experience** | Fitting cleanly into more application architectures — SDKs, scopes, composable peers, the CLI. | +| **Breadth of input** | Widening what Honcho can ingest and represent — multimodal and non-conversational data. | +| **Ubiquity** | Reachable wherever a developer already works — integrations, self-hosting, alternate vector-store and inference backends, local-first defaults. | +| **Reliability and cost** | Trustworthy in production — connection and concurrency hardening, queue throughput, cost per token. | + +In practice, **Ubiquity** and **Developer experience** are where outside contributions land +most easily. A new integration, a self-hosting rough edge, a vector-store or inference +backend, an SDK ergonomics fix — these are additive and rarely collide with work already in +flight. + +Changes to the reasoning pipeline itself — deriver prompts, dialectic tool design, dreamer +strategy — are the hardest to accept from outside. Not because they are unwelcome, but +because they are measured against eval results we run internally, and they frequently +conflict with in-flight work. Talk to us in Discord first, always. + +## If you're an agent + +If you are a coding agent working on this repository, read this section before writing code. +The most common failure we see is a well-formed, well-tested pull request against an issue +that was never approved. That gets closed, and the work is wasted. + +- **Check the gate first.** Before writing code: + + ```bash + gh issue view --repo plastic-labs/honcho --json number,title,state,labels + ``` + + Stop if there is no issue number, if the issue is closed, or if `maintainer-approved` is + not in the labels. Report that to the person you are working with instead of proceeding. + +- **Do not open a PR in order to establish the issue link afterwards.** The issue comes + first. + +- **Do not report checks you did not run.** If you did not execute the test command, say so. + A PR body claiming a green run that did not happen costs a maintainer more time than no + claim at all. + +- **Use the checklist.** [`skills/pre-pr/SKILL.md`](https://github.com/plastic-labs/honcho/blob/main/skills/pre-pr/SKILL.md) in this repo + encodes the gate, the test-layer matrix, and the PR body format. If your harness supports + skills, invoke it rather than reimplementing the checks. + +## How Honcho works + +Enough architecture to find your way around. For the user-facing model — what a Peer is, what +`get_context` returns — see [Core Concepts](https://github.com/plastic-labs/honcho#core-concepts) and the +[documentation](https://honcho.dev/docs/). + +### Two processes + +Honcho runs as two cooperating processes over a shared Postgres database and Redis cache. + +| | API server | Deriver worker | +| --- | --- | --- | +| Start | `uv run fastapi dev src/main.py` | `uv run python -m src.deriver` | +| Entry | `src/main.py` | `src/deriver/__main__.py` | +| Does | Serves HTTP, enqueues background work, returns immediately | Consumes the queue: Deriver, Summarizer, Dreamer, Reconciler | +| Hosts | The Dialectic agent, inline on the request path | Everything else | + +The split is the load-bearing design decision: **an HTTP request never blocks on LLM work**, +with the single exception of the Dialectic chat endpoint, which is synchronous by nature. +If you are adding something slow, it belongs in the worker. + +The deriver is a separate process. If messages go in and nothing ever comes out, the usual +cause is that nobody started it. + +### The path of a message + +Worth tracing once, because it crosses most of the codebase: + +1. `POST /v3/workspaces/{w}/sessions/{s}/messages` lands in `src/routers/messages.py`. +2. The row is written, then `enqueue()` in `src/deriver/enqueue.py` creates `queue_item` + rows — one set of work per observing peer. +3. `src/deriver/queue_manager.py` polls the queue, claiming work units so that messages in a + session are processed in order. +4. `process_item()` in `src/deriver/consumer.py` dispatches on task type — representation, + summary, deletion, reconciliation. +5. For a representation task, `process_representation_tasks_batch()` in + `src/deriver/deriver.py` makes **one structured-output LLM call for the whole batch** and + writes the resulting conclusions into the collection keyed by the + `(observer, observed)` peer pair. +6. Later, `src/dialectic/` reads those conclusions back at recall time to answer a chat + request. + +Embedding is deliberately *not* on this path. `MessageEmbedding` rows are written with +`sync_state='pending'` and embedded asynchronously by the Reconciler +(`src/reconciler/sync_vectors.py`), which runs on a scheduler inside the deriver process. + +### The four agents + +They share tool definitions in `src/utils/agent_tools.py` and the provider-agnostic LLM +client in `src/llm/`. Each has its own `MODEL_CONFIG` with a fallback chain in +`src/config.py`. + +| Agent | Where | Shape | +| --- | --- | --- | +| **Deriver** | `src/deriver/` | A single structured-output call per message batch. Not a tool loop — this is a deliberate cost and latency tradeoff. | +| **Dialectic** | `src/dialectic/` | The one tool-using agent on the request path. Loops over tools until it can answer. Five reasoning tiers from `minimal` to `max`, each with its own model and tool set. | +| **Dreamer** | `src/dreamer/` | Off-queue consolidation. Two specialist phases (deduction, then induction) that build reasoning trees over existing conclusions. | +| **Summarizer** | `src/utils/summarizer.py` | Direct LLM call, no tools. Two tiers — short and long summaries at different message counts. | + +Prompts live in `src/deriver/prompts.py`, `src/dialectic/prompts.py`, and +`src/dreamer/specialists.py`. + +### A note on naming + +What the public API and documentation call **conclusions** are called **observations** +throughout the code — `create_observations`, `get_observation_context`, and so on. Likewise +**collections** and **documents** are internal storage concepts that are not exposed +directly through the API. Do not rename across that boundary in a drive-by change; the +public and internal vocabularies are being reconciled deliberately. + +## Where to change what + +| I want to change... | Start here | +| --- | --- | +| An HTTP endpoint | `src/routers/` — one module per resource | +| A database query | `src/crud/` — mirrors the router layout | +| The database schema | `src/models.py`, plus a migration in `migrations/versions/` | +| A configuration value | `src/config.py`, and add it to `config.toml.example` and `.env.template` | +| A tool an agent can call | `src/utils/agent_tools.py` — definitions plus the per-agent tool lists | +| A prompt | `src/deriver/prompts.py`, `src/dialectic/prompts.py`, `src/dreamer/specialists.py` | +| LLM provider behavior | `src/llm/backends/` — `anthropic.py`, `gemini.py`, `openai.py` | +| Embeddings or vector storage | `src/embedding_client.py`, `src/vector_store/` | +| Telemetry or metrics | `src/telemetry/` — see the notes in `CLAUDE.md` before adding an event type | +| Authentication and scoping | `src/security.py`, `src/dependencies.py` | +| The Python or TypeScript SDK | `sdks/python/`, `sdks/typescript/` | +| The CLI | `honcho-cli/` | +| The MCP server | `mcp/` | +| Public documentation | `docs/v3/` — Mintlify; nav lives in `docs/docs.json` | + +Tests in `tests/` mirror `src/`. `CLAUDE.md` at the repo root has more detail on house +conventions, and is worth skimming even if you are not using an agent. + +## Local setup + +Get a stack running first — [Self-hosting](/v3/contributing/self-hosting) covers +both the Docker path and a manual Postgres setup. Then, for development: + +```bash +uv sync # create the venv and install dependencies +uv run alembic upgrade head # apply migrations +``` + +Run both processes, in separate terminals: + +```bash +uv run fastapi dev src/main.py # API server, reloads on change +uv run python -m src.deriver # background worker +``` + +Everything Python goes through `uv run`. Redis is optional for local development; without it +caching is simply disabled. + +## Making the change + +### Branches and commits ```bash git checkout -b feature/your-feature-name -# or -git checkout -b fix/your-bug-fix-name ``` -**Branch naming conventions:** -- `feature/description` - for new features -- `fix/description` - for bug fixes -- `docs/description` - for documentation updates -- `refactor/description` - for code refactoring -- `test/description` - for adding or updating tests +Prefixes: `feature/`, `fix/`, `docs/`, `refactor/`, `test/`. -### 3. Make Your Changes +Commits follow [Conventional Commits](https://www.conventionalcommits.org/), enforced by a +`commit-msg` hook: -- Write clean, readable code that follows our coding standards (see below) -- Add tests for new functionality -- Update documentation as needed -- Make sure your changes don't break existing functionality - -### 4. Commit Your Changes - -We follow conventional commit standards. Format your commit messages as: - -``` -type(scope): description - -[optional body] - -[optional footer] -``` - -**Types:** -- `feat`: A new feature -- `fix`: A bug fix -- `docs`: Documentation only changes -- `style`: Changes that do not affect the meaning of the code -- `refactor`: A code change that neither fixes a bug nor adds a feature -- `test`: Adding missing tests or correcting existing tests -- `chore`: Changes to the build process or auxiliary tools - -**Examples:** ```bash git commit -m "feat(api): add new dialectic endpoint for user insights" git commit -m "fix(db): resolve connection pool timeout issue" -git commit -m "docs(readme): update installation instructions" ``` -### 5. Submit a Pull Request +Types: `feat`, `fix`, `docs`, `style`, `refactor`, `test`, `chore`. -1. Push your branch to your fork: - ```bash - git push origin your-branch-name - ``` +### Pre-commit hooks -2. Create a pull request on GitHub from your branch to the `main` branch +Install them. CI runs the same checks, and it is much faster to find out locally. -3. Fill out the pull request template with: - - A clear description of what changes you've made - - The motivation for the changes - - Any relevant issue numbers (use "Closes #123" to auto-close issues) - - Screenshots or examples if applicable +```bash +uv run pre-commit install \ + --hook-type pre-commit \ + --hook-type commit-msg \ + --hook-type pre-push +``` -## Coding Standards +At **commit** time: ruff lint and format, biome for TypeScript, basedpyright, bandit, +markdownlint, and file hygiene. At **push** time: pytest, the alembic migration tests, and +the SDK builds. -### Python Code Style +That split matters — **a clean commit is not a clean push.** The test suite only runs at +`pre-push`, so the first time you see test failures may be well after you thought you were +done. -- Follow [PEP 8](https://www.python.org/dev/peps/pep-0008/) style guidelines -- Use [Black](https://black.readthedocs.io/) for code formatting (we may add this to CI in the future) -- Use type hints where possible -- Write docstrings for functions and classes using Google style docstrings +Run them by hand at any time: -### Code Organization +```bash +uv run pre-commit run --all-files +uv run pre-commit run ruff --all-files +``` -- Keep functions focused and single-purpose -- Use meaningful variable and function names -- Add comments for complex logic -- Follow existing patterns in the codebase +Or the individual tools: -### Testing +```bash +uv run ruff check src/ +uv run ruff format src/ +uv run basedpyright +``` -- Write unit tests for new functionality -- Ensure existing tests pass before submitting -- Use descriptive test names that explain what is being tested -- Mock external dependencies appropriately +### Tests + +Write tests for new functionality, in the directory under `tests/` that mirrors the code you +changed. Which layer you need depends on what you touched: + +| What you changed | What to run | +| --- | --- | +| Anything in `src/` | Unit tests in the matching `tests/` tree — `uv run pytest tests/...` | +| Deriver, dialectic, dreamer, or the LLM path | Unit tests, and consider `tests/live_llm/` (gated behind `--live-llm`) | +| Queue behavior, config hierarchy, multi-turn flows, SDK contracts | `uv run python -m tests.unified.run` | +| A `/v3` endpoint or deriver queue behavior | Actually run the stack and exercise it — not just pytest | +| A migration | `uv run python scripts/run_alembic_tests.py`; every revision needs a test file | + +The TypeScript SDK tests need a running server with a database and Redis, which pytest +orchestrates. Run them with `uv run pytest tests/ -k typescript` from the repo root — +`bun test` on its own will fail. To type-check the SDK alone: +`cd sdks/typescript && bun run tsc --noEmit`. ### Documentation -- Update relevant documentation for new features -- Include examples in docstrings where helpful -- Keep README and other docs up to date with changes +Update docs in the same PR when you change a public surface: `/v3` endpoints, SDK exports, +or anything in `config.toml` / settings. Docs live in `docs/v3/`, and new pages need an entry +in `docs/docs.json` or they will not appear in the nav. -## Review Process +## Opening the pull request -1. **Automated checks** - Your PR will run through automated checks including tests and linting -2. **Project maintainer review** - A project maintainer will review your code for: - - Code quality and adherence to standards - - Functionality and correctness - - Test coverage - - Documentation completeness -3. **Discussion and iteration** - You may be asked to make changes or clarifications -4. **Approval and merge** - Once approved, your PR will be merged into `main` +### Leave "Allow edits by maintainers" checked -## Types of Contributions +This is the single most useful thing you can do to get your PR merged quickly. -We welcome various types of contributions: +Most contributor PRs arrive nearly right, needing a rename, a missing test, or a lint fix. +If we can push that commit ourselves, it merges the same day. If we cannot, it becomes a +review comment, and then we wait — sometimes for weeks — for a round trip on a two-line +change. -- **Bug fixes** - Help us squash bugs and improve stability -- **New features** - Add functionality that benefits the community -- **Documentation** - Improve or expand our documentation -- **Tests** - Increase test coverage and reliability -- **Performance improvements** - Help make Honcho faster and more efficient -- **Examples and tutorials** - Help other developers use Honcho +GitHub checks the box by default when you fork. Leave it checked. -## Issue Reporting +One caveat worth knowing: **the option does not exist on forks owned by an organization.** +If you have the choice, fork from your personal account. -When reporting bugs or requesting features: +### Fill out the template -1. Check if the issue already exists -2. Use the appropriate [issue template](https://github.com/plastic-labs/honcho/issues/new/choose) (bug, memory/recall quality, feature, integration, or documentation) -3. Provide clear reproduction steps for bugs -4. Include relevant environment information (managed vs self-hosted, server version, SDK) -5. Be specific about expected vs actual behavior -6. Redact secrets, JWTs, and production user content +`.github/pull_request_template.md` asks for a description, proofs, and the issue checkbox. -## Questions and Support +"Proofs" means evidence the change works: the command you ran and its result, a log snippet, +a screenshot, the failing case before and after. This is the section that most determines +how fast your PR gets reviewed. Do not add sections to the template. -- **General questions** - Join our [Discord](https://discord.gg/honcho) -- **Bug reports** - GitHub issues → Bug report template -- **Memory / recall quality** - GitHub issues → Memory / recall quality template -- **Feature requests** - GitHub issues → Feature request template -- **Integrations / plugins / app-store listings** - GitHub issues → Integration request template -- **Documentation issues** - GitHub issues → Documentation issue template -- **Security issues** - Report **privately** only — see [`SECURITY.md`](https://github.com/plastic-labs/honcho/blob/main/SECURITY.md) (GitHub Private Vulnerability Reporting or email). Do not open a public issue. +Link the issue so the gate can see it: `Fixes #123` in the description, or the +**Development** section of the sidebar. The gate reads GitHub's own resolved issue links, so +either route works — but a bare `#123` mention is only a reference and does not count. + +### Review + +1. Automated checks run — tests, linting, static analysis, and the issue gate. +2. A maintainer reviews for correctness, test coverage, and fit with the surrounding code. + `.github/CODEOWNERS` routes the request to whoever owns the area you touched. +3. You may be asked for changes. Or we may just push them, if you left edits enabled. +4. Once approved, we merge to `main`. + +If a PR goes quiet, nudge us in [Discord](https://discord.gg/honcho). + +## Reporting bugs and requesting features + +Use the [issue templates](https://github.com/plastic-labs/honcho/issues/new/choose). There is +one per kind of report, and picking the right one is most of what gets an issue triaged +quickly: + +- **Bug report** — something is broken or behaves incorrectly +- **Memory / recall quality** — the deriver or dialectic returns poor, wrong, or missing context +- **Feature request** — a new capability or API surface +- **Integration request** — plugins, framework integrations, app-store listings +- **Documentation issue** — anything wrong or missing in the docs +- **General questions** — not an issue at all; ask in [Discord](https://discord.gg/honcho) + +Before opening one, search existing issues, including closed ones. + +A good bug report has the Honcho version or commit, whether you are self-hosted or on +`api.honcho.dev`, the steps to reproduce, and what you expected instead. If it involves the +deriver, logs from the worker process are usually the thing we ask for first. + +**Redact before you post.** Issues are public, and Honcho stores conversational data — strip +API keys, JWTs, and production user content out of any log or payload you attach. + +## Security + +Do not open a public issue for a suspected vulnerability. Report it privately through +[GitHub Private Vulnerability Reporting](https://github.com/plastic-labs/honcho/security/advisories/new), +which is the preferred channel, or by email. See [SECURITY.md](https://github.com/plastic-labs/honcho/blob/main/SECURITY.md) for what +to include, and note that Honcho does not operate a bug bounty. ## License -By contributing to Honcho, you agree that your contributions will be licensed under the same [AGPL-3.0 License](./license) that covers the project. +By contributing to Honcho, you agree that your contributions will be licensed under the same +[AGPL-3.0 License](./license) that covers the project. Thank you for helping make Honcho better! 🫡