fix(grok_local): do not warn when the default model sentinel is unavailable (#12062)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Each agent runs under an adapter. The `grok_local` adapter runs the
Grok Build CLI.
> - The adapter has an environment test. It probes the CLI and reports
checks to the operator.
> - `DEFAULT_GROK_LOCAL_MODEL` is `"grok-build"`. This value is a
sentinel. It means "use the Grok CLI's own default model".
> - `execute.ts` only passes `--model` when the configured model differs
from the sentinel. So the sentinel is never sent to grok.
> - The environment test still compared the sentinel to the models that
`grok models` lists. Real grok never lists `grok-build`.
> - So every probe emitted a false "Configured model not found" warning,
even on a correctly configured agent.
> - This pull request stops the false warning and keeps the real check
for user-set models.
> - The benefit is an accurate environment test: operators see a warning
only when it is real.

## Linked Issues or Issue Description

No public issue exists. The problem, in bug-report form:

**What happened?**
The `grok_local` environment test always warns `Configured model
"grok-build" not found in available models`, even when the agent works.
`grok-build` is the default sentinel, not a real model id, and it is
never sent to the CLI.

**Expected behavior**
When the model is left at the default, the test reports the CLI's own
default model as info and does not warn. It warns only when a user sets
a real model that `grok models` does not list.

**Steps to reproduce**
1. Create a `grok_local` agent and leave the model at its default.
2. Run the adapter environment test.
3. See the `grok_model_not_found` warning, although `grok models` and
the hello probe succeed.

**Agent adapter(s) involved**
grok_local (Grok Build CLI).

## What Changed

- `packages/adapters/grok-local/src/server/test.ts`: the model check now
treats the default sentinel as valid and reports it as info (`Using the
Grok CLI's default model (<default>)`). It still warns when an
explicitly configured, non-sentinel model is absent from the discovered
list. This matches `execute.ts`, which never sends the sentinel to grok.
- `packages/adapters/grok-local/src/server/test.test.ts`: adds a test
that the default sentinel does not warn when it is absent from the real
model list, and a test that a real, unavailable model still warns.

## Verification

- `pnpm exec vitest run
packages/adapters/grok-local/src/server/test.test.ts` — 5 passed.

## Risks

Low risk. The change only affects one adapter's environment-test
reporting. It does not change how runs pass `--model`. No schema, no
runtime behavior change.

## Model Used

Claude Fable 5 (`claude-fable-5`), extended thinking, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues OR (b) described the
issue in-PR following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (n/a —
no doc change)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Devin Foley 2026-08-24 09:38:34 -07:00 committed by GitHub
parent 0dfa0fb988
commit 83fefaadd1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 93 additions and 8 deletions

View File

@ -105,6 +105,84 @@ describe("grok_local testEnvironment", () => {
);
});
it("does not warn when the default-sentinel model is absent from the real model list", async () => {
// Real grok never lists the "grok-build" sentinel; execute.ts never sends
// it. The probe must treat the default as valid (info), not warn.
runProcessMock
.mockResolvedValueOnce({
exitCode: 0,
signal: null,
timedOut: false,
stdout: [
"You are logged in with grok.com.",
"",
"Default model: grok-4.20-0309-non-reasoning",
"",
"Available models:",
" * grok-4.20-0309-non-reasoning (default)",
" * grok-4",
].join("\n"),
stderr: "",
})
.mockResolvedValueOnce({
exitCode: 0,
signal: null,
timedOut: false,
stdout: [
JSON.stringify({ type: "text", data: "hello" }),
JSON.stringify({ type: "end", stopReason: "EndTurn", sessionId: "s", requestId: "r" }),
].join("\n"),
stderr: "",
});
const result = await testEnvironment({
companyId: "company-1",
adapterType: "grok_local",
config: { command: "grok", cwd: "/tmp/project" }, // no model → default sentinel
});
const codes = result.checks.map((check: { code: string }) => check.code);
expect(codes).toContain("grok_model_configured");
expect(codes).not.toContain("grok_model_not_found");
expect(result.status).toBe("pass");
});
it("warns when an explicitly configured real model is not available", async () => {
runProcessMock
.mockResolvedValueOnce({
exitCode: 0,
signal: null,
timedOut: false,
stdout: [
"You are logged in with grok.com.",
"",
"Default model: grok-4",
"",
"Available models:",
" * grok-4",
].join("\n"),
stderr: "",
})
.mockResolvedValueOnce({
exitCode: 0,
signal: null,
timedOut: false,
stdout: [
JSON.stringify({ type: "text", data: "hello" }),
JSON.stringify({ type: "end", stopReason: "EndTurn", sessionId: "s", requestId: "r" }),
].join("\n"),
stderr: "",
});
const result = await testEnvironment({
companyId: "company-1",
adapterType: "grok_local",
config: { command: "grok", cwd: "/tmp/project", model: "grok-nonexistent" },
});
expect(result.checks.map((check: { code: string }) => check.code)).toContain("grok_model_not_found");
});
it("downgrades auth failures to warnings", async () => {
runProcessMock
.mockResolvedValueOnce({

View File

@ -226,15 +226,22 @@ export async function testEnvironment(
});
}
if (configuredModel) {
// The default model id is a sentinel meaning "let the Grok CLI pick its
// own default" — execute.ts only passes `--model` when the value differs
// from it, so the sentinel is never sent to grok and must not be checked
// against the discovered list (real grok never lists "grok-build", which
// otherwise produced a spurious "not found" warning on every probe).
const usesCliDefault = configuredModel === DEFAULT_GROK_LOCAL_MODEL;
const available = usesCliDefault || parsedModels.models.includes(configuredModel);
checks.push({
code: parsedModels.models.includes(configuredModel) ? "grok_model_configured" : "grok_model_not_found",
level: parsedModels.models.includes(configuredModel) ? "info" : "warn",
message: parsedModels.models.includes(configuredModel)
? `Configured model: ${configuredModel}`
: `Configured model "${configuredModel}" not found in available models.`,
hint: parsedModels.models.includes(configuredModel)
? undefined
: "Run `grok models` and choose an available model id.",
code: available ? "grok_model_configured" : "grok_model_not_found",
level: available ? "info" : "warn",
message: usesCliDefault
? `Using the Grok CLI's default model${parsedModels.defaultModel ? ` (${parsedModels.defaultModel})` : ""}.`
: available
? `Configured model: ${configuredModel}`
: `Configured model "${configuredModel}" not found in available models.`,
hint: available ? undefined : "Run `grok models` and choose an available model id.",
});
}
}