fix(agents): disable cheap model profiles by default (#10019)
## Thinking Path
> - Paperclip is the control plane people use to create and govern
AI-agent companies
> - Agent creation persists runtime configuration that controls which
model profiles future runs may select
> - Adapters can expose a `cheap` profile, and existing creation paths
implicitly left that profile available when operators made no choice
> - That made a newly created agent eligible for a lower-cost model
without an explicit operator opt-in
> - The UI also dropped an explicit opt-in when the operator selected
the adapter's default cheap model rather than a custom model ID
> - Codex additionally hardcoded `gpt-5.3-codex-spark` into its cheap
profile and static fallback model list, making Paperclip choose an
auth-dependent model rather than requiring an operator choice
> - This pull request makes new-agent creation disable an available
cheap profile by default while preserving explicit opt-in from the UI or
API
> - The Codex cheap profile now remains available for explicit
configuration but supplies no model default, so an unconfigured cheap
request stays on the primary model
> - The benefit is predictable model quality for new agents and an
intentional, auditable choice before lower-cost routing is enabled
## Linked Issues or Issue Description
**Problem**
New agents created with an adapter that exposes a `cheap` model profile
can inherit that profile without the operator explicitly enabling it. In
the UI, enabling the adapter-default cheap model is also omitted because
runtime configuration is only written when a custom model ID is present.
**Expected behavior**
- New agents default an available `cheap` model profile to `{ enabled:
false }` when the caller does not specify it.
- Explicit API configuration remains authoritative.
- UI opt-in persists even when the adapter default model is used.
- Codex does not advertise or automatically select
`gpt-5.3-codex-spark`; operators must explicitly configure any
lower-cost Codex model.
**Related public work**
- Refs #4881, which introduced cheap model profiles for local adapters.
- Supersedes the default-selection portions of #8032 and #10004 by
removing the Codex model default instead of replacing it with another
hardcoded model.
## What Changed
- Detect whether the selected adapter exposes a `cheap` model profile
during agent creation and hiring.
- Persist `runtimeConfig.modelProfiles.cheap.enabled = false` only when
the caller did not explicitly configure the profile.
- Preserve UI cheap-profile opt-in when using the adapter's default
model by writing an empty adapter config.
- Remove `gpt-5.3-codex-spark` from the Codex static model list.
- Keep the Codex `cheap` profile explicitly configurable while giving it
an empty adapter config, so Paperclip never chooses a cheap Codex model
automatically.
- Verify that a Codex cheap request without an explicit model leaves the
primary model unchanged.
- Extend server route and UI runtime-config tests for default-disable
and explicit-opt-in behavior.
## Verification
- `env -u PAPERCLIP_IN_WORKTREE -u PAPERCLIP_WORKTREE_NAME -u
PAPERCLIP_CONFIG -u PAPERCLIP_HOME -u PAPERCLIP_INSTANCE_ID -u
PAPERCLIP_CONTEXT pnpm exec vitest run
packages/adapters/codex-local/src/index.test.ts
packages/adapters/codex-local/src/server/codex-args.test.ts
server/src/__tests__/adapter-models.test.ts
server/src/__tests__/adapter-registry.test.ts
server/src/__tests__/heartbeat-model-profile.test.ts
server/src/__tests__/agent-permissions-routes.test.ts
ui/src/lib/new-agent-runtime-config.test.ts`
- Result: 7 test files passed, 105 tests passed.
- GitHub `Typecheck + Release Registry` check passed on the final head.
- `git diff --check public-gh/master...HEAD`
## Risks
- Low behavioral risk: only newly created or hired agents are
normalized; existing agents are unchanged.
- Explicit `cheap` profile settings remain untouched, including explicit
opt-in.
- Codex users who explicitly opt into the cheap lane must choose a
model; requests without a configured override intentionally continue on
the primary model.
- Adapter profile discovery is now awaited during creation, adding a
small amount of adapter metadata lookup work.
- The source branch name is automation-provided and retained as required
by the task, so it does not satisfy the preferred public branch naming
convention.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI `gpt-5.4` via Codex CLI, with reasoning, repository editing,
terminal execution, and GitHub/Paperclip tool access. The runtime did
not expose a context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
GitHub
parent
71a6535792
commit
1426494ab8
|
|
@ -14,5 +14,6 @@ describe("codex local adapter metadata", () => {
|
|||
]);
|
||||
expect(isCodexLocalFastModeSupported(DEFAULT_CODEX_LOCAL_MODEL)).toBe(true);
|
||||
expect(modelIds).not.toContain("gpt-5.3-codex");
|
||||
expect(modelIds).not.toContain("gpt-5.3-codex-spark");
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -43,7 +43,6 @@ export const models = [
|
|||
{ id: "gpt-5.6-luna", label: "gpt-5.6-luna" },
|
||||
{ id: "gpt-5.4", label: "gpt-5.4" },
|
||||
{ id: "gpt-5.4-mini", label: "gpt-5.4-mini" },
|
||||
{ id: "gpt-5.3-codex-spark", label: "gpt-5.3-codex-spark" },
|
||||
{ id: "gpt-5", label: "gpt-5" },
|
||||
{ id: "o3", label: "o3" },
|
||||
{ id: "o4-mini", label: "o4-mini" },
|
||||
|
|
@ -57,12 +56,8 @@ export const modelProfiles: AdapterModelProfileDefinition[] = [
|
|||
{
|
||||
key: "cheap",
|
||||
label: "Cheap",
|
||||
description: "Use the lowest-cost known Codex local model lane without changing the primary model.",
|
||||
adapterConfig: {
|
||||
model: "gpt-5.3-codex-spark",
|
||||
// Spark is the cheap lane by model price; high effort keeps Codex coding behavior usable for delegated work.
|
||||
modelReasoningEffort: "high",
|
||||
},
|
||||
description: "Use an explicitly configured lower-cost Codex model without changing the primary model.",
|
||||
adapterConfig: {},
|
||||
source: "adapter_default",
|
||||
},
|
||||
];
|
||||
|
|
|
|||
|
|
@ -89,9 +89,9 @@ describe("buildCodexExecArgs", () => {
|
|||
]);
|
||||
});
|
||||
|
||||
it("ignores fast mode for unsupported models", () => {
|
||||
it("ignores fast mode for known unsupported models", () => {
|
||||
const result = buildCodexExecArgs({
|
||||
model: "gpt-5.3-codex-spark",
|
||||
model: "gpt-5",
|
||||
fastMode: true,
|
||||
});
|
||||
|
||||
|
|
@ -104,7 +104,7 @@ describe("buildCodexExecArgs", () => {
|
|||
"exec",
|
||||
"--json",
|
||||
"--model",
|
||||
"gpt-5.3-codex-spark",
|
||||
"gpt-5",
|
||||
"-",
|
||||
]);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -52,6 +52,7 @@ describe("adapter model listing", () => {
|
|||
expect(models.some((model) => model.id === "gpt-5.6-sol")).toBe(true);
|
||||
expect(models.some((model) => model.id === "gpt-5.6-terra")).toBe(true);
|
||||
expect(models.some((model) => model.id === "gpt-5.6-luna")).toBe(true);
|
||||
expect(models.some((model) => model.id === "gpt-5.3-codex-spark")).toBe(false);
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -268,7 +268,7 @@ describe("server adapter registry", () => {
|
|||
await expect(listAdapterModelProfiles("codex_local")).resolves.toEqual([
|
||||
expect.objectContaining({
|
||||
key: "cheap",
|
||||
adapterConfig: expect.objectContaining({ model: "gpt-5.3-codex-spark" }),
|
||||
adapterConfig: {},
|
||||
source: "adapter_default",
|
||||
}),
|
||||
]);
|
||||
|
|
|
|||
|
|
@ -972,7 +972,7 @@ describe.sequential("agent permission routes", () => {
|
|||
.send({
|
||||
name: "Builder",
|
||||
role: "engineer",
|
||||
adapterType: "process",
|
||||
adapterType: "codex_local",
|
||||
adapterConfig: {},
|
||||
runtimeConfig: {
|
||||
heartbeat: {
|
||||
|
|
@ -991,11 +991,79 @@ describe.sequential("agent permission routes", () => {
|
|||
intervalSec: 3600,
|
||||
maxConcurrentRuns: 20,
|
||||
},
|
||||
modelProfiles: {
|
||||
cheap: { enabled: false },
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("creates agents when optional adapter model profile discovery fails", async () => {
|
||||
const { registerServerAdapter, unregisterServerAdapter } = await import("../adapters/index.js");
|
||||
registerServerAdapter({
|
||||
type: "failing_profile_discovery",
|
||||
execute: async () => ({ exitCode: 0, signal: null, timedOut: false }),
|
||||
testEnvironment: async () => ({
|
||||
adapterType: "failing_profile_discovery",
|
||||
status: "pass",
|
||||
checks: [],
|
||||
testedAt: new Date(0).toISOString(),
|
||||
}),
|
||||
listModelProfiles: async () => {
|
||||
throw new Error("profile discovery unavailable");
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
const app = await createApp({
|
||||
type: "board",
|
||||
userId: "board-user",
|
||||
source: "local_implicit",
|
||||
isInstanceAdmin: true,
|
||||
companyIds: [companyId],
|
||||
});
|
||||
|
||||
const res = await requestApp(app, (baseUrl) => request(baseUrl)
|
||||
.post(`/api/companies/${companyId}/agents`)
|
||||
.send({
|
||||
name: "Builder",
|
||||
role: "engineer",
|
||||
adapterType: "failing_profile_discovery",
|
||||
adapterConfig: {},
|
||||
runtimeConfig: {
|
||||
modelProfiles: {
|
||||
cheap: {
|
||||
enabled: true,
|
||||
adapterConfig: {},
|
||||
},
|
||||
},
|
||||
},
|
||||
}));
|
||||
|
||||
expect(res.status, JSON.stringify(res.body)).toBe(201);
|
||||
expect(mockAgentService.create).toHaveBeenCalledWith(
|
||||
companyId,
|
||||
expect.objectContaining({
|
||||
runtimeConfig: {
|
||||
heartbeat: {
|
||||
enabled: false,
|
||||
maxConcurrentRuns: 20,
|
||||
},
|
||||
modelProfiles: {
|
||||
cheap: {
|
||||
enabled: true,
|
||||
adapterConfig: {},
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
unregisterServerAdapter("failing_profile_discovery");
|
||||
}
|
||||
});
|
||||
|
||||
it("seeds opencode agent creation with the static default model without live discovery", async () => {
|
||||
mockEnsureOpenCodeModelConfiguredAndAvailable.mockRejectedValue(
|
||||
new Error("`opencode models` should not be called during creation"),
|
||||
|
|
@ -1082,7 +1150,7 @@ describe.sequential("agent permission routes", () => {
|
|||
.send({
|
||||
name: "Builder",
|
||||
role: "engineer",
|
||||
adapterType: "process",
|
||||
adapterType: "codex_local",
|
||||
adapterConfig: {},
|
||||
runtimeConfig: {
|
||||
heartbeat: {
|
||||
|
|
@ -1101,6 +1169,9 @@ describe.sequential("agent permission routes", () => {
|
|||
intervalSec: 3600,
|
||||
maxConcurrentRuns: 20,
|
||||
},
|
||||
modelProfiles: {
|
||||
cheap: { enabled: false },
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ const cheapProfile: AdapterModelProfileDefinition = {
|
|||
};
|
||||
|
||||
describe("heartbeat model profile application", () => {
|
||||
it("uses the Codex local adapter cheap default when the agent has no runtime override", async () => {
|
||||
it("keeps Codex on its primary model when cheap has no explicit model override", async () => {
|
||||
const modelProfile = resolveModelProfileApplication({
|
||||
adapterModelProfiles: await listAdapterModelProfiles("codex_local"),
|
||||
agentRuntimeConfig: {},
|
||||
|
|
@ -29,17 +29,21 @@ describe("heartbeat model profile application", () => {
|
|||
contextSnapshot: {},
|
||||
});
|
||||
|
||||
const merged = mergeModelProfileAdapterConfig({
|
||||
baseConfig: { model: "primary" },
|
||||
modelProfile,
|
||||
issueAdapterConfig: null,
|
||||
});
|
||||
|
||||
expect(modelProfile).toMatchObject({
|
||||
requested: "cheap",
|
||||
requestedBy: "issue_override",
|
||||
applied: "cheap",
|
||||
configSource: "adapter_default",
|
||||
fallbackReason: null,
|
||||
adapterConfig: {
|
||||
model: "gpt-5.3-codex-spark",
|
||||
modelReasoningEffort: "high",
|
||||
},
|
||||
adapterConfig: {},
|
||||
});
|
||||
expect(merged).toEqual({ model: "primary" });
|
||||
});
|
||||
|
||||
it("applies cheap profile patches before explicit issue adapter config overrides", () => {
|
||||
|
|
|
|||
|
|
@ -67,6 +67,7 @@ import type { AdapterExecutionTarget } from "@paperclipai/adapter-utils/executio
|
|||
import type {
|
||||
AdapterEnvironmentCheck,
|
||||
AdapterEnvironmentTestResult,
|
||||
AdapterModelProfileDefinition,
|
||||
} from "@paperclipai/adapter-utils";
|
||||
import { skillVersionSelectionMap } from "../services/runtime-skill-selections.js";
|
||||
import { secretService } from "../services/secrets.js";
|
||||
|
|
@ -104,6 +105,7 @@ import { recoveryService } from "../services/recovery/service.js";
|
|||
import { resolveCoreTrustPreset } from "../services/trust-preset-resolver.js";
|
||||
import { readObject } from "../lib/objects.js";
|
||||
import { listInvalidOrgChainDescendantIds } from "../services/agent-invokability.js";
|
||||
import { logger } from "../middleware/logger.js";
|
||||
import {
|
||||
AGENT_PROFILE_CHANGE_CONSENT_FIELDS,
|
||||
agentInstructionsChangeTargetKey,
|
||||
|
|
@ -1107,7 +1109,24 @@ export function agentRoutes(
|
|||
};
|
||||
}
|
||||
|
||||
function normalizeNewAgentRuntimeConfig(runtimeConfig: unknown): Record<string, unknown> {
|
||||
async function listNewAgentAdapterModelProfiles(
|
||||
adapterType: string,
|
||||
): Promise<AdapterModelProfileDefinition[]> {
|
||||
try {
|
||||
return await listAdapterModelProfiles(adapterType);
|
||||
} catch (error) {
|
||||
logger.warn(
|
||||
{ err: error, adapterType },
|
||||
"Failed to discover adapter model profiles while normalizing a new agent; continuing without profile defaults",
|
||||
);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
async function normalizeNewAgentRuntimeConfig(
|
||||
adapterType: string,
|
||||
runtimeConfig: unknown,
|
||||
): Promise<Record<string, unknown>> {
|
||||
const parsedRuntimeConfig = asRecord(runtimeConfig);
|
||||
const normalizedRuntimeConfig = parsedRuntimeConfig ? { ...parsedRuntimeConfig } : {};
|
||||
const parsedHeartbeat = asRecord(normalizedRuntimeConfig.heartbeat);
|
||||
|
|
@ -1121,6 +1140,19 @@ export function agentRoutes(
|
|||
}
|
||||
|
||||
normalizedRuntimeConfig.heartbeat = heartbeat;
|
||||
|
||||
const parsedModelProfiles = asRecord(normalizedRuntimeConfig.modelProfiles);
|
||||
const modelProfiles = parsedModelProfiles ? { ...parsedModelProfiles } : {};
|
||||
if (!Object.prototype.hasOwnProperty.call(modelProfiles, "cheap")) {
|
||||
const adapterModelProfiles = await listNewAgentAdapterModelProfiles(adapterType);
|
||||
if (adapterModelProfiles.some((profile) => profile.key === "cheap")) {
|
||||
modelProfiles.cheap = { enabled: false };
|
||||
}
|
||||
}
|
||||
if (Object.keys(modelProfiles).length > 0) {
|
||||
normalizedRuntimeConfig.modelProfiles = modelProfiles;
|
||||
}
|
||||
|
||||
return normalizedRuntimeConfig;
|
||||
}
|
||||
|
||||
|
|
@ -1191,7 +1223,7 @@ export function agentRoutes(
|
|||
): Promise<Record<string, unknown>> {
|
||||
const entries = listRuntimeModelProfileAdapterConfigs(runtimeConfig);
|
||||
if (entries.length === 0) return runtimeConfig;
|
||||
const adapterModelProfiles = await listAdapterModelProfiles(adapterType);
|
||||
const adapterModelProfiles = await listNewAgentAdapterModelProfiles(adapterType);
|
||||
|
||||
const normalizedRuntimeConfig = { ...runtimeConfig };
|
||||
const modelProfiles = asRecord(runtimeConfig.modelProfiles) ?? {};
|
||||
|
|
@ -2391,7 +2423,7 @@ export function agentRoutes(
|
|||
const normalizedRuntimeConfig = await normalizeRuntimeConfigAdapterConfigsForPersistence(
|
||||
companyId,
|
||||
hireInput.adapterType,
|
||||
normalizeNewAgentRuntimeConfig(hireInput.runtimeConfig),
|
||||
await normalizeNewAgentRuntimeConfig(hireInput.adapterType, hireInput.runtimeConfig),
|
||||
normalizedAdapterConfig,
|
||||
);
|
||||
const normalizedHireInput = {
|
||||
|
|
@ -2586,7 +2618,7 @@ export function agentRoutes(
|
|||
const normalizedRuntimeConfig = await normalizeRuntimeConfigAdapterConfigsForPersistence(
|
||||
companyId,
|
||||
createInput.adapterType,
|
||||
normalizeNewAgentRuntimeConfig(createInput.runtimeConfig),
|
||||
await normalizeNewAgentRuntimeConfig(createInput.adapterType, createInput.runtimeConfig),
|
||||
normalizedAdapterConfig,
|
||||
);
|
||||
await assertAgentEnvironmentSelection(companyId, createInput.adapterType, createInput.defaultEnvironmentId);
|
||||
|
|
|
|||
|
|
@ -58,6 +58,18 @@ describe("buildNewAgentRuntimeConfig", () => {
|
|||
expect(config.modelProfiles).toBeUndefined();
|
||||
});
|
||||
|
||||
it("persists explicit cheap-profile opt-in when using the adapter default", () => {
|
||||
const config = buildNewAgentRuntimeConfig({
|
||||
cheapModelEnabled: true,
|
||||
});
|
||||
expect(config.modelProfiles).toEqual({
|
||||
cheap: {
|
||||
enabled: true,
|
||||
adapterConfig: {},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("omits modelProfiles when cheap model is set but explicitly disabled", () => {
|
||||
const config = buildNewAgentRuntimeConfig({
|
||||
cheapModel: "claude-sonnet-4-6",
|
||||
|
|
|
|||
|
|
@ -20,11 +20,11 @@ export function buildNewAgentRuntimeConfig(input?: {
|
|||
|
||||
const cheapModel = input?.cheapModel?.trim() ?? "";
|
||||
const cheapEnabled = input?.cheapModelEnabled ?? false;
|
||||
if (cheapModel && cheapEnabled) {
|
||||
if (cheapEnabled) {
|
||||
config.modelProfiles = {
|
||||
cheap: {
|
||||
enabled: true,
|
||||
adapterConfig: { model: cheapModel },
|
||||
adapterConfig: cheapModel ? { model: cheapModel } : {},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in New Issue