fix(ui): keep the rich editor available for angle-bracket prose (#12290)
## Thinking Path
> - Paperclip helps people manage AI agents and their work
> - People use the rich markdown editor to write task descriptions,
comments, and agent instructions
> - The editor disables HTML processing, but the markdown parser still
treats some angle-bracket text as HTML
> - A value such as `<name>` could therefore stop the rich editor and
show the raw-source fallback
> - The retry action used the same input, so it could not recover
> - This pull request escapes unsupported angle brackets only while the
editor processes the value
> - The benefit is that the rich editor works and stored markdown stays
unchanged
## Linked Issues or Issue Description
Fixes: #12197
Related prior attempt: #2696
**What happened?**
The rich markdown editor did not start when prose contained a bare angle
bracket, such as `<name>`. It showed the raw-source fallback. The retry
action repeated the same failure.
**Expected behavior**
The rich editor starts for normal prose. It stores the text in its clean
form because agent prompts can use this text.
**Steps to reproduce**
1. Open a task description, comment, or other field that uses the rich
markdown editor.
2. Enter `Rename <name> to the real name`.
3. Reload the field.
4. Observe the raw-source fallback.
**Paperclip version or commit**
`master` at d785b1921.
**Deployment mode**
All modes. The defect is in the UI package.
## What Changed
- Escape only angle brackets that the parser treats as unsupported HTML
constructs.
- Restore the clean markdown before the editor sends a value to its
parent.
- Keep code, autolinks, link destinations, and existing escapes
unchanged.
- Apply the same conversion to pasted and inserted markdown.
- Make the empty-editor fallback wait for editor initialization and
confirm the empty state.
- Re-arm the initial empty-change guard when the user retries the
editor.
- Add `MDE-PARSE`, `MDE-RENDER`, and `MDE-EMPTY` fallback codes.
- Correct the editor mock and add regression tests for parsing, round
trips, paste, insert, retry, and fallback behavior.
## Verification
- `npx vitest run ui/src/lib/angle-bracket-markdown.test.ts
ui/src/lib/blockquote-markdown.test.ts
ui/src/components/MarkdownEditor.test.tsx` — 106 tests passed.
- `pnpm --filter @paperclipai/ui typecheck` — passed.
- `pnpm -r typecheck` — passed.
- `pnpm build` — passed.
- `pnpm check:token-gates` — reports nine existing color findings in
`ui/src/components/onboarding/PillGuy.tsx`. This pull request does not
change that file.
- `pnpm test:run` — the local run found four existing port-allocation
failures in unrelated workspace-runtime tests. The focused editor tests
passed. The pull request CI runs these suites on clean workers.
## Risks
- Low data risk. The conversion applies only while MDXEditor processes a
value. Stored markdown keeps the clean form.
- The scanner skips code, autolinks, link destinations, and existing
escapes to prevent content changes.
- The raw-source fallback can appear about 300 ms later because it now
confirms that the editor stayed empty.
- There is no database or API change.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Original implementation: Anthropic Claude Opus, 1M context, with
agentic tool use in Claude Code.
- PR preparation and verification: OpenAI Codex with GPT-5, reasoning,
code execution, and tool use. The runtime did not expose the
context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin Foley
GitHub
parent
c8a136fb02
commit
2a3c86f91d
|
|
@ -11,6 +11,7 @@ import {
|
|||
isSameAutocompleteSession,
|
||||
issueMentionTitle,
|
||||
MarkdownEditor,
|
||||
type MarkdownEditorRef,
|
||||
type MentionOption,
|
||||
placeCaretAfterMentionAnchor,
|
||||
shouldAcceptAutocompleteKey,
|
||||
|
|
@ -22,12 +23,28 @@ const mdxEditorMockState = vi.hoisted(() => ({
|
|||
emitMountParseError: false,
|
||||
emitMountSilentEmptyState: false,
|
||||
throwOnRender: false,
|
||||
/** Markdown the mock emits through `onChange` once mounted, as the real editor would export it. */
|
||||
emitMountChange: null as string | null,
|
||||
/** Milliseconds the mock waits before painting its content, to model a slow import. */
|
||||
populateDelayMs: 0,
|
||||
markdownValues: [] as string[],
|
||||
/** Every string handed to the editor's imperative `insertMarkdown`. */
|
||||
insertedMarkdownValues: [] as string[],
|
||||
suppressHtmlProcessingValues: [] as boolean[],
|
||||
}));
|
||||
|
||||
function containsHtmlLikeTag(markdown: string) {
|
||||
return /<\/?[A-Za-z][A-Za-z0-9:-]*(?:\s[^>]*)?\/?>/.test(markdown);
|
||||
/**
|
||||
* Stand-in for the real importer's HTML tokenizer. MDXEditor runs with
|
||||
* `suppressHtmlProcessing`, which *removes* the HTML visitor — so a bare `<`
|
||||
* that opens an HTML construct is what throws
|
||||
* `UnrecognizedMarkdownConstructError`, regardless of the flag. Only a
|
||||
* backslash escape (`\<`) hides the bracket from the parser, which is exactly
|
||||
* what `escapeUnsupportedAngleBrackets` produces.
|
||||
*/
|
||||
function containsUnescapedHtmlLikeTag(markdown: string) {
|
||||
// Consume escape pairs first so `\<` counts as literal text, then look for a
|
||||
// surviving `<` that would open an HTML construct.
|
||||
return /(?:^|[^\\])(?:\\\\)*<[A-Za-z!/?]/.test(markdown);
|
||||
}
|
||||
|
||||
vi.mock("@mdxeditor/editor", async () => {
|
||||
|
|
@ -59,7 +76,11 @@ vi.mock("@mdxeditor/editor", async () => {
|
|||
suppressHtmlProcessing?: boolean;
|
||||
className?: string;
|
||||
},
|
||||
forwardedRef: React.ForwardedRef<{ setMarkdown: (value: string) => void; focus: () => void } | null>,
|
||||
forwardedRef: React.ForwardedRef<{
|
||||
setMarkdown: (value: string) => void;
|
||||
insertMarkdown: (value: string) => void;
|
||||
focus: (callback?: () => void) => void;
|
||||
} | null>,
|
||||
) {
|
||||
if (mdxEditorMockState.throwOnRender) {
|
||||
throw new Error("Rich editor render crashed");
|
||||
|
|
@ -68,22 +89,44 @@ vi.mock("@mdxeditor/editor", async () => {
|
|||
mdxEditorMockState.suppressHtmlProcessingValues.push(Boolean(suppressHtmlProcessing));
|
||||
const [content, setContent] = React.useState(markdown);
|
||||
const editableRef = React.useRef<HTMLDivElement>(null);
|
||||
const onErrorRef = React.useRef(onError);
|
||||
onErrorRef.current = onError;
|
||||
const handle = React.useMemo(() => ({
|
||||
setMarkdown: (value: string) => setContent(value),
|
||||
focus: () => editableRef.current?.focus(),
|
||||
insertMarkdown: (value: string) => {
|
||||
mdxEditorMockState.insertedMarkdownValues.push(value);
|
||||
// Inserted markdown goes through the same importer as the mounted
|
||||
// document, so an unescaped tag fails here in exactly the same way.
|
||||
if (containsUnescapedHtmlLikeTag(value)) {
|
||||
onErrorRef.current?.({ error: "Unrecognized markdown construct: html", source: value });
|
||||
return;
|
||||
}
|
||||
setContent((previous) => `${previous}${value}`);
|
||||
},
|
||||
// The real `focus` runs its callback once a selection exists.
|
||||
focus: (callback?: () => void) => {
|
||||
editableRef.current?.focus();
|
||||
callback?.();
|
||||
},
|
||||
}), []);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (!suppressHtmlProcessing && containsHtmlLikeTag(markdown)) {
|
||||
if (containsUnescapedHtmlLikeTag(markdown)) {
|
||||
setContent("");
|
||||
onError?.({
|
||||
error: "Error parsing markdown: HTML-like formatting requires suppressHtmlProcessing",
|
||||
error: "Unrecognized markdown construct: html",
|
||||
source: markdown,
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (mdxEditorMockState.populateDelayMs > 0) {
|
||||
// Model an import that paints its content some time after mount.
|
||||
setContent("");
|
||||
const timer = window.setTimeout(() => setContent(markdown), mdxEditorMockState.populateDelayMs);
|
||||
return () => window.clearTimeout(timer);
|
||||
}
|
||||
setContent(markdown);
|
||||
}, [markdown, onError, suppressHtmlProcessing]);
|
||||
}, [markdown, onError]);
|
||||
|
||||
React.useEffect(() => {
|
||||
setForwardedRef(forwardedRef, null);
|
||||
|
|
@ -103,6 +146,9 @@ vi.mock("@mdxeditor/editor", async () => {
|
|||
source: markdown,
|
||||
});
|
||||
}
|
||||
if (mdxEditorMockState.emitMountChange !== null) {
|
||||
onChange?.(mdxEditorMockState.emitMountChange);
|
||||
}
|
||||
}, 0);
|
||||
return () => {
|
||||
window.clearTimeout(timer);
|
||||
|
|
@ -118,7 +164,8 @@ vi.mock("@mdxeditor/editor", async () => {
|
|||
contentEditable
|
||||
suppressContentEditableWarning
|
||||
>
|
||||
{content || placeholder || ""}
|
||||
{/* The real editor paints resolved text, never the escapes that carried it in. */}
|
||||
{content.replace(/\\</g, "<") || placeholder || ""}
|
||||
</div>
|
||||
);
|
||||
});
|
||||
|
|
@ -167,6 +214,28 @@ async function flush() {
|
|||
});
|
||||
}
|
||||
|
||||
function clickRetryRichEditor(scope: HTMLElement) {
|
||||
const button = Array.from(scope.querySelectorAll("button")).find((candidate) =>
|
||||
candidate.textContent?.includes("Retry rich editor"),
|
||||
);
|
||||
if (!button) throw new Error('"Retry rich editor" button not found');
|
||||
button.dispatchEvent(new MouseEvent("click", { bubbles: true }));
|
||||
}
|
||||
|
||||
function fallbackCode(scope: HTMLElement) {
|
||||
return scope.querySelector('[data-testid="markdown-editor-fallback-code"]')?.textContent;
|
||||
}
|
||||
|
||||
/**
|
||||
* The DOM-empty heuristic checks at 100ms and confirms 200ms later, so a test
|
||||
* that wants to prove no fallback happened has to outlast both phases.
|
||||
*/
|
||||
async function waitPastEmptyHeuristic() {
|
||||
await act(async () => {
|
||||
await new Promise((resolve) => setTimeout(resolve, 400));
|
||||
});
|
||||
}
|
||||
|
||||
function createFileDragEvent(type: string) {
|
||||
const event = (
|
||||
typeof DragEvent === "function"
|
||||
|
|
@ -244,7 +313,10 @@ describe("MarkdownEditor", () => {
|
|||
mdxEditorMockState.emitMountParseError = false;
|
||||
mdxEditorMockState.emitMountSilentEmptyState = false;
|
||||
mdxEditorMockState.throwOnRender = false;
|
||||
mdxEditorMockState.emitMountChange = null;
|
||||
mdxEditorMockState.populateDelayMs = 0;
|
||||
mdxEditorMockState.markdownValues = [];
|
||||
mdxEditorMockState.insertedMarkdownValues = [];
|
||||
mdxEditorMockState.suppressHtmlProcessingValues = [];
|
||||
});
|
||||
|
||||
|
|
@ -464,6 +536,7 @@ describe("MarkdownEditor", () => {
|
|||
expect(textarea).not.toBeNull();
|
||||
expect(textarea?.value).toBe("Affected versions: <= v0.3.1");
|
||||
expect(container.textContent).toContain("Rich editor unavailable for this markdown");
|
||||
expect(fallbackCode(container)).toBe("MDE-PARSE");
|
||||
expect(handleChange).not.toHaveBeenCalled();
|
||||
|
||||
await act(async () => {
|
||||
|
|
@ -494,6 +567,7 @@ describe("MarkdownEditor", () => {
|
|||
expect(textarea).not.toBeNull();
|
||||
expect(textarea?.value).toBe("5. python3 circleback/sync_insights.py --input <tmp> -- writes insights/<group>/*.md");
|
||||
expect(container.textContent).toContain("Rich editor unavailable for this markdown");
|
||||
expect(fallbackCode(container)).toBe("MDE-RENDER");
|
||||
expect(consoleError).toHaveBeenCalledWith(
|
||||
"Markdown rich editor failed; falling back to raw textarea",
|
||||
expect.objectContaining({
|
||||
|
|
@ -532,6 +606,7 @@ describe("MarkdownEditor", () => {
|
|||
expect(textarea).not.toBeNull();
|
||||
expect(textarea?.value).toBe("Affected versions: <= v0.3.1");
|
||||
expect(container.textContent).toContain("Rich editor unavailable for this markdown");
|
||||
expect(fallbackCode(container)).toBe("MDE-EMPTY");
|
||||
expect(handleChange).not.toHaveBeenCalled();
|
||||
|
||||
await act(async () => {
|
||||
|
|
@ -539,6 +614,293 @@ describe("MarkdownEditor", () => {
|
|||
});
|
||||
});
|
||||
|
||||
it("keeps prose with bare angle brackets in the rich editor", async () => {
|
||||
const value = "Affected versions: <= v0.3.1\n\nRename <name> to the real name.";
|
||||
const handleChange = vi.fn();
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<MarkdownEditor value={value} onChange={handleChange} placeholder="Markdown body" />,
|
||||
);
|
||||
});
|
||||
|
||||
await flush();
|
||||
await waitPastEmptyHeuristic();
|
||||
|
||||
expect(container.querySelector("textarea")).toBeNull();
|
||||
expect(container.textContent).not.toContain("Rich editor unavailable for this markdown");
|
||||
|
||||
// The editor is handed the escaped form: the placeholder is hidden from the
|
||||
// HTML tokenizer, while `<=` cannot open a tag and is left exactly as typed.
|
||||
const received = mdxEditorMockState.markdownValues.at(-1);
|
||||
expect(received).toContain("Rename \\<name> to the real name.");
|
||||
expect(received).toContain("Affected versions: <= v0.3.1");
|
||||
|
||||
// What the user sees is still the markdown they wrote.
|
||||
expect(container.textContent).toContain("Rename <name> to the real name.");
|
||||
expect(handleChange).not.toHaveBeenCalled();
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it("returns editor output to the parent with the transport escaping removed", async () => {
|
||||
const value = "Affected versions: <= v0.3.1\n\nRename <name> to the real name.";
|
||||
// What the real exporter emits for this document: `<` re-escaped, `<=` bare.
|
||||
mdxEditorMockState.emitMountChange =
|
||||
"Affected versions: <= v0.3.1\n\nRename \\<name> to the real name.";
|
||||
const handleChange = vi.fn();
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<MarkdownEditor value={value} onChange={handleChange} placeholder="Markdown body" />,
|
||||
);
|
||||
});
|
||||
|
||||
await flush();
|
||||
|
||||
// The parent stores the clean form — these fields feed agent prompts, so a
|
||||
// stray `\<` would leak into a prompt.
|
||||
expect(handleChange).toHaveBeenCalledWith(value);
|
||||
expect(handleChange.mock.calls.every(([next]) => !String(next).includes("\\<"))).toBe(true);
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it("does not notify the parent when a prop sync echoes escaped markdown back", async () => {
|
||||
const handleChange = vi.fn();
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<MarkdownEditor value="" onChange={handleChange} placeholder="Markdown body" />,
|
||||
);
|
||||
});
|
||||
|
||||
// The editor echoes an imperative `setMarkdown` back through `onChange` in
|
||||
// editor space, while the component compares in stored space.
|
||||
mdxEditorMockState.emitMountChange = "Rename \\<name> here";
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<MarkdownEditor
|
||||
value="Rename <name> here"
|
||||
onChange={handleChange}
|
||||
placeholder="Markdown body"
|
||||
/>,
|
||||
);
|
||||
});
|
||||
|
||||
await flush();
|
||||
await waitPastEmptyHeuristic();
|
||||
|
||||
expect(handleChange).not.toHaveBeenCalled();
|
||||
expect(container.querySelector("textarea")).toBeNull();
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it("recovers the rich editor on retry after a transient parse failure", async () => {
|
||||
mdxEditorMockState.emitMountParseError = true;
|
||||
const handleChange = vi.fn();
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<MarkdownEditor
|
||||
value="Rename <name> to the real name."
|
||||
onChange={handleChange}
|
||||
placeholder="Markdown body"
|
||||
/>,
|
||||
);
|
||||
});
|
||||
|
||||
await flush();
|
||||
await vi.waitFor(() => {
|
||||
expect(container.querySelector("textarea")).not.toBeNull();
|
||||
});
|
||||
expect(fallbackCode(container)).toBe("MDE-PARSE");
|
||||
|
||||
// The failure was transient; the next mount imports the same markdown fine.
|
||||
mdxEditorMockState.emitMountParseError = false;
|
||||
|
||||
await act(async () => {
|
||||
clickRetryRichEditor(container);
|
||||
});
|
||||
await flush();
|
||||
await waitPastEmptyHeuristic();
|
||||
|
||||
expect(container.querySelector("textarea")).toBeNull();
|
||||
expect(container.textContent).not.toContain("Rich editor unavailable for this markdown");
|
||||
expect(container.textContent).toContain("Rename <name> to the real name.");
|
||||
expect(handleChange).not.toHaveBeenCalledWith("");
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it("re-arms the empty-onChange guard when the editor is retried", async () => {
|
||||
// The sequence this guards: the editor mounts, reports an edit (spending the
|
||||
// one-shot mount guard), then fails. The user retries, and the fresh mount
|
||||
// emits the empty onChange that the guard exists to swallow — which would
|
||||
// otherwise wipe the parent's value.
|
||||
mdxEditorMockState.emitMountChange = "Rename \\<name> to the real name.";
|
||||
mdxEditorMockState.emitMountParseError = true;
|
||||
const handleChange = vi.fn();
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<MarkdownEditor
|
||||
value="Rename <name> to the real name."
|
||||
onChange={handleChange}
|
||||
placeholder="Markdown body"
|
||||
/>,
|
||||
);
|
||||
});
|
||||
|
||||
await flush();
|
||||
await vi.waitFor(() => {
|
||||
expect(container.querySelector("textarea")).not.toBeNull();
|
||||
});
|
||||
|
||||
mdxEditorMockState.emitMountChange = null;
|
||||
mdxEditorMockState.emitMountParseError = false;
|
||||
mdxEditorMockState.emitMountEmptyReset = true;
|
||||
|
||||
await act(async () => {
|
||||
clickRetryRichEditor(container);
|
||||
});
|
||||
await flush();
|
||||
await waitPastEmptyHeuristic();
|
||||
|
||||
expect(handleChange).not.toHaveBeenCalledWith("");
|
||||
expect(container.querySelector("textarea")).toBeNull();
|
||||
expect(container.textContent).toContain("Rename <name> to the real name.");
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it("does not fall back again while a retried editor is still populating", async () => {
|
||||
mdxEditorMockState.emitMountParseError = true;
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<MarkdownEditor
|
||||
value="Rename <name> to the real name."
|
||||
onChange={() => {}}
|
||||
placeholder="Markdown body"
|
||||
/>,
|
||||
);
|
||||
});
|
||||
|
||||
await flush();
|
||||
await vi.waitFor(() => {
|
||||
expect(container.querySelector("textarea")).not.toBeNull();
|
||||
});
|
||||
|
||||
// The retried editor paints its content between the first check and the
|
||||
// confirming re-check, so the heuristic must not call it empty.
|
||||
mdxEditorMockState.emitMountParseError = false;
|
||||
mdxEditorMockState.populateDelayMs = 150;
|
||||
|
||||
await act(async () => {
|
||||
clickRetryRichEditor(container);
|
||||
});
|
||||
await flush();
|
||||
await waitPastEmptyHeuristic();
|
||||
|
||||
expect(container.querySelector("textarea")).toBeNull();
|
||||
expect(container.textContent).not.toContain("Rich editor unavailable for this markdown");
|
||||
expect(container.textContent).toContain("Rename <name> to the real name.");
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it("escapes angle brackets in markdown inserted through the ref", async () => {
|
||||
const editorRef = { current: null as MarkdownEditorRef | null };
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<MarkdownEditor
|
||||
ref={editorRef}
|
||||
value="Intro."
|
||||
onChange={() => {}}
|
||||
placeholder="Markdown body"
|
||||
/>,
|
||||
);
|
||||
});
|
||||
await flush();
|
||||
|
||||
await act(async () => {
|
||||
editorRef.current?.insertMarkdown("\n\nRename <name> to the real name.");
|
||||
});
|
||||
await flush();
|
||||
await waitPastEmptyHeuristic();
|
||||
|
||||
expect(mdxEditorMockState.insertedMarkdownValues).toEqual([
|
||||
"\n\nRename \\<name> to the real name.",
|
||||
]);
|
||||
expect(container.querySelector("textarea")).toBeNull();
|
||||
expect(container.textContent).not.toContain("Rich editor unavailable for this markdown");
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it("escapes angle brackets in pasted markdown", async () => {
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<MarkdownEditor value="Intro." onChange={() => {}} placeholder="Markdown body" />,
|
||||
);
|
||||
});
|
||||
await flush();
|
||||
|
||||
const scope = container.querySelector('[data-testid="mdx-editor"]')?.parentElement;
|
||||
const pasted = "## Setup\n\n- Rename <name> to the real name\n";
|
||||
const event = new Event("paste", { bubbles: true, cancelable: true });
|
||||
Object.defineProperty(event, "clipboardData", {
|
||||
configurable: true,
|
||||
value: {
|
||||
types: ["text/plain"],
|
||||
getData: (type: string) => (type === "text/plain" ? pasted : ""),
|
||||
},
|
||||
});
|
||||
|
||||
await act(async () => {
|
||||
scope?.dispatchEvent(event);
|
||||
});
|
||||
await flush();
|
||||
await waitPastEmptyHeuristic();
|
||||
|
||||
expect(mdxEditorMockState.insertedMarkdownValues).toEqual([
|
||||
"## Setup\n\n- Rename \\<name> to the real name\n",
|
||||
]);
|
||||
expect(container.querySelector("textarea")).toBeNull();
|
||||
expect(container.textContent).not.toContain("Rich editor unavailable for this markdown");
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it("shows the editor-scoped dropzone by default when files are dragged over it", async () => {
|
||||
const root = createRoot(container);
|
||||
|
||||
|
|
|
|||
|
|
@ -51,6 +51,10 @@ import { MentionAwareLinkNode, mentionAwareLinkNodeReplacement } from "../lib/me
|
|||
import { mentionDeletionPlugin } from "../lib/mention-deletion";
|
||||
import { looksLikeMarkdownPaste } from "../lib/markdownPaste";
|
||||
import { normalizeMarkdown } from "../lib/normalize-markdown";
|
||||
import {
|
||||
escapeUnsupportedAngleBrackets,
|
||||
unescapeAngleBracketEscapes,
|
||||
} from "../lib/angle-bracket-markdown";
|
||||
import { unescapeBlockquoteMarkers } from "../lib/blockquote-markdown";
|
||||
import { pasteNormalizationPlugin } from "../lib/paste-normalization";
|
||||
import { cn } from "../lib/utils";
|
||||
|
|
@ -149,12 +153,33 @@ function convertHtmlImagesToMarkdown(text: string): string {
|
|||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert a stored value into the exact markdown handed to MDXEditor.
|
||||
*
|
||||
* The angle-bracket escape runs last, after the `<img>` rewrite, because that
|
||||
* rewrite has to match a bare `<img` tag before the escape hides it.
|
||||
*/
|
||||
function prepareMarkdownForEditor(value: string): string {
|
||||
const normalizedLineEndings = value.replace(/\r\n/g, "\n").replace(/\r/g, "\n");
|
||||
// Recover escaped blockquotes (`\>`) so `>`-prefixed content renders as a real
|
||||
// blockquote in the editor as well as on display (keeps import/export in sync).
|
||||
const withBlockquotes = unescapeBlockquoteMarkers(normalizedLineEndings);
|
||||
return convertHtmlImagesToMarkdown(withBlockquotes);
|
||||
const withImages = convertHtmlImagesToMarkdown(withBlockquotes);
|
||||
// The editor runs with `suppressHtmlProcessing`, so a bare `<` that opens an
|
||||
// HTML construct would throw and drop the whole component to raw source.
|
||||
// Escaping those brackets keeps ordinary prose — `<name>`, `</close>` — in the
|
||||
// rich editor. `toStoredMarkdown` reverses it on the way out.
|
||||
return escapeUnsupportedAngleBrackets(withImages);
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert markdown exported by MDXEditor back into the form this product
|
||||
* stores. Inverse of the rewrites `prepareMarkdownForEditor` applies, so the
|
||||
* stored value never carries the editor's transport escaping — these fields
|
||||
* feed agent prompts and must stay in the clean, human-authored form.
|
||||
*/
|
||||
function toStoredMarkdown(markdown: string): string {
|
||||
return unescapeAngleBracketEscapes(unescapeBlockquoteMarkers(markdown));
|
||||
}
|
||||
|
||||
function escapeRegExp(value: string): string {
|
||||
|
|
@ -222,6 +247,30 @@ function richEditorErrorMessage(error: unknown): string {
|
|||
return "Rich editor failed to render";
|
||||
}
|
||||
|
||||
/**
|
||||
* Why the rich editor was abandoned for this value. Surfaced in the fallback
|
||||
* header so a bug report names the path that failed rather than describing the
|
||||
* same generic message for three unrelated causes.
|
||||
*
|
||||
* - `MDE-PARSE` the markdown importer rejected the source
|
||||
* - `MDE-RENDER` the editor threw while rendering
|
||||
* - `MDE-EMPTY` the editor mounted but never painted the content
|
||||
*/
|
||||
type RichEditorErrorCode = "MDE-PARSE" | "MDE-RENDER" | "MDE-EMPTY";
|
||||
|
||||
interface RichEditorError {
|
||||
code: RichEditorErrorCode;
|
||||
message: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The editor populates its DOM asynchronously, so "looks empty" is only
|
||||
* trustworthy after it has had time to settle and then stayed empty. The first
|
||||
* check debounces behind mutations; the second confirms the verdict once.
|
||||
*/
|
||||
const RICH_EDITOR_EMPTY_CHECK_MS = 100;
|
||||
const RICH_EDITOR_EMPTY_CONFIRM_MS = 200;
|
||||
|
||||
/* ---- Mention detection helpers ---- */
|
||||
|
||||
interface MentionState {
|
||||
|
|
@ -663,7 +712,7 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
const echoIgnoreMarkdownRef = useRef<string | null>(null);
|
||||
const [uploadError, setUploadError] = useState<string | null>(null);
|
||||
const [isDragOver, setIsDragOver] = useState(false);
|
||||
const [richEditorError, setRichEditorError] = useState<string | null>(null);
|
||||
const [richEditorError, setRichEditorError] = useState<RichEditorError | null>(null);
|
||||
const dragDepthRef = useRef(0);
|
||||
|
||||
// Stable ref for imageUploadHandler so plugins don't recreate on every render
|
||||
|
|
@ -735,7 +784,10 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
// (an editor that was never focused). Focus first — the callback runs
|
||||
// once focus (and a selection: caret kept, else rootEnd) is in place.
|
||||
const editor = ref.current;
|
||||
editor.focus(() => editor.insertMarkdown(markdown), { defaultSelection: "rootEnd" });
|
||||
// Inserted markdown reaches the same importer as the mounted document, so
|
||||
// it needs the same angle-bracket escaping to survive it.
|
||||
const editorMarkdown = escapeUnsupportedAngleBrackets(markdown);
|
||||
editor.focus(() => editor.insertMarkdown(editorMarkdown), { defaultSelection: "rootEnd" });
|
||||
return;
|
||||
}
|
||||
const textarea = fallbackTextareaRef.current;
|
||||
|
|
@ -781,22 +833,44 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
const container = containerRef.current;
|
||||
if (!container) return;
|
||||
|
||||
let timeoutId = 0;
|
||||
let checkTimeoutId = 0;
|
||||
let confirmTimeoutId = 0;
|
||||
let mutationCount = 0;
|
||||
|
||||
const looksEmpty = () => {
|
||||
const editable = container.querySelector('[contenteditable="true"]');
|
||||
if (!(editable instanceof HTMLElement)) return false;
|
||||
const activeElement = document.activeElement;
|
||||
// A focused editor is the user's, not ours to second-guess.
|
||||
if (activeElement === editable || editable.contains(activeElement)) return false;
|
||||
return isRichEditorDomEmpty(editable, editorValue, placeholder);
|
||||
};
|
||||
|
||||
// Two phases. Mounting (and especially remounting after "Retry rich
|
||||
// editor", where focus sits on the button rather than the editor) leaves
|
||||
// the editable momentarily empty, so a single immediate check reads a
|
||||
// still-populating editor as a broken one. Wait, then confirm once; any
|
||||
// mutation in between means content arrived and restarts the whole thing.
|
||||
const scheduleCheck = () => {
|
||||
window.clearTimeout(timeoutId);
|
||||
timeoutId = window.setTimeout(() => {
|
||||
const editable = container.querySelector('[contenteditable="true"]');
|
||||
if (!(editable instanceof HTMLElement)) return;
|
||||
const activeElement = document.activeElement;
|
||||
if (activeElement === editable || editable.contains(activeElement)) return;
|
||||
if (isRichEditorDomEmpty(editable, editorValue, placeholder)) {
|
||||
setRichEditorError("Rich editor failed to load content");
|
||||
}
|
||||
}, 0);
|
||||
window.clearTimeout(checkTimeoutId);
|
||||
window.clearTimeout(confirmTimeoutId);
|
||||
checkTimeoutId = window.setTimeout(() => {
|
||||
if (!looksEmpty()) return;
|
||||
const mutationsAtCheck = mutationCount;
|
||||
confirmTimeoutId = window.setTimeout(() => {
|
||||
if (mutationCount !== mutationsAtCheck) return;
|
||||
if (!looksEmpty()) return;
|
||||
setRichEditorError({
|
||||
code: "MDE-EMPTY",
|
||||
message: "Rich editor failed to load content",
|
||||
});
|
||||
}, RICH_EDITOR_EMPTY_CONFIRM_MS);
|
||||
}, RICH_EDITOR_EMPTY_CHECK_MS);
|
||||
};
|
||||
|
||||
scheduleCheck();
|
||||
const observer = new MutationObserver(() => {
|
||||
mutationCount += 1;
|
||||
scheduleCheck();
|
||||
});
|
||||
observer.observe(container, {
|
||||
|
|
@ -806,7 +880,8 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
});
|
||||
|
||||
return () => {
|
||||
window.clearTimeout(timeoutId);
|
||||
window.clearTimeout(checkTimeoutId);
|
||||
window.clearTimeout(confirmTimeoutId);
|
||||
observer.disconnect();
|
||||
};
|
||||
}, [editorValue, placeholder, richEditorError]);
|
||||
|
|
@ -836,7 +911,9 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
latestValueRef.current = updated;
|
||||
echoIgnoreMarkdownRef.current = updated;
|
||||
ref.current?.setMarkdown(updated);
|
||||
onChange(updated);
|
||||
// `updated` derives from `latestValueRef`, which is editor
|
||||
// space; the parent only ever sees stored space.
|
||||
onChange(toStoredMarkdown(updated));
|
||||
requestAnimationFrame(() => {
|
||||
ref.current?.focus(undefined, { defaultSelection: "rootEnd" });
|
||||
});
|
||||
|
|
@ -1065,7 +1142,9 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
latestValueRef.current = next;
|
||||
echoIgnoreMarkdownRef.current = next;
|
||||
ref.current?.setMarkdown(next);
|
||||
onChange(next);
|
||||
// `next` derives from `latestValueRef`, which is editor space; the
|
||||
// parent only ever sees stored space.
|
||||
onChange(toStoredMarkdown(next));
|
||||
}
|
||||
|
||||
const restoreSelection = (attemptsRemaining: number) => {
|
||||
|
|
@ -1163,11 +1242,15 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
if (!looksLikeMarkdownPaste(rawText)) return;
|
||||
|
||||
event.preventDefault();
|
||||
ref.current.insertMarkdown(normalizeMarkdown(rawText));
|
||||
ref.current.insertMarkdown(escapeUnsupportedAngleBrackets(normalizeMarkdown(rawText)));
|
||||
}, []);
|
||||
|
||||
const handleRichEditorError = useCallback((error: unknown) => {
|
||||
setRichEditorError(richEditorErrorMessage(error));
|
||||
const handleRichEditorRenderError = useCallback((error: unknown) => {
|
||||
setRichEditorError({ code: "MDE-RENDER", message: richEditorErrorMessage(error) });
|
||||
}, []);
|
||||
|
||||
const handleRichEditorParseError = useCallback((error: unknown) => {
|
||||
setRichEditorError({ code: "MDE-PARSE", message: richEditorErrorMessage(error) });
|
||||
}, []);
|
||||
|
||||
const mentionMenuPosition = mentionState
|
||||
|
|
@ -1189,11 +1272,20 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
)}
|
||||
>
|
||||
<div className="flex items-start justify-between gap-3 px-3 pt-2 text-xs text-muted-foreground">
|
||||
<p>Rich editor unavailable for this markdown. Showing raw source instead.</p>
|
||||
<p>
|
||||
Rich editor unavailable for this markdown. Showing raw source instead.{" "}
|
||||
<span data-testid="markdown-editor-fallback-code" className="font-mono">
|
||||
{richEditorError.code}
|
||||
</span>
|
||||
</p>
|
||||
<button
|
||||
type="button"
|
||||
className="shrink-0 underline underline-offset-2 hover:text-foreground"
|
||||
onClick={() => {
|
||||
// The retry remounts MDXEditor, so re-arm the mount-time guard:
|
||||
// a fresh mount can emit an empty onChange that would otherwise
|
||||
// wipe the parent's value.
|
||||
initialChildOnChangeRef.current = true;
|
||||
setRichEditorError(null);
|
||||
}}
|
||||
>
|
||||
|
|
@ -1336,7 +1428,7 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
}}
|
||||
onPasteCapture={handlePasteCapture}
|
||||
>
|
||||
<MarkdownEditorRichErrorBoundary onError={handleRichEditorError}>
|
||||
<MarkdownEditorRichErrorBoundary onError={handleRichEditorRenderError}>
|
||||
<MDXEditor
|
||||
ref={setEditorRef}
|
||||
markdown={editorValue}
|
||||
|
|
@ -1346,19 +1438,22 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
readOnly={readOnly}
|
||||
onChange={(rawNext) => {
|
||||
if (readOnly) return;
|
||||
// Recover blockquotes the exporter escaped as `\>` (see
|
||||
// unescapeBlockquoteMarkers) so a `>`-prefixed line the user typed
|
||||
// always survives as a real blockquote, even when the WYSIWYG
|
||||
// shortcut didn't fire.
|
||||
const next = unescapeBlockquoteMarkers(rawNext);
|
||||
// Reverse the editor-only rewrites: blockquotes the exporter escaped
|
||||
// as `\>` (so a `>`-prefixed line the user typed survives even when
|
||||
// the WYSIWYG shortcut didn't fire), and the `\<` transport escaping
|
||||
// that keeps bare angle brackets off the HTML parser.
|
||||
const next = toStoredMarkdown(rawNext);
|
||||
const echo = echoIgnoreMarkdownRef.current;
|
||||
if (echo !== null && next === echo) {
|
||||
echoIgnoreMarkdownRef.current = null;
|
||||
latestValueRef.current = next;
|
||||
return;
|
||||
}
|
||||
if (echo !== null) {
|
||||
echoIgnoreMarkdownRef.current = null;
|
||||
// `echo` is what we handed to `setMarkdown`, so it is in editor
|
||||
// space while `next` is in stored space. Accept either form —
|
||||
// otherwise every prop sync of a value containing an escaped
|
||||
// bracket reads as a real edit and notifies the parent.
|
||||
if (next === echo || next === toStoredMarkdown(echo)) {
|
||||
latestValueRef.current = echo;
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
if (initialChildOnChangeRef.current) {
|
||||
|
|
@ -1369,12 +1464,16 @@ export const MarkdownEditor = forwardRef<MarkdownEditorRef, MarkdownEditorProps>
|
|||
return;
|
||||
}
|
||||
}
|
||||
latestValueRef.current = next;
|
||||
// `latestValueRef` is compared against `editorValue`, so it has to
|
||||
// hold editor space; storing `next` would make every edit containing
|
||||
// an escaped bracket look like a pending prop sync and trigger a
|
||||
// redundant `setMarkdown` that resets the caret.
|
||||
latestValueRef.current = prepareMarkdownForEditor(next);
|
||||
onChange(next);
|
||||
}}
|
||||
onBlur={() => onBlur?.()}
|
||||
onError={(payload) => {
|
||||
handleRichEditorError(payload.error);
|
||||
handleRichEditorParseError(payload.error);
|
||||
}}
|
||||
className={cn("paperclip-mdxeditor", !bordered && "paperclip-mdxeditor--borderless")}
|
||||
contentEditableClassName={cn(
|
||||
|
|
|
|||
|
|
@ -0,0 +1,283 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
escapeUnsupportedAngleBrackets,
|
||||
unescapeAngleBracketEscapes,
|
||||
} from "./angle-bracket-markdown";
|
||||
|
||||
describe("escapeUnsupportedAngleBrackets", () => {
|
||||
it("leaves markdown without angle brackets untouched", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("")).toBe("");
|
||||
expect(escapeUnsupportedAngleBrackets("plain prose")).toBe("plain prose");
|
||||
});
|
||||
|
||||
it("escapes a placeholder written in prose", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("Rename <name> to the real name")).toBe(
|
||||
"Rename \\<name> to the real name",
|
||||
);
|
||||
});
|
||||
|
||||
it("escapes closing tags, comments and processing instructions", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("a </close> b")).toBe("a \\</close> b");
|
||||
expect(escapeUnsupportedAngleBrackets("a <!-- note --> b")).toBe("a \\<!-- note --> b");
|
||||
expect(escapeUnsupportedAngleBrackets("a <?php b")).toBe("a \\<?php b");
|
||||
});
|
||||
|
||||
it("escapes every eligible bracket on a line", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("<a> and <b>")).toBe("\\<a> and \\<b>");
|
||||
});
|
||||
|
||||
it("leaves a bracket that cannot open HTML alone", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("Affected versions: <= v0.3.1")).toBe(
|
||||
"Affected versions: <= v0.3.1",
|
||||
);
|
||||
expect(escapeUnsupportedAngleBrackets("2 < 3 and 4 > 1")).toBe("2 < 3 and 4 > 1");
|
||||
expect(escapeUnsupportedAngleBrackets("a <1 b")).toBe("a <1 b");
|
||||
expect(escapeUnsupportedAngleBrackets("trailing <")).toBe("trailing <");
|
||||
});
|
||||
|
||||
it("escapes a bracket next to one that cannot open HTML", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("use <= or <name>")).toBe("use <= or \\<name>");
|
||||
});
|
||||
|
||||
it("does not double-escape an already-escaped bracket", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("Rename \\<name> here")).toBe("Rename \\<name> here");
|
||||
});
|
||||
|
||||
it("escapes a bracket that follows a literal backslash", () => {
|
||||
// `\\` is an escaped backslash, so the `<` after it is still bare markup.
|
||||
expect(escapeUnsupportedAngleBrackets("a \\\\<name> b")).toBe("a \\\\\\<name> b");
|
||||
});
|
||||
|
||||
it("skips inline code spans", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("Use `<name>` here")).toBe("Use `<name>` here");
|
||||
expect(escapeUnsupportedAngleBrackets("Use ``a <b> c`` here")).toBe("Use ``a <b> c`` here");
|
||||
});
|
||||
|
||||
it("escapes outside a code span on the same line", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("`<a>` then <b>")).toBe("`<a>` then \\<b>");
|
||||
});
|
||||
|
||||
it("treats an unmatched backtick as literal text, not a code span", () => {
|
||||
// CommonMark only forms a code span from a matched pair of equal-length
|
||||
// backtick runs, so the bracket after a lone backtick is still markup.
|
||||
expect(escapeUnsupportedAngleBrackets("a ` b <name> c")).toBe("a ` b \\<name> c");
|
||||
expect(escapeUnsupportedAngleBrackets("a ``b` c <name>")).toBe("a ``b` c \\<name>");
|
||||
});
|
||||
|
||||
it("treats a backslash-escaped backtick as literal text", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("a \\`x` b <name>")).toBe("a \\`x` b \\<name>");
|
||||
});
|
||||
|
||||
it("skips a code span that wraps across a soft line break", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("a `one\ntwo <x>` b <y>")).toBe(
|
||||
"a `one\ntwo <x>` b \\<y>",
|
||||
);
|
||||
});
|
||||
|
||||
it("does not carry a code span across a blank line", () => {
|
||||
// A blank line ends the paragraph, so the opening backtick never matches.
|
||||
expect(escapeUnsupportedAngleBrackets("a `one <x>\n\ntwo` b <y>")).toBe(
|
||||
"a `one \\<x>\n\ntwo` b \\<y>",
|
||||
);
|
||||
});
|
||||
|
||||
it("skips fenced code blocks", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("```\n<name>\n```\n<after>")).toBe(
|
||||
"```\n<name>\n```\n\\<after>",
|
||||
);
|
||||
expect(escapeUnsupportedAngleBrackets("~~~ts\n<name>\n~~~")).toBe("~~~ts\n<name>\n~~~");
|
||||
});
|
||||
|
||||
it("skips a fenced code block nested in a blockquote", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("> ```\n> <name>\n> ```\n<after>")).toBe(
|
||||
"> ```\n> <name>\n> ```\n\\<after>",
|
||||
);
|
||||
});
|
||||
|
||||
it("skips a fenced code block nested in a list", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("- ```\n <name>\n ```\n<after>")).toBe(
|
||||
"- ```\n <name>\n ```\n\\<after>",
|
||||
);
|
||||
});
|
||||
|
||||
it("does not close a fence on a shorter run of the same character", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("````\n<a>\n```\n<b>\n````\n<c>")).toBe(
|
||||
"````\n<a>\n```\n<b>\n````\n\\<c>",
|
||||
);
|
||||
});
|
||||
|
||||
it("skips indented code blocks", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("intro\n\n <name>\n\nafter <x>")).toBe(
|
||||
"intro\n\n <name>\n\nafter \\<x>",
|
||||
);
|
||||
expect(escapeUnsupportedAngleBrackets("\t<name>")).toBe("\t<name>");
|
||||
});
|
||||
|
||||
it("escapes an indented lazy paragraph continuation", () => {
|
||||
// Indented code cannot interrupt a paragraph, so this line is prose.
|
||||
expect(escapeUnsupportedAngleBrackets("para\n more <name> here")).toBe(
|
||||
"para\n more \\<name> here",
|
||||
);
|
||||
});
|
||||
|
||||
it("escapes prose inside a blockquote but skips quoted indented code", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("> Rename <name> here")).toBe(
|
||||
"> Rename \\<name> here",
|
||||
);
|
||||
expect(escapeUnsupportedAngleBrackets(">\n> <name>")).toBe(">\n> <name>");
|
||||
});
|
||||
|
||||
it("skips autolinks", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("See <https://example.com> for more")).toBe(
|
||||
"See <https://example.com> for more",
|
||||
);
|
||||
expect(escapeUnsupportedAngleBrackets("Mail <a.b@example.com> now")).toBe(
|
||||
"Mail <a.b@example.com> now",
|
||||
);
|
||||
});
|
||||
|
||||
it("escapes a tag that only looks like an autolink", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("<notascheme>")).toBe("\\<notascheme>");
|
||||
});
|
||||
|
||||
it("skips pointed-bracket link destinations", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("[a](<my file.md>)")).toBe("[a](<my file.md>)");
|
||||
expect(escapeUnsupportedAngleBrackets("")).toBe("");
|
||||
});
|
||||
|
||||
it("still escapes prose around a pointed link destination", () => {
|
||||
expect(escapeUnsupportedAngleBrackets("[a](<f.md>) then <name>")).toBe(
|
||||
"[a](<f.md>) then \\<name>",
|
||||
);
|
||||
});
|
||||
|
||||
it("is idempotent", () => {
|
||||
const inputs = [
|
||||
"Rename <name> to the real name",
|
||||
"Use `<name>` and <other>",
|
||||
"a \\\\<name> b",
|
||||
"```\n<name>\n```\n<after>",
|
||||
"See <https://example.com> and <name>",
|
||||
"[a](<f.md>) then <name>",
|
||||
];
|
||||
for (const input of inputs) {
|
||||
const once = escapeUnsupportedAngleBrackets(input);
|
||||
expect(escapeUnsupportedAngleBrackets(once)).toBe(once);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("unescapeAngleBracketEscapes", () => {
|
||||
it("leaves markdown without escapes untouched", () => {
|
||||
expect(unescapeAngleBracketEscapes("")).toBe("");
|
||||
expect(unescapeAngleBracketEscapes("Rename <name> here")).toBe("Rename <name> here");
|
||||
});
|
||||
|
||||
it("drops the backslash from an escaped html-like bracket", () => {
|
||||
expect(unescapeAngleBracketEscapes("Rename \\<name> here")).toBe("Rename <name> here");
|
||||
expect(unescapeAngleBracketEscapes("a \\</close> b")).toBe("a </close> b");
|
||||
expect(unescapeAngleBracketEscapes("a \\<!-- note --> b")).toBe("a <!-- note --> b");
|
||||
expect(unescapeAngleBracketEscapes("a \\<?php b")).toBe("a <?php b");
|
||||
});
|
||||
|
||||
it("leaves an escape that does not hide an html construct alone", () => {
|
||||
expect(unescapeAngleBracketEscapes("a \\<= b")).toBe("a \\<= b");
|
||||
expect(unescapeAngleBracketEscapes("a \\<1 b")).toBe("a \\<1 b");
|
||||
});
|
||||
|
||||
it("leaves other backslash escapes alone", () => {
|
||||
expect(unescapeAngleBracketEscapes("a \\> b and \\* c and \\<name>")).toBe(
|
||||
"a \\> b and \\* c and <name>",
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps backslash parity", () => {
|
||||
// `\\` is a literal backslash; the `<name>` after it is already bare.
|
||||
expect(unescapeAngleBracketEscapes("a \\\\<name> b")).toBe("a \\\\<name> b");
|
||||
// `\\` then `\<` is a literal backslash followed by an escaped bracket.
|
||||
expect(unescapeAngleBracketEscapes("a \\\\\\<name> b")).toBe("a \\\\<name> b");
|
||||
});
|
||||
|
||||
it("keeps an escape that would otherwise open an autolink", () => {
|
||||
// `\<https://example.com>` renders as the literal text `<https://example.com>`.
|
||||
// Dropping the backslash would publish a link the author escaped on purpose,
|
||||
// and the escape half never escapes an autolink, so there is nothing to undo.
|
||||
expect(unescapeAngleBracketEscapes("See \\<https://example.com> here")).toBe(
|
||||
"See \\<https://example.com> here",
|
||||
);
|
||||
expect(unescapeAngleBracketEscapes("Mail \\<a.b@example.com> now")).toBe(
|
||||
"Mail \\<a.b@example.com> now",
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps an escape that would otherwise open a pointed link destination", () => {
|
||||
expect(unescapeAngleBracketEscapes("[a](\\<my file.md>)")).toBe("[a](\\<my file.md>)");
|
||||
});
|
||||
|
||||
it("still unescapes a tag that only looks like an autolink", () => {
|
||||
expect(unescapeAngleBracketEscapes("\\<notascheme>")).toBe("<notascheme>");
|
||||
// The scheme needs at least two characters, so this is not an autolink.
|
||||
expect(unescapeAngleBracketEscapes("\\<x:y>")).toBe("<x:y>");
|
||||
});
|
||||
|
||||
it("skips code spans and code blocks", () => {
|
||||
expect(unescapeAngleBracketEscapes("Use `\\<name>` here")).toBe("Use `\\<name>` here");
|
||||
expect(unescapeAngleBracketEscapes("```\n\\<name>\n```\n\\<after>")).toBe(
|
||||
"```\n\\<name>\n```\n<after>",
|
||||
);
|
||||
expect(unescapeAngleBracketEscapes("intro\n\n \\<name>\n\nafter \\<x>")).toBe(
|
||||
"intro\n\n \\<name>\n\nafter <x>",
|
||||
);
|
||||
});
|
||||
|
||||
it("is idempotent", () => {
|
||||
const inputs = [
|
||||
"Rename \\<name> here",
|
||||
"a \\\\\\<name> b",
|
||||
"Use `\\<name>` and \\<other>",
|
||||
"```\n\\<name>\n```\n\\<after>",
|
||||
];
|
||||
for (const input of inputs) {
|
||||
const once = unescapeAngleBracketEscapes(input);
|
||||
expect(unescapeAngleBracketEscapes(once)).toBe(once);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("angle bracket escape round trip", () => {
|
||||
const cases = [
|
||||
"Rename <name> to the real name",
|
||||
"Affected versions: <= v0.3.1",
|
||||
"5. python3 sync.py --input <tmp> -- writes insights/<group>/*.md",
|
||||
"a </close> and <!-- note --> and <?php",
|
||||
"Use `<name>` but not <other>",
|
||||
"```\n<name>\n```\n<after>",
|
||||
"intro\n\n <name>\n\nafter <x>",
|
||||
"See <https://example.com> and mail <a@b.com>, then <name>",
|
||||
"[a](<my file.md>) then <name>",
|
||||
"a \\\\<name> b",
|
||||
"> quoted <name>",
|
||||
"- item <name>\n- other <x>",
|
||||
"para\n more <name> here",
|
||||
"a `one\ntwo <x>` b <y>",
|
||||
"plain prose with no brackets",
|
||||
// An author's escaped autolink must survive an edit as literal text.
|
||||
"See \\<https://example.com> here",
|
||||
"Mail \\<a.b@example.com> now",
|
||||
"",
|
||||
];
|
||||
|
||||
it("returns the original markdown after escaping and unescaping", () => {
|
||||
for (const input of cases) {
|
||||
expect(unescapeAngleBracketEscapes(escapeUnsupportedAngleBrackets(input))).toBe(input);
|
||||
}
|
||||
});
|
||||
|
||||
it("normalizes a pre-escaped bracket to its clean form", () => {
|
||||
// `\<name>` and `<name>` are the same literal text; the clean form is what
|
||||
// this product stores, and the rewrite only happens on a real edit.
|
||||
expect(unescapeAngleBracketEscapes(escapeUnsupportedAngleBrackets("Rename \\<name>"))).toBe(
|
||||
"Rename <name>",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,317 @@
|
|||
/**
|
||||
* The rich markdown editor mounts MDXEditor with `suppressHtmlProcessing`, which
|
||||
* strips the HTML visitor out of the import pipeline. Anything the markdown
|
||||
* parser tokenizes as an HTML construct therefore arrives with no visitor able
|
||||
* to handle it and throws `UnrecognizedMarkdownConstructError`, dropping the
|
||||
* whole component into its raw-source fallback.
|
||||
*
|
||||
* The parser opens an HTML construct on a bare `<` followed by a letter, `!`,
|
||||
* `/` or `?` — which is ordinary prose, not markup: a `<name>` placeholder, an
|
||||
* inline `</close>`, an `<!-- note -->`. `escapeUnsupportedAngleBrackets`
|
||||
* rewrites exactly those `<` to `\<` on the way into the editor so the parser
|
||||
* sees literal text, and `unescapeAngleBracketEscapes` reverses it on the way
|
||||
* out so the stored value keeps its clean, human-authored form. These fields
|
||||
* feed agent prompts, so the escape is a transport detail that must never
|
||||
* survive into storage.
|
||||
*
|
||||
* The pair is byte-lossless across the editor's own parse/serialize round trip:
|
||||
* `mdast-util-to-markdown` re-escapes `<` to `\<` in precisely the contexts
|
||||
* escaped here, and in no others (`<=` stays bare; code spans and code blocks
|
||||
* are never escaped).
|
||||
*
|
||||
* One deliberate normalization: markdown that *already* contains `\<name>` is
|
||||
* left alone on the way in and comes back out as `<name>`, because both forms
|
||||
* mean the same literal text and the clean form is what this product stores.
|
||||
* That only happens when the user actually edits the field.
|
||||
*
|
||||
* Contexts skipped, because the parser does not open an HTML construct there
|
||||
* and rewriting would corrupt the content:
|
||||
* - fenced code blocks (including fences carried by blockquote/list prefixes)
|
||||
* - indented code blocks
|
||||
* - inline code spans (matched backtick runs only — an unmatched or
|
||||
* backslash-escaped backtick is literal text and protects nothing)
|
||||
* - autolinks: `<scheme:...>` and `<user@host>`
|
||||
* - pointed-bracket link destinations: `](<...>)`
|
||||
* - an already-escaped `\<`
|
||||
*
|
||||
* Fence tracking mirrors `blockquote-markdown.ts`. The two scanners stay
|
||||
* separate on purpose: that one only inspects the leading marker of a line,
|
||||
* while this one has to classify every line as code or inline content and then
|
||||
* walk the inline runs character by character.
|
||||
*/
|
||||
|
||||
/** Characters that make the markdown parser treat a `<` as opening HTML. */
|
||||
const HTML_OPENER_RE = /[A-Za-z!/?]/;
|
||||
|
||||
// An opening fence may follow blockquote/list container markers. Capture the
|
||||
// whole prefix so the closing scan can preserve that container context.
|
||||
const FENCE_OPEN_RE =
|
||||
/^( {0,3}(?:(?:> ?|(?:[-+*]|\d{1,9}[.)]) +))*)(`{3,}|~{3,})(.*)$/;
|
||||
const LIST_MARKER_RE = /(?:[-+*]|\d{1,9}[.)]) +/g;
|
||||
const BLOCKQUOTE_MARKER_RE = />/g;
|
||||
const FENCE_CLOSE_RE = /^( *)(`{3,}|~{3,})[ \t]*$/;
|
||||
/** Blockquote container markers, stripped before measuring block indent. */
|
||||
const BLOCKQUOTE_PREFIX_RE = /^ {0,3}(?:> ?)+/;
|
||||
/** Four spaces or a tab: the indent that opens a CommonMark indented code block. */
|
||||
const INDENTED_CODE_RE = /^(?: {4}|\t)/;
|
||||
|
||||
// CommonMark autolinks. The scheme needs 2–32 characters, which is why `<x:y>`
|
||||
// is not an autolink (it is also not HTML, so escaping it is harmless).
|
||||
const AUTOLINK_URI_RE = /^<[A-Za-z][A-Za-z0-9+.-]{1,31}:[^\s<>\x00-\x1f]*>/;
|
||||
const AUTOLINK_EMAIL_RE =
|
||||
/^<[A-Za-z0-9.!#$%&'*+/=?^_`{|}~-]+@[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?(?:\.[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?)*>/;
|
||||
/** A pointed-bracket link destination may hold spaces but never `<` or `>`. */
|
||||
const POINTED_DESTINATION_RE = /^<[^<>\n]*>/;
|
||||
|
||||
type Mode = "escape" | "unescape";
|
||||
|
||||
function isHtmlOpener(char: string | undefined): boolean {
|
||||
return char !== undefined && HTML_OPENER_RE.test(char);
|
||||
}
|
||||
|
||||
function stripBlockquotePrefix(line: string, depth: number): string | null {
|
||||
let rest = line;
|
||||
|
||||
for (let i = 0; i < depth; i += 1) {
|
||||
const marker = /^ {0,3}> ?/.exec(rest);
|
||||
if (!marker) return null;
|
||||
rest = rest.slice(marker[0].length);
|
||||
}
|
||||
|
||||
return rest;
|
||||
}
|
||||
|
||||
/**
|
||||
* Index just past the backtick run that closes a code span opened at
|
||||
* `contentStart` with `runLength` backticks, or -1 when nothing closes it.
|
||||
*
|
||||
* Only a run of exactly the same length closes the span, and backslashes are
|
||||
* literal inside a code span — so the closing scan must not skip a backtick
|
||||
* that happens to follow one.
|
||||
*/
|
||||
function findCodeSpanEnd(text: string, contentStart: number, runLength: number): number {
|
||||
let i = contentStart;
|
||||
|
||||
while (i < text.length) {
|
||||
if (text[i] !== "`") {
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
let run = 1;
|
||||
while (text[i + run] === "`") run += 1;
|
||||
if (run === runLength) return i + run;
|
||||
i += run;
|
||||
}
|
||||
|
||||
return -1;
|
||||
}
|
||||
|
||||
/**
|
||||
* The non-HTML constructs a `<` can open: an autolink, or a pointed-bracket
|
||||
* link destination. Returns the whole construct, or null when the `<` opens
|
||||
* neither.
|
||||
*
|
||||
* Both modes consult this one definition on purpose. Escape mode leaves these
|
||||
* brackets bare, so unescape mode must refuse to *create* one — otherwise the
|
||||
* pair is not an inverse, and an author's escaped `\<https://x>` would silently
|
||||
* become a live link the first time the document is edited.
|
||||
*
|
||||
* `rest` starts at the `<`. `emitted` is the output so far, whose tail is what
|
||||
* identifies a link destination.
|
||||
*/
|
||||
function matchProtectedConstruct(rest: string, emitted: string): string | null {
|
||||
const autolink = AUTOLINK_URI_RE.exec(rest) ?? AUTOLINK_EMAIL_RE.exec(rest);
|
||||
if (autolink) return autolink[0];
|
||||
|
||||
if (emitted.endsWith("](")) {
|
||||
const destination = POINTED_DESTINATION_RE.exec(rest);
|
||||
if (destination) return destination[0];
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk one run of inline content (one or more consecutive non-code lines) and
|
||||
* rewrite the angle brackets the parser would treat as HTML.
|
||||
*/
|
||||
function transformInlineChunk(text: string, mode: Mode): string {
|
||||
let out = "";
|
||||
let i = 0;
|
||||
|
||||
while (i < text.length) {
|
||||
const char = text[i];
|
||||
|
||||
if (char === "`") {
|
||||
let run = 1;
|
||||
while (text[i + run] === "`") run += 1;
|
||||
const end = findCodeSpanEnd(text, i + run, run);
|
||||
if (end !== -1) {
|
||||
// A matched code span: its contents are literal, copy them through.
|
||||
out += text.slice(i, end);
|
||||
i = end;
|
||||
continue;
|
||||
}
|
||||
// An unmatched run is ordinary text and protects nothing after it.
|
||||
out += text.slice(i, i + run);
|
||||
i += run;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (char === "\\") {
|
||||
if (
|
||||
mode === "unescape"
|
||||
&& text[i + 1] === "<"
|
||||
&& isHtmlOpener(text[i + 2])
|
||||
&& matchProtectedConstruct(text.slice(i + 1), out) === null
|
||||
) {
|
||||
out += "<";
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
// Copy the escape pair through untouched. This is what keeps backslash
|
||||
// parity intact: `\\<name>` is a literal backslash followed by markup, so
|
||||
// the `<` after it is still eligible for escaping, while the `<` in an
|
||||
// already-escaped `\<name>` is skipped.
|
||||
out += text.slice(i, i + 2);
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (mode === "escape" && char === "<" && isHtmlOpener(text[i + 1])) {
|
||||
const protectedConstruct = matchProtectedConstruct(text.slice(i), out);
|
||||
if (protectedConstruct) {
|
||||
out += protectedConstruct;
|
||||
i += protectedConstruct.length;
|
||||
continue;
|
||||
}
|
||||
|
||||
out += "\\<";
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
out += char;
|
||||
i += 1;
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
|
||||
function transformAngleBrackets(markdown: string, mode: Mode): string {
|
||||
const lines = markdown.split("\n");
|
||||
const out: string[] = [];
|
||||
let chunk: string[] = [];
|
||||
|
||||
const flushChunk = () => {
|
||||
if (chunk.length === 0) return;
|
||||
out.push(transformInlineChunk(chunk.join("\n"), mode));
|
||||
chunk = [];
|
||||
};
|
||||
|
||||
let fenceChar = ""; // "" when not inside a fenced code block
|
||||
let fenceLen = 0;
|
||||
let fenceBlockquoteDepth = 0;
|
||||
let fenceCloseIndentMax = 3;
|
||||
// A paragraph is open while the previous line held inline content. Indented
|
||||
// code cannot interrupt a paragraph, so a 4-space-indented line is a lazy
|
||||
// continuation of the prose above it rather than code.
|
||||
let paragraphOpen = false;
|
||||
|
||||
for (const line of lines) {
|
||||
if (fenceChar) {
|
||||
flushChunk();
|
||||
out.push(line);
|
||||
|
||||
const closeCandidate = stripBlockquotePrefix(line, fenceBlockquoteDepth);
|
||||
const closeMatch = closeCandidate ? FENCE_CLOSE_RE.exec(closeCandidate) : null;
|
||||
|
||||
if (closeMatch) {
|
||||
const indent = closeMatch[1].length;
|
||||
const run = closeMatch[2];
|
||||
if (indent <= fenceCloseIndentMax && run[0] === fenceChar && run.length >= fenceLen) {
|
||||
fenceChar = "";
|
||||
fenceLen = 0;
|
||||
fenceBlockquoteDepth = 0;
|
||||
fenceCloseIndentMax = 3;
|
||||
}
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
const fenceMatch = FENCE_OPEN_RE.exec(line);
|
||||
|
||||
if (fenceMatch) {
|
||||
const prefix = fenceMatch[1];
|
||||
const run = fenceMatch[2];
|
||||
const char = run[0];
|
||||
const rest = fenceMatch[3];
|
||||
|
||||
// A backtick info string may not itself contain a backtick (CommonMark);
|
||||
// such a line is not a valid opening fence.
|
||||
if (!(char === "`" && rest.includes("`"))) {
|
||||
const listMarkers = prefix.match(LIST_MARKER_RE) ?? [];
|
||||
flushChunk();
|
||||
out.push(line);
|
||||
fenceChar = char;
|
||||
fenceLen = run.length;
|
||||
fenceBlockquoteDepth = (prefix.match(BLOCKQUOTE_MARKER_RE) ?? []).length;
|
||||
// A list's continuation indent includes its marker and following spaces.
|
||||
// A closing fence may add CommonMark's normal 0–3 spaces after that.
|
||||
fenceCloseIndentMax =
|
||||
(listMarkers.length > 0 ? listMarkers.reduce((sum, marker) => sum + marker.length, 0) : 0) + 3;
|
||||
paragraphOpen = false;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
// Block structure is measured on the line's own content, after any
|
||||
// blockquote markers, so quoted prose and quoted code are classified the
|
||||
// same way as their unquoted equivalents.
|
||||
const content = line.replace(BLOCKQUOTE_PREFIX_RE, "");
|
||||
|
||||
if (content.trim().length === 0) {
|
||||
flushChunk();
|
||||
out.push(line);
|
||||
paragraphOpen = false;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!paragraphOpen && INDENTED_CODE_RE.test(content)) {
|
||||
flushChunk();
|
||||
out.push(line);
|
||||
continue;
|
||||
}
|
||||
|
||||
paragraphOpen = true;
|
||||
chunk.push(line);
|
||||
}
|
||||
|
||||
flushChunk();
|
||||
return out.join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Rewrite `<` to `\<` wherever the markdown parser would otherwise open an HTML
|
||||
* construct, leaving code, autolinks, link destinations and existing escapes
|
||||
* untouched. Idempotent.
|
||||
*/
|
||||
export function escapeUnsupportedAngleBrackets(markdown: string): string {
|
||||
if (!markdown.includes("<")) return markdown;
|
||||
return transformAngleBrackets(markdown, "escape");
|
||||
}
|
||||
|
||||
/**
|
||||
* The inverse of {@link escapeUnsupportedAngleBrackets}: drop the backslash
|
||||
* from `\<` wherever it only exists to hide an HTML construct from the parser.
|
||||
*
|
||||
* An escape is kept when removing it would open an autolink or a pointed link
|
||||
* destination, because those are constructs the escape half deliberately leaves
|
||||
* bare — unescaping one would turn an author's literal `\<https://x>` into a
|
||||
* live link rather than reversing anything. Idempotent.
|
||||
*/
|
||||
export function unescapeAngleBracketEscapes(markdown: string): string {
|
||||
if (!markdown.includes("\\<")) return markdown;
|
||||
return transformAngleBrackets(markdown, "unescape");
|
||||
}
|
||||
Loading…
Reference in New Issue