Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
fix(markdown): a fenced-code line with trailing text is content, not …
…a closing fence

scanFenceSpans accepted any line starting with >=3 matching fence markers as a
closing fence, ignoring trailing text after the marker. Per CommonMark a closing
fence may be followed only by whitespace, so a code-content line such as
"``` not a close" was wrongly treated as a close: the block ended early, the
following lines were reported as outside any fence, and the trailing marker line
became a new unclosed opener.

That made isSafeFenceBreak() return true for offsets inside the real code block
and findFenceSpanAt() return undefined, so chunkers (chunkMarkdownText, the
embedded-agent block chunker) could split inside a fenced code block — the exact
thing this module exists to prevent.

Require the closing fence's trailing text to be whitespace-only. Opening info
strings, bare closes, and longer same-marker closes are unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
  • Loading branch information
2 people authored and vincentkoc committed Jun 26, 2026
commit 0540f59c50a466694e103925f144d8fede75b185
26 changes: 26 additions & 0 deletions packages/markdown-core/src/fences.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// Tests fenced-code-block span scanning used to keep chunk breaks out of code blocks.
import { describe, expect, it } from "vitest";
import { isSafeFenceBreak, parseFenceSpans } from "./fences.js";

describe("parseFenceSpans closing-fence rules", () => {
it("treats a marker line with trailing text as code content, not a closing fence", () => {
// CommonMark: a closing fence may be followed only by whitespace, so "``` not a close" is code
// content and the block stays open until the real closing fence. Reporting an interior offset
// as a safe break would let a chunker split inside the code block.
const text = "```\ncode\n``` not a close\nmore code\n```\n";
const spans = parseFenceSpans(text);

expect(spans).toHaveLength(1);
expect(isSafeFenceBreak(spans, text.indexOf("more code") + 1)).toBe(false);
});

it("still closes on a bare fence, a longer same-marker fence, and keeps an opener info string", () => {
expect(parseFenceSpans("```\ncode\n```\nafter\n")).toHaveLength(1);
expect(parseFenceSpans("```\ncode\n````` \nafter\n")).toHaveLength(1);
expect(parseFenceSpans("```python\nx = 1\n```\n")).toHaveLength(1);

const closed = "```\ncode\n```\nafter\n";
const spans = parseFenceSpans(closed);
expect(isSafeFenceBreak(spans, closed.indexOf("after") + 1)).toBe(true);
});
});
11 changes: 8 additions & 3 deletions packages/markdown-core/src/fences.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,14 @@ export function scanFenceSpans(
marker,
indent,
};
} else if (open.markerChar === markerChar && markerLen >= open.markerLen) {
// CommonMark allows a closing fence to be longer than the opener, but
// it must use the same marker character to avoid crossing fence kinds.
} else if (
open.markerChar === markerChar &&
markerLen >= open.markerLen &&
match[3].trim() === ""
) {
// CommonMark: a closing fence uses the same marker character, may be longer than the
// opener, and may be followed only by whitespace. A marker line carrying trailing text
// (e.g. "``` note") is code content, not a close, so it must not end the block.
const end = lineEnd;
spans.push({
start: open.start,
Expand Down