Skip to content

fix(markdown): a fenced-code line with trailing text is content, not a closing fence#96687

Closed
ly-wang19 wants to merge 1 commit into
openclaw:mainfrom
ly-wang19:fix/fence-close-trailing-text
Closed

fix(markdown): a fenced-code line with trailing text is content, not a closing fence#96687
ly-wang19 wants to merge 1 commit into
openclaw:mainfrom
ly-wang19:fix/fence-close-trailing-text

Conversation

@ly-wang19

Copy link
Copy Markdown
Contributor

What Problem This Solves

scanFenceSpans/parseFenceSpans (packages/markdown-core/src/fences.ts) maps where fenced code blocks start and end so chunkers never split inside one. It accepted any line that begins with ≥3 matching fence markers as a closing fence, ignoring trailing text after the marker. Per CommonMark a closing fence may be followed only by whitespace, so a code-content line like ``` not a close was wrongly treated as a close: the block ended early, the following lines were reported as outside any fence, and the trailing real ``` became a new unclosed opener.

The consequence: isSafeFenceBreak() returns true for offsets inside the actual code block and findFenceSpanAt() returns undefined, so chunkMarkdownText and the embedded-agent block chunker can split inside a fenced code block — the exact thing this module exists to prevent. Code blocks that demonstrate markdown/fences (a line starting with ``` followed by text) are ordinary content.

Why This Change Was Made

The closing branch already checked the marker character and length (its comment cites CommonMark) but missed the trailing-text rule. The fix adds that one condition: a closing fence's trailing text must be whitespace-only. The reference implementation (micromark) confirms the sample input is a single code block. This makes fence detection stricter — the safe direction (it keeps more text inside a fence, so a chunker is less likely to split inside code).

User Impact

Outbound/agent text containing a fenced code block whose body has a ```-prefixed line is no longer mis-parsed, so chunkers won't break the code block at that point. Opening info strings, bare closes, and longer same-marker closes are unaffected. No API change.

Evidence

oxfmt, oxlint, tsgo:core pass. A new colocated fences.test.ts regression test passes and fails when the fix is reverted (fail-before). Both downstream suites — src/auto-reply/chunk.test.ts and src/agents/embedded-agent-block-chunker.test.ts — stay green.

Real behavior proof

Behavior addressed: a fenced-code content line beginning with the fence marker but carrying trailing text (``` not a close) was treated as a closing fence, so isSafeFenceBreak reported an offset inside the code block as a safe chunk break, letting chunkers split inside the block.

Real environment tested: Ran the exported parseFenceSpans / isSafeFenceBreak / findFenceSpanAt from packages/markdown-core/src/fences.ts via tsx on Node v22.22.1 (no mocks), before and after the patch, plus the colocated Vitest test and the two downstream chunker suites via node scripts/run-vitest.mjs.

Exact steps or command run after this patch: parseFenceSpans("```\\ncode\\n``` not a close\\nmore code\\n```\\n") and isSafeFenceBreak(spans, <offset of "more code">); then node scripts/run-vitest.mjs packages/markdown-core/src/fences.test.ts src/auto-reply/chunk.test.ts src/agents/embedded-agent-block-chunker.test.ts, node scripts/run-oxlint.mjs, pnpm tsgo:core.

Evidence after fix:

BEFORE (origin/main):
  parseFenceSpans(...)          -> [{0,24},{35,39}]  (block closed early; "more code" outside any fence)
  isSafeFenceBreak(spans, 27)   -> true   (WRONG: 27 = 'r' in "more code", inside the real code block)
  findFenceSpanAt(spans, 27)    -> undefined  (WRONG)
AFTER:
  parseFenceSpans(...)          -> [{0,38}]   (one code block, as micromark renders it)
  isSafeFenceBreak(spans, 27)   -> false
  findFenceSpanAt(spans, 27)    -> the enclosing span
  control "```\ncode\n```\nafter" -> still one span; "after" is a safe break (true)
fences.test.ts -> exit 0 ; fail-before (revert source) -> the new test fails
chunk.test.ts -> exit 0 ; embedded-agent-block-chunker.test.ts -> exit 0 ; oxlint -> 0 ; tsgo:core -> 0

Observed result after fix: a marker line with trailing text stays inside the code block (one span), interior offsets are no longer reported as safe breaks, and valid closes (bare, longer same-marker, with trailing whitespace) plus opening info strings still behave correctly; the downstream chunkers are unchanged on their existing tests.

What was not tested: a full end-to-end channel render of a chunked message containing such a code block (no live channel in this environment); the fix is a pure parser change proven at the function and unit-test level, cross-checked against the CommonMark reference (micromark) for the single-code-block interpretation.

…a closing fence

scanFenceSpans accepted any line starting with >=3 matching fence markers as a
closing fence, ignoring trailing text after the marker. Per CommonMark a closing
fence may be followed only by whitespace, so a code-content line such as
"``` not a close" was wrongly treated as a close: the block ended early, the
following lines were reported as outside any fence, and the trailing marker line
became a new unclosed opener.

That made isSafeFenceBreak() return true for offsets inside the real code block
and findFenceSpanAt() return undefined, so chunkers (chunkMarkdownText, the
embedded-agent block chunker) could split inside a fenced code block — the exact
thing this module exists to prevent.

Require the closing fence's trailing text to be whitespace-only. Opening info
strings, bare closes, and longer same-marker closes are unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@openclaw-barnacle openclaw-barnacle Bot added size: XS r: too-many-prs Auto-close: author has more than twenty active PRs. labels Jun 25, 2026
@openclaw-barnacle

Copy link
Copy Markdown

Closing this PR because the author has more than 20 active PRs in this repo. Please reduce the active PR queue and reopen or resubmit once it is back under the limit. You can close your own PRs to get back under the limit.

@ly-wang19

Copy link
Copy Markdown
Contributor Author

Reopening — I've trimmed my active-PR queue back under the repo limit. This fix is self-contained and verified (reproduce → fix → fail-before-test → green CI), with full proof in the PR body; no live-channel environment needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

r: too-many-prs Auto-close: author has more than twenty active PRs. size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant