fix(anthropic): surface reasoning and pre-tool commentary on Discord#96106
Conversation
312e99c to
a181314
Compare
|
Codex review: needs changes before merge. Reviewed June 30, 2026, 8:13 PM ET / 00:13 UTC. Summary PR surface: Source +579, Tests +682, Docs +17, Generated 0, Other 0. Total +1278 across 42 files. Reproducibility: yes. The linked issue and inspected screenshots show Anthropic reasoning/commentary present in the run but missing from Discord before the fix, and source inspection shows the pre-tool text classification and delivery gates this branch changes. Review metrics: 3 noteworthy metrics.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Copy recommended automerge instructionNext step before merge
Security Review findings
Review detailsBest possible solution: Land after converting the stale progress.thinking tests to the session-level reasoning gate and after maintainers accept isCommentary/commentaryPayloadsEnabled as the plugin-facing commentary contract. Do we have a high-confidence way to reproduce the issue? Yes. The linked issue and inspected screenshots show Anthropic reasoning/commentary present in the run but missing from Discord before the fix, and source inspection shows the pre-tool text classification and delivery gates this branch changes. Is this the best way to solve the issue? Yes, with the API acceptance and test cleanup noted above. Classification belongs in the Anthropic transport/subscriber path, shared dispatch keeps hidden payloads default-suppressed, and Discord owns visible presentation. Full review comments:
Overall correctness: patch is correct AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 5e572dcf781a. Label changesLabel justifications:
Evidence reviewedPR surface: Source +579, Tests +682, Docs +17, Generated 0, Other 0. Total +1278 across 42 files. View PR surface stats
Acceptance criteria:
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
…(opus + sonnet)
Real behavior proof — live Discord capturesComplementing the boundary-injection test in the PR body, here are redacted live Discord captures on two Anthropic models with Build disclosure: captured on the 2026.6.9 line — before = Before — opus-4-8, After — opus-4-8, After — sonnet-4-6, |
5592905 to
d5b50b4
Compare
d5b50b4 to
703fabe
Compare
|
Refreshed this PR — the earlier revision was built/captured against a misread local build state, so the diff and proof didn't reflect the intended behavior. This revision is the correct implementation, verified live on Discord ( |
The streamReasoning gate in the shared embedded-agent-subscribe was widened from reasoningMode === "stream" to !== "on", which also fires the onReasoningStream hook in "off" mode for EVERY channel. Sibling channels (Slack when status/tool-progress is on, Mattermost in progress mode) install that hook and render it on their own progress surfaces, so an operator with reasoning display off would see thinking text leak into the progress draft — a regression vs main. Restore main's stream-only behavior by default and gate the widening behind a new explicit opt-in param streamReasoningInNonStreamModes, threaded alongside onReasoningStream through the reply/runner chain. Only the Discord handler sets it true (where it is further gated by channels.discord.streaming.progress .thinking). Slack/Mattermost/Telegram/Feishu/msteams/codex are byte-identical to main. Adds table-driven regression coverage asserting the hook is NOT invoked in off mode without the opt-in, preserved in stream mode, and invoked under the Discord opt-in. Addresses the ClawSweeper [P1] 'keep non-stream reasoning dispatch Discord-scoped' finding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…t render The previous commit opted Discord into non-stream reasoning unconditionally (streamReasoningInNonStreamModes: true). With progress.thinking off (default), the onReasoningStream callback still fired in plain mode and ran side effects — statusReactions.setThinking() and windowReasoningOpen — before the compositor's render gate, so a thinking status reaction leaked despite the config being off. (The compositor render + 🧠 counter were already gated; the reaction was not.) Make channels.discord.streaming.progress.thinking the single source of truth: the draft-preview controller now exposes reasoningProgressEnabled (the resolved flag it already used for the compositor reasoningGate), and the Discord handler passes it as streamReasoningInNonStreamModes. In plain mode with the config off the callback no longer fires at all; stream mode keeps main's behavior. Adds resolver coverage asserting progress.thinking defaults off (unset, explicit false, missing entry, and non-progress stream mode) and is true only when explicitly enabled in progress mode. Also the first time the Discord extension typecheck (against built dist) was run for this change set; it passes. Addresses the ClawSweeper [P1] 'gate the Discord reasoning opt-in before side effects' finding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…c placeholder
extractAssistantThinking substituted a diagnostic string ("Native reasoning was
produced; no summary text was returned.") onto the reasoning lane whenever a
thinking block carried a signature but no summary text. That fires on GPT
constantly via the codex harness and about half the time via the openclaw
harness, and on Anthropic when display:"summarized" still returns empty —
surfacing a placeholder bubble on every channel. It is diagnostic, not reasoning
content; return "" so the downstream .filter(Boolean) drops the bubble. The
signed block stays on the message for API replay; this only governs display.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…alize Anthropic has no native commentary channel; the only signal that leading text is pre-tool narration is that a tool_use block follows it. The finalize loop tagged it only once stop_reason:tool_use landed at message end — after the unphased text already streamed. Tag it the moment the first tool_use content block opens, so the commentary phase rides that partial and the subscriber can route it to the narration lane far earlier. Finalize stays as an idempotent backstop for replay/non-streaming assembly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…is known On block-reply surfaces (onBlockReply + blockReplyBreak "text_end" — the default) the subscriber durably sends a text block at its text_end. For a tool turn with leading narration, that fired before the commentary tag was applied at the next event (tool_use content_block_start), leaking pre-tool narration as a final reply that the later tag could not retract. Progress mode was unaffected (no mid-stream durable send), which is why it looked clean. Hold each leading text block's text_end in the transport until the disposition is known: flush it right after tagPendingCommentaryText at the tool_use boundary (so it carries the commentary phase and the subscriber suppresses it from the durable reply), or at message_delta / finalize for a non-tool turn (where the text is the answer and is emitted unphased). Cleared on error/abort. Text deltas still stream live, so the preview is unchanged; only the text_end (and thus the durable commit) waits one event. Subscriber and all channels are untouched — they now simply see a correctly-phased text_end, so every streaming-capable channel inherits the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The transport now defers a pre-tool text block's text_end until it is tagged commentary, but the earlier text_delta chunks were still appended to the block-reply buffer (handleMessageUpdate). On block-reply channels (blockReplyBreak "text_end" — Feishu/SMS/IRC and other append-style surfaces) that buffer is drained mid-stream by the chunker and flushed before tool execution (handleToolExecutionStart), so the narration could still post as a durable answer that the later commentary tag cannot retract. Gate the durable block-reply append for phase-pending Anthropic text: while an anthropic-messages assistant block is unphased mid-stream (no deliveryPhase, no streamItemId), do not append it to the block-reply buffer. A tool turn's deferred commentary-tagged text_end then suppresses it; a non-tool answer is delivered in full from handleMessageEnd (lastBlockReplyText stays null, so the message_end safety-send fires). Unlike the OpenAI phase-pending guard this does not early-return — the live preview keeps streaming; only the durable append is withheld. Discord progress mode and other providers are unaffected. Unit tests reproduce the leak (narration posted without the gate) and prove the non-tool answer is still delivered in full. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… payloads Mirror json's reasoningPayloadsEnabled seam (openclaw#97875) for the 💬 commentary lane: add GetReplyOptions.commentaryPayloadsEnabled and an isCommentary payload tag next to isReasoning across the payload types; default-suppress isCommentary in dispatch-from-config (block stream + final loops) unless the channel opts in; exclude commentary from TTS exactly like reasoning; thread the tag through the block-reply coalescer, reply-delivery direct-lane, and realtime consult so it cannot be dropped mid-pipeline. Discord opts in when its commentary lane is active. No durable producer is wired yet (preserves the obviyus-approved 3-part Anthropic withhold invariant; handlers.messages.ts unchanged). Also regenerates docs/.generated/config-baseline.sha256 and bumps the plugin-SDK surface budgets to actual post-openclaw#97875-rebase values. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…unts only streamed items Reasoning durable/window selection now follows the session /reasoning level (off|on|stream) instead of the streaming.progress.thinking config flag: on -> durable blockquote, stream -> live window lane, off -> nothing. reasoningPayloadsEnabled is opted in only for 'on'; the window opt-in only for 'stream'. Collapse-bar counters increment only when a line is actually streamed to the window (post-yield), so durable 🧠/💬 lanes are never counted. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Anthropic commentary has no native item id, so the transport re-emits the full commentary text under an empty itemId on every later stream event. The verbose standalone commentary lane dedups only on a defined matching itemId, so each snapshot posted a duplicate persistent message (observed 8x). Skip re-emitting an unchanged commentary snapshot per item key. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…italic The progress-draft compositor wraps reasoning and commentary window lines in _..._ for italics. Telegram renders that as italics; Discord shows a literal underscore on commentary. Add a commentaryItalics compositor flag (default on) and opt Discord out: streamed thinking stays italic, commentary renders plain. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…in live transport (opus-4-8 was sent no display → blank summary); mirror llm/providers/anthropic.ts (cherry picked from commit 378a06da3a6061a867213148e7f0de7d216b0618)
…rs (pioneer) Restores 363f544c4e (dropped in the min-rebase), re-applied onto the tag-at-tool-boundary structure: - stream-resolution: route anthropic-messages models with a non-"anthropic" provider onto the boundary-aware managed transport (the only place commentary gets tagged); the base SDK stream never tags it. Scoped to provider!=anthropic so direct-anthropic paths are unchanged. - transport backstops: tag when the turn CONTAINS a toolCall, not only when stop_reason maps to toolUse — Bedrock/Vertex routes report end_turn on tool turns. No-op for direct Anthropic (already toolUse when a tool call is present). - regression test for the stop_reason-mislabel case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Aligns the two assertions with the commentaryItalics:false opt-out; reasoning window lines stay italic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restores e0c838314b (dropped in the min-rebase), adapted to the count-on-stream bar: when /reasoning on posts durable 🧠 messages, they land after the early progress draft, so editing the draft in place strands the summary at the top above the thoughts. Track persistentReasoningDelivered and, when set, clear the early draft and post the summary fresh below the thoughts (thoughts → summary → answer). Durable 🧠 stays uncounted in the bar (no progressReasoningSteps bump). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… non-work tools out of bar [P2] reasoning-level default now resolves the agent's own reasoningDefault first (mirrors core get-reply-directives + Telegram), then global, then off — an agent-scoped default no longer runs reasoning on while Discord drops the durable 🧠 lane. [P3] exclude non-work tools (message/react/typing) from the collapse-bar tool count via the canonical isChannelProgressDraftWorkToolName predicate, so they no longer inflate the bar. (The earlier render-result approach was wrong — the compositor returns false for stored-but-deferred lines, which undercounts.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…surface Reasoning display is gated by the session /reasoning level after the rework, so the streaming.progress.thinking config key/resolver/docs were a no-op (ClawSweeper P2). Remove the key, type, zod, generated metadata, resolver, tests, and docs mention; regenerate the docs map. progress.commentary and session-level reasoning unchanged. Re-pin the plugin-sdk public-surface budgets (10406->10401 / 5224->5220) to match the source counts after removing the exported resolver. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fuller-stack-dev
left a comment
There was a problem hiding this comment.
The prior Discord summary leak is fixed on the current head, and the Anthropic commentary classification path is covered by focused tests. One shared-channel merge blocker remains.
LOC: +1903/-625 across 42 files (production +894/-315, tests/QA +989/-307, docs +20/-3).
Best-fix verdict: this is not landable as a Discord-scoped patch until the shared compositor behavior is either scoped to Discord or explicitly adopted and proved for the sibling channels that use it.
Verification on exact head bf0d31ef7de9a3c89b82765023e49a478dcae035 against upstream/main 1289abddcb966cfa1535c7b82a08f555d9848445:
- 670 focused tests passed across the shared compositor, Discord, Anthropic transport, embedded runner, auto-reply dispatch, Telegram, and Mattermost.
git diff --checkpassed.- Exact-head GitHub checks are green.
| params.active && resolveChannelStreamingPreviewToolProgress(params.entry); | ||
| const commentaryProgressEnabled = | ||
| params.active && resolveChannelStreamingProgressCommentary(params.entry); | ||
| const thinkingProgressEnabled = params.active && (params.reasoningGate ?? true); |
There was a problem hiding this comment.
[P1] Preserve the existing default for non-Discord callers
reasoningGate defaults to true, and this change also removes the previewToolProgressEnabled guard around storing reasoning. Telegram and Mattermost both construct this shared compositor without the new parameters, so streaming.progress.toolProgress: false plus reasoning streaming now renders reasoning text where main intentionally kept the draft label-only; resetting reasoningRawText on each stored tool line also changes their burst interleaving. The replacement shared test explicitly reverses main's "keeps reasoning details hidden when tool progress lines are hidden" assertion, while neither sibling has changed coverage. This contradicts the PR's Discord-scoped claim. Please preserve current-main defaults and opt Discord into the independent/interleaved reasoning lane, or expand the scope and add Telegram/Mattermost behavior tests, proof, and docs.
|
Landed via squash onto main.
Thanks @Marvinthebored! |



What Problem This Solves
On Discord, Anthropic models' reasoning (🧠) and pre-tool commentary (💬) don't surface correctly alongside tool calls:
💬 0 · 🧠 0even though the content existed in the session transcript).🧠 → 💬 → 🛠️ → 💬 → 🛠️as the turn actually unfolded.Why This Change Was Made
mainalready renders reasoning/commentary lanes via the shared progress-draft compositor, but the Anthropic embedded path had three gaps, plus two delivery/counting gates that suppressed the lanes outside the narrowstream/verbosecases:stream:"assistant",phase:"commentary") reaches the agent-event bus but is never rebuilt into apreambleitem event — sonoteWindowCommentarynever counts it and the draft never renders it.reasoningMode === "stream", so plain mode (reasoning off) dropped reasoning from the window draft and counters. Window thinking is presentation — it should stream for any mode except"on"(which uses durable 🧠 persistence instead).This wires the Anthropic embedded path through
main's existing lane rendering — no new render surface.User Impact
Across plain / verbose / reasoning modes on Discord, Anthropic turns now surface their reasoning and pre-tool commentary, interleaved with tool calls in arrival order, and the collapse-summary bar counts all three lanes (
🧠 N thoughts · 💬 N notes · 🛠️ N tool calls). No config changes; no new render surface.Evidence
Live, on a build carrying these commits (Discord,
anthropic/claude-sonnet-4-6), verified by reading the bot's actual channel messages:Plain mode (reasoning off + verbose off):
🧠 _…reasoning…_ → 💬 _First, the time:_ → 🛠️ date → 💬 _Next, the two largest files — sorting by size:_ → 🛠️ list files— reasoning + commentary interleaved with tools in arrival order.🧠 1 thought · 💬 3 notes · 🛠️ 4 tool calls · ⏱️ 32s— all three lanes counted (was💬 0 · 🧠 0before the fix).Verbose + reasoning mode: commentary renders as durable messages interleaved with tools:
💬 → 🛠️ → 💬 → 🛠️ → 💬 → 🛠️. No regression.After-fix evidence (plain mode, reasoning off + verbose off):
Streaming draft — reasoning + commentary interleaved with the tool in arrival order:
Collapse summary — all three lanes counted:
Persistent (verbose
_rv_) mode (reasoning on + verbose on): reasoning + commentary render as durable messages interleaved with tools in arrival order — the first thought surfaces as a 🧠 blockquote before the tool:Validation:
tsgo(core + test-types) pass; lane suites green (message-handler.process.test.ts,embedded-agent-subscribe.handlers.messages.test.ts,progress-draft-compositor.test.tsincluding the"interleaves reasoning bursts with tool calls in arrival order"assertion);build-allexit 0.Linked context