fix(gateway): report omitted chat-history messages in truncation log by ZengWen-DT · Pull Request #96788 · openclaw/openclaw

ZengWen-DT · 2026-06-25T13:02:47Z

What Problem This Solves

enforceChatHistoryFinalBudget (the final byte-budget step of the chat.history Gateway response) returned placeholderCount: 0 in the branch that keeps only the last message while silently dropping every earlier one:

const last = messages.at(-1);
if (last && jsonUtf8Bytes([last]) <= maxBytes) {
  return { messages: [last], placeholderCount: 0 }; // dropped N-1 messages, reported 0
}

The caller logs truncation only when that count is positive:

if (placeholderCount > 0) {
  logLargePayload({ surface: "gateway.chat.history", action: "truncated", ... });
}

So whenever history was trimmed down to its last message, the count was 0, the gate never fired, and the truncation was invisible to operators. The front byte cap (capArrayByJsonBytes) also drops the oldest messages without reporting a count, so that omission went unlogged too. Closes #96783.

Why This Change Was Made

The count was being assembled from per-step partial counts that did not compose: a step's "placeholders inserted" is not the same as "history omitted", and summing replacedCount + front-cap drops + final-budget count double-counts a message that is first replaced with a placeholder and then trimmed by the byte cap.

The fix gives the omission count a single owner that sees the whole pipeline. enforceChatHistoryFinalBudget now returns only its surviving messages (its message behavior is unchanged). A new reportOmittedChatHistory helper counts, by message identity, how many of the original messages lost their verbatim representation anywhere in the replace → cap → final-budget pipeline, and emits the diagnostic. Identity membership counts each omitted original exactly once.

User Impact

Operators now get the payload.large / truncated diagnostic (and the debug line) whenever chat.history discards older history for any reason — drop-to-last, placeholder replacement, or front byte-cap trimming — with an accurate, non-inflated count. No API/response shape change for clients; this is purely diagnostic accounting. The internal field rename touched only the helper, its single Gateway caller, and the focused test (the TUI / embedded-stub callers read only messages).

Evidence

Real Gateway request (after fix). A full chat.history request issued over a booted Gateway WebSocket server against a real on-disk transcript now emits the truncation diagnostic. The new regression test chat-history-omission-request.test.ts boots the gateway via the standard server harness, seeds a session store + transcript on disk (12 messages, budget lowered to 8 KB through the existing setMaxChatHistoryMessagesBytesForTest seam), then drives rpcReq(ws, "chat.history", …) end-to-end through the real handleChatHistoryRequest (not the budget helpers in isolation) and captures the real diagnostic event bus output:

chat.history real-request diagnostic: returned=3 event={
  "type":"payload.large","surface":"gateway.chat.history","action":"truncated",
  "bytes":25167,"limitBytes":8000,"count":9,"reason":"chat_history_budget", ...
}

12 messages in, 3 survive the byte budget, 9 older messages omitted → count: 9.

Before (negative control). Checking out pre-fix main's chat.ts and running the same real-WS test reproduces the bug: the request still omits 9 messages but emits no diagnostic, so the assertion fails. The test imports only the WebSocket harness and the diagnostic bus (no changed symbols), so it compiles and runs against both main and the PR head — the only difference is whether the diagnostic fires.

$ git checkout 2e6e17f7c5 -- src/gateway/server-methods/chat.ts   # pre-fix main
$ node scripts/run-vitest.mjs run \
    src/gateway/server-methods/chat-history-omission-request.test.ts
  AssertionError: expected [] to have a length of 1 but got +0
     78|       expect(captured).toHaveLength(1);
  Test Files  2 failed (2)
$ git checkout HEAD -- src/gateway/server-methods/chat.ts          # restore fix → passes

Premise. On main the caller computed placeholderCount = replaced.replacedCount + bounded.placeholderCount. The front byte cap (capArrayByJsonBytes) and the keep-last branch of enforceChatHistoryFinalBudget both omit messages without contributing to either term, so the if (placeholderCount > 0) gate never fired for those paths. The fix routes all omission accounting through reportOmittedChatHistory, which counts originals missing from the final survivor set by identity (each omitted original once).

Focused tests exercise the real exported functions and the real WS request path:

node scripts/run-vitest.mjs \
  src/gateway/server-methods/chat-history-omission-request.test.ts \
  src/gateway/server-methods/chat-history-omission-logging.test.ts \
  src/gateway/server-methods/chat-history-budget.test.ts

chat-history-omission-request.test.ts: a real Gateway WS chat.history request emits payload.large / truncated with the unique omitted count (output above); reproduces the missing diagnostic on pre-fix main.
chat-history-omission-logging.test.ts: the diagnostic fires when history is trimmed, does not fire when nothing is omitted, and a message that is replaced and then front-capped is counted once (strictly less than the old additive replacedCount + frontCapDropped sum).
chat-history-budget.test.ts: enforceChatHistoryFinalBudget returns the right surviving messages (pass-through, keep-last preserving the original reference, oversized placeholder, never-empty sentinel).

Formatting verified with oxfmt.

AI-assisted.

enforceChatHistoryFinalBudget reported placeholderCount: 0 when it kept only the last message and silently dropped all earlier ones, so the chat.history caller's "log only when count > 0" gate never fired and operators had no signal that history was truncated. Rename the field to omittedCount and have every omitting branch report the number of input messages that lost their verbatim representation. Also derive the front byte-cap drop count at the caller so that omission source is logged too.

clawsweeper · 2026-06-25T13:05:53Z

Codex review: passed. Reviewed June 25, 2026, 4:16 PM ET / 20:16 UTC.

Summary
The PR moves Gateway chat.history omission accounting to a whole-pipeline reporter and adds focused helper plus real WebSocket request regression tests.

PR surface: Source +35, Tests +219. Total +254 across 4 files.

Reproducibility: yes. Current-main source shows the zero-count keep-last helper branch and the positive-count caller gate, and the PR body includes a negative-control real WebSocket run where the same request test fails before the fix.

Review metrics: 1 noteworthy metric.

Open overlapping fix PRs: 1 open. The remaining narrower candidate targets the same linked Gateway bug, so maintainers should keep this PR as the canonical landing path or explicitly choose otherwise.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #96783
Summary: This PR is the strongest current candidate fix for the canonical Gateway chat-history truncation-accounting issue; one open sibling remains as a narrower overlapping candidate.

Members:

canonical: enforceChatHistoryFinalBudget misreports placeholderCount: 0 when messages are silently dropped #96783 - The issue reports the same zero-count final-budget branch and Gateway truncation logging gate that this PR changes.
superseded: fix(gateway): report dropped message count in enforceChatHistoryFinalBudget (fixes #96783) (AI-assisted) #96792 - That narrower helper-only candidate was closed as superseded by this whole-pipeline PR.
partial_overlap: fix(gateway): report dropped message count in enforceChatHistoryFinalBudget placeholderCount #96803 - That open PR targets the same linked issue but only changes the direct helper branch and does not cover the production front-cap path as completely.
adjacent_distinct: enforceChatHistoryFinalBudget silently returns empty messages when a single message exceeds 6MB #96760 - That closed issue touched the same final-budget helper but concerned the empty-transcript sentinel fallback rather than omitted-count diagnostics.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] One narrower overlapping PR remains open for the same linked issue and should be superseded or closed if this PR lands.

Maintainer options:

Decide the mitigation before merge
Land this whole-pipeline Gateway omission-accounting fix as the canonical path, then close the linked issue and supersede the remaining narrower duplicate PR.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P2] No repair lane is needed; the exact-head automerge path can continue gating this branch on review, CI, and mergeability.

Security
Cleared: The diff changes Gateway TypeScript diagnostic accounting and focused tests only, with no concrete security or supply-chain concern found.

Review details

Best possible solution:

Land this whole-pipeline Gateway omission-accounting fix as the canonical path, then close the linked issue and supersede the remaining narrower duplicate PR.

Do we have a high-confidence way to reproduce the issue?

Yes. Current-main source shows the zero-count keep-last helper branch and the positive-count caller gate, and the PR body includes a negative-control real WebSocket run where the same request test fails before the fix.

Is this the best way to solve the issue?

Yes. Counting omitted originals after replacement, front-capping, and final-budget enforcement is the best fix I saw because it covers the production caller path and avoids double-counting partial-stage accounting.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d2da8c79d9b8.

Label changes

Label changes:

add status: 🚀 automerge armed: This PR is in ClawSweeper's automerge lane. Sufficient (terminal): The PR body includes after-fix terminal output from a real booted Gateway WebSocket chat.history request over an on-disk transcript, plus a negative-control failure against pre-fix main.
remove status: 👀 ready for maintainer look: Current PR status label is status: 🚀 automerge armed.

Label justifications:

P2: The PR addresses a bounded Gateway observability bug where omitted chat-history messages can avoid truncation diagnostics.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 🚀 automerge armed: This PR is in ClawSweeper's automerge lane. Sufficient (terminal): The PR body includes after-fix terminal output from a real booted Gateway WebSocket chat.history request over an on-disk transcript, plus a negative-control failure against pre-fix main.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal output from a real booted Gateway WebSocket chat.history request over an on-disk transcript, plus a negative-control failure against pre-fix main.

Evidence reviewed

PR surface:

Source +35, Tests +219. Total +254 across 4 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	57	22	+35
Tests	3	224	5	+219
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	4	281	27	+254

What I checked:

Repository policy read: Root AGENTS.md and scoped Gateway AGENTS.md files were read; the review applied the Gateway full-path and test-harness guidance. (AGENTS.md:1, d2da8c79d9b8)
Current main still has the bug: Current main still returns placeholderCount: 0 when the final-budget helper keeps only the last message, and the caller logs truncation only when the combined count is positive. (src/gateway/server-methods/chat.ts:1688, d2da8c79d9b8)
Release remains affected: Latest release v2026.6.10 contains the same zero-count keep-last branch and positive-count logging gate, so the PR is not obsolete on shipped code. (src/gateway/server-methods/chat.ts:1667, aa69b12d0086)
PR head centralizes accounting: PR head adds reportOmittedChatHistory, which counts original normalized messages missing from the final survivor set by identity and emits the truncation diagnostic with the unique omitted count. (src/gateway/server-methods/chat.ts:1711, 414f885880f0)
Caller uses the reporter after the full pipeline: The Gateway request path now runs replacement, front byte-cap, final-budget enforcement, and then reports omissions against the original normalized messages and final bounded messages. (src/gateway/server-methods/chat.ts:2806, 414f885880f0)
Front-cap behavior checked: capArrayByJsonBytes trims items from the front and returns only survivors plus bytes, which is why whole-pipeline omission accounting is the better fix than changing only the final-budget helper. (src/gateway/session-utils.fs.ts:846, d2da8c79d9b8)

Likely related people:

Hidetsugu55: PR and commit metadata show this author added the focused chat-history final-budget sentinel behavior and budget tests that this PR refines. (role: introduced adjacent final-budget behavior; confidence: high; commits: e1ab3fba73ad, f2fa246ab79c, 50a4bb00e557; files: src/gateway/server-methods/chat.ts, src/gateway/server-methods/chat-history-budget.test.ts)
vincentkoc: PR metadata for the adjacent final-budget sentinel change shows this user triggered the automerge path for the same helper surface. (role: merge-route signal; confidence: medium; commits: 50a4bb00e557; files: src/gateway/server-methods/chat.ts, src/gateway/server-methods/chat-history-budget.test.ts)
lin-hongkuan: Current-main blame in this checkout attributes the helper, caller, and focused test lines to the recent squash commit for fix(media-generation): preserve trimmed default model flag #96430; this is a routing signal because local history is shallow/grafted. (role: recent area contributor; confidence: low; commits: 2e6e17f7c502; files: src/gateway/server-methods/chat.ts, src/gateway/server-methods/chat-history-budget.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

…nostic Address review: move omission accounting to a single owner that sees the whole replace/cap/final pipeline. enforceChatHistoryFinalBudget now returns only its surviving messages; the new reportOmittedChatHistory helper counts, by message identity, how many original messages lost their verbatim form and emits the truncated diagnostic. Identity membership counts a message that is first replaced and then trimmed exactly once, fixing the double-count in the previous additive sum. Add a real-behavior test that runs the production helpers and asserts the real payload.large/truncated diagnostic event fires with the unique count.

ZengWen-DT · 2026-06-25T13:27:48Z

@clawsweeper re-review

Addressed both findings:

Real behavior proof: added chat-history-omission-logging.test.ts, which runs the production budget pipeline and captures the real diagnostic event bus output. The PR body now shows the emitted event and debug line:
payload.large {action:"truncated", reason:"chat_history_budget", count:2, surface:"gateway.chat.history"}.
Double-count (chat.ts:2774): removed the additive replacedCount + frontCapDropped + bounded.count sum. Omission accounting now lives in a single owner (reportOmittedChatHistory) that counts, by message identity over the whole replace/cap/final pipeline, how many originals lost their verbatim form — so a message that is first replaced and then trimmed is counted exactly once. A test asserts the emitted count is strictly less than the old additive sum in that overlap case.

clawsweeper · 2026-06-25T13:27:50Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Drive a real chat.history request over a booted Gateway WebSocket server against a real on-disk transcript and assert the payload.large/truncated diagnostic fires with the unique omitted count. Imports only the WS harness and diagnostic bus, so the same test reproduces the missing diagnostic on pre-fix main.

Takhoffman · 2026-06-25T20:09:31Z

@clawsweeper automerge

clawsweeper · 2026-06-25T20:09:34Z

🦞✅
ClawSweeper merged this PR after the passing review.

Source: clawsweeper[bot]
Feedback: structured ClawSweeper verdict: pass (sha=414f885880f061508d5149a859266d7095363081)
Merge status: merged by ClawSweeper automerge
Merged at: 2026-06-25T20:17:28Z
Merge commit: c68484acc4b1

What merged:

The PR moves Gateway chat.history omission accounting to a whole-pipeline reporter and adds focused helper plus real WebSocket request regression tests.
PR surface: Source +35, Tests +219. Total +254 across 4 files.
Reproducibility: yes. Current-main source shows the zero-count keep-last helper branch and the positive-coun ... he PR body includes a negative-control real WebSocket run where the same request test fails before the fix.

Automerge notes:

PR branch already contained follow-up commit before automerge: fix(gateway): count unique omitted chat-history messages + prove diag…
PR branch already contained follow-up commit before automerge: test(gateway): prove chat.history request emits omission diagnostic

The automerge loop is complete.

Automerge progress:

2026-06-25 20:10:38 UTC review queued 414f885880f0 (queued)

2026-06-25 20:17:07 UTC review passed 414f885880f0 (structured ClawSweeper verdict: pass (sha=414f885880f061508d5149a859266d7095363...)

2026-06-25 20:17:31 UTC merged 414f885880f0 (merged by ClawSweeper automerge)

…penclaw#96788) Summary: - The PR moves Gateway `chat.history` omission accounting to a whole-pipeline reporter and adds focused helper plus real WebSocket request regression tests. - PR surface: Source +35, Tests +219. Total +254 across 4 files. - Reproducibility: yes. Current-main source shows the zero-count keep-last helper branch and the positive-coun ... he PR body includes a negative-control real WebSocket run where the same request test fails before the fix. Automerge notes: - PR branch already contained follow-up commit before automerge: fix(gateway): count unique omitted chat-history messages + prove diag… - PR branch already contained follow-up commit before automerge: test(gateway): prove chat.history request emits omission diagnostic Validation: - ClawSweeper review passed for head 414f885. - Required merge gates passed before the squash merge. Prepared head SHA: 414f885 Review: openclaw#96788 (comment) Co-authored-by: ZengWen-DT <ceng.wen@xydigit.com> Approved-by: takhoffman

openclaw-barnacle Bot added app: web-ui App: web-ui gateway Gateway runtime size: S labels Jun 25, 2026

clawsweeper Bot mentioned this pull request Jun 25, 2026

fix(gateway): report dropped message count in enforceChatHistoryFinalBudget (fixes #96783) (AI-assisted) #96792

Closed

1 task

openclaw-barnacle Bot added size: M and removed size: S labels Jun 25, 2026

clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jun 25, 2026

harjothkhara mentioned this pull request Jun 25, 2026

fix(gateway): report dropped message count in enforceChatHistoryFinalBudget placeholderCount #96803

Closed

clawsweeper Bot merged commit c68484a into openclaw:main Jun 25, 2026
143 of 151 checks passed

github-actions Bot mentioned this pull request Jun 25, 2026

📡 Upstream Digest — 2026-06-25 21:23 UTC curtismercier/openclaw-mods#1152

Open

Haderach-Ram mentioned this pull request Jun 26, 2026

Ecosystem Digest — 2026-06-26 Haderach-Ram/openclaw-radar#62

Open

clawsweeper Bot mentioned this pull request Jun 26, 2026

Umbrella: duplicate transcript, replay, and context assembly across channels #69208

Open

clawsweeper Bot mentioned this pull request Jun 28, 2026

fix(gateway): report dropped message count in enforceChatHistoryFinalBudget placeholderCount #97427

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(gateway): report omitted chat-history messages in truncation log#96788

fix(gateway): report omitted chat-history messages in truncation log#96788
clawsweeper[bot] merged 3 commits into
openclaw:mainfrom
ZengWen-DT:fix/96783-chat-history-budget-omitted-count

ZengWen-DT commented Jun 25, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

Uh oh!

ZengWen-DT commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

Uh oh!

Takhoffman commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

ZengWen-DT commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZengWen-DT commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Takhoffman commented Jun 25, 2026

Uh oh!

clawsweeper Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ZengWen-DT commented Jun 25, 2026 •

edited

Loading

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading

clawsweeper Bot commented Jun 25, 2026 •

edited

Loading