fix(agents): preserve embedded OpenAI completions usage#96523
Conversation
|
Codex review: needs maintainer review before merge. Reviewed July 2, 2026, 2:54 AM ET / 06:54 UTC. Summary PR surface: Source +138, Tests +244. Total +382 across 9 files. Reproducibility: yes. at source level: current main lets a present zeroed final assistant snapshot mask nonzero streamed or current-attempt usage in embedded run metadata and message-end persistence. I did not run a live current-main custom provider in this read-only review. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Land the narrowed usage-preservation fix if maintainers accept the session-state behavior, and keep #96463 open for the broader provider request-policy decision. Do we have a high-confidence way to reproduce the issue? Yes at source level: current main lets a present zeroed final assistant snapshot mask nonzero streamed or current-attempt usage in embedded run metadata and message-end persistence. I did not run a live current-main custom provider in this read-only review. Is this the best way to solve the issue? Yes for the narrowed fix: the PR repairs the local owner paths that consume streamed usage, build final metadata, and persist the assistant snapshot. The broader default request-policy question belongs on #96463, not in this branch. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against be5906e19dd7. Label changesLabel justifications:
Evidence reviewedPR surface: Source +138, Tests +244. Total +382 across 9 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
|
@clawsweeper re-review Addressed the latest review on head |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Addressed the session-owned message finding on head Fresh local proof after the fix:
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Addressed the latest type/lint findings on head
Fresh local proof:
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Follow-up after the previous review on head |
4b520d6 to
279638e
Compare
Co-authored-by: ben.li <li.yang6@xydigit.com>
279638e to
a66a69a
Compare
|
Maintainer repair is land-ready on exact head What changed:
Proof:
No provider live test was required because this repair does not change provider request shaping, defaults, config, or endpoint behavior. It only preserves usage already delivered by the stream. Issue #96463 remains open for the separate product decision about whether generic custom OpenAI-compatible providers should request |
|
Merged via squash.
|
Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
What Problem This Solves
Refs #96463. This PR fixes the embedded runner usage-accounting path; any broader custom-provider request-policy decision can remain tracked on the issue.
When
openclaw agent --local --jsonuses an OpenAI-compatible provider through the embedded runner, the provider stream can report nonzero token usage while the final assistant snapshot still carries a zeroed usage object. The run metadata then treated that zero object as authoritative, soagentMeta.usage,agentMeta.lastCallUsage, session listings, and related JSON output stayed at0even though the stream had usage.Why This Change Was Made
The OpenAI completions provider already parses streaming
usagechunks. The embedded run aggregation path now treats a zeroed assistant usage snapshot as missing for run-level accounting, keeps latest-call usage separate from accumulated attempt totals, and preserves the streamed nonzero usage on the final assistant snapshot when the provider's final message snapshot is zeroed.The final metadata helper follows the same rule for
lastCallUsage, so zero snapshots no longer mask known nonzero usage in either returned JSON metadata or the persisted session assistant artifact.User Impact
Custom OpenAI-compatible providers using the embedded runner now report nonzero token usage in
--jsonoutput and persisted assistant session records when the provider stream includes usage, instead of always showing zeros.Evidence
src/agents/embedded-agent-subscribe.handlers.messages.test.tsasserting that a zeroed final assistant message object is updated in place tousage.totalTokens: 12beforenoteLastAssistant/recordAssistantUsageconsume it.node scripts/run-vitest.mjs src/agents/embedded-agent-subscribe.handlers.messages.test.ts src/agents/embedded-agent-runner/run/helpers.test.ts src/agents/embedded-agent-runner/usage-reporting.test.ts src/llm/providers/openai-completions.test.tsnode_modules\.bin\oxlint.CMD src\agents\embedded-agent-subscribe.handlers.messages.ts src\agents\embedded-agent-subscribe.handlers.messages.test.tsnode scripts/run-tsgo.mjs -p tsconfig.core.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/core.tsbuildinfonode scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.core.test.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/core-test.tsbuildinfogit diff --checkLoopback OpenAI-compatible provider proof, redacted:
Observed
--jsonoutput excerpt:{ "payloads": [{ "text": "loopback usage ok", "mediaUrl": null }], "meta": { "agentMeta": { "provider": "loopback", "model": "loopback-model", "usage": { "input": 7, "output": 5, "total": 12 }, "lastCallUsage": { "input": 7, "output": 5, "cacheRead": 0, "cacheWrite": 0, "total": 12 }, "promptTokens": 7 }, "finalAssistantVisibleText": "loopback usage ok" } }Observed session JSONL assistant artifact excerpt:
{ "type": "message", "message": { "role": "assistant", "api": "openai-completions", "provider": "loopback", "model": "loopback-model", "usage": { "input": 7, "output": 5, "cacheRead": 0, "cacheWrite": 0, "totalTokens": 12, "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0, "total": 0 } } } }Loopback server log confirmed the embedded runner called
POST /v1/chat/completionson the custom OpenAI-compatible provider and received an SSEusagechunk withprompt_tokens: 7,completion_tokens: 5, andtotal_tokens: 12.Autoreview was attempted with
python .agents\skills\autoreview\scripts\autoreview --mode branch --base origin/main. One run hit the Codex account usage limit, and later retries timed out after 10 minutes without returning findings.