fix(agents): classify provider upstream_error as fallbackable transient failure#95523
fix(agents): classify provider upstream_error as fallbackable transient failure#95523Bartok9 wants to merge 1 commit into
Conversation
|
Thanks for the context here. I swept through the related work, and this is now duplicate or superseded. Close as superseded: the branch is a useful matcher-only slice, but it does not cover the embedded provider-payload fallback boundary, while #95542 is an open, mergeable, proof-positive canonical fix for the same linked fallback bug. Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Canonical path: Use #95542 as the canonical landing path for the linked fallback bug, then close the narrower matcher-only siblings as superseded. So I’m closing this here and keeping the remaining discussion on #95542. Review detailsBest possible solution: Use #95542 as the canonical landing path for the linked fallback bug, then close the narrower matcher-only siblings as superseded. Do we have a high-confidence way to reproduce the issue? Yes, at source level: current main routes embedded error payload text through an auth/auth_permanent/billing-only helper, and a null classification is treated as a completed model attempt. I did not run a live provider outage repro in this read-only review. Is this the best way to solve the issue? No: adding shared matcher strings is useful, but this branch is not the best fix because the linked runtime path also needs the embedded payload classifier to accept the structured upstream/server provider failure. Security review: Security review cleared: The diff only changes internal agent classifier source/tests and does not add dependencies, workflows, scripts, secrets handling, or code execution paths. AGENTS.md: found and applied where relevant. What I checked:
Likely related people:
Codex review notes: model internal, reasoning high; reviewed against e6f41a4df005. |
|
Complementary observations on top of ClawSweeper's review: Same shape as #95521 — adds upstream_error / upstream_request_failed patterns to
|
…nt failure
When a provider returns an error payload shaped like
{"type":"upstream_error","message":"Upstream request failed"} the failover
classifier left it unclassified, so the turn ended with "LLM request failed"
and no configured fallback model was attempted.
The serverError pattern list already covered "upstream error" and
"upstream connect error" but not the "upstream_error" type token or the
"Upstream request failed" message phrasing, so these provider-side transient
failures fell through to the unclassified path that skips model fallback.
Add both patterns to serverError so they classify as a transient server_error
(mapped to the timeout failover reason), letting the fallback chain rotate to
the next candidate. OpenClaw's own generic "LLM request failed" text is
unaffected and stays unclassified to avoid fallback loops on internal failures.
Closes openclaw#95519
23f1231 to
c57dccd
Compare
|
ClawSweeper applied the proposed close for this PR.
|
|
Got it — thanks ClawSweeper. Agreed this is superseded; the fallback-classification intent is carried elsewhere. No objection to the close. 👍 |
Summary
upstream_error/ "Upstream request failed" responses as a transient server failure so the configured fallback chain is attempted.upstream_errornow fails over to the next configured model instead of ending the turn withLLM request failed.Motivation
Closes #95519.
When the primary provider returns an error payload shaped like:
{ "error": { "message": "Upstream request failed", "type": "upstream_error", "param": "", "code": null } }the failover classifier returned
null(unclassified). Unclassified failures do not enter the model-fallback path, so the run ended withstopReason: "error"and the user-facing textLLM request failed, with nomodel.fallback_steprecorded — even whenagents.defaults.model.fallbackswas configured.The
serverErrormatcher list already covered"upstream error"and"upstream connect error", but not theupstream_errortype token nor the"Upstream request failed"message phrasing emitted by this provider. So this transient upstream failure slipped through.The fix adds
"upstream_error"and"upstream request failed"to theserverErrorpatterns. These classify as a transientserver_error(mapped to thetimeoutfailover reason), which is the existing fallbackable transient path.OpenClaw's own generic assistant text
"LLM request failed"(GENERIC_ASSISTANT_ERROR_TEXT) is deliberately left unclassified to avoid a fallback loop on internal/non-provider failures.Verification
node scripts/run-vitest.mjs run src/agents/embedded-agent-helpers/failover-matches.test.ts— 30 passed (5 new)node scripts/run-vitest.mjs run src/agents/failover-error.test.ts src/agents/embedded-agent-helpers.isbillingerrormessage.test.ts— 234 passednode scripts/run-vitest.mjs run src/agents/model-fallback.test.ts src/agents/embedded-agent-helpers/provider-error-patterns.test.ts— 121 passed"LLM request failed"text classification (staysnull).Real behavior proof
{type:"upstream_error", message:"Upstream request failed"}payload from the issue was classifiednull→ no fallback. After the fix it classifies astimeout(fallbackable transient).payload= full issue JSON;bare= "Upstream request failed";generic= OpenClaw's own "LLM request failed", correctly left unclassified.)null(verified onupstream/mainbefore editing).src/agents/embedded-agent-helpers/failover-matches.test.ts→describe("upstream provider transient errors (#95519)"). It asserts the new classifications AND that the generic text staysnull; it fails without the source change.model-fallback.test.tsreason→candidate-rotation tests, which still pass); this PR is scoped to the classification gap that gates entry into that path.What Problem This Solves
When the primary provider returns
{type:"upstream_error", message:"Upstream request failed"}, the failover classifier returnednull(unclassified), so the run ended withLLM request failedand never tried the configured fallback models — even withagents.defaults.model.fallbacksset. This PR classifies that payload as a transientserver_errorso the existing fallback path is entered.Evidence
node scripts/run-vitest.mjs run src/agents/embedded-agent-helpers/failover-matches.test.ts→ 30 passed (5 new, including the#95519regression describe block). Re-verified on a clean rebase onto currentupstream/main: 30/30 pass.npx oxlint src/agents/embedded-agent-helpers/failover-matches.ts failover-matches.test.ts→ 0 errors / 0 warnings.classifyFailoverReason()on the exact issue payload returns"timeout"(fallbackable) AFTER the fix; returnednullBEFORE. OpenClaw's own generic"LLM request failed"text correctly staysnull(no fallback loop on internal errors).model-fallback.test.ts.