Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/sandbox-common-smoke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,10 @@ jobs:
BASE_IMAGE="openclaw-sandbox-smoke-base:bookworm-slim" \
TARGET_IMAGE="openclaw-sandbox-common-smoke:bookworm-slim" \
PACKAGES="ca-certificates" \
INSTALL_PNPM=0 \
INSTALL_BUN=0 \
INSTALL_BREW=0 \
FINAL_USER=sandbox \
scripts/sandbox-common-setup.sh

u="$(timeout --kill-after=30s 2m docker run --rm openclaw-sandbox-common-smoke:bookworm-slim sh -lc 'id -un')"
test "$u" = "sandbox"
timeout --kill-after=30s 2m docker run --rm openclaw-sandbox-common-smoke:bookworm-slim sh -lc \
'set -e; test "$(id -un)" = sandbox; node --version; pnpm --version'
4 changes: 2 additions & 2 deletions docs/.generated/config-baseline.sha256
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
9246475f5771612a5fd12de38b153783c4a4cbb8b2682a5c40115916661c90f2 config-baseline.json
6349131baaa1828f2a071f42e4d7b17c8966c59b6588c8a4c1a32ea5ea4dcd5e config-baseline.core.json
f5a5855ddd7aa8c23a732f257eceaa20fd163b1d5f342c909f4aef15aa8643cf config-baseline.json
b8dffdb1a328aaf728a0707ab04d21c65f1a225a2360042e10832aa608699716 config-baseline.core.json
671979e86e4c4f59415d0a20879e838f9bbd883b3d29eeb02cb5131db8d187fe config-baseline.channel.json
94529978588d6e3776a86780b22cf9ff46a6f9957f2f178d3829403fad451ca7 config-baseline.plugin.json
62 changes: 62 additions & 0 deletions docs/concepts/usage-tracking.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,68 @@ title: "Usage tracking"
- CLI: `openclaw channels list` prints the same usage snapshot alongside provider config (use `--no-usage` to skip).
- macOS menu bar: "Usage" section under Context (only if available).

## Default usage footer mode

`/usage off|tokens|full` sets the footer for a session and is remembered for that
session. `messages.responseUsage` seeds that mode for sessions that have not
chosen one, so the footer can be on by default without typing `/usage` each time.

Set one mode for every channel, or a per-channel map with a `default` fallback:

```jsonc
{
"messages": {
"responseUsage": "tokens",
// or: { "default": "off", "discord": "full" }
},
}
```

### Three distinct session states

A session's `responseUsage` field has three representable states, each with
different semantics:

| State | Stored value | Effective mode |
| ------------------- | ------------------------------- | --------------------------------------------------------------------- |
| **Unset / inherit** | `undefined` (absent) | Falls through to `messages.responseUsage` config default, then `off`. |
| **Explicit off** | `"off"` (stored) | Always off — a non-off config default cannot re-enable the footer. |
| **Explicit on** | `"tokens"` or `"full"` (stored) | That mode, regardless of config default. |

### Precedence

Effective mode = session override → channel config entry → `default` → `off`.

An explicit `/usage off` is **persisted** as the literal value `"off"` in the
session, not the same as "unset." This means a non-off `messages.responseUsage`
default cannot turn the footer back on once the user has explicitly disabled it.

### Resetting vs. turning off

- `/usage off` — forces the footer off and persists that choice. A configured
non-off default cannot override this.
- `/usage reset` (aliases: `inherit`, `clear`, `default`) — clears the session
override. The session then **inherits** the effective config default
(`messages.responseUsage`). If no default is configured, the footer is off
(unchanged from before). Use this to "go back to default" without explicitly
turning the footer on.
- A full session reset (`/reset` or `/new`) or a session rollover **preserves**
the explicit usage-mode preference so the user's display choice survives
session rollovers. Only `/usage reset` (and its aliases) actually clears the
override.

### Toggle behavior

`/usage` with no arguments cycles: off → tokens → full → off. The starting point
for the cycle is the **effective** current mode (session override falling through
to the config default when unset), so the cycle is always consistent with what
the user sees in the footer.

### Config

With no config the prior behavior holds (footer off until `/usage`). Use
`/usage reset` to clear a session override and re-inherit the configured default.

## Custom `/usage full` footer

`/usage full` shows a built-in compact footer with model, reasoning, fast/slow,
Expand Down
2 changes: 1 addition & 1 deletion docs/gateway/sandboxing.md
Original file line number Diff line number Diff line change
Expand Up @@ -415,7 +415,7 @@ If you installed OpenClaw via `npm install -g openclaw`, use the inline `docker

</Step>
<Step title="Optional: build the common image">
For a more functional sandbox image with common tooling (for example `curl`, `jq`, `nodejs`, `python3`, `git`):
For a more functional sandbox image with common tooling (for example `curl`, `jq`, Node 24, pnpm, `python3`, and `git`):

From a source checkout:

Expand Down
2 changes: 2 additions & 0 deletions docs/reference/token-use.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ Use these in chat:
configured for the active model.
- `/usage off|tokens|full` → appends a **per-response usage footer** to every reply.
- Persists per session (stored as `responseUsage`).
- `/usage reset` (aliases: `inherit`, `clear`, `default`) — clears the session
override so the session re-inherits the configured default.
- `/usage full` shows estimated cost only when OpenClaw has usage metadata and
local pricing for the active model. Otherwise it shows tokens only.
- `/usage cost` → shows a local cost summary from OpenClaw session logs.
Expand Down
2 changes: 1 addition & 1 deletion docs/tools/slash-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ plugins.
| `/tasks` | List active/recent background tasks for the current session |
| `/context [list\|detail\|map\|json]` | Explain how context is assembled |
| `/whoami` | Show your sender id. Alias: `/id` |
| `/usage off\|tokens\|full\|cost` | Control the per-response usage footer or print a local cost summary |
| `/usage off\|tokens\|full\|reset\|cost` | Control the per-response usage footer (`reset`/`inherit`/`clear`/`default` clears the session override to re-inherit the configured default) or print a local cost summary |
</Accordion>

<Accordion title="Skills, allowlists, approvals">
Expand Down
2 changes: 1 addition & 1 deletion docs/web/tui.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ Session controls:
- `/verbose <on|full|off>`
- `/trace <on|off>`
- `/reasoning <on|off|stream>`
- `/usage <off|tokens|full>`
- `/usage <off|tokens|full|reset>` (`reset`/`inherit`/`clear`/`default` clears the session override)
- `/goal [status] | /goal start <objective> | /goal pause|resume|complete|block|clear`
- `/elevated <on|off|ask|full>` (alias: `/elev`)
- `/activation <mention|always>`
Expand Down
58 changes: 43 additions & 15 deletions extensions/duckduckgo/src/ddg-client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,21 +36,49 @@ type DuckDuckGoResult = {
};

function decodeHtmlEntities(text: string): string {
return text
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&apos;/g, "'")
.replace(/&#39;/g, "'")
.replace(/&#x27;/g, "'")
.replace(/&#x2F;/g, "/")
.replace(/&nbsp;/g, " ")
.replace(/&ndash;/g, "-")
.replace(/&mdash;/g, "--")
.replace(/&hellip;/g, "...")
.replace(/&#(\d+);/g, (_, code) => String.fromCodePoint(Number(code)))
.replace(/&#x([0-9a-f]+);/gi, (_, code) => String.fromCodePoint(Number.parseInt(code, 16)));
return text.replace(
/&(?:lt|gt|quot|apos|#39|#x27|#x2F|nbsp|ndash|mdash|hellip|amp|#\d+|#x[0-9a-f]+);/gi,
(entity) => {
const normalized = entity.toLowerCase();
if (normalized === "&lt;") {
return "<";
}
if (normalized === "&gt;") {
return ">";
}
if (normalized === "&quot;") {
return '"';
}
if (normalized === "&apos;" || normalized === "&#39;" || normalized === "&#x27;") {
return "'";
}
if (normalized === "&#x2f;") {
return "/";
}
if (normalized === "&nbsp;") {
return " ";
}
if (normalized === "&ndash;") {
return "-";
}
if (normalized === "&mdash;") {
return "--";
}
if (normalized === "&hellip;") {
return "...";
}
if (normalized === "&amp;") {
return "&";
}
if (normalized.startsWith("&#x")) {
return String.fromCodePoint(Number.parseInt(normalized.slice(3, -1), 16));
}
if (normalized.startsWith("&#")) {
return String.fromCodePoint(Number.parseInt(normalized.slice(2, -1), 10));
}
return entity;
},
);
}

function stripHtml(html: string): string {
Expand Down
11 changes: 11 additions & 0 deletions extensions/duckduckgo/src/ddg-search-provider.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,17 @@ describe("duckduckgo web search provider", () => {
);
});

it("does not double-decode escaped entities (decodes &amp; last)", () => {
// A result whose text literally shows "&lt;" arrives double-encoded as
// "&amp;lt;". Decoding &amp; first would re-decode it into "<", corrupting
// the snippet; &amp; must be decoded last.
expect(ddgClientTesting.decodeHtmlEntities("How to escape &amp;lt; in HTML")).toBe(
"How to escape &lt; in HTML",
);
expect(ddgClientTesting.decodeHtmlEntities("a&amp;#39;b")).toBe("a&#39;b");
expect(ddgClientTesting.decodeHtmlEntities("a&#x26;amp;b")).toBe("a&amp;b");
});

it("parses results when href appears before class", () => {
const html = `
<a href="https://duckduckgo.com/l/?uddg=https%3A%2F%2Fexample.com" class="result__a">
Expand Down
17 changes: 15 additions & 2 deletions extensions/exa/src/exa-web-search-provider.runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import {
wrapWebContent,
writeCachedSearchPayload,
} from "openclaw/plugin-sdk/provider-web-search";
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
import {
normalizeOptionalLowercaseString,
normalizeOptionalString,
Expand All @@ -30,6 +31,10 @@ const EXA_SEARCH_TYPES = ["auto", "neural", "fast", "deep", "deep-reasoning", "i
const EXA_FRESHNESS_VALUES = ["day", "week", "month", "year"] as const;
const EXA_MAX_SEARCH_COUNT = 100;
const EXA_ERROR_BODY_LIMIT_BYTES = 8 * 1024;
// Exa search responses are untrusted external bodies. Cap the success JSON the
// same way other bundled providers do (16 MiB) so a misbehaving or hostile
// endpoint cannot stream an unbounded body into memory before we parse it.
const EXA_SEARCH_JSON_MAX_BYTES = 16 * 1024 * 1024;

type ExaConfig = {
apiKey?: string;
Expand Down Expand Up @@ -70,9 +75,17 @@ type ExaSearchResponse = {
results?: unknown;
};

async function readExaSearchResults(response: Response): Promise<ExaSearchResult[]> {
async function readExaSearchResults(
response: Response,
opts?: { maxBytes?: number },
): Promise<ExaSearchResult[]> {
const maxBytes = opts?.maxBytes ?? EXA_SEARCH_JSON_MAX_BYTES;
const bytes = await readResponseWithLimit(response, maxBytes, {
onOverflow: ({ maxBytes: maxBytesLocal }) =>
new Error(`Exa API response exceeds ${maxBytesLocal} bytes`),
});
try {
return normalizeExaResults(await response.json());
return normalizeExaResults(JSON.parse(new TextDecoder().decode(bytes)));
} catch (cause) {
throw new Error("Exa API returned malformed JSON", { cause });
}
Expand Down
48 changes: 48 additions & 0 deletions extensions/exa/src/exa-web-search-provider.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,33 @@ function cancelTrackedResponse(
};
}

function streamingJsonResponse(params: { chunkCount: number; chunkSize: number }): {
response: Response;
getReadCount: () => number;
} {
// Streaming fixture proves an oversized success body stops being read before
// the whole payload is buffered into memory.
let reads = 0;
const encoder = new TextEncoder();
const stream = new ReadableStream<Uint8Array>({
pull(controller) {
if (reads >= params.chunkCount) {
controller.close();
return;
}
reads += 1;
controller.enqueue(encoder.encode("a".repeat(params.chunkSize)));
},
});
return {
response: new Response(stream, {
status: 200,
headers: { "content-type": "application/json" },
}),
getReadCount: () => reads,
};
}

describe("exa web search provider", () => {
it("exposes the expected metadata and selection wiring", () => {
const provider = createExaWebSearchProvider();
Expand Down Expand Up @@ -265,6 +292,27 @@ describe("exa web search provider", () => {
);
});

it("parses well-formed Exa search JSON under the byte cap", async () => {
const response = new Response(
JSON.stringify({ results: [{ url: "https://example.com", title: "Example" }] }),
{ status: 200, headers: { "content-type": "application/json" } },
);

await expect(testing.readExaSearchResults(response)).resolves.toEqual([
{ url: "https://example.com", title: "Example" },
]);
});

it("caps oversized Exa search JSON instead of buffering the whole body", async () => {
const streamed = streamingJsonResponse({ chunkCount: 64, chunkSize: 1024 });

await expect(
testing.readExaSearchResults(streamed.response, { maxBytes: 4096 }),
).rejects.toThrow(/Exa API response exceeds 4096 bytes/);

expect(streamed.getReadCount()).toBeLessThan(64);
});

it("bounds Exa API error bodies without using response.text()", async () => {
const tracked = cancelTrackedResponse(`${"exa upstream unavailable ".repeat(1024)}tail`, {
status: 503,
Expand Down
16 changes: 0 additions & 16 deletions extensions/lmstudio/src/models.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,21 +59,6 @@ describe("lmstudio-models", () => {
}
return JSON.parse(init.body) as unknown;
};
// The model fetch/load helpers now read bodies through the shared byte-capped
// reader, so success-path mocks must be real Response objects with a body
// stream rather than bare `{ ok, json }` placeholders.
const jsonResponse = (payload: unknown, init?: ResponseInit): Response =>
new Response(JSON.stringify(payload), {
status: 200,
headers: { "content-type": "application/json" },
...init,
});
const malformedJsonResponse = (init?: ResponseInit): Response =>
new Response("{ this is not valid json", {
status: 200,
headers: { "content-type": "application/json" },
...init,
});
const cancelTrackedResponse = (
text: string,
init: ResponseInit,
Expand Down Expand Up @@ -104,7 +89,6 @@ describe("lmstudio-models", () => {
}) =>
vi.fn(async (url: string | URL, _init?: RequestInit) => {
const key = params?.key ?? "qwen3-8b-instruct";
void init;
if (String(url).endsWith("/api/v1/models")) {
return jsonResponse({
models: [
Expand Down
24 changes: 24 additions & 0 deletions extensions/memory-core/src/concept-vocabulary.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,30 @@ describe("concept vocabulary", () => {
expect(tags).not.toContain("2026-04-04.md");
});

it("preserves short protected-glossary terms past the latin minimum-length gate", () => {
const tags = deriveConceptTags({
path: "memory/2026-04-04.md",
snippet: "Store the session in kv and back up to s3 nightly.",
});

// "kv" and "s3" are 2-char latin glossary entries that the generic min-length-3 gate would drop.
expect(tags).toContain("kv");
expect(tags).toContain("s3");
});

it("does not surface short glossary terms that only appear inside longer words", () => {
const tags = deriveConceptTags({
path: "memory/2026-04-04.md",
snippet: "Played the mkv recording and tuned the css3 layout.",
});

// "kv"/"s3" are substrings of "mkv"/"css3"; whole-word matching must not emit them as tags.
expect(tags).not.toContain("kv");
expect(tags).not.toContain("s3");
expect(tags).toContain("mkv");
expect(tags).toContain("css3");
});

it("extracts protected and segmented CJK concept tags", () => {
const tags = deriveConceptTags({
path: "memory/2026-04-04.md",
Expand Down
Loading
Loading