From a9cf4d9378fde68b061e9c6f3476a89a4b96fe1f Mon Sep 17 00:00:00 2001 From: DavidBabinec Date: Sat, 11 Jul 2026 15:41:36 +0200 Subject: [PATCH] fix(agent): preserve preview backgrounds across breakpoints --- docs/features/agent.md | 6 +- docs/features/canvas-iframe-per-frame.md | 16 +- server/ai/tools/site/writeTools.ts | 2 +- .../agent/agentBreakpointCapture.test.tsx | 546 ++++++++++++++++++ src/__tests__/agent/renderEvidence.test.ts | 312 +++++++++- src/__tests__/base-modules.test.ts | 77 ++- .../canvas/bodyPresentation.test.tsx | 359 ++++++++++++ .../canvas/canvasPreviewReadiness.test.ts | 64 ++ .../media/hooks/useCmsMediaAssetByPath.ts | 10 +- src/admin/pages/site/agent/executor.ts | 4 +- src/admin/pages/site/agent/index.ts | 8 +- src/admin/pages/site/agent/renderEvidence.ts | 286 +++++++-- .../site/agent/renderSnapshotAtBreakpoint.ts | 83 +++ .../pages/site/agent/renderSnapshotTool.ts | 2 + .../site/canvas/AgentSnapshotFrame.module.css | 8 + .../pages/site/canvas/AgentSnapshotFrame.tsx | 234 ++++++++ .../pages/site/canvas/CanvasComposedTree.tsx | 71 ++- src/admin/pages/site/canvas/CanvasContexts.ts | 4 + .../site/canvas/CanvasPreviewReadiness.ts | 63 ++ src/admin/pages/site/canvas/CanvasRoot.tsx | 21 +- .../pages/site/canvas/IframeFrameSurface.tsx | 72 ++- .../site/canvas/iframeBodyPresentation.ts | 117 ++++ .../pages/site/canvas/iframeBodyReset.ts | 18 +- .../pages/site/canvas/useLoopPreviewItems.ts | 16 +- .../site/hooks/useTemplatePreviewContext.ts | 35 +- .../pages/site/preview/PreviewOverlay.tsx | 2 +- .../pages/site/store/slices/canvasSlice.ts | 19 + src/modules/base/body/BodyEditor.tsx | 84 +-- src/modules/base/body/index.ts | 5 +- 29 files changed, 2371 insertions(+), 173 deletions(-) create mode 100644 src/__tests__/agent/agentBreakpointCapture.test.tsx create mode 100644 src/__tests__/canvas/bodyPresentation.test.tsx create mode 100644 src/__tests__/canvas/canvasPreviewReadiness.test.ts create mode 100644 src/admin/pages/site/agent/renderSnapshotAtBreakpoint.ts create mode 100644 src/admin/pages/site/canvas/AgentSnapshotFrame.module.css create mode 100644 src/admin/pages/site/canvas/AgentSnapshotFrame.tsx create mode 100644 src/admin/pages/site/canvas/CanvasPreviewReadiness.ts create mode 100644 src/admin/pages/site/canvas/iframeBodyPresentation.ts diff --git a/docs/features/agent.md b/docs/features/agent.md index 46d32aa68..07e25e855 100644 --- a/docs/features/agent.md +++ b/docs/features/agent.md @@ -497,7 +497,7 @@ The agent works **design-system-first**: it establishes or reuses tokens, then r | Tool | Input | Success `data` | What it does | |-------------------|-----------------------|----------------|------------------------------------------------------------------| -| `site_render_snapshot` | `{ breakpointId?, nodeId? }` | `{ breakpointId, nodeId?, label, width, capturedAt, layout, screenshot }` + optional `images[]` | Inspect the rendered canvas: always returns a layout report (viewport, per-node bounding boxes, overflow / broken-image / invisible warnings); when the active provider supports native image-bearing tool results, a PNG is attached via the tool-output **image channel**. `breakpointId` picks the frame (defaults to active); `nodeId` scopes the capture to that node's subtree — image and report cover only that section, with coordinates relative to its box, and the report carries the same `nodeId`. Omit `nodeId` for the whole page; an unknown `nodeId` returns an `aiToolError` | +| `site_render_snapshot` | `{ breakpointId?, nodeId? }` | `{ breakpointId, nodeId?, label, width, capturedAt, layout, screenshot }` + optional `images[]` | Inspect the rendered canvas: always returns a layout report (viewport, per-node bounding boxes, overflow / broken-image / invisible warnings); when the active provider supports native image-bearing tool results, a PNG is attached via the tool-output **image channel**. `breakpointId` picks any configured viewport at its exact configured width (defaults to active). Evidence always renders through a deterministic one-shot frame, so Live mode, collapsed frames, and `previewFrame:false` do not change the result or the user's visible canvas state. `nodeId` crops the rendered document to a smaller, sharper model image while preserving HTML/body/ancestor paint; the report covers the same section with coordinates relative to its box. Unknown breakpoint/node ids return `aiToolError` | ### Auto-navigation @@ -550,6 +550,10 @@ The content system prompt is markdown-native: it tells the model to exchange bod 4. **`site_read_document` is cleaned and paged before it reaches the model.** `renderAgentDocument` strips pathological strings from the broad read surface: long base64/data URLs become `data:;base64,[omitted N chars]`, and very long URLs are middle-truncated. The returned object always includes `pageInfo` with `part`, `totalParts`, `nextPart`, `ranges`, `serializedChars`, and cleanup counts. The hard budget is measured against `JSON.stringify({ html, css, pageInfo }).length`, because that is the text providers receive as the tool result. If `nextPart` is not `null`, the agent calls `site_read_document({ document, part: nextPart })` to continue. For exact node-level markup, use the `uid` with `site_get_node_html`. 5. **Stale evidence is elided.** Within one tool loop, only the **most recent** heavy result per tool name (`site_render_snapshot`, `site_read_document`, `site_get_node_html`, or anything with an image) is replayed at full fidelity; earlier ones are rewritten to a one-line breadcrumb (`"Earlier output removed… Call again…"`). Older snapshots describe page state the model has since mutated, so they carry no value. See `applyHeavyElision` in `server/ai/drivers/http/toolLoop.ts`. +Snapshot pixels come from the iframe document's authored rendering. Full-page captures rasterise `` at the exact iframe viewport width and full document height, so a narrow/transformed body cannot shrink the reported viewport or omit document gutters. Node-scoped captures rasterise that same document painting context and crop to the node rectangle, so transparent sections retain HTML/body/ancestor colors, gradients, and background images. A white browser-default fallback is composited only behind pixels the authored page leaves transparent — it is never written onto the cloned document. + +Every request uses `AgentSnapshotFrame`, an offscreen one-shot `IframeFrameSurface` at the configured width. Before it becomes capturable, a revisioned barrier waits for template preview rows, nested loop data, media metadata, web fonts, the resulting React commit, and a quiet DOM window. Readiness lives on the host iframe, never on authored ``/``, so user attribute selectors cannot distinguish Agent evidence from the published page. Lazy `` resources are left authored as-is; `html-to-image` makes its private clones eager and embeds image/background resources before `toCanvas()` resolves. The frame deliberately does not execute authored runtime scripts, and it is released after capture without changing `activeBreakpointId`, `canvasView`, or collapsed-frame state. Parallel requests are serialized so they cannot replace the single transient frame mid-capture. + --- ## System prompt diff --git a/docs/features/canvas-iframe-per-frame.md b/docs/features/canvas-iframe-per-frame.md index f5665b639..201ea3355 100644 --- a/docs/features/canvas-iframe-per-frame.md +++ b/docs/features/canvas-iframe-per-frame.md @@ -8,9 +8,10 @@ Each viewport frame runs in its own `