Skip to content

Commit e579b4d

Browse files
DavidBabinecDavidBabinec
authored
fix(import): stream site transfer bundles and unify review flow (CoreBunch#77)
* fix(data): stream site transfer zip archives * fix(import): unify site bundle review flow * fix(import): route site bundles through shared wizard * fix(import): stage archive media before data import
1 parent 6643353 commit e579b4d

30 files changed

Lines changed: 4213 additions & 826 deletions

‎docs/features/site-import.md‎

Lines changed: 18 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,21 @@
11
# Site Import
22

3-
`src/admin/modals/SiteImport` is the canonical import surface. It routes static-site bundles (HTML pages, CSS files, images, fonts, JS) through `src/core/siteImport`, and routes CMS-exported `SiteBundle` JSON files through the CMS transfer endpoints for full import/export parity.
3+
`src/admin/modals/SiteImport` is the canonical import surface. It routes static-site bundles (HTML pages, CSS files, images, fonts, JS) through `src/core/siteImport`, and routes CMS-exported site-transfer ZIP bundles through the CMS transfer endpoints for full import/export parity.
44

5-
The static-site pipeline has two parts: a pure analysis function (`buildImportPlan`) that produces an `ImportPlan` preview, and an async commit function (`commitImportPlan`) that uploads assets and writes to the store. CMS bundle imports keep their native semantics: validate the `SiteBundle`, preview against `/admin/api/cms/import/preview`, then apply through `/admin/api/cms/import`.
5+
The static-site pipeline has two parts: a pure analysis function (`buildImportPlan`) that produces an `ImportPlan` preview, and an async commit function (`commitImportPlan`) that uploads assets and writes to the store. CMS bundle imports keep their native semantics: validate the `SiteBundle`, preview against `/admin/api/cms/import/preview`, resolve any row slug conflicts in the shared Conflicts step, then apply through `/admin/api/cms/import` or `/admin/api/cms/import/archive`. The modal uses the same Review category navigator and Import progress surface for CMS bundles, so tables, media, folders, redirects, conflict resolution, and completion all live in the same picker pattern as HTML/CSS/media imports.
66

77
---
88

99
## TL;DR
1010

11-
- Entry: global admin-shell modal, opened from Spotlight or workspace actions. Drop files, a folder, a `.zip`, or a CMS-exported `.json` bundle. Static files use the four-stage modal (Drop → Review → Conflicts → Import, with completion shown inside the Import stage). CMS bundles use Drop → Review bundle → Import.
11+
- Entry: global admin-shell modal, opened from Spotlight or workspace actions. Drop files, a folder, a static `.zip`, or a CMS-exported `.zip` bundle. Static files and CMS bundles both use the four-stage modal (Drop → Review → Conflicts → Import, with completion shown inside the Import stage); Conflicts is skipped when there is nothing to resolve.
1212
- `buildImportPlan({ fileMap, currentSite, options })` — pure, synchronous — produces an `ImportPlan` with pages, style rules, kept stylesheet files, media, color tokens, custom fonts, Google font install requests, font tokens, and scripts.
1313
- **Per-stylesheet import modes:** each top-level linked stylesheet either converts to editable style rules (default) or imports verbatim as a page-scoped `SiteFile` stylesheet (`options.stylesheetModes`, picked in the Review step). There are no generated scope classes — page isolation comes from the kept file's runtime scope.
1414
- `commitImportPlan(plan, adapter)` — uploads assets, then wraps all store writes in a single `adapter.commit` call → one Cmd+Z reverts the whole import.
1515
- Static imports load the current CMS draft into the editor store on demand when launched outside `/admin/site`; if no draft exists, the modal creates an empty site before analysis.
1616
- Conflict resolution: rename with a numeric suffix (default), overwrite, skip, or custom-rename — per page slug, per class name, per design token (colour / font CSS variable), and per divergent cross-stylesheet class definition, with category-level bulk actions. Token renames rewrite `var(--x)` references so imports stay faithful.
1717
- What imports: pages, linked CSS plus unconditional local CSS `@import` graphs, `kind:'class'` and `kind:'ambient'` style rules, stylesheets kept as page-scoped files, `@keyframes`, uploadable media/font files, root CSS color tokens, root CSS font tokens, `@font-face` families, known external font stylesheet imports, safe extra HTML attributes on base modules, body-level classes/attributes/style metadata, bare DOM text nodes in mixed content, and executable HTML scripts as page-scoped runtime scripts.
18-
- CMS bundle import preserves exported tables, rows, optional site shell, and embedded media using the same merge strategies as site transfer (`replace`, `merge-add`, `merge-overwrite`).
18+
- CMS bundle import preserves selected exported tables, rows, optional site shell, media, folders, and redirects using the same merge strategies as site transfer (`replace`, `merge-add`, `merge-overwrite`).
1919
- HTML forms import through the shared HTML importer as first-class form primitives (`base.form`, controls, labels, submit buttons), not as custom containers.
2020
- What cannot be modeled: `@layer`, conditional local CSS `@import`, and arbitrary external `@import` — surfaced as warnings when the CSS engine exposes them, never silently dropped.
2121
- Headless: `src/core/siteImport/` carries no admin, React, or server imports (gated by `siteImport-headless.test.ts`).
@@ -51,16 +51,19 @@ src/core/siteImport/
5151
src/admin/modals/SiteImport/
5252
├── index.ts
5353
├── SiteImportModal.tsx — canonical import wizard shell + CMS bundle router
54+
├── SiteImportFooter.tsx — shared wizard footer actions
5455
├── SiteImportModal.module.css
5556
├── steps/
5657
│ ├── DropStep.tsx — full-modal drop zone (files, folder, .zip)
57-
│ ├── AnalyzeStep.tsx — category navigator (left) + detail pane (right)
58-
│ ├── CmsBundleReviewStep.tsx — CMS bundle diff + merge strategy review
58+
│ ├── AnalyzeStep.tsx — category navigator (left) + detail pane (right) for static imports
59+
│ ├── CmsBundleAnalyzeStep.tsx — category navigator (left) + detail pane (right) for CMS bundles
5960
│ ├── ConflictsStep.tsx — page-slug + class-name + design-token conflict resolution rows
61+
│ ├── CmsBundleConflictsStep.tsx — row slug conflict resolution for CMS bundles
6062
│ └── ImportStep.tsx — determinate progress surface + complete/failed states
6163
└── shared/
6264
├── createSiteImportAdapter.ts — wires adapter to editor store + media API
6365
├── useCmsBundleImport.ts — CMS bundle parse/preview/import flow
66+
├── cmsBundleFlow.ts — CMS bundle selection counts + conflict-resolution helpers
6467
├── ConflictRow.tsx — single slug / class-name / token-variable conflict row with resolution picker
6568
├── ImportStepper.tsx — shared four-stage progress rail (Review + Import)
6669
└── importProgress.ts — RunProgress model used by ImportStep
@@ -71,11 +74,12 @@ src/admin/modals/SiteImport/
7174
## Data flow
7275

7376
```text
74-
User drops files / folder / .zip / CMS bundle JSON
77+
User drops files / folder / static .zip / CMS bundle .zip
7578
│
76-
├─ valid SiteBundle JSON → previewSiteBundle → CmsBundleReviewStep
79+
├─ valid CMS bundle → previewSiteBundle → Review navigator
7780
│ │
78-
│ └─ importSiteBundle(strategy)
81+
│ ├─ row slug conflicts? → Conflicts step
82+
│ └─ ImportStep → importSiteBundle/importSiteBundleArchive(strategy, selection)
7983
│
8084
▼
8185
ingestInput(input)
@@ -281,9 +285,9 @@ The conflict wizard renders bulk controls in each of the three conflict categori
281285

282286
The modal is mounted once at the authenticated admin shell (`AuthenticatedAdmin.tsx`) behind `useAdminUi().siteImportOpen`. It is not owned by the Site editor route. The Site editor, Data workspace, and Spotlight command all open the same shell-level modal state, so importing works from any admin workspace with the required capability.
283287

284-
**Drop** — full-modal drop zone. Accepts loose files, a folder, a `.zip`, or a CMS-exported `.json` bundle. A single JSON file is first checked with `parseSiteBundle`; valid bundles route to the CMS bundle review path. Everything else goes through `ingestInput`, which normalizes static import input shapes to `FileMap`. Static import analysis needs a `currentSite`; when the modal opens outside the Site editor, it loads the CMS draft through `cmsAdapter.loadSite('default')` before calling `buildImportPlan`. Size guards: 1 GB aggregate, 10 k files, 5 GB uncompressed (zip-bomb guard).
288+
**Drop** — full-modal drop zone. Accepts loose files, a folder, a static `.zip`, or a CMS-exported `.zip` bundle. A single ZIP is classified before analysis: an Instatic transfer archive has `.instatic/site-bundle.json` as its first stored entry and routes to the CMS bundle review path; any other ZIP is treated as a static-site import and normalized through `ingestInput` to `FileMap`. JSON `SiteBundle` files are still accepted by the internal parser for tests and direct API work, but the exported user-facing artifact is ZIP. Static import analysis needs a `currentSite`; when the modal opens outside the Site editor, it loads the CMS draft through `cmsAdapter.loadSite('default')` before calling `buildImportPlan`. Size guards: 1 GB aggregate, 10 k files, 5 GB uncompressed (zip-bomb guard).
285289

286-
**CMS bundle review** — shown when the dropped file validates as `SiteBundle`. The wizard calls `previewSiteBundle` to render a diff against the local site, then lets the user pick `replace`, `merge-add`, or `merge-overwrite`. Commit calls `importSiteBundle`; on success the modal closes and the caller can refresh workspace data.
290+
**CMS bundle review** — shown when the dropped archive validates as an Instatic transfer archive. The wizard reads only the manifest for preview, calls `previewSiteBundle` to render a diff against the local site, then lets the user pick `replace`, `merge-add`, or `merge-overwrite` and include/exclude the shell, rows, media, folders, and redirects. Continue routes through the same Conflicts and Import steps as static import. Commit calls `importSiteBundleArchive` with the original ZIP `File`, so media assets stream through `/admin/api/cms/import/archive` instead of expanding into browser memory.
287291

288292
**Analyze (Review)** — category navigator. Left column: one nav entry per import category with its count and include-toggle, plus "Add more files" (files can be added at any point — re-ingests and rebuilds the plan) and a "Can't import" entry for skipped items. Right pane: detail view per category:
289293
- **Pages** — checkbox + inline slug editor per page.
@@ -294,9 +298,9 @@ The modal is mounted once at the authenticated admin shell (`AuthenticatedAdmin.
294298
- **Scripts** — Switch per JS file.
295299
- **Can't import** — list of `unusedCss` + `droppedAtRules` with reasons.
296300

297-
**Conflicts** — shown only when conflicts exist. Page-slug rows and class-name rows each use a segmented control: `Rename | Skip | Overwrite | Custom`.
301+
**Conflicts** — shown only when conflicts exist. Static imports resolve page slugs, class names, design tokens, and cross-sheet classes. CMS bundle imports resolve row slug collisions reported by `/admin/api/cms/import/preview`; the default action renames the incoming row to the server-provided next slug, but users can skip or custom-rename each row before import. Those decisions travel as `selection.rowSlugOverrides` to the archive endpoint while the browser still uploads the original ZIP unchanged.
298302

299-
**Import** (`ImportStep`) — a calm, determinate progress surface (no terminal log). A headline activity (phase verb + N of M), a determinate bar with a travelling shimmer, a one-line current-item ticker, and a per-category breakdown mirroring the Review navigator (pending ring → spinner → mint check, with a tint-washed progress fill). Everything is driven by real pipeline state: media (asset uploads) is the only incremental phase, so it dominates the bar; the other categories land together at the atomic commit. The commit phase is uncancellable; the upload phase is cancellable (orphaned uploads are harmless).
303+
**Import** (`ImportStep`) — a calm, determinate progress surface (no terminal log). A headline activity (phase verb + N of M), a determinate bar with a travelling shimmer, a one-line current-item ticker, and a per-category breakdown mirroring the Review navigator (pending ring → spinner → mint check, with a tint-washed progress fill). Static imports are driven by real pipeline state: media (asset uploads) is the only incremental phase, so it dominates the bar; the other categories land together at the atomic commit. CMS bundle imports show the same surface with CMS-native categories (site, rows, media, folders, redirects) while the server streams the archive. The commit phase is uncancellable; the upload phase is cancellable (orphaned uploads are harmless).
300304

301305
On success the same step switches to its **complete** state — a success mark, an "Imported into <site>" summary, and every category shown as done. Footer actions: **View import log** (reveals per-category counts + warnings) and **Open site →** (jumps to the first imported page). On failure it shows an inline error surface, and the failure is also surfaced via toast.
302306

@@ -332,7 +336,7 @@ On success the same step switches to its **complete** state — a success mark,
332336
## Related
333337

334338
- [docs/features/html-import.md](html-import.md) — `@core/htmlImport` is used by `htmlPagePlan.ts` to parse each HTML file's body into a `PageNode` fragment
335-
- [docs/features/site-transfer.md](site-transfer.md) — CMS bundle export/import format and server endpoints used by the JSON branch of this modal
339+
- [docs/features/site-transfer.md](site-transfer.md) — CMS bundle export/import archive format and server endpoints used by the CMS branch of this modal
336340
- [docs/reference/page-tree.md](../reference/page-tree.md) — `NodeTree<PageNode>`, `ImportFragment` shape
337341
- [docs/reference/typebox-patterns.md](../reference/typebox-patterns.md) — boundary validation
338342
- Source-of-truth files:

0 commit comments

Comments
 (0)