@bettercms-ai/convert 0.1.0 → 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,90 @@
+# @bettercms-ai/convert
+
+## 0.2.0
+
+### Minor Changes
+
+- `--componentize`: every derived SECTION becomes a component the CMS can reorder.
+
+  The repo half of the componentize lane. Given the brief and the plan from
+  `get_componentize_plan`, it takes each section's lowest common ancestor in the page's
+  tier-1 AST as the section ROOT and lifts it into
+  `src/components/bcms/<Name>.<astro|tsx>` — markup verbatim, P2's `data-bcms-field`
+  and `bcms(...)` reads included, plus `data-bcms-block={blockId}` for the editor to
+  hit-test. The page's contiguous run of roots becomes one `<Sections page="<slug>" />`.
+
+  `src/lib/bcms-sections.ts` reads `blocks` off the committed snapshots and
+  `bcms-content/components.json`; `Sections.<astro|tsx>` is the registry. **When the CMS
+  list is empty the registry renders the order the repository shipped** — the committed
+  stub, a fresh clone and every not-yet-componentised page all look like that, and
+  rendering nothing there would blank the site on the first build.
+
+  Which file a route is comes from the ROUTER's convention, not from a search for the field
+  group: `hero` is on every landing page, and picking the file by hit count let one route's
+  codemod rewrite another route's page. Everything the markup uses travels with it — the
+  page's imports, re-based, and its module-scope declarations, transitively.
+
+  Every refusal has a name — `SECTION_ROOT_AMBIGUOUS`, `SECTION_NOT_CONTIGUOUS`,
+  `SECTION_FREE_IDENTIFIERS`, `COMPONENT_CONFLICT`, `REGISTRY_CONFLICT`, `ROUTE_FILE_AMBIGUOUS`,
+  `DIALECT_UNSUPPORTED`, `HELPER_CONFLICT`, `SNAPSHOT_INVALID`, `NOT_IN_SOURCE` — and the
+  arithmetic is asserted:
+  `plan sections === extracted + inlineOnly + alreadyExtracted + pending.length`.
+
+  Re-running is safe in both directions: completion is decided per ROUTE from that page's own
+  bytes, and the registry and sections library are read back before being rebuilt, so a run that
+  componentizes one new page reuses the existing components and keeps every other page's shipped
+  order.
+  A second run over an already-componentised tree writes nothing and reports
+  `extracted: 0, pending: []`, recognising its own output by a marker comment rather than
+  by a path.
+
+  jsx and astro only. svelte, vue and html pages are `DIALECT_UNSUPPORTED` and nothing is
+  written for them.
+
+- Helper v3: `bcmsSection(page, bind, overrides)` — `overrides ?? pageGroup(bind) ??` the
+  fallbacks the template already carries. An existing v1 or v2 helper this package wrote
+  upgrades in place; anything else is still `HELPER_CONFLICT`. `overrides` wins whenever it is a
+  record, an EMPTY one included: `{}` is a section an editor cleared, and treating it as absent
+  put the page's copy back into it.
+
+- A plan spanning two frameworks gets its own components, shipped-order table and `Sections` per
+  dialect. One registry would import an `.astro` component into a `.tsx` module, and the second
+  dialect's pages would import a `<Sections>` nothing wrote.
+
+- **The brief and the plan are validated at ingress.** A page slug becomes
+  `bcms-content/<slug>.json`, so `../package` named the repository's own manifest — from inside
+  `--root`, where the CLI's boundary check cannot see it. Slugs, group keys and proposed slugs must
+  each be one safe path segment, routes must be absolute URL paths, and every consumed field is
+  typed; a failure refuses the whole run with `BRIEF_INVALID` / `PLAN_INVALID` before any write.
+  `convertSources` checks the same brief for the same reason.
+
+### Patch Changes
+
+- Route matching reads each FILE's route rather than generating candidate paths, so Next route
+  groups (`app/(marketing)/about/page.tsx`) and parallel-route segments resolve; two files serving
+  one route is `ROUTE_FILE_AMBIGUOUS` rather than a silent pick.
+
+- `importsIn` distinguishes default, named and NAMESPACE imports. `import * as Icons` was recorded
+  as a named one, so a lane re-emitting it wrote `import { Icons }` — a binding nobody exports.
+
+- Astro shorthand and spread attributes carry an expression range on the neutral tree. The compiler
+  reports `{title}` and `{...props}` at the identifier, so a reader looking for the first `{` inside
+  the attribute's span found none and treated the attribute as holding no code.
+
+- The CLI's snapshot read treats only ENOENT as "there are none". Every other failure was read as an
+  empty directory, and the run then wrote a stub over content it had been unable to open.
+
+- The astro parser remaps `@astrojs/compiler`'s **byte** offsets to UTF-16 units. The
+  compiler is a Go program behind WebAssembly and counts bytes, so one em dash in a page's
+  copy shifted every offset after it by two: elements downstream stopped matching their own
+  closing tag, lost their `inner`, and the values after that dash silently did not convert.
+
+- Every parser now reports an element's `end` on the neutral tree, so a subtree can be
+  lifted out of a page without re-deriving where it stops.
+
+## 0.1.0
+
+### Minor Changes
+
+- First release. The P2 codemod: an imported site's text-matched bindings become declared
+  ones, in one `magic-string` pass per file, with a receipt of everything it could not do.

package/README.md

@@ -116,8 +116,153 @@ reading `.slug` off one is `undefined` on every request. A `"use client"` page c
 and a signature this cannot edit is not guessed at: both are `DYNAMIC_PARAMS_UNAVAILABLE` and are
 left alone. A repeater on a dynamic route passes the same parameter to `bcmsRows`.
 
+## Componentize
+
+The second mode. P2 above makes a page's copy editable in place; `--componentize` makes its
+SECTIONS movable — each band becomes a component the CMS can reorder, duplicate and swap, while its
+copy stays exactly where P2 put it.
+
+The division of labour is the one that already works by hand: **the CMS owns STRUCTURE, the repo
+owns RENDERING.** The server decides which field groups are sections (`get_componentize_plan` →
+`componentize_sections`); this decides which bytes render them.
+
+```bash
+# after get_componentize_plan and componentize_sections
+npx @bettercms-ai/convert --componentize --brief brief.json --plan componentize.json --root .
+git diff
+```
+
+```
+add    src/components/bcms/Hero.astro
+add    src/components/bcms/Features.astro
+add    src/components/bcms/FAQ.astro
+add    src/components/bcms/Sections.astro
+add    src/lib/bcms-sections.ts
+modify src/pages/index.astro
+modify bcms-content/home.json
+add    bcms-content/components.json
+
+5 sections extracted (0 inline-only, 0 already extracted, 0 pending)
+```
+
+`--plan <file>` is required and is only read in this mode. `--dry-run`, `--receipt`, `--strict` and
+`--overwrite-helper` mean what they mean above; `--strict` exits 2 on any pending SECTION.
+
+### What is written
+
+- **`src/components/bcms/<Name>.<astro|tsx>`** — the section root's markup **verbatim**, P2's
+  `data-bcms-field` and `bcms(...)` reads included, plus `data-bcms-block={blockId}` on the root
+  (the editor hit-tests that for section chrome and reorder). Its copy comes from a new
+  `bcmsSection(page, bind, overrides)` in the committed helper: `overrides ?? pageGroup(bind) ??`
+  the fallbacks the template already carries. One component serves every page whose group has the
+  same shape, which is what the plan's `shapeHash` is for.
+- **`src/lib/bcms-sections.ts`** — `pageSections(pageSlug)` over the committed snapshots'
+  `blocks` plus `bcms-content/components.json`, returning `[{ blockId, slug, bind, overrides }]`.
+  An instance naming a component this repo does not have is **skipped, never a hole**.
+- **`src/components/bcms/Sections.<astro|tsx>`** — the registry, slug → component. **When the CMS
+  list is empty it falls back to the order the repository shipped, and that is load-bearing:** an
+  empty `blocks` is what a fresh clone, the committed stub and every not-yet-componentised page all
+  look like, so rendering nothing there would blank the site on the first build after the codemod
+  ran, with nothing in the diff to explain it. The fallback is decided on the **raw** block list:
+  a page whose blocks all name components this repo does not have has still been *arranged*, and
+  putting the shipped bands back would undo an editor's work on the next build.
+- **The page** — the contiguous run of section roots replaced by one `<Sections page="<slug>" />`,
+  with the import added.
+- **The stubs** — `blocks: []` added to each page's snapshot (keys it already has are kept), and
+  `bcms-content/components.json` = `[]`.
+
+### Which file is a route?
+
+The router's convention, not a search. Every candidate file says which route IT serves —
+`src/pages/about.astro`, `app/about/page.tsx`, `src/routes/about/+page.svelte` all answer `/about`,
+and so does `src/app/(marketing)/about/page.tsx`, because a Next route group and a parallel-route
+segment are not in the URL. Asking the SOURCE instead ("which file mentions this field group?")
+answers a different question: `hero` is on every landing page, and one route's codemod then rewrote
+another route's file. A route no file serves is `NOT_IN_SOURCE`; a route TWO files serve is
+`ROUTE_FILE_AMBIGUOUS` — the repository's own router has to pick, and a codemod that picks
+differently edits a page nobody serves.
+
+### What it will not accept
+
+The brief and the plan are validated at ingress, before a byte is written. **A page slug becomes a
+path** — `bcms-content/<slug>.json` — so `../package` names the repository's own manifest, from
+inside `--root`, where the CLI's boundary check cannot see it. Slugs, group keys and proposed slugs
+must each be one safe path segment (`[A-Za-z0-9_-]{1,64}`), routes must be absolute URL paths, and
+every field the codemod consumes is typed. A value that fails refuses the whole run with
+`BRIEF_INVALID` or `PLAN_INVALID` — a throw, not a receipt, because there is no partial run to
+report on.
+
+### What travels with the markup
+
+Every name the section's CODE uses — expression attributes, expression nodes, component tags; never
+its TEXT — is accounted for. The page's imports are carried and re-based onto the new file, and the
+page's module-scope declarations are carried **transitively**: `{plans.map(…)}` brings `const plans`,
+and `plans` brings the `const currency` it reads. A name nothing can supply — a local of the page's
+own component body, an expression this version cannot parse — is `SECTION_FREE_IDENTIFIERS`, which
+is a refusal that names the identifier rather than a component file that does not build.
+
+Scope is LEXICAL: `{rows.map((row) => …)}` binds `row` for the markup inside that expression and
+for nothing else, so a later sibling `{row}` is free and says so. Astro's shorthand and spread
+attributes (`{title}`, `{...rest}`) are read through the parser's own expression span rather than
+by looking for a `{` — the compiler reports both at the identifier, four characters in. A namespace
+import travels as `import * as X`, not as a named one.
+
+### What it refuses, and what it says
+
+| reason | |
+|---|---|
+| `SECTION_ROOT_AMBIGUOUS` | the group's fields sit under no single element, one section's root contains another's, or the group is rendered somewhere other than this route's page file. |
+| `SECTION_NOT_CONTIGUOUS` | the page's section roots share no container at all. **The components are still written**; the page is left exactly as its author wrote it. |
+| `SECTION_FREE_IDENTIFIERS` | the markup uses a name the component file cannot be given. The identifier is named in the message. |
+| `COMPONENT_CONFLICT` | a file already sits at `src/components/bcms/<Name>` and this package did not write it. Decided on our marker, never on the path — that is a name a person may reasonably have chosen first. |
+| `REGISTRY_CONFLICT` | the same, for `Sections.astro`/`Sections.tsx` or `src/lib/bcms-sections.ts`. Both are rebuilt whole on every run, so an unmarked file there would be destroyed. `--overwrite-helper` proceeds. |
+| `ROUTE_FILE_AMBIGUOUS` | two files serve one route. |
+| `DIALECT_UNSUPPORTED` | the page is svelte, vue or html. Nothing is written for it. |
+| `HELPER_CONFLICT` | `src/bcms-content.ts` is a module this package did not write. `--overwrite-helper` proceeds. |
+| `SNAPSHOT_INVALID` | `bcms-content/<page>.json` is not JSON this can add `blocks` to. Refused before a byte is written for that route. |
+| `NOT_IN_SOURCE` | no file we were given renders the group's fields, or the route names no page file. |
+
+A plan spanning TWO frameworks gets two of everything that is dialect-shaped: its own components,
+its own shipped-order table and its own `Sections`, per dialect. Only `src/lib/bcms-sections.ts` is
+shared, because it is plain TypeScript over the same snapshots.
+
+### Running it again
+
+Completion is decided per ROUTE, from that page's own bytes — it imports our registry and calls
+`<Sections page="<its slug>" />`, or it calls our components in place. Not from "some file
+somewhere carries our marker", which is true of every group that has ever been extracted and left a
+brand-new page untouched.
+
+The registry and the sections library are rebuilt WHOLE each run, so they are first read back:
+components an earlier run wrote are reused rather than rewritten, and the shipped order of every
+page already served is carried forward. Without that, componentizing one new page emitted a
+registry that had never heard of the others, and they rendered blank.
+
+A run whose roots DO share a container but have a foreign sibling between them is not refused: each
+root becomes its own `<Hero …/>` where its markup was, counted `inlineOnly`. The markup is
+componentised and the ORDER stays the repository's — looping that page would move the sibling too.
+
+The arithmetic is checked rather than hoped for:
+
+```
+plan sections === extracted + inlineOnly + alreadyExtracted + pending.length
+```
+
+`alreadyExtracted` is the fourth bucket the plan document does not name, and it is what makes the
+round trip observable: a second run over an already-componentised tree reports `extracted: 0,
+pending: []` **and** still accounts for every section. Our own output is recognised by a marker
+comment, never by its path — `src/components/bcms/` is a directory anybody may put a component in.
+
+Known leftovers, stated rather than hidden: a page whose every read moved into a component keeps
+the now-unused `bcms`/snapshot imports P2 added. They are harmless at build time and removing them
+would mean editing statements no section asked about.
+
 ## Not here yet
 
 `promoteSharedChrome`, and prop drilling through a svelte or vue component (both parse, and both
 bind their own copy; only `findPropTargets` is jsx/astro-only, so a drilled prop there is
 `PROP_TARGET_NOT_FOUND` rather than a wrong edit).
+
+Componentize: svelte and vue extraction, and Phase B of the lane — copy owned by the INSTANCE
+(`overrides`) with the page-lane bindings rewritten to the block lane. `bcmsSection` already
+prefers `overrides`, so the repo half of Phase B is a data change rather than another codemod.