@bettercms-ai/convert 0.11.0 → 0.12.1

package/CHANGELOG.md

@@ -1,5 +1,57 @@
 # @bettercms-ai/convert
 
+## 0.12.1
+
+### Patch Changes
+
+- 6d826cf: The trunk-zero rule is per-country data, not a regex
+
+  `bcmsPhone` stripped a leading zero from every national number with a bare
+  `.replace(/^0/, "")`. That is only correct for countries that actually use a trunk
+  prefix. Where the leading digit is part of the number, the submitted value was
+  silently corrupted — one digit short, and wrong in a way nobody notices until a
+  customer cannot be called back.
+
+  The country list now carries a `trunk` flag derived from libphonenumber's own example
+  numbers, rendered as `data-trunk` on each option, and the zero is dropped only when
+  that country says it is a trunk prefix.
+
+  The inline script exists once per vanilla lane — the Astro component, the converter and
+  the server renderer — because each ships as inline script text on someone else's page
+  and cannot import at runtime. All three carry the identical change, which
+  `src/__tests__/content/form-script-pin.test.ts` pins byte for byte.
+
+## 0.12.0
+
+### Minor Changes
+
+- a243594: Every pending path leaves with an exact fix, and a locator resolves each path sharing a literal.
+
+  `paths.pending[]` rows now carry `fix: { file, line, col?, snippet, action, why? }` — one action
+  out of a closed set (`wrap-span`, `declare-attr`, `bind-expression`, `declare-richtext`,
+  `bind-data`, `manual`), so "rerun until nothing is pending" is a loop an agent can finish instead
+  of a list of category names. A literal one element renders on several routes is named as shared
+  chrome rather than sent for an attribute that could only ever be one route's.
+
+  Positional resolution is now per PATH rather than per literal: one path with no locator used to
+  refuse the whole literal, so a page whose rows carried locators converted none of them.
+
+  Two paths whose locators cannot be told apart now place NEITHER. Resolution counts the targets
+  wanting each element before it assigns any, so it no longer depends on brief order: `plan` runs
+  per file, and claiming greedily let the first of two indistinguishable paths win in every file —
+  writing one path's `data-bcms-field` onto the other's element and reporting it bound.
+
+  An action that edits an element (`wrap-span`, `declare-attr`, `declare-richtext`) is never issued
+  at a position inside a frontmatter array or a `<script>`; there is no element there, so the fix is
+  `bind-data` on the row the copy actually lives in. A fix for a dynamic route names the template
+  that generates it instead of an empty file. Files that talk ABOUT the site — `*.test.*`,
+  `*.spec.*`, `__tests__/`, `__mocks__/`, root `*.config.*` — are no longer read, so a test
+  asserting on the hero line stops holding an already-declared path pending.
+
+  Fix positions are measured against the bytes the run WRITES, not the ones it read — the same run
+  inserts a helper import above most pending lines, so a receipt minted from the originals named a
+  line one or more short of the element on 616 of 948 Shatter rows.
+
 ## 0.11.0
 
 ### Minor Changes

package/README.md

@@ -184,6 +184,60 @@ the receipt's arithmetic is checked, not hoped for:
 paths.declared === paths.rewritten + paths.alreadyDeclared + paths.pending.length
 ```
 
+### What pending means — every row carries the edit, not the diagnosis
+
+A reason is a diagnosis and nobody can act on one. So every `paths.pending[]` row also carries a
+`fix`: the file, the 1-based line and column, the bytes that are there now, and ONE action out of
+a closed set. That is what makes "run it again until nothing is pending" a loop an agent can
+actually finish — and what `get_next_steps` prints, one line per path.
+
+```jsonc
+{ "route": "/about", "scope": "page", "path": "p-founded-in-2019-in", "kind": "text",
+  "reason": "DIALECT_UNSUPPORTED",
+  "fix": { "file": "src/data/facts.js", "line": 3, "col": 25,
+           "snippet": "export const founded = \"Founded in 2019 in Bristol.\";",
+           "action": "bind-data",
+           "why": "src/data/facts.js holds this copy as data — bind the element that renders it." } }
+```
+
+| action | what to type |
+|---|---|
+| `wrap-span` | the value shares its element with something else, or is rendered in a position no attribute reaches — give it a `<span data-bcms-field="…">` of its own. |
+| `declare-attr` | the element is there and nothing said which path it is — add `data-bcms-field` to it at `file:line`. |
+| `declare-richtext` | the container is fed markup (`set:html`, `dangerouslySetInnerHTML`) — bind it with the richtext helper. |
+| `bind-expression` | the value is computed in the markup (a ternary, a template literal) — read the path there instead. |
+| `bind-data` | the copy lives in a data module, a frontmatter array or a content-collection entry — bind THAT field, and the element that renders it. |
+| `manual` | none of the above is true; `why` says what a human has to decide. |
+
+Reason → action, and the two places the answer is not the obvious one:
+
+| reason | action |
+|---|---|
+| `AMBIGUOUS_LITERAL`, one route | `declare-attr` — `why` names every candidate file. |
+| `AMBIGUOUS_LITERAL`, **several routes** | `manual` — one element cannot carry N routes' bindings. This is SHARED CHROME the derive lane did not promote: a Layout field is one value for the whole site, and an attribute here would bind one route and silently freeze the rest. |
+| `IN_EXPRESSION` in the markup | `bind-expression`, or `declare-richtext` for a richtext value. |
+| `IN_EXPRESSION` **in the frontmatter or a `<script>`** | `bind-data` — the copy is a row in the file's own data, and the array is what gets bound, not the `{l.label}` that reads it. |
+| `DIALECT_UNSUPPORTED` on `.md`/`.mdx` | `bind-data` — a content-collection body is the entry's `document` field. |
+| `NOT_IN_SOURCE` / `NO_ORIGINAL` | `bind-data` when the copy is in a data module, else `manual`: the built page formats it and the source holds the input. |
+| `PROP_TARGET_NOT_FOUND`, `PROP_DRILLED_DEEP`, `SUBSTRING_ONLY` | `wrap-span`. |
+| `REPEATER_AMBIGUOUS`, `REPEATER_FIXED_LENGTH` | `declare-attr` on the row's own element. |
+| `IMAGE_ASSET_UNRESOLVED`, `IN_DATA_FILE` | `bind-data`. |
+| `SUBTREE_NOT_HTML` | `declare-richtext`. |
+| `KIND_MISMATCH`, `IN_SCRIPT_OR_COMMENT`, `DYNAMIC_PARAMS_UNAVAILABLE`, `BRIEF_META_UNPLACED`, `PARSE_ERROR`, `TIER3_UNVERIFIABLE`, `CANVAS_BRIDGE_MANUAL`, `SSR_DRAFT_ROUTE` | `manual`, with the reason quoted in `why` — read the row's `message` for what blocked it. |
+
+**A file that talks ABOUT the site is not a source file.** A path is rewritten only when every
+located occurrence was, so a test asserting on the hero line, or a build config quoting the font
+stack, used to hold an already-declared path pending. `*.test.*`, `*.spec.*`, `__tests__/`,
+`__mocks__/` and root-level `*.config.*` are never opened, alongside `node_modules/` and `dist/`.
+A data module under `src/` — `src/site.config.ts` — is not a build config and stays bindable.
+
+**`--xray` and the receipt exclude the same things.** The xray measures a BUILT site's visible
+copy and the receipt measures the brief's paths in SOURCE, so they never have the same
+denominator — but neither counts a decorative glyph, a `<script>` or a `<style>` as copy
+(`isDecorativeGlyph` is the one reader), and the two lane rows `CANVAS_BRIDGE_MANUAL` and
+`SSR_DRAFT_ROUTE` ride `paths.pending` without entering the arithmetic above. A path the receipt
+calls pending is copy the xray will still count as unaddressed; a path it calls rewritten is not.
+
 A path counts as rewritten only when EVERY located occurrence of it was. A value rendered by both
 the home page and a shared header is two places a reader sees it, and converting one of them reads
 as a finished job.

package/dist/cli.js

@@ -5312,6 +5312,138 @@ function assertReceipt(receipt) {
   return receipt;
 }
 
+// src/fix.ts
+var MAX_SNIPPET = 160;
+var MAX_WORDS = 12;
+var escapeRegExp2 = (value) => value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+function findLiteral(content, literal) {
+  const words = literal.split(/\s+/).filter(Boolean).slice(0, MAX_WORDS);
+  if (words.length === 0) return null;
+  const pattern = new RegExp(words.map(escapeRegExp2).join("(?:\\s|<[^>]*>)+"), "u");
+  const match = pattern.exec(content);
+  return match ? { index: match.index, length: match[0].length } : null;
+}
+function placeOf(content, index) {
+  const before = content.slice(0, index);
+  const line = before.split("\n").length;
+  const start = before.lastIndexOf("\n") + 1;
+  const end = content.indexOf("\n", index);
+  return {
+    line,
+    col: index - start + 1,
+    snippet: content.slice(start, end < 0 ? content.length : end).trim().slice(0, MAX_SNIPPET)
+  };
+}
+function fileHead(content) {
+  return { line: 1, snippet: (content ?? "").split("\n", 1)[0].trim().slice(0, MAX_SNIPPET) };
+}
+var isMarkdown = (file) => /\.mdx?$/i.test(file);
+var isDataModule = (file) => /\.(json|ya?ml|[mc]?[jt]s)$/i.test(file) && !/\.d\.ts$/i.test(file);
+function inDataRegion(content, index) {
+  if (content.startsWith("---")) {
+    const close = content.indexOf("\n---", 3);
+    if (close >= 0 && index < close) return true;
+  }
+  for (const match of content.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script>/gi)) {
+    const start = match.index + match[0].indexOf(">") + 1;
+    if (index >= start && index < start + match[1].length) return true;
+  }
+  return false;
+}
+var ELEMENT_ACTIONS = /* @__PURE__ */ new Set([
+  "wrap-span",
+  "declare-attr",
+  "declare-richtext"
+]);
+function actionFor(reason, input, file, inData) {
+  const chosen = reasonAction(reason, input, file, inData);
+  if (inData && ELEMENT_ACTIONS.has(chosen.action)) {
+    return { action: "bind-data", why: "The copy is a row in this file's own data \u2014 bind that field." };
+  }
+  return chosen;
+}
+function reasonAction(reason, input, file, inData) {
+  const richtext = input.kind === "richtext" || input.inline === true;
+  switch (reason) {
+    case "AMBIGUOUS_LITERAL":
+      if (input.routes > 1) {
+        return {
+          action: "manual",
+          why: `${file} renders this value on ${input.routes} routes, so it is shared chrome rather than page copy: promote it to the project Layout (a Layout field is one value for the whole site) and rerun, or give each route its own element.`
+        };
+      }
+      return {
+        action: "declare-attr",
+        why: input.candidates.length > 1 ? `This copy appears in ${input.candidates.join(", ")}; declare the one this route renders.` : "Two paths want this element and nothing says which \u2014 declare the right one by hand."
+      };
+    case "REPEATER_AMBIGUOUS":
+    case "REPEATER_FIXED_LENGTH":
+      return {
+        action: "declare-attr",
+        why: "Declare this row's leaf on the element itself, or rewrite the loop to read the CMS array."
+      };
+    case "IN_EXPRESSION":
+      if (inData) {
+        return { action: "bind-data", why: "The copy is a row in this file's own data \u2014 bind that field." };
+      }
+      return richtext ? { action: "declare-richtext", why: "The container is fed markup \u2014 bind it with the richtext helper." } : { action: "bind-expression", why: "Replace the computed value with the helper's read for this path." };
+    case "DIALECT_UNSUPPORTED":
+      if (isMarkdown(file)) {
+        return {
+          action: "bind-data",
+          why: "This is a content-collection body: bind the element that renders it and make the entry the source."
+        };
+      }
+      return isDataModule(file) ? { action: "bind-data", why: `${file} holds this copy as data \u2014 bind the element that renders it.` } : { action: "manual", why: `No parser here reads ${file}, so the declaration has to be written by hand.` };
+    case "NOT_IN_SOURCE":
+    case "NO_ORIGINAL":
+      return isDataModule(file) || inData ? { action: "bind-data", why: "The template computes this from data \u2014 bind the data field instead of the text." } : {
+        action: "manual",
+        why: "The built page renders this and no source file holds it verbatim; find what formats it."
+      };
+    case "IN_DATA_FILE":
+    case "IMAGE_ASSET_UNRESOLVED":
+      return { action: "bind-data", why: "Bind the data field this value comes from." };
+    case "PROP_TARGET_NOT_FOUND":
+    case "PROP_DRILLED_DEEP":
+      return {
+        action: "wrap-span",
+        why: "The component renders this in a position no attribute reaches \u2014 wrap it in a span of its own."
+      };
+    case "SUBSTRING_ONLY":
+      return { action: "wrap-span", why: "Only part of the element's text is this value \u2014 give it its own span." };
+    case "SUBTREE_NOT_HTML":
+      return { action: "declare-richtext", why: "The value is the element's whole subtree \u2014 bind it as richtext." };
+    default:
+      return { action: "manual", why: `${reason} \u2014 see the receipt's message for what blocked this path.` };
+  }
+}
+function fixFor(row, input) {
+  const file = row.file ?? input.candidates[0] ?? "";
+  const content = input.contents.get(file);
+  const at = content ? findLiteral(content, input.literal) : null;
+  const place = at ? placeOf(content, at.index) : { ...fileHead(content), col: void 0 };
+  const { action, why } = actionFor(row.reason, input, file, at !== null && inDataRegion(content, at.index));
+  return {
+    file,
+    line: row.line ?? place.line,
+    ...place.col !== void 0 ? { col: place.col } : {},
+    snippet: place.snippet,
+    action,
+    ...why ? { why } : {}
+  };
+}
+function routeCounts(pending) {
+  const routes = /* @__PURE__ */ new Map();
+  for (const row of pending) {
+    const id = `${row.scope}|${row.path}`;
+    const held = routes.get(id) ?? /* @__PURE__ */ new Set();
+    held.add(row.route);
+    routes.set(id, held);
+  }
+  return new Map([...routes].map(([id, set]) => [id, set.size]));
+}
+
 // src/sites/html.ts
 import { parse as parseHtml } from "parse5";
 var RAW_TAGS = /* @__PURE__ */ new Set(["script", "style", "noscript", "template"]);
@@ -6189,6 +6321,7 @@ import MagicString4 from "magic-string";
 var WIRABLE = ["html", "astro", "jsx"];
 var CONTROLS = /* @__PURE__ */ new Set(["input", "select", "textarea"]);
 var FORM_SCRIPT = `<script>
+function bcmsPhone(cc,raw,trunk){var d=String(raw||'').trim();if(!d)return '';if(d.charAt(0)==='+'){var i=d.replace(/D/g,'');return i?'+'+i:'';}if(!cc)return d;var n=d.replace(/D/g,'');if(trunk&&n.charAt(0)==='0')n=n.slice(1);return n?'+'+cc+n:'';}
 document.querySelectorAll('form[data-bcms-form]').forEach(function(f){
   f.addEventListener('submit',function(e){
     e.preventDefault();
@@ -6204,6 +6337,15 @@ document.querySelectorAll('form[data-bcms-form]').forEach(function(f){
       if(Object.prototype.hasOwnProperty.call(data,k))data[k]=(Array.isArray(data[k])?data[k]:[data[k]]).concat(v);
       else data[k]=v;
     });
+    // Country-picker phone fields post ONE key. The __country select is scaffolding: it is
+    // composed into the field's own value here, AFTER collection, so a showIf that points at a
+    // phone key still compares the raw digits. Then it is removed from the body.
+    f.querySelectorAll('select[name$="__country"]').forEach(function(sel){
+      var k=sel.name.slice(0,-9);
+      var o=sel.options[sel.selectedIndex];
+      data[k]=bcmsPhone(o?(o.getAttribute('data-cc')||''):'',data[k],o?o.getAttribute('data-trunk')==='1':false);
+      delete data[k+'__country'];
+    });
     var body={data:data};
     var tok=f.querySelector('[name="cf-turnstile-response"]');
     if(tok&&tok.value)body['cf-turnstile-response']=tok.value;
@@ -6428,7 +6570,9 @@ var SKIP_DIRS = [
   ".vercel/",
   ".cache/",
   "coverage/",
-  "vendor/"
+  "vendor/",
+  "__tests__/",
+  "__mocks__/"
 ];
 var SKIP_FILES = ["package-lock.json", "bun.lock", "bun.lockb", "yarn.lock", "pnpm-lock.yaml"];
 var SOURCE_EXTENSIONS = [
@@ -6466,11 +6610,13 @@ var TEMPLATE_DIRS = /* @__PURE__ */ new Set([
   "_includes",
   "_layouts"
 ]);
+var isAboutTheSite = (lower) => /\.(test|spec)\.[^./]+$/.test(lower) || /^[^/]*\.config\.[^./]+$/.test(lower);
 function isSourceCandidate(path) {
   const lower = path.toLowerCase();
   if (SKIP_DIRS.some((dir) => lower === dir.slice(0, -1) || lower.startsWith(dir) || lower.includes(`/${dir}`)))
     return false;
   if (SKIP_FILES.includes(lower.split("/").pop() ?? "")) return false;
+  if (isAboutTheSite(lower)) return false;
   if (/(^|\/)[jt]sconfig\.json$/.test(lower)) return true;
   if (!SOURCE_EXTENSIONS.some((ext) => lower.endsWith(ext))) return false;
   if (!PROSE_EXTENSIONS.some((ext) => lower.endsWith(ext))) return true;
@@ -8411,7 +8557,7 @@ var ConvertError = class extends Error {
   code;
 };
 var keyOf = (route, scope, path) => scope === "layout" ? `layout::${path}` : `${route}::page::${path}`;
-var escapeRegExp2 = (value) => value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+var escapeRegExp3 = (value) => value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
 var identifierFor2 = (slug) => `bcms${slug.replace(/[^A-Za-z0-9]+(.)?/g, (_, c) => c ? c.toUpperCase() : "").replace(/^./, (c) => c.toUpperCase())}`;
 var UNPLACED = /* @__PURE__ */ new Set(["NOT_IN_SOURCE", "IN_EXPRESSION"]);
 async function drillProp(ctx, fromFile, component, prop, kind, hop) {
@@ -8758,6 +8904,33 @@ async function convertSources(briefIn, sources, options) {
     else outFiles.push({ path, operation: "modify", content });
   }
   pending.push(...canvas.pending);
+  for (const file of outFiles) if (file.operation === "modify") latest.set(file.path, file.content);
+  const pageFilesOrTemplate = (route, files) => {
+    const all = [...files];
+    const exact = pageFilesFor(route, all);
+    if (exact.length > 0) return exact;
+    const wanted = route.split("/").filter(Boolean);
+    return all.filter((file) => {
+      if (!isDynamicRoute(file)) return false;
+      const pattern = routeOfFile(file)?.split("/").filter(Boolean);
+      if (!pattern) return false;
+      const rest = pattern.some((segment) => segment.startsWith("[..."));
+      if (rest ? wanted.length < pattern.length : wanted.length !== pattern.length) return false;
+      return pattern.every((segment, i) => segment.startsWith("[") || segment === wanted[i]);
+    }).sort();
+  };
+  const routes = routeCounts(pending);
+  for (const row of pending) {
+    const target = targets.get(keyOf(row.route, row.scope, row.path));
+    row.fix = fixFor(row, {
+      literal: target?.literal ?? "",
+      kind: row.kind,
+      ...target?.inline ? { inline: true } : {},
+      candidates: locatedByKey.get(keyOf(row.route, row.scope, row.path))?.files ?? pageFilesOrTemplate(row.route, latest.keys()),
+      contents: latest,
+      routes: routes.get(`${row.scope}|${row.path}`) ?? 1
+    });
+  }
   const receipt = assertReceipt({
     briefDigest: digest,
     paths: {
@@ -8970,13 +9143,13 @@ async function plan(file, content, dialect, targets, sites, importAt, outcomes,
       for (const target of sharing) outcomes.set(target.key, missing(literal));
       continue;
     }
-    const assigned = sharing.length === 1 ? /* @__PURE__ */ new Map([[sharing[0].key, forLiteral]]) : resolvePositionally(sharing, forLiteral);
-    if (!assigned) {
-      for (const target of sharing) outcomes.set(target.key, "AMBIGUOUS_LITERAL");
-      continue;
-    }
+    const resolved = sharing.length === 1 ? { assigned: /* @__PURE__ */ new Map([[sharing[0].key, forLiteral]]), ambiguous: /* @__PURE__ */ new Set() } : resolvePositionally(sharing, forLiteral);
     for (const target of sharing) {
-      const mine = assigned.get(target.key) ?? [];
+      if (resolved.ambiguous.has(target.key)) {
+        outcomes.set(target.key, "AMBIGUOUS_LITERAL");
+        continue;
+      }
+      const mine = resolved.assigned.get(target.key) ?? [];
       const rewritable = mine.filter((s) => s.where !== "script" && s.where !== "comment" && !s.partial);
       if (rewritable.length === 0) {
         outcomes.set(
@@ -9321,19 +9494,25 @@ function repeatGroups(targets) {
 }
 function resolvePositionally(targets, sites) {
   const assigned = /* @__PURE__ */ new Map();
-  const claimed = /* @__PURE__ */ new Set();
+  const ambiguous = /* @__PURE__ */ new Set();
+  const picked = /* @__PURE__ */ new Map();
+  const wanters = /* @__PURE__ */ new Map();
   for (const target of targets) {
     const locator = target.locator;
-    if (!locator) return null;
-    const candidates = sites.filter(
-      (s) => s.siblingIndex === locator.siblingIndex && isSubsequence(locator.ancestors, s.ancestors)
-    );
-    const site = locator.repeat ? candidates[locator.repeat.index] : candidates.length === 1 ? candidates[0] : void 0;
-    if (!site || claimed.has(site)) return null;
-    claimed.add(site);
-    assigned.set(target.key, [site]);
+    const candidates = locator ? sites.filter((s) => s.siblingIndex === locator.siblingIndex && isSubsequence(locator.ancestors, s.ancestors)) : [];
+    const site = locator?.repeat ? candidates[locator.repeat.index] : candidates.length === 1 ? candidates[0] : void 0;
+    if (!site) {
+      ambiguous.add(target.key);
+      continue;
+    }
+    picked.set(target.key, site);
+    wanters.set(site, (wanters.get(site) ?? 0) + 1);
+  }
+  for (const [key, site] of picked) {
+    if ((wanters.get(site) ?? 0) > 1) ambiguous.add(key);
+    else assigned.set(key, [site]);
   }
-  return assigned;
+  return { assigned, ambiguous };
 }
 function isSubsequence(wanted, chain) {
   let at = 0;
@@ -9356,7 +9535,7 @@ async function tierThree(file, content, dialect, targets, llmFallback) {
   if (elements.length !== targets.length) return { ok: false, reason: "unverifiable" };
   const locals = helperLocals(reparsed, dialect, file);
   const reads = DIALECT_RULES[dialect].text !== null;
-  const boundary = (name) => `(?<![\\w$])${escapeRegExp2(name)}(?![\\w$])`;
+  const boundary = (name) => `(?<![\\w$])${escapeRegExp3(name)}(?![\\w$])`;
   const claimed = /* @__PURE__ */ new Set();
   for (const element of elements) {
     const candidates = targets.filter(
@@ -9364,7 +9543,7 @@ async function tierThree(file, content, dialect, targets, llmFallback) {
     );
     const match = candidates.find((target) => {
       if (!reads) return true;
-      const quoted = escapeRegExp2(JSON.stringify(target.path));
+      const quoted = escapeRegExp3(JSON.stringify(target.path));
       if (target.scope === "layout") {
         return locals.bcmsLayout.some(
           (name) => new RegExp(`${boundary(name)}\\s*\\(\\s*${quoted}`).test(element.own)
@@ -9426,7 +9605,7 @@ function emitStubs2(dialects, slugs, contents, out) {
 // package.json
 var package_default = {
   name: "@bettercms-ai/convert",
-  version: "0.11.0",
+  version: "0.12.1",
   type: "module",
   description: "Deterministic codemod that turns an imported site's text-matched bindings into declared ones \u2014 parser offsets in, one magic-string pass out, plus a receipt of everything it could not do.",
   bin: {
@@ -9466,7 +9645,7 @@ var package_default = {
   },
   devDependencies: {
     "@types/node": "^20",
-    esbuild: "^0.27.7",
+    esbuild: "^0.28.1",
     tsup: "^8.5.1",
     typescript: "^5",
     vitest: "^4.1.4"