@mapled/cli 0.8.0 → 0.10.0

package/README.md

@@ -97,7 +97,7 @@ npx @mapled/cli scan
 npx @mapled/cli scan --write
 ```
 
-The scanner reads the site's source and writes down what it reads from Mapled: the pages (from the framework's file layout — Next.js App and Pages Router, Astro, SvelteKit, Nuxt, Remix) and one binding per place a collection or field is used.
+The scanner reads the site's source and writes down what it reads from Mapled: the pages (from the framework's file layout — Next.js App and Pages Router, Astro, SvelteKit, Nuxt, Remix — or a plain HTML site's `.html` files) and one binding per place a collection or field is used.
 
 ```text
 Scanned 42 files with TypeScript — 3 pages, 14 bindings.
@@ -113,6 +113,21 @@ No mapled/manifest.json yet — run `mapled scan --write` to create it, then `ma
 
 It follows `getRecords` / `getRecord` / `getRecordBySlug` / `getSingle` calls through variables, destructuring, `.map()` callbacks and into the components a record is passed to as a prop, and through helpers in `lib/` that return a read. The parser is your repository's own TypeScript; without it the scanner still finds the collections, but not the fields. Source code never leaves your machine — only the manifest does, when you push it.
 
+**A plain HTML site** (`framework` is `plain-html` in `mapled.json` — a folder with an `index.html` and no framework in `package.json`) has no reads in code: the scanner reads the `data-mapled-*` attributes of its `.html` files the way the [`@mapled/vanilla`](https://www.npmjs.com/package/@mapled/vanilla) script does, in the tree a browser builds from the file (the CLI carries the standard HTML parser, parse5, inside — nothing to install). Each file is a page where a static host serves it (`index.html` → `/`, `blog/index.html` → `/blog/`, `post.html` → `/post.html`, `public/` from the root); a record read by `data-mapled-slug-param` has its own address with that parameter in the query, so **Preview** in the record's editor opens the right page — and the scan says when Preview can't open an address. The target of a binding is what the attribute renders — `data-mapled-format="markdown"` rich text, an `<img>` an image, `data-mapled-bind-href` a link — and each different way a field is read is a binding of its own, so a field bound where it can't render shows up as a mismatch even when another element reads it well. With the schema at hand it also names the paths nothing in Mapled answers to: on the page, a misspelled field and an empty one look the same.
+
+```text
+Scanned 2 HTML files — 2 pages, 11 bindings.
+
+/                       index.html
+  home   headline, intro, tagline
+  posts  list • date, slug, title
+/post.html?slug=[slug]  post.html
+  home   headline
+  posts  by slug • body, title
+
+⚠ This path names nothing in the schema: home.tagline (index.html). A misspelled field empties its place on the page, a misspelled single leaves the page's own HTML.
+```
+
 `--write` creates or updates `mapled/manifest.json`. Bindings the scan can't see (written by your AI agent, or by hand) are kept and listed; `--prune` drops them. Commit the file.
 
 ## Validate and push the manifest
@@ -175,7 +190,7 @@ Mapled doctor — Dolphin Landing
 ✓ Environment          MAPLED_KEY and MAPLED_WEBHOOK_SECRET in .env.local
 ✓ Secrets in git       No env files or Mapled secrets are tracked.
 ✓ @mapled/next         0.8.0 (current)
-✓ mapled CLI           0.8.0 (current)
+✓ mapled CLI           0.9.0 (current)
 ✓ Revalidation route   app/api/mapled/revalidate/route.ts
 ⚠ Preview route        app/api/mapled/preview/route.ts is missing — mount createPreviewHandler from "@mapled/next/server" so Preview from Mapled works.
 ✓ Site reads content   Last read 3m ago
@@ -188,7 +203,7 @@ Mapled doctor — Dolphin Landing
 
 The repository side: the link, the sign-in, the generated types, the schema pin, the site manifest (and whether it matches what was pushed), the integration hash — one line saying whether the pin and the manifest here describe the same integration Mapled holds, and if not, what moved (the schema, the bindings, or edits made in Mapled) and which command catches up — `MAPLED.md` (and whether it is what Mapled would render now), `MAPLED_KEY` and `MAPLED_WEBHOOK_SECRET` in your env files (names only — values are never read out), env files or Mapled secrets tracked by git, the installed `@mapled/next` and CLI versions, the revalidation and preview routes of a Next.js site. The Mapled side, from the same status the AI agent's `check_integration` tool reads: whether the site has read content with the delivery key, the publish webhook and its last delivery, the preview route on the deployed site, the bindings health.
 
-A site rendered in the visitor's browser — a React single-page app (Vite and the like), plain HTML — is checked for what it has: the package line is `@mapled/react` (or `@mapled/client`), Environment wants `VITE_MAPLED_KEY` and no webhook secret, and the routes and the webhook read ○ «Not needed» — such a site shows a publish on its next load and trades Mapled's preview link on the page itself. doctor takes the framework from `mapled.json`, or works it out from `package.json`. Whatever the framework, a server key or a signing secret under a public prefix (`VITE_…`, `NEXT_PUBLIC_…`) is a ✗: it would ship to every visitor.
+A site rendered in the visitor's browser — a React single-page app (Vite and the like), plain HTML — is checked for what it has: the package line is `@mapled/react`, Environment wants `VITE_MAPLED_KEY` and no webhook secret, and the routes and the webhook read ○ «Not needed» — such a site shows a publish on its next load and trades Mapled's preview link on the page itself. Plain HTML installs nothing: the line is `@mapled/vanilla`, read from the script tag in the pages (or from a copy of the file next to them) — ⚠ when the address has no version or an older one — and Generated types and Environment read ○: the fields are named in attributes, the delivery key sits in the page. doctor takes the framework from `mapled.json`, or works it out from `package.json` (no `package.json` and an `index.html` is plain HTML). Whatever the framework, a server key or a signing secret under a public prefix (`VITE_…`, `NEXT_PUBLIC_…`) is a ✗: it would ship to every visitor.
 
 Exit code 1 when something is marked ✗; `--json` prints the checks for scripts and agents.
 

package/dist/commands.js

@@ -12,6 +12,7 @@ import { formatChecks, GLYPH, paint, summarize, useColor } from "./output.js";
 import { diffSchema, formatChanges, pinSchema, readPin, SCHEMA_FILE, summarizeChanges, writePin } from "./pin.js";
 import { checkMapledMd, MD_FILE, mergeMd } from "./md.js";
 import { loadTypeScript, mergeManifest, scanRepository } from "./scan.js";
+import { scanHtml } from "./scanHtml.js";
 import { generateTypes } from "./types.js";
 /** The OAuth client id for this API — registered once per machine, and
     again when the server no longer knows it. */
@@ -490,8 +491,8 @@ export async function scan(ctx, flags) {
         throw new CliError(`The scan needs the schema to name the fields — sign in with \`mapled auth login\` or run \`mapled schema pull\` first.`);
     }
     const framework = found.config.framework ?? (await dr.detectFramework(found.dir)) ?? null;
-    const ts = loadTypeScript(found.dir);
-    const result = await scanRepository(found.dir, { framework, schema, ts });
+    // a plain HTML site declares its reads in attributes, not in code
+    const result = framework === "plain-html" ? await scanHtml(found.dir, { schema }) : await scanRepository(found.dir, { framework, schema, ts: loadTypeScript(found.dir) });
     const existing = await loadManifest(found.dir);
     const merged = mergeManifest(existing?.manifest ?? null, result, { prune: Boolean(flags.prune) });
     if (flags.json) {
@@ -500,7 +501,7 @@ export async function scan(ctx, flags) {
             await writeManifestFile(found.dir, merged.manifest);
         return;
     }
-    ctx.out(`Scanned ${plural(result.files, "file")} with ${result.parser === "typescript" ? "TypeScript" : "the tokenizer"} — ${plural(result.pages.length, "page")}, ${plural(result.bindings.length, "binding")}.`);
+    ctx.out(`${result.parser === "html" ? `Scanned ${plural(result.files, "HTML file")}` : `Scanned ${plural(result.files, "file")} with ${result.parser === "typescript" ? "TypeScript" : "the tokenizer"}`} — ${plural(result.pages.length, "page")}, ${plural(result.bindings.length, "binding")}.`);
     if (result.bindings.length > 0) {
         ctx.out("");
         for (const line of describeScan(result, schema))
@@ -581,9 +582,10 @@ export async function doctor(ctx, flags) {
     const checks = [];
     checks.push(dr.checkLink(found ? { path: path.relative(ctx.cwd, found.path) || CONFIG_FILE } : null, status?.project.name ?? conn?.projectName ?? null));
     checks.push(dr.checkAuth({ conn: conn ?? null, status, error: authError }));
+    const framework = found?.config.framework ?? (await dr.detectFramework(dir));
     const typesPath = found?.config.types ?? DEFAULT_TYPES_PATH;
     const existing = await readFile(path.join(dir, typesPath), "utf8").catch(() => null);
-    checks.push(dr.checkTypes(existing, schema ? generateTypes(schema, { projectName: status?.project.name, sdk: dr.sdkOf(found?.config.framework ?? (await dr.detectFramework(dir))) }) : null, typesPath));
+    checks.push(dr.checkTypes(existing, schema ? generateTypes(schema, { projectName: status?.project.name, sdk: dr.sdkOf(framework) }) : null, typesPath, framework));
     const pin = found ? await readPin(dir).catch(() => null) : null;
     const livePin = schema && found ? pinSchema(schema, found.config.project) : null;
     const pinChanges = pin && livePin ? diffSchema(pin.pin.collections, livePin.collections) : null;
@@ -601,13 +603,13 @@ export async function doctor(ctx, flags) {
     checks.push(dr.checkManifestFile(localManifest ? { manifest: localManifest.manifest, problems: localManifest.problems } : null, remoteManifest));
     checks.push(dr.checkIntegration({ pin: pin?.pin ?? null, manifest: localManifest?.manifest ?? null }, conn && status ? { integration: status.integration, manifest: remoteManifest } : null));
     checks.push(checkMapledMd(await readFile(path.join(dir, MD_FILE), "utf8").catch(() => null), guide));
-    const framework = found?.config.framework ?? (await dr.detectFramework(dir));
     // a site rendered in the browser has no webhook, no routes and no secret to check
     const browser = dr.rendersInBrowser(framework);
     const sdk = dr.sdkOf(framework);
     checks.push(dr.checkEnv(await dr.envNames(dir), ctx.env, framework));
     checks.push(dr.checkSecrets(await dr.gitFacts(dir)));
-    checks.push(dr.checkSdk(await dr.installedVersion(dir, sdk), status?.sdk[sdk], sdk));
+    // plain HTML installs nothing: the runtime comes with a script tag
+    checks.push(framework === "plain-html" ? dr.checkScriptTag(await dr.findScriptTag(dir), status?.sdk[sdk]) : dr.checkSdk(await dr.installedVersion(dir, sdk), status?.sdk[sdk], sdk));
     checks.push(dr.checkCli(ctx.version, status?.sdk["@mapled/cli"]));
     checks.push(dr.checkRoute("revalidate", await dr.findRoute(dir, "revalidate"), framework));
     checks.push(dr.checkRoute("preview", await dr.findRoute(dir, "preview"), framework));

package/dist/doctor.d.ts

@@ -100,6 +100,18 @@ export declare function installedVersion(dir: string, pkg: string): Promise<Pack
 export declare const rendersInBrowser: (framework: string | null | undefined) => boolean;
 /** The first-party package a site of this framework reads through. */
 export declare const sdkOf: (framework: string | null | undefined) => string;
+/** How a plain HTML site loads "@mapled/vanilla": a script tag in its
+    pages — from a CDN, where the version is in the address, or a copy of
+    the single file next to the pages, where it is in the file's first
+    line. */
+export type ScriptTag = {
+    /** the page or the copy the version was read from */
+    file: string;
+    /** null — a CDN address without a version: it follows every release */
+    version: string | null;
+};
+export declare function findScriptTag(dir: string): Promise<ScriptTag | null>;
+export declare function checkScriptTag(tag: ScriptTag | null, latest: string | undefined): Check;
 export declare function detectFramework(dir: string): Promise<string | undefined>;
 /** -1, 0 or 1 comparing dotted numeric versions; anything else compares by number. */
 export declare function compareVersions(a: string, b: string): number;
@@ -127,7 +139,7 @@ export declare function checkAuth(state: {
     status: IntegrationStatus | null;
     error: string | null;
 }): Check;
-export declare function checkTypes(existing: string | null, generated: Generated | null, typesPath: string): Check;
+export declare function checkTypes(existing: string | null, generated: Generated | null, typesPath: string, framework?: string | null): Check;
 export declare function checkEnv(names: Map<string, string>, env: NodeJS.ProcessEnv, framework?: string | null): Check;
 export declare function checkSecrets(facts: GitFacts): Check;
 export declare function checkSdk(pkg: PackageVersion, latest: string | undefined, name?: string): Check;

package/dist/doctor.js

@@ -1,5 +1,5 @@
 import { execFile } from "node:child_process";
-import { access, readFile } from "node:fs/promises";
+import { access, readdir, readFile } from "node:fs/promises";
 import path from "node:path";
 import { promisify } from "node:util";
 import { integrationHash } from "./integrationHash.js";
@@ -108,7 +108,65 @@ export async function installedVersion(dir, pkg) {
     keys are the ones Mapled knows by (lib/siteStack.ts of the API). */
 export const rendersInBrowser = (framework) => framework === "react-spa" || framework === "plain-html";
 /** The first-party package a site of this framework reads through. */
-export const sdkOf = (framework) => !framework || framework === "nextjs" ? "@mapled/next" : framework === "react-spa" ? "@mapled/react" : "@mapled/client";
+export const sdkOf = (framework) => !framework || framework === "nextjs" ? "@mapled/next" : framework === "react-spa" ? "@mapled/react" : framework === "plain-html" ? "@mapled/vanilla" : "@mapled/client";
+const SKIPPED_DIRS = new Set(["node_modules", ".git", ".next", ".vercel", ".netlify", "vendor"]);
+export async function findScriptTag(dir) {
+    // pages first, breadth first, and not forever: a site's pages sit near its root
+    const queue = [{ abs: dir, rel: "", depth: 0 }];
+    const copies = [];
+    let unpinned = null;
+    let seen = 0;
+    while (queue.length > 0 && seen < 2000) {
+        const at = queue.shift();
+        const entries = await readdir(at.abs, { withFileTypes: true }).catch(() => []);
+        for (const entry of entries.sort((a, b) => a.name.localeCompare(b.name))) {
+            seen += 1;
+            const rel = at.rel ? `${at.rel}/${entry.name}` : entry.name;
+            if (entry.isDirectory()) {
+                if (at.depth < 3 && !SKIPPED_DIRS.has(entry.name) && !entry.name.startsWith("."))
+                    queue.push({ abs: path.join(at.abs, entry.name), rel, depth: at.depth + 1 });
+                continue;
+            }
+            if (/^mapled(\.min)?\.js$/.test(entry.name))
+                copies.push(rel);
+            if (!/\.html?$/i.test(entry.name))
+                continue;
+            const text = await readFile(path.join(at.abs, entry.name), "utf8").catch(() => "");
+            const pinned = /@mapled\/vanilla@(\d+\.\d+\.\d+)/.exec(text);
+            if (pinned)
+                return { file: rel, version: pinned[1] };
+            if (unpinned === null && /@mapled\/vanilla(@latest)?\//.test(text))
+                unpinned = rel;
+        }
+    }
+    for (const rel of copies) {
+        const head = (await readFile(path.join(dir, rel), "utf8").catch(() => "")).slice(0, 200);
+        const banner = /^\/\*! @mapled\/vanilla (\d+\.\d+\.\d+) /.exec(head);
+        if (banner)
+            return { file: rel, version: banner[1] };
+    }
+    return unpinned === null ? null : { file: unpinned, version: null };
+}
+export function checkScriptTag(tag, latest) {
+    const key = "sdk";
+    const label = "@mapled/vanilla";
+    const at = latest ?? "x.y.z";
+    if (!tag) {
+        return {
+            key,
+            label,
+            status: "failed",
+            detail: `No page loads it yet. Put <script src="https://cdn.jsdelivr.net/npm/@mapled/vanilla@${at}/dist/mapled.js" data-mapled-key="…" defer></script> into the pages.`,
+        };
+    }
+    if (tag.version === null) {
+        return { key, label, status: "warning", detail: `${tag.file} loads it without a version — that address follows every release. Pin it: @mapled/vanilla@${at}.` };
+    }
+    if (latest && compareVersions(tag.version, latest) < 0) {
+        return { key, label, status: "warning", detail: `${tag.version} in ${tag.file}, ${latest} available — change the version in the script tag.` };
+    }
+    return { key, label, status: "passed", detail: latest ? `${tag.version} in ${tag.file} (current)` : `${tag.version} in ${tag.file}` };
+}
 export async function detectFramework(dir) {
     const hasIndexHtml = () => access(path.join(dir, "index.html")).then(() => true, () => false);
     let deps;
@@ -234,8 +292,11 @@ export function checkAuth(state) {
         detail: `${state.status.connection.name} • ${state.status.project.name}`,
     };
 }
-export function checkTypes(existing, generated, typesPath) {
+export function checkTypes(existing, generated, typesPath, framework) {
     const label = "Generated types";
+    if (framework === "plain-html" && existing === null) {
+        return { key: "types", label, status: "skipped", detail: "Not needed: plain HTML names its fields in attributes, not in TypeScript." };
+    }
     if (!generated)
         return { key: "types", label, status: "skipped", detail: "Sign in to compare with the schema." };
     if (existing === null) {

package/dist/html5.d.ts

@@ -0,0 +1,66 @@
+import { type Token, type TreeAdapter, type TreeAdapterTypeMap } from "parse5";
+/** The HTML parser of `mapled scan` for a plain HTML site: parse5, the
+    WHATWG parsing algorithm — the tree a browser builds, and the one
+    jsdom builds for the runtime's tests. Tag soup decides which scope a
+    binding lands in (a <p> a <div> closes, a table's foster parent, a
+    template an end tag can't leave), so the scan can't read a tree of its
+    own. build.mjs bundles parse5 (pinned) into dist/html5.js: the CLI
+    still installs without dependencies.
+
+    What a page can make the parser do is held linear in its size — a
+    page is the site's to write, and the scan reads up to 2 MB of it:
+    - open elements: at most MAX_OPEN (Guarded.onItemPush). Every start tag
+      looks down them for a scope; 100,000 unclosed <div>s cost minutes.
+    - a tag's attributes: a set of names (Guarded, _leaveAttrName). parse5
+      compared each new one with every earlier one — 64,000 on one <div>: 4 s.
+    - the tree: children in linked lists (`linked` below), every operation
+      O(1). parse5's own tree keeps arrays, and foster parenting (text
+      before a <table>: indexOf), the adoption agency (a misnested </b>:
+      splice from the front) and repeated <html>/<body> tags (every
+      attribute so far) walk them — 160,000 "a<br>" in a table: 11 s.
+    The worst page within the cap is the depth times its start tags. */
+/** How deep elements may be open at once — Chrome's own limit of the tree it builds. */
+export declare const MAX_OPEN = 512;
+export declare class TooDeep extends Error {
+}
+export type Html5Attr = Token.Attribute;
+/** A node of the tree: an element (tagName), text, a comment, a doctype,
+    the document or a template's content — children linked both ways. Its
+    fields are named as parse5's own tree names them; `childNodes` is made
+    when asked for (the parser never asks). */
+export declare class Html5Node {
+    readonly nodeName: string;
+    tagName?: string;
+    namespaceURI?: string;
+    attrs?: Html5Attr[];
+    /** a <template>'s content: what the page's author wrote inside it */
+    content?: Html5Node;
+    parentNode: Html5Node | null;
+    firstChild: Html5Node | null;
+    lastChild: Html5Node | null;
+    previousSibling: Html5Node | null;
+    nextSibling: Html5Node | null;
+    /** a text's */
+    value?: string;
+    /** a comment's */
+    data?: string;
+    /** a doctype's */
+    name?: string;
+    publicId?: string;
+    systemId?: string;
+    /** the document's */
+    mode?: string;
+    /** the names an <html> or <body> has, as repeated tags add to them */
+    names?: Set<string>;
+    location?: Token.ElementLocation | null;
+    constructor(nodeName: string, extra?: Partial<Html5Node>);
+    /** none on a text, a comment or a doctype, as in parse5's tree */
+    get childNodes(): Html5Node[] | undefined;
+}
+type Map_ = TreeAdapterTypeMap<Html5Node, Html5Node, Html5Node, Html5Node, Html5Node, Html5Node, Html5Node, Html5Node, Html5Node, Html5Node>;
+/** parse5's default tree adapter, method for method, on linked children. */
+export declare const linked: TreeAdapter<Map_>;
+/** The document, with scripting on — as the runtime sees it (a <noscript>
+    is text). Throws TooDeep past MAX_OPEN open elements. */
+export declare function parseDocument(text: string): Html5Node;
+export {};