@botlr/cli 0.2.0 → 0.3.0

package/CHANGELOG.md

@@ -1,5 +1,21 @@
 # @botlr/cli
 
+## 0.3.0
+
+### Minor Changes
+
+- `botlr schema pull` and `botlr schema push`: the explicit way to resolve props schema drift in either direction. A code push still never changes the schema.
+
+  - `botlr schema pull <name...>` (or `--all` for every manifest in `sectionsDir`) fetches the section definition's props schema from the dashboard and writes it as `schema` into `<name>.botlr.json`, in the CLI's fixed key order and in whichever format the dashboard has (field DSL or FormBuilder). It says "unchanged" when the manifest already holds the same schema, and regenerates `<name>.props.ts` afterwards unless `--no-types` or `"types": false`.
+  - `botlr schema push <name> [--force] [--yes]` reads `schema` from the manifest (an error when it is missing), fetches the dashboard's current schema, prints both formats and a diff of the field keys (added, removed, kept; node names for FormBuilder schemas), asks for confirmation unless `--yes`, then replaces the dashboard's schema (needs `BOTLR_PAT`). When the two schemas are in different formats the server answers 409 and the CLI prints both formats and says to rerun with `--force`; a 422 (not an object of fields, or the definition is not project-scoped) prints the server's message. A successful push regenerates the props file.
+  - The drift warning of `push` and `sections status` now names both commands: `botlr schema pull <name>` to take the dashboard's schema, `botlr schema push <name>` to send the code's. A manifest `schema` in the FormBuilder format now takes part in the drift comparison too (by node names), since `schema pull` can write one.
+  - `@botlr/sdk`: `client.sections.getSchema(name)` (site token, `GET /api/sections/{name}/schema`) and `client.sections.putSchema(name, { schema, force })` (personal token, `PUT /api/sections/{name}/schema`), with the `SectionSchemaFormat`, `SectionSchemaGetResponse`, `SectionSchemaPutBody` and `SectionSchemaPutResponse` types, `schemaFormat(schema)`, `SectionDefinition.format`, `BotlrSchemaFormatConflictError` (the 409 with `currentFormat` and `newFormat`) and `botlrErrorMessage(error)` to read a server's `detail` message. `SectionManifest.schema` is typed as `SectionSchema` (either format).
+
+### Patch Changes
+
+- Updated dependencies
+  - @botlr/sdk@0.5.0
+
 ## 0.2.0
 
 ### Minor Changes

package/README.md

@@ -28,8 +28,9 @@ component registry `pull` keeps up to date and `sections init` reads.
 `types` (default true) makes `pull` write `<name>.props.ts` from the
 definition's props schema; `false` turns it off for the app.
 
-Environment: `BOTLR_API_KEY` (site token, for pull/list/status) and
-`BOTLR_PAT` (personal access token with `write:sections`, for push).
+Environment: `BOTLR_API_KEY` (site token, for pull/list/status and `schema
+pull`) and `BOTLR_PAT` (personal access token with `write:sections`, for
+`push` and `schema push`).
 `BOTLR_BASE_URL`, `BOTLR_APP_ID`, `BOTLR_FRAMEWORK` and `BOTLR_SECTIONS_DIR`
 override the file; `--framework` and `--dir` override both.
 
@@ -41,6 +42,8 @@ botlr push <name>             [-m msg] [--shared] [--create] [--force]
 botlr sections list
 botlr sections status [name...] [--offline]
 botlr sections init           [--force] [--dry-run]
+botlr schema pull <name...> | --all  [--no-types]
+botlr schema push <name>      [--force] [-y, --yes] [--no-types]
 ```
 
 - `pull` writes every file of the section under `sectionsDir`, plus
@@ -76,6 +79,10 @@ botlr sections init           [--force] [--dry-run]
   `.optional()`, `.nullable()` or `.default(...)`). Fields it cannot read are
   left out, and so is `schema` when nothing could be read. Review them, then
   `botlr push <name> --create` per section.
+- `schema pull` copies the dashboard's props schema into the manifest's
+  `schema`; `schema push` sends the manifest's `schema` to the dashboard.
+  These are the only commands that change a definition's schema; see
+  "Schema drift" below.
 
 ## Prop types
 
@@ -117,22 +124,58 @@ export function Hero(props: HeroProps) { ... }
 
 ## Schema drift
 
-A manifest may carry a `schema` object in the field DSL (the same shape the
-dashboard stores): the props the code expects. The CLI never writes it to
-the server's definition except as the seed for `push --create`. On `push`
-and `sections status` it compares that object's keys with the definition's
-schema and warns:
+A manifest may carry a `schema` object (normally the field DSL, the same
+shape the dashboard stores): the props the code expects. A code `push`
+never writes it to the server's definition except as the seed for
+`push --create`. On `push` and `sections status` the CLI compares that
+object's keys with the definition's schema and warns:
 
 ```
 warning: props schema of 'hero' differs between the code and the dashboard
   the code expects props the dashboard does not define: ctaLink
   the dashboard defines props the code does not list: image
-  the dashboard owns the schema: edit it there, or update 'schema' in hero.botlr.json
+  the dashboard owns the schema; a code push never changes it. To resolve:
+    botlr schema pull hero   takes the dashboard's schema into hero.botlr.json
+    botlr schema push hero   sends the code's schema to the dashboard
 ```
 
-The warning changes nothing; fix the schema in the dashboard, pull to
-refresh `<name>.props.ts`, and update the manifest's `schema` (or drop it)
-when the code catches up.
+The warning changes nothing. Resolve it in one of two directions:
+
+```bash
+botlr schema pull hero          # the dashboard is right: copy its schema into hero.botlr.json
+botlr schema push hero          # the code is right: replace the dashboard's schema with the manifest's
+```
+
+`schema pull <name...>` (or `--all` for every manifest in `sectionsDir`)
+fetches the definition's props schema and writes it as `schema` into
+`<name>.botlr.json`, in the fixed key order and in whichever format the
+dashboard has. It prints `schema unchanged` when the manifest already says
+the same, and regenerates `<name>.props.ts` afterwards unless `--no-types`
+(or `"types": false`). A section the app has no definition for is skipped
+with a notice.
+
+`schema push <name>` reads `schema` from the manifest (an error when it is
+missing), fetches the dashboard's current schema and prints the formats and
+a diff of the field keys, then asks before sending (`--yes` skips the
+question, and is required when there is no terminal):
+
+```
+schema of 'hero': dashboard has field-dsl (2 fields), code has field-dsl (3 fields)
+  + added    ctaLink, subheading
+  - removed  image
+  = kept     heading
+replace the dashboard's schema of 'hero' with the code's? [y/N]
+```
+
+For a FormBuilder schema (`{ nodes: [...] }`) the keys are the node names.
+When the dashboard's schema and the manifest's are in different formats the
+server refuses with 409; the CLI prints both formats and says to rerun with
+`--force`, which replaces the schema whatever its format. A 422 (the
+`schema` is not an object of field definitions, or the definition is not
+project-scoped and the template has to be installed into the project first)
+prints the server's message. After a successful push the CLI regenerates
+`<name>.props.ts` from the stored schema. `schema push` needs `BOTLR_PAT`,
+like `push`.
 
 ## Manifest format
 

package/dist/cli.js

@@ -446,14 +446,14 @@ function parseZodSchema(source, schemaName) {
     }
   }
   if (end < 0) return null;
-  const schema = {};
+  const schema2 = {};
   for (const entry of splitTopLevel(source.slice(open, end))) {
     const m = entry.match(/^(?:"([^"]+)"|'([^']+)'|([A-Za-z_$][A-Za-z0-9_$]*))\s*:\s*([\s\S]+)$/);
     if (!m) continue;
     const field = zodFieldToSchemaField(m[4]);
-    if (field) schema[m[1] ?? m[2] ?? m[3]] = field;
+    if (field) schema2[m[1] ?? m[2] ?? m[3]] = field;
   }
-  return Object.keys(schema).length ? schema : null;
+  return Object.keys(schema2).length ? schema2 : null;
 }
 function parseImports(source) {
   const internal = /* @__PURE__ */ new Set();
@@ -488,7 +488,7 @@ async function deriveManifests(sectionsDir, options) {
     if (!entryFile) continue;
     const source = await readFile4(entryFile, "utf8");
     const meta = parseDefinition(source);
-    const schema = meta?.schemaName ? parseZodSchema(source, meta.schemaName) : null;
+    const schema2 = meta?.schemaName ? parseZodSchema(source, meta.schemaName) : null;
     const imports = parseImports(source);
     const relative = path4.relative(sectionsDir, entryFile).split(path4.sep).join("/");
     const npm = {};
@@ -510,7 +510,7 @@ async function deriveManifests(sectionsDir, options) {
         ...Object.keys(npm).length ? { npm } : {},
         ...imports.internal.length ? { internal: imports.internal } : {}
       },
-      ...schema ? { schema } : {}
+      ...schema2 ? { schema: schema2 } : {}
     };
     out.push({ name: entry.slug, manifest, entryFile });
   }
@@ -734,9 +734,9 @@ function docFor(field) {
   }
   return doc;
 }
-function collectDslFields(schema) {
+function collectDslFields(schema2) {
   const out = [];
-  for (const [name, value] of Object.entries(schema)) {
+  for (const [name, value] of Object.entries(schema2)) {
     if (!value || typeof value !== "object" || Array.isArray(value)) continue;
     const field = value;
     out.push({ name, type: tsTypeForField(field), required: field.required === true, doc: docFor(field) });
@@ -759,13 +759,13 @@ function collectFormBuilderFields(nodes) {
   }
   return out;
 }
-function collectFields(schema) {
-  if (isFormBuilderSchema(schema)) return collectFormBuilderFields(schema.nodes);
-  if (isFieldSchema(schema)) return collectDslFields(schema);
+function collectFields(schema2) {
+  if (isFormBuilderSchema(schema2)) return collectFormBuilderFields(schema2.nodes);
+  if (isFieldSchema(schema2)) return collectDslFields(schema2);
   return [];
 }
-function schemaFieldNames(schema) {
-  return collectFields(schema).map((f) => f.name);
+function schemaFieldNames(schema2) {
+  return collectFields(schema2).map((f) => f.name);
 }
 var IDENTIFIER_RE = /^[A-Za-z_$][A-Za-z0-9_$]*$/;
 function propertyKey(name) {
@@ -783,8 +783,8 @@ function propsInterfaceName(manifest) {
   const base = manifest.exportName && IDENTIFIER_RE.test(manifest.exportName) ? manifest.exportName : pascalCase(manifest.section);
   return `${base}Props`;
 }
-function generatePropsSource(schema, options) {
-  const fields = collectFields(schema);
+function generatePropsSource(schema2, options) {
+  const fields = collectFields(schema2);
   const lines = [
     `// Generated by @botlr/cli from the Botlr section definition "${options.section}".`,
     "// Do not edit: the props schema lives in the dashboard, and `botlr pull` rewrites this file.",
@@ -810,10 +810,10 @@ function generatePropsSource(schema, options) {
 function propsPath(sectionsDir, name) {
   return path6.join(sectionsDir, `${name}${PROPS_SUFFIX}`);
 }
-async function writePropsTypes(sectionsDir, manifest, schema) {
+async function writePropsTypes(sectionsDir, manifest, schema2) {
   const interfaceName = propsInterfaceName(manifest);
   const file = propsPath(sectionsDir, manifest.section);
-  const text = generatePropsSource(schema, { section: manifest.section, interfaceName });
+  const text = generatePropsSource(schema2, { section: manifest.section, interfaceName });
   try {
     if (await readFile6(file, "utf8") === text) return { file, interfaceName, changed: false };
   } catch {
@@ -900,15 +900,15 @@ import {
 } from "@botlr/sdk";
 
 // src/drift.ts
-import { isFieldSchema as isFieldSchema2 } from "@botlr/sdk";
+import { schemaFormat } from "@botlr/sdk";
 function manifestSchema(manifest) {
-  const schema = manifest.schema;
-  return isFieldSchema2(schema) ? schema : null;
+  const schema2 = manifest.schema;
+  return schemaFormat(schema2) ? schema2 : null;
 }
 function compareSchemas(manifest, definitionSchema) {
   const code = manifestSchema(manifest);
   if (!code) return null;
-  const codeKeys = new Set(Object.keys(code).filter((k) => code[k] && typeof code[k] === "object"));
+  const codeKeys = new Set(schemaFieldNames(code));
   const dashboardKeys = new Set(schemaFieldNames(definitionSchema));
   const missingInDashboard = [...codeKeys].filter((k) => !dashboardKeys.has(k)).sort();
   const missingInCode = [...dashboardKeys].filter((k) => !codeKeys.has(k)).sort();
@@ -923,7 +923,9 @@ function formatDriftWarning(name, drift) {
   if (drift.missingInCode.length) {
     lines.push(`  the dashboard defines props the code does not list: ${drift.missingInCode.join(", ")}`);
   }
-  lines.push(`  the dashboard owns the schema: edit it there, or update 'schema' in ${name}.botlr.json`);
+  lines.push(`  the dashboard owns the schema; a code push never changes it. To resolve:`);
+  lines.push(`    botlr schema pull ${name}   takes the dashboard's schema into ${name}.botlr.json`);
+  lines.push(`    botlr schema push ${name}   sends the code's schema to the dashboard`);
   return lines.join("\n");
 }
 
@@ -1093,6 +1095,149 @@ async function initSections(config, options = {}) {
   return results;
 }
 
+// src/commands/schema.ts
+import { BotlrError, BotlrSchemaFormatConflictError, botlrErrorMessage } from "@botlr/sdk";
+
+// src/schema-diff.ts
+import { schemaFormat as schemaFormat2 } from "@botlr/sdk";
+function schemaKeys(schema2) {
+  return [...new Set(schemaFieldNames(schema2))];
+}
+function diffSchemaKeys(current, next) {
+  const currentKeys = new Set(schemaKeys(current));
+  const nextKeys = new Set(schemaKeys(next));
+  return {
+    added: [...nextKeys].filter((k) => !currentKeys.has(k)).sort(),
+    removed: [...currentKeys].filter((k) => !nextKeys.has(k)).sort(),
+    kept: [...nextKeys].filter((k) => currentKeys.has(k)).sort()
+  };
+}
+function previewSchemaPush(name, current, next, formats = {}) {
+  return {
+    name,
+    current,
+    currentFormat: formats.currentFormat ?? schemaFormat2(current),
+    next,
+    nextFormat: formats.nextFormat ?? schemaFormat2(next),
+    diff: diffSchemaKeys(current, next)
+  };
+}
+function plural(n, word) {
+  return `${n} ${word}${n === 1 ? "" : "s"}`;
+}
+function formatName(format) {
+  return format ?? "unknown";
+}
+function formatSchemaDiff(preview) {
+  const { diff } = preview;
+  const header = `schema of '${preview.name}': dashboard has ${formatName(preview.currentFormat)} (${plural(schemaKeys(preview.current).length, "field")}), code has ${formatName(preview.nextFormat)} (${plural(schemaKeys(preview.next).length, "field")})`;
+  const lines = [header];
+  if (diff.added.length) lines.push(`  + added    ${diff.added.join(", ")}`);
+  if (diff.removed.length) lines.push(`  - removed  ${diff.removed.join(", ")}`);
+  lines.push(`  = kept     ${diff.kept.length ? diff.kept.join(", ") : "none"}`);
+  if (preview.currentFormat && preview.nextFormat && preview.currentFormat !== preview.nextFormat) {
+    lines.push(`  the formats differ: the push needs --force to replace ${preview.currentFormat} with ${preview.nextFormat}`);
+  }
+  if (!diff.added.length && !diff.removed.length) lines.push("  the field keys match; only field settings may change");
+  return lines.join("\n");
+}
+
+// src/commands/schema.ts
+function isNotFound(error) {
+  return error instanceof BotlrError && error.status === 404;
+}
+async function pullSchemas(config, options) {
+  const client = clientFor(config);
+  const names = options.all ? await listManifests(config.sectionsDir) : options.names;
+  const results = [];
+  for (const name of names) {
+    let manifest;
+    try {
+      manifest = await readManifest(config.sectionsDir, name);
+    } catch (error) {
+      results.push({ name, changed: false, file: "", error: error.message });
+      continue;
+    }
+    let definition;
+    try {
+      definition = (await client.sections.getSchema(name)).definition;
+    } catch (error) {
+      if (!isNotFound(error)) throw error;
+      results.push({ name, changed: false, file: "", error: `the app has no section type '${name}' (botlr push ${name} --create adds one)` });
+      continue;
+    }
+    const updated = { ...manifest, schema: definition.schema };
+    const written = await writeManifestIfChanged(config.sectionsDir, name, updated);
+    const types = config.types && !options.noTypes ? await writePropsTypes(config.sectionsDir, manifest, definition.schema) : void 0;
+    results.push({ name, changed: written.changed, file: written.file, format: definition.format, types });
+  }
+  return results;
+}
+var SchemaFormatConflict = class extends Error {
+  constructor(sectionName, currentFormat, newFormat, message) {
+    super(message ?? `the schema of '${sectionName}' is stored as ${currentFormat}, the manifest's is ${newFormat}`);
+    this.sectionName = sectionName;
+    this.currentFormat = currentFormat;
+    this.newFormat = newFormat;
+    this.name = "SchemaFormatConflict";
+  }
+  sectionName;
+  currentFormat;
+  newFormat;
+};
+var SchemaRejected = class extends Error {
+  constructor(sectionName, message) {
+    super(message);
+    this.sectionName = sectionName;
+    this.name = "SchemaRejected";
+  }
+  sectionName;
+};
+async function pushSchema(config, options) {
+  const client = clientFor(config, { needPersonalToken: true });
+  const { name } = options;
+  const manifest = await readManifest(config.sectionsDir, name);
+  const next = manifest.schema;
+  if (!next || typeof next !== "object" || Array.isArray(next)) {
+    throw new ManifestError(
+      `${name}.botlr.json has no 'schema' object to push; add one, or run 'botlr schema pull ${name}' to take the dashboard's`
+    );
+  }
+  let current;
+  try {
+    current = (await client.sections.getSchema(name)).definition;
+  } catch (error) {
+    if (!isNotFound(error)) throw error;
+    throw new ManifestError(`the app has no section type '${name}'; push the code first with 'botlr push ${name} --create'`);
+  }
+  const preview = previewSchemaPush(name, current.schema, next, { currentFormat: current.format });
+  if (options.confirm && !await options.confirm(preview)) {
+    return { name, preview, pushed: false };
+  }
+  let response;
+  try {
+    response = await client.sections.putSchema(name, { schema: next, force: options.force ?? false });
+  } catch (error) {
+    if (error instanceof BotlrSchemaFormatConflictError) {
+      throw new SchemaFormatConflict(name, error.currentFormat, error.newFormat, botlrErrorMessage(error));
+    }
+    if (error instanceof BotlrError && error.status === 422) {
+      throw new SchemaRejected(name, botlrErrorMessage(error) ?? "the server rejected the schema (422)");
+    }
+    throw error;
+  }
+  const types = config.types && !options.noTypes ? await writePropsTypes(config.sectionsDir, manifest, response.definition.schema) : void 0;
+  return {
+    name,
+    preview,
+    pushed: true,
+    changed: response.changed,
+    previousFormat: response.previousFormat,
+    format: response.definition.format,
+    types
+  };
+}
+
 // src/output.ts
 function table(rows, header) {
   const all = header ? [header, ...rows] : rows;
@@ -1110,6 +1255,7 @@ function fail(message) {
 }
 
 // src/cli.ts
+import { createInterface } from "readline/promises";
 var require2 = createRequire(import.meta.url);
 var { version } = require2("../package.json");
 function parseVersion(value) {
@@ -1133,6 +1279,17 @@ async function run(action) {
   latest on server: ${latest}
   pull first, or push --force to overwrite`);
     }
+    if (error instanceof SchemaFormatConflict) {
+      fail(
+        `${error.message}
+  dashboard: ${error.currentFormat}
+  code:      ${error.newFormat}
+  rerun with --force to replace the ${error.currentFormat} schema: botlr schema push ${error.sectionName} --force`
+      );
+    }
+    if (error instanceof SchemaRejected) {
+      fail(`the server rejected the schema of '${error.sectionName}': ${error.message}`);
+    }
     fail(error.message);
   }
 }
@@ -1256,5 +1413,67 @@ withCommon(
 review the manifests, then push each section: botlr push <name> --create`);
   })
 );
+var schema = program.command("schema").description("copy a section's props schema between the dashboard and <name>.botlr.json");
+withCommon(
+  schema.command("pull [names...]").description("write the dashboard's props schema into <name>.botlr.json as 'schema', then regenerate <name>.props.ts").option("--all", "every section with a manifest in the sections directory").option("--no-types", "do not write <name>.props.ts afterwards")
+).action(
+  (names, opts) => run(async () => {
+    if (names.length === 0 && !opts.all) fail("name a section or pass --all");
+    const config = await loadConfig(process.cwd(), overrides(opts));
+    const results = await pullSchemas(config, { names, all: opts.all, noTypes: opts.types === false });
+    if (results.length === 0) {
+      console.log(`no *.botlr.json manifests in ${config.sectionsDir}`);
+      return;
+    }
+    for (const r of results) {
+      if (r.error) {
+        console.log(`skip  ${r.name}: ${r.error}`);
+        process.exitCode = 1;
+        continue;
+      }
+      const format = r.format ? ` (${r.format})` : "";
+      if (r.changed) console.log(`pull  ${r.name}: wrote schema${format} to ${path7.relative(process.cwd(), r.file)}`);
+      else console.log(`pull  ${r.name}: schema unchanged${format}`);
+      if (r.types?.changed) console.log(`      types: wrote ${r.types.interfaceName} to ${path7.relative(process.cwd(), r.types.file)}`);
+    }
+  })
+);
+async function confirmOnTerminal(preview) {
+  console.log(formatSchemaDiff(preview));
+  if (!process.stdin.isTTY) {
+    fail(`not a terminal: rerun with --yes to push the schema of '${preview.name}' without asking`);
+  }
+  const rl = createInterface({ input: process.stdin, output: process.stderr });
+  try {
+    const answer = await rl.question(`replace the dashboard's schema of '${preview.name}' with the code's? [y/N] `);
+    return /^y(es)?$/i.test(answer.trim());
+  } finally {
+    rl.close();
+  }
+}
+withCommon(
+  schema.command("push <name>").description("send the manifest's 'schema' to the dashboard as the definition's props schema (needs BOTLR_PAT)").option("--force", "replace the schema even when the dashboard's is in the other format").option("-y, --yes", "push without asking").option("--no-types", "do not write <name>.props.ts afterwards")
+).action(
+  (name, opts) => run(async () => {
+    const config = await loadConfig(process.cwd(), overrides(opts));
+    const result = await pushSchema(config, {
+      name,
+      force: opts.force,
+      noTypes: opts.types === false,
+      confirm: opts.yes ? (preview) => {
+        console.log(formatSchemaDiff(preview));
+        return true;
+      } : confirmOnTerminal
+    });
+    if (!result.pushed) {
+      console.log(`nothing pushed`);
+      return;
+    }
+    const formats = result.previousFormat && result.format && result.previousFormat !== result.format ? ` (${result.previousFormat} -> ${result.format})` : result.format ? ` (${result.format})` : "";
+    if (result.changed) console.log(`pushed schema of ${result.name}${formats}`);
+    else console.log(`schema of ${result.name} already matches, nothing changed${formats}`);
+    if (result.types?.changed) console.log(`      types: wrote ${result.types.interfaceName} to ${path7.relative(process.cwd(), result.types.file)}`);
+  })
+);
 program.parseAsync(process.argv).catch((error) => fail(error.message));
 //# sourceMappingURL=cli.js.map