@malloy-publisher/create-malloy-package 0.0.2 → 0.0.3

package/README.md

@@ -10,10 +10,33 @@ instructions, and the Malloy agent skills copied in), so you can go from nothing
 
 ```bash
 mkdir my-data && cd my-data
-npm create @malloy-publisher/malloy-package sales
+npm create @malloy-publisher/malloy-package@latest sales
 npm start
 ```
 
+Keep the `@latest`. `npm create` and `npx` both resolve through npm's npx cache, and an
+unversioned name is satisfied by any copy already in it, so on a machine that has run
+the command before npm reuses that copy instead of asking the registry. Without
+`@latest` you can scaffold from an old scaffolder, which pins an old server, silently.
+
+Because that is silent, the scaffolder also checks for itself. After it has finished
+writing, it asks the npm registry which version is `latest` and prints a note if the one
+you are running is older. That is the only network request this package makes: a plain
+unauthenticated GET of a public package document, sending nothing about you or your
+files. It is bounded at 1.5 seconds and fails open, so no network, a proxy, or a
+registry outage costs you a moment of silence and nothing else.
+
+The request is not made at all when `CI` or `NO_UPDATE_NOTIFIER` is set, so a build
+never pays for advice nobody will read, and a machine that has already switched off
+update notices stays switched off. Anywhere else, `CREATE_MALLOY_PACKAGE_NO_UPDATE_CHECK`
+does the same thing:
+
+```bash
+CREATE_MALLOY_PACKAGE_NO_UPDATE_CHECK=1 npm create @malloy-publisher/malloy-package@latest sales
+```
+
+Set any of the three to a value other than `0` or `false`.
+
 The workspace files land in the current directory and the package in `./sales`, so you
 run `npm start` from where you created it (no need to `cd` into the package). The
 scaffolder's own last lines are the ones to keep: the app URL, the MCP endpoint, and a
@@ -48,6 +71,13 @@ whatever that file names, and the generated agent briefing spells out the URLs f
 this workspace. That briefing is `AGENTS.md`, or `AGENTS.malloy.md` in a directory
 that already had an `AGENTS.md` of its own; see "Running it again" below.
 
+A directory nobody has trusted yet is a second gate, separate from connecting the MCP
+server: Claude Code lists the `malloy_*` tools and then refuses every call, and a
+`.claude/settings.json` allowlist is discarded rather than merged. Start Claude Code
+interactively in the directory once and answer the trust prompt, which is asked once per
+directory. A headless run is never asked, so it cannot clear the gate either. You will
+know it worked when the agent's first Malloy query returns data.
+
 ## Query it
 
 The web UI at http://localhost:4000 is the quickest look. For a check you can script,
@@ -86,12 +116,13 @@ no route falls through to the web app and answers `200` with an HTML page, which
 as success until you look at the body.
 
 Agents should reach the same models through MCP rather than curl, which buys them
-schema discovery, compile checks and a reload that needs no restart. The generated
-briefing lists those tools.
+schema discovery and compile checks. The generated briefing lists those tools. It also
+gives the REST route that recompiles a package after an edit, for an agent with nobody
+around to reconnect its MCP client.
 
 ## What it creates
 
-Running `npm create @malloy-publisher/malloy-package sales` in an empty directory produces:
+Running `npm create @malloy-publisher/malloy-package@latest sales` in an empty directory produces:
 
 ```
 publisher.config.json    the server config, with your package registered
@@ -282,7 +313,7 @@ and binds `0.0.0.0`. Write the flag and the address with a space between them.
 ## Options
 
 ```bash
-npm create @malloy-publisher/malloy-package [name] -- [options]
+npm create @malloy-publisher/malloy-package@latest [name] -- [options]
 ```
 
 `npm create` parses the command line with npm's own config parser before handing
@@ -291,26 +322,26 @@ it npm swallows `--force` as one of its own settings and turns `--data mydata.cs
 into two stray positional arguments:
 
 ```bash
-npm create @malloy-publisher/malloy-package sales -- --data mydata.csv
-npm create @malloy-publisher/malloy-package sales -- --client cursor
-npm create @malloy-publisher/malloy-package sales -- --force
+npm create @malloy-publisher/malloy-package@latest sales -- --data mydata.csv
+npm create @malloy-publisher/malloy-package@latest sales -- --client cursor
+npm create @malloy-publisher/malloy-package@latest sales -- --force
 ```
 
 Running the published bin directly takes the flags as-is, with no separator:
 
 ```bash
-npx @malloy-publisher/create-malloy-package sales --data mydata.csv
+npx @malloy-publisher/create-malloy-package@latest sales --data mydata.csv
 ```
 
 - `name` (positional): the package name. Omit it to only set up the agent workspace in
   the current directory (write the MCP connection, agent instructions, and skills)
   without scaffolding a package.
-- `--data <file>`: seed the package from your own CSV, Parquet, or XLSX file instead of
-  the built-in sample. The file is copied into the package and the starter model points
-  at it. DuckDB reads all three formats in place; an Excel file is read as its first
-  sheet. It seeds a new package, so it requires a package name: it cannot be combined
-  with the setup-only mode above, and passing it without a name is an error rather than
-  a silently ignored flag.
+- `--data <file>`: seed the package from your own CSV, Parquet, JSON, NDJSON, or XLSX
+  file instead of the built-in sample. The file is copied into the package and the
+  starter model points at it. DuckDB reads all of them in place, so nothing needs
+  converting first; an Excel file is read as its first sheet. It seeds a new package,
+  so it requires a package name: it cannot be combined with the setup-only mode above,
+  and passing it without a name is an error rather than a silently ignored flag.
 - `--client <claude-code|cursor>`: which agent client to wire up. Defaults to
   `claude-code`. `AGENTS.md` and the skills in `.claude/skills/` are written for every
   client; the MCP config file (`.mcp.json` for Claude Code, `.cursor/mcp.json` for

package/dist/index.js

@@ -246,6 +246,135 @@ function toMalloyIdentifier(name) {
   return identifier;
 }
 
+// src/registry_check.ts
+import * as http from "node:http";
+import * as https from "node:https";
+var LATEST_URL = "https://registry.npmjs.org/@malloy-publisher/create-malloy-package/latest";
+var TIMEOUT_MS = 1500;
+var MAX_BODY_BYTES = 1e6;
+var OPT_OUT_ENV = [
+  "CREATE_MALLOY_PACKAGE_NO_UPDATE_CHECK",
+  "CI",
+  "NO_UPDATE_NOTIFIER"
+];
+function updateCheckDisabled(env = process.env) {
+  return OPT_OUT_ENV.some((name) => {
+    const value = env[name]?.trim().toLowerCase();
+    return value !== undefined && value !== "" && value !== "0" && value !== "false";
+  });
+}
+function parseVersion(v) {
+  const m = /^(\d+)\.(\d+)\.(\d+)$/.exec(v.trim());
+  if (!m)
+    return;
+  return [Number(m[1]), Number(m[2]), Number(m[3])];
+}
+function isOlder(a, b) {
+  const left = parseVersion(a);
+  const right = parseVersion(b);
+  if (!left || !right)
+    return;
+  for (let i = 0;i < 3; i++) {
+    if (left[i] < right[i])
+      return true;
+    if (left[i] > right[i])
+      return false;
+  }
+  return false;
+}
+function getBody(url, timeoutMs, onCancel) {
+  return new Promise((resolve) => {
+    let settled = false;
+    let deadline;
+    const finish = (value) => {
+      if (settled)
+        return;
+      settled = true;
+      if (deadline !== undefined)
+        clearTimeout(deadline);
+      resolve(value);
+    };
+    try {
+      const mod = url.startsWith("http:") ? http : https;
+      const req = mod.get(url, { timeout: timeoutMs, headers: { accept: "application/json" } }, (res) => {
+        if (res.statusCode !== 200) {
+          res.resume();
+          finish(undefined);
+          return;
+        }
+        let body = "";
+        let bytes = 0;
+        res.setEncoding("utf8");
+        res.on("data", (chunk) => {
+          body += chunk;
+          bytes += Buffer.byteLength(chunk, "utf8");
+          if (bytes > MAX_BODY_BYTES) {
+            req.destroy();
+            finish(undefined);
+          }
+        });
+        res.on("end", () => finish(body));
+        res.on("error", () => finish(undefined));
+      });
+      deadline = setTimeout(() => {
+        req.destroy();
+        finish(undefined);
+      }, timeoutMs);
+      req.on("timeout", () => {
+        req.destroy();
+        finish(undefined);
+      });
+      req.on("error", () => finish(undefined));
+      onCancel(() => {
+        req.destroy();
+        finish(undefined);
+      });
+    } catch {
+      finish(undefined);
+    }
+  });
+}
+async function fetchLatestVersion(url = LATEST_URL, timeoutMs = TIMEOUT_MS, onCancel = () => {}) {
+  if (updateCheckDisabled())
+    return;
+  const body = await getBody(url, timeoutMs, onCancel);
+  if (body === undefined)
+    return;
+  try {
+    const parsed = JSON.parse(body);
+    if (typeof parsed !== "object" || parsed === null)
+      return;
+    const version = parsed.version;
+    return typeof version === "string" ? version.trim() : undefined;
+  } catch {
+    return;
+  }
+}
+function startVersionCheck(url = LATEST_URL, timeoutMs = TIMEOUT_MS) {
+  let cancel = () => {};
+  const result = fetchLatestVersion(url, timeoutMs, (fn) => {
+    cancel = fn;
+  });
+  return { result, cancel: () => cancel() };
+}
+function staleScaffolderWarning(running, latest) {
+  if (!latest)
+    return;
+  if (isOlder(running, latest) !== true)
+    return;
+  return [
+    `This is create-malloy-package ${running}, and npm has ${preview(latest)}.`,
+    "",
+    "The workspace above was written by the older one, so the server version it",
+    "pins is whatever that release pinned. If you did not choose this version on",
+    "purpose, name it explicitly and scaffold again:",
+    "  npm create @malloy-publisher/malloy-package@latest <name>",
+    "",
+    "(Set CREATE_MALLOY_PACKAGE_NO_UPDATE_CHECK=1 to skip this check entirely.)"
+  ].join(`
+`);
+}
+
 // src/scaffold.ts
 import * as fs4 from "node:fs";
 import * as path3 from "node:path";
@@ -614,7 +743,7 @@ function createPackage(options, result) {
 ` + forceDescription(name, modelFile, options.host) + `
 
 ` + `If you passed --force to \`npm create\`, npm read it as one of its ` + `own settings and it never reached this tool. Options need a \`--\` ` + `in front of them there:
-` + `  npm create @malloy-publisher/malloy-package ${name} -- --force`);
+` + `  npm create @malloy-publisher/malloy-package@latest ${name} -- --force`);
   }
   const dataDir = path3.join(packageDir, "data");
   assertWithinWorkspace(dataDir, options.cwd, `${name}/data`);
@@ -630,7 +759,11 @@ function createPackage(options, result) {
     if (destBase !== sourceBase) {
       result.dataFileRenamedFrom = sourceBase;
     }
-    writeFile(path3.join(packageDir, modelFile), renderTemplate("model.custom.malloy", { sourceName, dataPath }), options.cwd);
+    const siblings = findSiblingDataFiles(options.dataFile);
+    if (siblings.length > 0) {
+      result.siblingDataFiles = siblings;
+    }
+    writeFile(path3.join(packageDir, modelFile), renderTemplate(isSpreadsheet(dataPath) ? "model.custom.xlsx.malloy" : "model.custom.malloy", { sourceName, dataPath }), options.cwd);
   } else {
     copyFile(path3.join(templatesDir, "sales.csv"), path3.join(dataDir, "sales.csv"), options.cwd, `${name}/data/sales.csv`);
     dataPath = "data/sales.csv";
@@ -1160,6 +1293,36 @@ function describeMcpProblem(parses, parsed, config) {
 function renderAgentsFile(result, host, envPackages) {
   const mcpNote = !result.mcpWired ? `\`${result.mcpConfigPath}\` could not be updated automatically, so the \`malloy\` server may still need adding to it by hand.` : host === "cursor" ? "This workspace ships a `.cursor/mcp.json` that points Cursor at the endpoint." : "This workspace ships a `.mcp.json`, so Claude Code offers to connect on first run.";
   const reconnectNote = host === "cursor" ? "ask the user to reload the MCP servers from Cursor's settings (the `malloy` server, then Refresh), or to restart Cursor." : "ask the user to run `/mcp`, select the `malloy` server, and choose Reconnect, or to restart Claude Code. That panel reports `Auth: not authenticated` and offers `Authenticate` first, which is a red herring: this endpoint has no auth, and Reconnect is the one that works.";
+  const registrationNote = host === "cursor" ? "" : [
+    "",
+    "If they cannot, ask them to register the server so the directory stops deciding:",
+    "",
+    "```bash",
+    `claude mcp add --transport http malloy http://localhost:${result.mcpPort}/mcp -s user`,
+    "```",
+    "",
+    "That registration is stored per user rather than per project, so the tools follow the",
+    "agent everywhere. Skills need no such escape hatch: they are rescanned as the working",
+    "directory changes, so a session started further up picks them up once work moves into",
+    "this directory.",
+    ""
+  ].join(`
+`);
+  const trustNote = host === "cursor" ? "" : [
+    "",
+    "One gate can void the reconnect fix below, and you cannot clear it yourself: if the",
+    "`malloy_*` tools are listed but every call is refused, the usual cause is that this",
+    "workspace has not been trusted, which also discards a `.claude/settings.json` allowlist.",
+    "A human has to start Claude Code interactively in this directory once and answer the",
+    "trust prompt; a non-interactive run is never asked, so it cannot clear the gate either.",
+    "Tell the user and leave the choice to them. You will know the prompt was answered when a",
+    "`malloy_*` call returns data instead of being refused. If that does not change it, the",
+    "other cause is a `malloy` server whose own approval was never given, which the user",
+    "clears in the same `/mcp` panel named below. If the tools are not listed at all, this",
+    "gate is not the cause, so read on.",
+    ""
+  ].join(`
+`);
   const skillsCount = String(result.skillsInstalled);
   const skillsNote = result.skillsInstalled === 0 ? "This run installed no skills into `.claude/skills/` (the scaffolder's output says why), so there are none here to load. Pull the same guidance as MCP prompts from the endpoint above instead." : host === "cursor" ? `\`.claude/skills/\` holds ${skillsCount} Malloy agent skills as real files. Cursor reads \`AGENTS.md\`; these skills are the same guidance broken out by task, and any MCP client can also pull them as prompts from the endpoint above.` : `\`.claude/skills/\` holds ${skillsCount} Malloy agent skills as real files, so Claude Code auto-discovers them. Hosts that read \`AGENTS.md\` rather than Anthropic Agent Skills can pull the same guidance as MCP prompts from the endpoint above.`;
   const startCommand = result.hasStartScript ? "npm start" : result.startCommand;
@@ -1176,6 +1339,8 @@ function renderAgentsFile(result, host, envPackages) {
     packageSection: packageSection(result, envPackages),
     mcpNote,
     reconnectNote,
+    registrationNote,
+    trustNote,
     skillsNote
   });
 }
@@ -1299,14 +1464,36 @@ function claudeFile(agentsFile) {
 This directory already had an AGENTS.md when the Malloy workspace was ` + `scaffolded, so that file was left exactly as it was and carries none of ` + `the above. ${agentsFile} is the generated briefing; read that one for ` + `anything about Publisher, Malloy, or the MCP endpoint.
 `;
 }
+var LOADABLE_DATA_EXTENSIONS = [
+  ".csv",
+  ".parquet",
+  ".json",
+  ".ndjson",
+  ".xlsx"
+];
+function isLoadableDataFile(file) {
+  return LOADABLE_DATA_EXTENSIONS.includes(path3.extname(file).toLowerCase());
+}
+function isSpreadsheet(dataPath) {
+  return path3.extname(dataPath).toLowerCase() === ".xlsx";
+}
 function validateDataFile(dataFile) {
   if (!fs4.existsSync(dataFile) || !fs4.statSync(dataFile).isFile()) {
     throw new ScaffoldError(`--data file not found: ${printable(dataFile)}`);
   }
-  const ext = path3.extname(dataFile).toLowerCase();
-  if (ext !== ".csv" && ext !== ".parquet" && ext !== ".xlsx") {
-    throw new ScaffoldError(`--data must be a .csv, .parquet, or .xlsx file (got "${printable(path3.basename(dataFile))}").`);
+  if (!isLoadableDataFile(dataFile)) {
+    throw new ScaffoldError(`--data must be one of ${LOADABLE_DATA_EXTENSIONS.join(", ")} (got "${printable(path3.basename(dataFile))}").`);
+  }
+}
+function findSiblingDataFiles(dataFile) {
+  let entries;
+  try {
+    entries = fs4.readdirSync(path3.dirname(dataFile));
+  } catch {
+    return [];
   }
+  const chosen = path3.basename(dataFile);
+  return entries.filter((e) => e !== chosen && isLoadableDataFile(e)).sort().map(printable);
 }
 function toNpmName(base) {
   const cleaned = base.toLowerCase().replace(/[^a-z0-9._-]/g, "-").replace(/^[._]+/, "");
@@ -1357,12 +1544,13 @@ function resolveDataFile(cwd, data) {
     return;
   }
   if (data.trim() === "") {
-    throw new ScaffoldError(`--data was given an empty filename. Pass the CSV, Parquet, or XLSX ` + `file to seed the package from, or leave --data off to use the ` + `sample data.`);
+    throw new ScaffoldError(`--data was given an empty filename. Pass the CSV, Parquet, JSON, NDJSON, or XLSX ` + `file to seed the package from, or leave --data off to use the ` + `sample data.`);
   }
   return path4.resolve(cwd, data);
 }
-function run(name, options) {
+async function run(name, options) {
   const cwd = process.cwd();
+  const versionCheck = startVersionCheck();
   const before = snapshotArtifacts(cwd, name);
   try {
     const result = scaffold({
@@ -1373,7 +1561,14 @@ function run(name, options) {
       force: Boolean(options.force)
     });
     process.stdout.write(formatSuccess(result));
+    const warning = staleScaffolderWarning(version(), await versionCheck.result);
+    if (warning) {
+      process.stdout.write(`
+${yellow(warning)}
+`);
+    }
   } catch (err) {
+    versionCheck.cancel();
     process.stderr.write(formatFailure(err, changesSince(cwd, name, before), Boolean(options.force)));
     process.exitCode = 1;
   }
@@ -1664,6 +1859,11 @@ function formatSuccess(result) {
   if (result.dataFileRenamedFrom) {
     lines.push(dim(`  ${result.dataFileRenamedFrom} was copied in as ${result.dataPath}, ` + `renamed to the characters a Malloy table path can carry.`));
   }
+  if (result.siblingDataFiles) {
+    const shown2 = result.siblingDataFiles.slice(0, 5);
+    const rest = result.siblingDataFiles.length - shown2.length;
+    lines.push(dim(`  Not included: ${shown2.join(", ")}${rest > 0 ? ` and ${rest} more` : ""}. --data takes one file; copy any others into ` + `${result.packageName}/data/ and add a source for each.`));
+  }
   if (result.configExtended) {
     lines.push(`${green("✓")} Registered ${bold(result.packageName)} in the existing publisher.config.json`);
   }
@@ -1699,12 +1899,20 @@ function formatSuccess(result) {
       lines.push(`  ${printable(item)}`);
     }
   }
+  if (result.dataPath !== undefined && isSpreadsheet(result.dataPath)) {
+    lines.push("");
+    lines.push(yellow("Spreadsheets need one check first:"));
+    lines.push(`  Run the ${cyan("overview")} view and compare its row count against what you`);
+    lines.push(`  know is in ${printable(result.dataPath)}. If it comes back far too small, the header`);
+    lines.push("  is not on the first row and the model is reading a title or a banner");
+    lines.push(`  instead of your data. ${printable(result.modelFile ?? "the model file")} carries the fix inline.`);
+  }
   const url = `http://localhost:${result.publisherPort}`;
   const environmentIsEmpty = result.envPackageCount === 0;
   lines.push("");
   lines.push(bold("Next steps:"));
   if (environmentIsEmpty) {
-    lines.push(`  ${cyan("npx @malloy-publisher/create-malloy-package <name>")}   ${dim("add a package; the server has nothing to serve without one")}`);
+    lines.push(`  ${cyan("npx @malloy-publisher/create-malloy-package@latest <name>")}   ${dim("add a package; the server has nothing to serve without one")}`);
   }
   if (result.needsReset) {
     const served = `re-read the config, so ${result.packageName ?? "the new package"} is served`;
@@ -1746,6 +1954,9 @@ function formatSuccess(result) {
   lines.push(`  ${cyan(url)}   ${dim("explore in the browser")}`);
   lines.push("");
   lines.push(bold("Check it is ready:"));
+  lines.push(dim(`  In a second terminal: the boot command above keeps running until you
+` + `  stop it. Until the server is up these print nothing at all, not an
+` + "  error, because -s silences curl's connection failure."));
   lines.push(`  curl -s ${url}/api/v0/status`);
   const packagesUrl = `${url}/api/v0/environments/${result.envName}/packages`;
   if (environmentIsEmpty) {
@@ -1786,8 +1997,28 @@ function formatSuccess(result) {
       }
     }
   }
-  lines.push(dim(`  An agent that starts the server itself can't reconnect MCP in that
-` + `  session; ask the user to reconnect it. See ${result.agentsFile}.`));
+  const trustNamed = result.host !== "cursor";
+  if (trustNamed) {
+    lines.push(yellow(`  Trust this workspace: a human has to open Claude Code here interactively
+` + `  once and answer the trust prompt. Until then the malloy tools can report
+` + `  connected and still refuse every call, and a permissions allowlist in
+` + `  .claude/settings.json is ignored. You will know the prompt was answered
+` + "  when a malloy tool returns data instead of refusing."));
+    lines.push("");
+  }
+  lines.push(dim((trustNamed ? `  A different problem: an agent that starts the server itself can't
+  reconnect MCP in that session` : `  An agent that starts the server itself can't reconnect MCP in that
+  session`) + `; ask the user to reconnect it. See ${result.agentsFile}.`));
+  if (trustNamed) {
+    lines.push(dim(`  ${result.mcpConfigPath} is only picked up by an agent session that
+` + `  STARTED in this directory. Launch your agent from here. If you cannot,
+` + `  register the server so the directory stops mattering (.claude/skills
+` + "  needs nothing: it is rescanned as the working directory changes):"));
+    lines.push(`    ${cyan(`claude mcp add --transport http malloy http://localhost:${result.mcpPort}/mcp -s user`)}`);
+  } else {
+    lines.push(dim(`  ${result.mcpConfigPath} is only picked up by an agent session that
+` + "  STARTED in this directory. Launch your agent from here."));
+  }
   lines.push("");
   return lines.join(`
 `) + `
@@ -1806,7 +2037,7 @@ function startNote(result, environmentIsEmpty) {
   return "start Publisher (web UI + MCP) with this workspace's packages in watch mode";
 }
 function excessArgumentsHint() {
-  const npmCreateFix = `  ${cyan("npm create @malloy-publisher/malloy-package sales -- --data mydata.csv")}`;
+  const npmCreateFix = `  ${cyan("npm create @malloy-publisher/malloy-package@latest sales -- --data mydata.csv")}`;
   if (process.env.npm_command === "init") {
     return [
       "",
@@ -1826,13 +2057,13 @@ function excessArgumentsHint() {
     "",
     "With npx there is no separator: it forwards the flags as they are, and a",
     "-- would be passed through and counted as another argument.",
-    `  ${cyan("npx @malloy-publisher/create-malloy-package sales --data mydata.csv")}`,
+    `  ${cyan("npx @malloy-publisher/create-malloy-package@latest sales --data mydata.csv")}`,
     ""
   ].join(`
 `);
 }
 var program = new Command;
-program.name("create-malloy-package").description("Scaffold a Malloy Publisher package and a local agent workspace.").version(version(), "-v, --version").argument("[name]", "package name to create; omit to only wire the agent workspace into the current directory").option("--data <file>", "seed the package from a CSV, Parquet, or XLSX file").addOption(new Option("--client <client>", "agent client to wire up").choices(CLIENTS).default("claude-code")).option("--force", "overwrite existing workspace files instead of keeping them").allowExcessArguments(false).exitOverride((err) => {
+program.name("create-malloy-package").description("Scaffold a Malloy Publisher package and a local agent workspace.").version(version(), "-v, --version").argument("[name]", "package name to create; omit to only wire the agent workspace into the current directory").option("--data <file>", "seed the package from a CSV, Parquet, JSON, NDJSON, or XLSX file").addOption(new Option("--client <client>", "agent client to wire up").choices(CLIENTS).default("claude-code")).option("--force", "overwrite existing workspace files instead of keeping them").allowExcessArguments(false).exitOverride((err) => {
   if (err.code === "commander.excessArguments") {
     process.stderr.write(excessArgumentsHint());
   }
@@ -1844,8 +2075,8 @@ The flag that picks the agent client is ${cyan("--client")} (claude-code or curs
 `);
   }
   process.exit(err.exitCode);
-}).action((name, options) => {
-  run(name, options);
+}).action(async (name, options) => {
+  await run(name, options);
 });
 function startedAsProgram() {
   const entry = process.argv[1];
@@ -1859,7 +2090,10 @@ function startedAsProgram() {
   }
 }
 if (startedAsProgram()) {
-  program.parse();
+  program.parseAsync().catch((err) => {
+    process.stderr.write(formatFailure(err, { created: [], changed: [] }, false));
+    process.exitCode = 1;
+  });
 }
 export {
   snapshotArtifacts,

package/package.json

@@ -1,7 +1,7 @@
 {
    "name": "@malloy-publisher/create-malloy-package",
    "description": "Scaffold a Malloy Publisher package and a local agent workspace, so one command takes you from nothing to an agent that can query your data.",
-   "version": "0.0.2",
+   "version": "0.0.3",
    "license": "MIT",
    "type": "module",
    "engines": {
@@ -45,7 +45,7 @@
       "test:e2e": "bun test --timeout 180000 tests/e2e"
    },
    "dependencies": {
-      "@malloy-publisher/skills": "^0.1.1",
+      "@malloy-publisher/skills": "^0.1.2",
       "commander": "^12.1.0"
    },
    "devDependencies": {

package/templates/AGENTS.md

@@ -13,15 +13,24 @@ Everything below talks to a running server, so start it before anything else:
 {{startCommand}}
 ```
 
-That is meant to run Publisher on http://localhost:{{port}} (web UI and REST) with the
-MCP endpoint on http://localhost:{{mcpPort}}/mcp, and to mount this workspace's
-packages in watch mode so model edits recompile. All three of those come from flags in
-the command above (`--port`, `--mcp_port`, `--watch-env`), and where that command is an
-`npm` script, from the script `package.json` runs for it. Read them there before you
-trust anything below: different ports move every URL in this file, a `--server_root` or
-`--config` pointing outside this directory serves a different workspace entirely, and
-no `--watch-env` means an edit on disk is not picked up at all until you reload the
-package. The server settles all of it on boot, printing
+That is *meant* to run Publisher on http://localhost:{{port}} (web UI and REST) with
+the MCP endpoint on http://localhost:{{mcpPort}}/mcp, and to mount this workspace's
+packages in watch mode so model edits recompile. Meant to, because only some of that
+is written in the command, and this paragraph cannot see what yours says. Go and
+read it, and where it is an `npm` script, read the script `package.json` runs for it.
+Five flags decide whether the rest of this file is true:
+
+- `--port` and `--mcp_port` are absent from a command that uses the defaults above,
+  which is the usual case. Where either one is present, it moves every URL here.
+- `--watch-env` is not a default, so watch mode is on only if you can see it. Without
+  it, an edit on disk is not picked up at all until you reload the package.
+- `--server_root` and `--config` say which workspace is being served. Pointing either
+  one outside this directory serves a different workspace entirely, and then nothing
+  below describes what you are talking to.
+- `--host` decides whether the endpoint is reachable from off this machine. That one
+  has a section of its own further down; read it before you assume this is private.
+
+The server settles all of it on boot, printing
 `Publisher server listening at http://<address>:<port>` and a
 `MCP server listening at ...` line beside it, so read those two lines rather than
 taking this paragraph's word for it.
@@ -32,9 +41,9 @@ returning the old model's rows; it may instead make the source vanish, so every
 fails with `Reference to undefined object '<source>'`. Either way the failure goes only
 to the server's stdout, which you are not reading if you started it in the background.
 So a query that succeeds after an edit is not proof the edit compiled, and the rows you
-get back may be the old model's. Compile-check the edit with `malloy_compile`, or
-reload with `malloy_reloadPackage` afterwards (both described below); either one
-reports the failure that watch mode swallowed.
+get back may be the old model's. Compile-check the edit with `malloy_compile`, or reload with
+`malloy_reloadPackage` afterwards (both described below, and reload has a REST form for a
+script or an unattended run); either one reports the failure that watch mode swallowed.
 
 Poll until it reports serving rather than assuming a fixed wait; the first run
 downloads the server, so it can take a minute:
@@ -136,9 +145,13 @@ the model file's own errors too),
 watch-mode recompile that failed), and
 `malloy_searchDocs`. {{mcpNote}}
 
-REST, for a script or a check that does not need an agent: every model is queryable
-at `POST /api/v0/environments/<env>/packages/<package>/models/<model>/query`. This
-lists what the server has loaded:
+REST, for a script or a check that does not need an agent: every model is queryable at
+`POST /api/v0/environments/<env>/packages/<package>/models/<model>/query`, and after a
+model-file edit `GET /api/v0/environments/<env>/packages/<package>?reload=true` recompiles
+the package and comes back 424 with the compile errors when it does not compile. Any other
+non-2xx is a failed check too, and its `message` says what went wrong: a 404 for a package
+name this server does not serve, a 503 for a package it could not load at all. This lists
+what the server has loaded:
 
 ```bash
 curl -s http://localhost:{{port}}/api/v0/environments/{{envName}}/packages
@@ -149,16 +162,23 @@ package, the one the scaffolder created on the run that wrote this file. A works
 scaffolded more than once serves more than that, so take the packages, sources and
 views you do not see here from the list above and from `malloy_getContext`, never
 from their absence here.
-
+{{trustNote}}
 If you started this server yourself in this session, your `malloy_*` tools will not
 appear however long you wait: an MCP client fixes its tool list when it connects, so
 it never saw a server that did not yet exist. You cannot reconnect yourself. Say so,
 and {{reconnectNote}} Do not quietly switch to curl instead and call it done: it
 looks like it is working while hiding a problem the user can clear in seconds, and
-it gives up the grounded discovery, compile checks, and reload the tools exist to
-provide. If the user would rather keep going without reconnecting, the REST endpoint
-above runs the same models.
-
+it gives up the grounded discovery and compile checks the tools exist to provide. With
+nobody there to reconnect you, REST is the supported interface rather than a workaround,
+and the reload form above is the same check.
+
+There is a second reason the tools can be missing, and it looks identical from here:
+this workspace's MCP config is only discovered by an agent session that **started in
+this directory**. A session rooted anywhere else, a parent directory included, never
+sees the server, and no message anywhere reports it. The two causes need different
+fixes, so tell them apart before acting. If the server is running and the tools were
+never offered at all, ask the user to relaunch the agent from this directory.
+{{registrationNote}}
 ## Skills ({{skillsCount}} installed)
 
 {{skillsNote}}

package/templates/model.custom.xlsx.malloy

@@ -0,0 +1,147 @@
+// {{sourceName}}: a starter Malloy model over {{dataPath}}, served by Publisher.
+//
+// Check the row count before you trust this. A spreadsheet is only a table when
+// its first row is the header and every row under it is data. Plenty are not: a
+// report title, a "generated on" banner, a merged group header, or a blank line
+// above the header are all ordinary in an export, and none of them is visible
+// from here. DuckDB takes the first row it finds as the column names and stops
+// at the first blank row, so a workbook with thousands of rows can load as a
+// handful, or as one, and nothing anywhere reports a problem: the package loads,
+// the server says it is serving, and the query returns 200 with the wrong number.
+//
+// So run `overview` first and compare `record_count` against what you know is in
+// the file. If it is short, the read is wrong. Fixing it takes two steps.
+//
+// STEP 1: look at the sheet, because the fix needs three things this file cannot
+// tell you: which sheet, which row the header is on, and which column marks a
+// real row. Add this to the bottom of THIS FILE, reload the package, then run
+// `{{sourceName}}_probe -> peek`. Delete it once you have what you need.
+//
+//   source: {{sourceName}}_probe is duckdb.sql("""
+//     SELECT * FROM read_xlsx(
+//       '{{dataPath}}',
+//       header = false, range = 'A1:Z25', all_varchar = true
+//     )
+//   """) extend {
+//     view: peek is { select: *; limit: 25 }
+//   }
+//
+// It has to go in the file rather than being sent as a one-off query: Publisher
+// refuses raw SQL in an ad-hoc query ("`duckdb.sql(...)` cannot be used in a
+// restricted query"), and only allows it where it is part of a model.
+//
+// `peek` prints the first 25 rows exactly as they sit in the sheet, banner rows
+// and all, so you can count down to the real header row and see which column
+// carries an id.
+//
+// It does NOT tell you which sheet you are looking at. With no `sheet =`,
+// read_xlsx reads the first one and says nothing about it, so a workbook whose
+// first sheet is a cover page or a notes tab hands you a plausible-looking
+// answer about the wrong sheet. There is no function that lists the sheets, so
+// if the workbook has more than one tab, take the name from the tabs along the
+// bottom in Excel and pass `sheet = '...'`.
+//
+// A name that does not exist fails loudly, which is safe, but do not use it to
+// go fishing: the error suggests at most one sheet and it picks by spelling, so
+// it can name the tab nearest your typo rather than the one holding your data.
+//
+// STEP 2: read it properly. Name the sheet and the range so the header is found,
+// AND filter the decoration back out, because an explicit range reads every row
+// in it: the blank spacers, the subtotal rows, the total, and the footnotes
+// underneath. Replace the source below, keeping the `extend` block so `overview`
+// still exists to re-run:
+//
+//   source: {{sourceName}} is duckdb.sql("""
+//     SELECT * FROM read_xlsx(
+//       '{{dataPath}}',
+//       sheet = 'Sheet1',        -- EDIT: the sheet from step 1 (or drop this line
+//                                -- entirely if the workbook has only one)
+//       header = true,
+//       range = 'A5:Z100000',    -- EDIT BOTH BOUNDS: A5 is where the real header
+//                                -- is; Z and 100000 are a cap, not a guess at your
+//                                -- size. Columns past Z and rows past 100000 are
+//                                -- dropped silently, which is the failure above
+//                                -- all over again.
+//       all_varchar = true       -- see the mixed-column note below
+//     )
+//     WHERE "Order ID" LIKE 'SO-%'   -- EDIT: a column every real row has and the
+//                                    -- decoration does not, and what a real value
+//                                    -- looks like. An id is the usual choice.
+//   """) extend {
+//     measure: record_count is count()
+//     view: overview is { aggregate: record_count }
+//   }
+//
+// Then reload the package, check the reload itself came back OK, and only then
+// run `overview` again. A reload that fails to compile leaves the previously
+// compiled model serving, so `overview` answers 200 with the OLD number and it
+// looks as though your edit did nothing. The reason is in the reload response,
+// not in the query.
+//
+// If no column is unique to real rows, exclude the labels instead of matching
+// them, and check the count both ways rather than trusting either:
+//
+//   WHERE "Region" IS NOT NULL AND "Region" NOT IN ('Subtotal', 'Total')
+//
+// Getting this wrong is quiet: a filter that also admits the subtotal rows raises
+// `record_count` to something plausible and double-counts every measure built on
+// it. That is the same failure this whole comment is about, so re-run `overview`
+// and check the number, do not assume.
+//
+// Both halves matter. Without the range you get the title row as your header.
+// Without the filter you get the whole range, padded with empty rows.
+// Do not reach for `stop_at_empty`: it stops at the first blank spacer, which is
+// usually somewhere in the middle of the data.
+//
+// `duckdb.table()` takes a plain path and nothing else, which is why every option
+// above needs the SQL form. Other things a real spreadsheet does: a column mixing
+// numbers and text fails at query time rather than at load, which is what
+// `all_varchar = true` avoids; a cell someone typed as text, `$1,234` or `12%`,
+// stays text even where the column looks numeric; dates may be text or Excel
+// serial numbers; and every number is a double. `skill:malloy-gotchas-modeling`
+// has the full list.
+//
+// Note that `all_varchar = true` hands you every column as a string, so a measure
+// cannot just sum it. Do the conversion in the SQL above, where you already are,
+// rather than in the measure:
+//
+//   SELECT
+//     *,
+//     try_cast(regexp_replace("Units Sold", '[^0-9.-]', '', 'g') AS DOUBLE) AS units_sold_num
+//   FROM read_xlsx(...)
+//
+// then `measure: total_units is units_sold_num.sum()` in Malloy. `try_cast` is
+// what makes it survive: a real export has cells reading `N/A` or `12 (est)`, and
+// a plain cast throws on the first one at query time. Note the regex keeps the
+// digits out of `12 (est)`, so those rows count as 12; strip the whole cell
+// instead if an estimate should not count.
+//
+// Keep the `-` in that character class. Without it every negative comes back
+// POSITIVE: a refund of `-500` sums as `500`, and a revenue total that quietly
+// adds its credits back is the same class of silent wrongness as the bad read
+// above. Accounting parentheses need a step of their own, because stripping them
+// also leaves a positive:
+//
+//   (CASE WHEN "Amount" LIKE '(%)' THEN -1 ELSE 1 END)
+//     * try_cast(regexp_replace("Amount", '[^0-9.]', '', 'g') AS DOUBLE) AS amount_num
+//
+// The Malloy-side form is `sum(amount::number)`, not `amount::number.sum()`,
+// which does not parse. It is fine on a clean column and throws on a dirty one,
+// which is why the SQL layer is the better place.
+// (`ignore_errors = true` on read_xlsx is the other way: it keeps real types and
+// nulls the cells that do not fit.)
+//
+// Beyond that this is the same starting point as any seeded package: the
+// scaffolder does not read your columns, so it begins with a row count. Build it
+// out for your columns: group_by a column to break the count down, and define
+// measures over the numeric ones. Validate edits with malloy_compile and run new
+// views by name with malloy_reloadPackage. The malloy-modeling skill walks
+// through turning a raw table into a semantic model.
+
+source: {{sourceName}} is duckdb.table('{{dataPath}}') extend {
+  measure: record_count is count()
+
+  view: overview is {
+    aggregate: record_count
+  }
+}