@hobocode/thought-layer 0.4.2 → 0.6.0

package/README.md

@@ -17,7 +17,7 @@ This is open source and BYOK by design. The point is to help people build real t
 - **thought-layer-prd.** Draft the complete PRD — with a first-cut domain glossary and testable requirements — from the validated idea and business model. The plan the grill then hardens.
 - **thought-layer-grill.** The last design step: grills the draft PRD against the domain one question at a time, sharpening the glossary and hardening the requirements inline until it is build-ready. Runs after the PRD, not instead of the framework.
 - **thought-layer-naming.** Name the thing, with rationale and domain-ready slugs.
-- **thought-layer-build.** Build the hardened PRD into a static-first, deploy-ready artifact, verified to run, and leave a manifest the deploy step reads.
+- **thought-layer-build.** Build the hardened PRD into a static-first, deploy-ready artifact, verified to run, and leave a manifest the deploy step reads. When a requirement genuinely needs a server, it also emits a real backend (serverless functions, a `schema.sql`, a names-only `.env.example`, and a `BACKEND.md` guide), with Neon Postgres as the documented default.
 - **thought-layer-deploy.** Take the build live to a URL you own, with no lock-in: a Netlify token deploys into your own account, or the Netlify CLI handles a logged-in or anonymous deploy.
 - **thought-layer-speedrun.** A fast, unranked path to a build-ready spec when you do not need the full panel and score.
 - **Optional deep-dives**, pulled in when you want to go further than the backbone: `thought-layer-strategy`, `thought-layer-brand`, `thought-layer-market-research`, and `thought-layer-business-model`.
@@ -69,6 +69,8 @@ The hosted version of the rigor lives at [weareallproductmanagersnow.com](https:
 - **Done:** the rigor as portable skills; a Pi package with deterministic tools + slash commands; the portable progress file (`tl_state` / the `tl` CLI) shared with the web app; and the speedrun.
 - **Phase 3 (done):** a `build` step that turns the hardened PRD into a deploy-ready artifact, built by your own agent (`/tl-build`), with a deterministic `tl_scaffold` tool that writes an instantly-deployable branded static site as the floor.
 - **Phase 4 (done):** a `deploy` step (`/tl-deploy`, the `deploy` tool, or `tl deploy`) that takes the build live to a URL you own, closing the loop. With a Netlify token it deploys into your own account via the file-digest API (owned immediately, no claim step); with no token it delegates to your Netlify CLI - logged in it creates a site in your account, logged out it deploys anonymously with a one-hour claim link. BYOK, no central account, no lock-in. `--dry-run` shows the plan first.
+- **Backend-capable build (done):** when the three-question backend test shows a product genuinely needs a server, `/tl-build` emits a real backend alongside the static front end (serverless functions per backend requirement, a `schema.sql`, a names-only `.env.example`, an updated `netlify.toml`, and a `BACKEND.md` guide), with Neon Postgres as the documented default and overridable to any Postgres. Static stays the default, gated by the same backend test.
+- **Backend deploy (done):** when `build.json` declares a backend, `/tl-deploy` (the `deploy` tool, or `tl deploy`) ships it automatically alongside the front end: the functions go up via your Netlify CLI and the declared env var names are set on the site (values read only from your environment, BYOK). `DATABASE_URL` is bring-your-own by default; `--provision-db` (your own Neon key) and `--apply-schema` (psql) are opt in, and `--static-only` ships just the front end. Owned, no lock-in.
 
 ## Notes for contributors
 

package/SECURITY.md

@@ -11,7 +11,15 @@ follow from that.
   (`THOUGHT_LAYER_DOMAIN_KEY` / `RAPIDAPI_KEY`) are read from `process.env`. They
   are never accepted as tool or CLI parameters, never logged or printed, and
   never written to disk. The `deploy.json` record stores the resulting URLs and
-  ids, never the token.
+  ids, never the token. When a build emits a backend, the generated `.env.example`
+  is a names-only contract (every line is a bare `NAME=`): real values live only
+  in the host environment, never in a committed file. When the deploy sets those
+  env vars on your site, it reads the VALUES only from the deploy environment and
+  pushes them to your own Netlify account (in the API request body, or a
+  `0600`-mode temp file consumed by the Netlify CLI and deleted after); the
+  database connection string reaches `psql` through child-process environment
+  variables, never a command line. Values are never logged, never placed on a
+  command line, and the `deploy.json` record names env vars only, never values.
 - **Deploys go to your own account.** With a token, the deploy uses Netlify's
   file-digest API to publish into your account. With no token, it delegates to
   your installed Netlify CLI (a site in your account when logged in, or an

package/core/backend-io.ts

@@ -0,0 +1,231 @@
+// Node IO for the backend deploy automation: push env vars to the user's own
+// Netlify site, optionally provision Neon (the user's own key), and optionally
+// apply schema.sql. The deploy orchestration lives in deploy-io.ts; this file
+// holds the backend-specific side effects so deploy-io.ts stays focused and
+// core/backend.ts stays pure.
+//
+// SECRET DISCIPLINE (the whole point): secret VALUES are read ONLY from the
+// process environment inside these functions. A value never arrives as a
+// function/CLI parameter from the user, never lands in a deploy.json field,
+// never appears in a returned message, and never rides on a command line (argv).
+// Env values travel in an HTTPS request body (API) or a 0600 temp file
+// (CLI import); the database URL reaches psql through child-process env vars,
+// not argv. Env vars are recorded by NAME only.
+//
+// Copy rule for any user-facing string here: no em-dashes, no en-dashes, no
+// spaced hyphen dashes.
+
+import { spawnSync } from "node:child_process";
+import { mkdtempSync, writeFileSync, unlinkSync, existsSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import type { EnvVarPlanItem } from "./backend.ts";
+
+const NETLIFY_API = "https://api.netlify.com/api/v1";
+const NEON_API = "https://console.neon.tech/api/v2";
+
+type Env = Record<string, string | undefined>;
+
+export interface EnvPushResult {
+  method: "api" | "cli";
+  set: string[]; // names actually pushed (value present in env)
+  missing: string[]; // names declared but absent from the deploy environment
+  note: string;
+}
+
+// ---- env vars via the account-level Netlify API (token path) -----------------
+
+// GET the site to learn its account_slug, then POST the env vars (values pulled
+// from `env`) to /accounts/{slug}/env. The value lives only in the JSON body.
+export async function pushEnvVarsApi(
+  siteId: string,
+  token: string,
+  plan: EnvVarPlanItem[],
+  env: Env,
+  accountSlug?: string,
+): Promise<EnvPushResult> {
+  const set: string[] = [];
+  const missing: string[] = [];
+  const body = plan
+    .filter((p) => {
+      const has = typeof env[p.name] === "string" && env[p.name] !== "";
+      (has ? set : missing).push(p.name);
+      return has;
+    })
+    .map((p) => ({
+      key: p.name,
+      scopes: p.scopes,
+      is_secret: p.isSecret,
+      values: [{ value: env[p.name] as string, context: p.context }],
+    }));
+
+  if (body.length === 0) {
+    return { method: "api", set, missing, note: "no declared env var had a value in the deploy environment; nothing pushed" };
+  }
+
+  const slug = accountSlug || (await getAccountSlug(siteId, token));
+  const res = await fetch(`${NETLIFY_API}/accounts/${encodeURIComponent(slug)}/env?site_id=${encodeURIComponent(siteId)}`, {
+    method: "POST",
+    headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/json" },
+    body: JSON.stringify(body),
+  });
+  if (!res.ok) {
+    // Never echo the request body (it holds values); report status + names only.
+    throw new Error(`Netlify env API ${res.status} ${res.statusText} when setting ${set.join(", ")}`);
+  }
+  return { method: "api", set, missing, note: "set via the Netlify API (secret-capable, scoped to builds and functions)" };
+}
+
+async function getAccountSlug(siteId: string, token: string): Promise<string> {
+  const res = await fetch(`${NETLIFY_API}/sites/${encodeURIComponent(siteId)}`, {
+    headers: { Authorization: `Bearer ${token}` },
+  });
+  if (!res.ok) throw new Error(`Netlify site lookup ${res.status} for env push`);
+  const site = (await res.json()) as Record<string, unknown>;
+  const slug = String(site["account_slug"] || site["account_id"] || "");
+  if (!slug) throw new Error("could not resolve the account for env push");
+  return slug;
+}
+
+// ---- env vars via the Netlify CLI (no-token path) ----------------------------
+
+// Write the values to a 0600 temp file (NAME=value), import them, and delete the
+// file in a finally. The CLI cannot mark a var secret and applies all scopes;
+// the API path is preferred when a token is present.
+export function cliImportEnv(plan: EnvVarPlanItem[], env: Env, siteId?: string): EnvPushResult {
+  const set: string[] = [];
+  const missing: string[] = [];
+  const lines: string[] = [];
+  for (const p of plan) {
+    const v = env[p.name];
+    if (typeof v === "string" && v !== "") {
+      set.push(p.name);
+      lines.push(`${p.name}=${v}`);
+    } else {
+      missing.push(p.name);
+    }
+  }
+  if (lines.length === 0) {
+    return { method: "cli", set, missing, note: "no declared env var had a value in the deploy environment; nothing imported" };
+  }
+
+  const dir = mkdtempSync(join(tmpdir(), "tl-env-"));
+  const file = join(dir, ".env.import");
+  try {
+    writeFileSync(file, lines.join("\n") + "\n", { mode: 0o600 });
+    const args = ["env:import", "--force", file, ...(siteId ? ["--site", siteId] : [])];
+    const r = spawnSync("netlify", args, { encoding: "utf8", timeout: 60000 });
+    if (r.status !== 0) {
+      throw new Error(`netlify env:import failed (exit ${r.status}) for ${set.join(", ")}`);
+    }
+    return { method: "cli", set, missing, note: "imported via the Netlify CLI (applies all scopes; cannot mark secret)" };
+  } finally {
+    try { unlinkSync(file); } catch { /* best effort */ }
+  }
+}
+
+// ---- database url resolution (BYO, env only) ---------------------------------
+
+export interface DbUrlResult {
+  name: string | null; // which env var name carried the value
+  value: string | null; // the connection string (in memory only, never recorded)
+}
+
+// Prefer the portable DATABASE_URL, then the names Netlify managed Neon injects.
+export function resolveDbUrl(env: Env): DbUrlResult {
+  for (const name of ["DATABASE_URL", "NETLIFY_DATABASE_URL", "NETLIFY_DATABASE_URL_UNPOOLED"]) {
+    const v = env[name];
+    if (typeof v === "string" && v !== "") return { name, value: v };
+  }
+  return { name: null, value: null };
+}
+
+// ---- opt-in Neon provisioning (the user's own key) ---------------------------
+
+export interface ProvisionResult {
+  provisioned: boolean;
+  url: string | null; // connection string in memory only
+  note: string;
+}
+
+// Create a Neon project with the user's OWN NEON_API_KEY (BYOK; no central
+// account). Returns the connection string in memory. The default deploy path
+// does NOT call this; it runs only behind the explicit --provision-db flag.
+export async function provisionNeon(env: Env): Promise<ProvisionResult> {
+  const key = env["NEON_API_KEY"] || "";
+  if (!key) {
+    return { provisioned: false, url: null, note: "set NEON_API_KEY to provision, or set DATABASE_URL to bring your own database" };
+  }
+  try {
+    const res = await fetch(`${NEON_API}/projects`, {
+      method: "POST",
+      headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json", Accept: "application/json" },
+      body: JSON.stringify({ project: {} }),
+    });
+    if (!res.ok) {
+      return { provisioned: false, url: null, note: `Neon API ${res.status} ${res.statusText} when creating the project` };
+    }
+    const data = (await res.json()) as Record<string, unknown>;
+    const uris = Array.isArray(data["connection_uris"]) ? (data["connection_uris"] as Array<Record<string, unknown>>) : [];
+    const uri = uris.length ? String(uris[0]?.["connection_uri"] || "") : "";
+    if (!uri) return { provisioned: false, url: null, note: "Neon created the project but returned no connection string" };
+    return { provisioned: true, url: uri, note: "provisioned a Neon project in your account (connection string set for this deploy only)" };
+  } catch (e) {
+    return { provisioned: false, url: null, note: `Neon provisioning error: ${(e as Error).message}` };
+  }
+}
+
+// ---- opt-in schema apply via psql --------------------------------------------
+
+export interface SchemaResult {
+  applied: boolean;
+  note: string;
+}
+
+// Apply schema.sql to the database with psql. The connection string reaches psql
+// through libpq environment variables (PGHOST, PGUSER, ...), never argv, so the
+// value cannot leak onto a command line. Runs only behind the --apply-schema
+// flag. If psql is absent, print the manual command and report applied:false.
+export function applySchema(schemaPath: string, dbUrl: string | null, env: Env): SchemaResult {
+  if (!existsSync(schemaPath)) {
+    return { applied: false, note: `schema file not found at ${schemaPath}` };
+  }
+  if (!dbUrl) {
+    return { applied: false, note: "no database connection string available; set DATABASE_URL or use --provision-db" };
+  }
+  const probe = spawnSync("psql", ["--version"], { encoding: "utf8", timeout: 15000 });
+  if (probe.status !== 0) {
+    return { applied: false, note: `psql is not installed; apply it manually: psql "$DATABASE_URL" -f ${schemaPath}` };
+  }
+  let pg: Env;
+  try {
+    pg = libpqEnv(dbUrl);
+  } catch {
+    return { applied: false, note: "could not parse the database connection string" };
+  }
+  const r = spawnSync("psql", ["-v", "ON_ERROR_STOP=1", "-f", schemaPath], {
+    encoding: "utf8",
+    timeout: 120000,
+    env: { ...env, ...pg },
+  });
+  if (r.status !== 0) {
+    return { applied: false, note: `psql exited ${r.status} applying the schema` };
+  }
+  return { applied: true, note: "applied schema.sql with psql" };
+}
+
+// Split a Postgres connection string into libpq env vars so the secret never
+// touches argv. Throws on an unparseable URL.
+function libpqEnv(dbUrl: string): Env {
+  const u = new URL(dbUrl);
+  const pg: Env = {
+    PGHOST: u.hostname,
+    PGPORT: u.port || "5432",
+    PGDATABASE: decodeURIComponent(u.pathname.replace(/^\//, "")) || "neondb",
+  };
+  if (u.username) pg["PGUSER"] = decodeURIComponent(u.username);
+  if (u.password) pg["PGPASSWORD"] = decodeURIComponent(u.password);
+  const sslmode = u.searchParams.get("sslmode");
+  pg["PGSSLMODE"] = sslmode || "require";
+  return pg;
+}

package/core/backend.ts

@@ -0,0 +1,318 @@
+// The typed contract + deterministic boilerplate for the backend-capable build.
+//
+// Static stays the default. When the build skill's three-question backend test
+// says a product genuinely needs a server (a browser-unsafe secret, shared or
+// persistent state, or trusted server-side enforcement), the agent emits a real
+// serverless backend alongside the static front end. This module is the
+// deterministic floor for that path, mirroring scaffold.ts: the agent supplies
+// judgment (which functions, which tables), and these pure helpers render the
+// fixed, dash-free boilerplate (the env contract and the deploy guide) and
+// coerce a hand-written backend block into a clean shape.
+//
+// Pure TypeScript, no node imports. The chosen runtime DB driver
+// (`@neondatabase/serverless`) is a dependency of the GENERATED product, never
+// of the kit. Nothing here holds or provisions credentials: secrets live only
+// in the host environment, and the emitted .env.example carries names, never
+// values.
+//
+// Copy rule (enforced by tests): no em-dashes, no en-dashes, and no spaced
+// hyphen dashes (" - ", " -- ") in any generated prose. Use commas, colons,
+// periods, and parentheses instead.
+
+export type BackendKind = "serverless" | "server" | null;
+
+// One environment variable the backend reads. Names only: a value never lives
+// in this spec, the manifest, or the emitted .env.example. Real values stay in
+// the host environment.
+export interface EnvVarSpec {
+  name: string;
+  required: boolean;
+  description: string;
+}
+
+// Where the backend keeps shared or persistent state. The kit's documented
+// default is Neon Postgres, reached through the `DATABASE_URL` connection
+// string. Overridable to any Postgres by pointing `DATABASE_URL` elsewhere.
+export interface DatabaseSpec {
+  provider: string; // "neon" by default
+  schemaFile: string; // "schema.sql"
+  envVar: string; // "DATABASE_URL"
+}
+
+// The structured backend payload carried on the build manifest. The build skill
+// writes it when hasBackend is true; the deploy-automation follow-up consumes
+// it. Today the deploy step only reads it for messaging.
+export interface BackendMeta {
+  backendKind: BackendKind;
+  functionsDir: string; // "netlify/functions"
+  runtime: string; // "nodejs20.x"
+  nodeVersion: string; // "20"
+  envVars: EnvVarSpec[];
+  database: DatabaseSpec | null;
+  guide: string; // "BACKEND.md"
+}
+
+// The single documented default database. Neon is also exactly what managed
+// Netlify DB provisions, so it is both the Netlify native choice and portable.
+export const NEON_DEFAULT_DB: DatabaseSpec = {
+  provider: "neon",
+  schemaFile: "schema.sql",
+  envVar: "DATABASE_URL",
+};
+
+// ---- helpers -----------------------------------------------------------------
+
+// Coerce a name to a safe environment-variable identifier. Real env var names
+// are UPPER_SNAKE; this hardens the names-only invariant so a stray space or
+// lowercase letter can never produce a line that fails the `^[A-Z0-9_]+=$`
+// security check.
+function envName(raw: string): string {
+  const cleaned = String(raw || "")
+    .trim()
+    .toUpperCase()
+    .replace(/[^A-Z0-9_]+/g, "_")
+    .replace(/^_+|_+$/g, "");
+  return cleaned || "VAR";
+}
+
+// Dedupe by safe name and sort, so the same set of vars always renders byte for
+// byte the same regardless of input order.
+function dedupeSortEnv(envVars: EnvVarSpec[]): EnvVarSpec[] {
+  const seen = new Set<string>();
+  const out: EnvVarSpec[] = [];
+  for (const v of envVars || []) {
+    const name = envName(v?.name ?? "");
+    if (seen.has(name)) continue;
+    seen.add(name);
+    out.push({ name, required: v?.required !== false, description: String(v?.description ?? "") });
+  }
+  return out.sort((a, b) => a.name.localeCompare(b.name));
+}
+
+// ---- the names-only env contract ---------------------------------------------
+
+// Render a deterministic .env.example. Every variable line is exactly `NAME=`
+// with nothing after the equals sign. The security invariant (no value is ever
+// written) is locked by a unit test that matches `^[A-Z0-9_]+=$` on every
+// non-comment, non-blank line.
+export function renderEnvExample(envVars: EnvVarSpec[]): string {
+  const vars = dedupeSortEnv(envVars);
+  const lines: string[] = [
+    "# Environment variables for this project.",
+    "# Names only. Real values live in your host environment (the Netlify UI, or netlify env:set), never in this file.",
+    "# Copy this file to .env for local work and fill the values there. .env is gitignored; this .env.example is committed so the contract travels with the repo.",
+  ];
+  if (vars.length === 0) {
+    lines.push("", "# No backend environment variables were declared for this build.");
+    return lines.join("\n") + "\n";
+  }
+  for (const v of vars) {
+    const note = v.description.trim() ? v.description.trim() : "Set this value in your host environment.";
+    lines.push("", `# ${note} (${v.required ? "required" : "optional"})`, `${v.name}=`);
+  }
+  return lines.join("\n") + "\n";
+}
+
+// ---- the deploy guide --------------------------------------------------------
+
+export interface BackendEndpoint {
+  name: string; // function file name without extension, glossary-named
+  rid: string; // the backend requirement id it implements
+  summary: string; // one line of what it does
+}
+
+export interface BackendGuideInput {
+  brandName?: string;
+  backendKind?: BackendKind;
+  functionsDir?: string;
+  runtime?: string;
+  database?: DatabaseSpec | null;
+  envVars?: EnvVarSpec[];
+  endpoints?: BackendEndpoint[];
+}
+
+const cell = (s: string): string => String(s ?? "").replace(/\|/g, "\\|").replace(/\n+/g, " ").trim();
+
+// Render a deterministic BACKEND.md for the Neon-on-Netlify path. The agent
+// supplies the endpoint and env lists; the prose, section order, and the honest
+// "deploy automation is a follow-up" status are fixed and dash-free.
+export function renderBackendGuide(input: BackendGuideInput): string {
+  const brand = (input.brandName || "This project").trim() || "This project";
+  const functionsDir = (input.functionsDir || "netlify/functions").trim() || "netlify/functions";
+  const db = input.database ?? NEON_DEFAULT_DB;
+  const envVars = dedupeSortEnv(input.envVars || []);
+  const endpoints = [...(input.endpoints || [])].sort(
+    (a, b) => String(a.rid).localeCompare(String(b.rid)) || String(a.name).localeCompare(String(b.name)),
+  );
+
+  const envRows =
+    envVars.length > 0
+      ? envVars
+          .map((v) => `| \`${v.name}\` | ${v.required ? "yes" : "no"} | ${cell(v.description) || "Set in your host environment."} |`)
+          .join("\n")
+      : `| \`${db.envVar}\` | yes | ${db.provider === "neon" ? "Neon Postgres connection string" : "Database connection string"} |`;
+
+  const fnRows =
+    endpoints.length > 0
+      ? endpoints.map((e) => `| \`${cell(e.name)}\` | ${cell(e.rid)} | ${cell(e.summary) || "See the function source."} |`).join("\n")
+      : "| (none emitted) | n/a | This build declared a backend but wrote no functions; see DECISIONS.md. |";
+
+  return `# Backend deploy guide
+
+${brand} needs a server side, so the build emitted a backend alongside the static front end. This guide covers what is in the repo, the honest status of automated backend deploy, and the one-time manual steps to take it live yourself.
+
+## What is in the repo
+
+- \`${functionsDir}/\`: one serverless function per backend requirement (the table further down maps each to its R-ID). The front end calls a function at \`/.netlify/functions/<name>\`.
+- \`${db.schemaFile}\`: the database schema. Tables and columns use the project glossary terms. Apply it once against your database.
+- \`.env.example\`: the environment variable names this backend reads. Names only, no values. Copy it to \`.env\` for local work and set real values in your host environment.
+- \`netlify.toml\`: declares the functions directory (\`[functions] directory = "${functionsDir}"\`) so Netlify bundles and serves the functions next to the static publish directory.
+
+## Status: automated backend deploy is a follow-up
+
+\`tl deploy\` publishes the static front end today. It does not yet provision the database or ship the functions for you; that automation is the next piece of work. Until it lands, do the steps below once. The front end deploys and works now; the functions go live when you complete these steps.
+
+## 1. Provision the database
+
+The documented default is Neon Postgres. Netlify DB is managed Neon, so this is the Netlify native choice as well as a portable one.
+
+1. Create a Neon project at neon.tech, or run \`netlify db init\` to provision managed Neon from your own Netlify account.
+2. Copy the connection string. It looks like \`postgresql://USER:PASSWORD@HOST/DB?sslmode=require\`.
+3. Apply the schema once: \`psql "$${db.envVar}" -f ${db.schemaFile}\`.
+
+When you provision managed Neon through Netlify, Netlify sets \`NETLIFY_DATABASE_URL\` for you. The code reads \`${db.envVar}\` as the portable name, so set that (or map one to the other) in the next step.
+
+To use a different Postgres provider, point \`${db.envVar}\` at it instead. Any standard Postgres works with the Neon serverless driver, so this is not locked to one vendor.
+
+## 2. Set the environment variables
+
+Set these in your host environment (the Netlify UI under Site settings, Environment variables, or \`netlify env:set NAME value\`). Never commit real values; \`.env.example\` carries names only.
+
+| Variable | Required | Purpose |
+| --- | --- | --- |
+${envRows}
+
+## 3. The functions
+
+Each backend requirement maps to one function. The front end reaches it at \`/.netlify/functions/<name>\`.
+
+| Function | Requirement | Does |
+| --- | --- | --- |
+${fnRows}
+
+## 4. Deploy with the functions present
+
+1. Confirm \`netlify.toml\` declares the functions directory.
+2. Set the environment variables from step 2 on the target site.
+3. From the project root, with the functions in place, run \`netlify deploy --prod\`. (Once the backend deploy automation lands, \`tl deploy\` will do this for you.)
+
+\`netlify deploy\` ships both the static publish directory and the functions in one go. After it completes, call a function once to confirm it responds, then use the live site.
+`;
+}
+
+// ---- defensive normalizer ----------------------------------------------------
+
+function normalizeDatabase(raw: unknown): DatabaseSpec | null {
+  if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null;
+  const r = raw as Record<string, unknown>;
+  const str = (v: unknown, fb: string): string => (typeof v === "string" && v.trim() ? v.trim() : fb);
+  return {
+    provider: str(r["provider"], "neon"),
+    schemaFile: str(r["schemaFile"], "schema.sql"),
+    envVar: str(r["envVar"], "DATABASE_URL"),
+  };
+}
+
+function normalizeEnvVars(raw: unknown): EnvVarSpec[] {
+  if (!Array.isArray(raw)) return [];
+  const seen = new Set<string>();
+  const out: EnvVarSpec[] = [];
+  for (const item of raw) {
+    if (!item || typeof item !== "object") continue;
+    const r = item as Record<string, unknown>;
+    const name = typeof r["name"] === "string" ? r["name"].trim() : "";
+    if (!name) continue; // drop nameless vars
+    if (seen.has(name)) continue; // dedupe
+    seen.add(name);
+    out.push({
+      name,
+      required: typeof r["required"] === "boolean" ? r["required"] : true,
+      description: typeof r["description"] === "string" ? r["description"] : "",
+    });
+  }
+  return out;
+}
+
+// Coerce a hand-written backend block (the agent fills it into build.json) into
+// a clean BackendMeta, or null when there is nothing recognizable. Mirrors the
+// progress.ts normalizers: forgiving on input, strict on output.
+export function normalizeBackendMeta(raw: unknown): BackendMeta | null {
+  if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null;
+  const r = raw as Record<string, unknown>;
+
+  const kindRaw = r["backendKind"];
+  const kind: BackendKind = kindRaw === "server" ? "server" : kindRaw === "serverless" ? "serverless" : null;
+  const envVars = normalizeEnvVars(r["envVars"]);
+  const database = normalizeDatabase(r["database"]);
+  const hasFunctionsDir = typeof r["functionsDir"] === "string" && (r["functionsDir"] as string).trim().length > 0;
+
+  // Nothing the deploy step could act on: treat as no backend.
+  if (kind === null && envVars.length === 0 && database === null && !hasFunctionsDir) return null;
+
+  const str = (v: unknown, fb: string): string => (typeof v === "string" && v.trim() ? v.trim() : fb);
+  return {
+    backendKind: kind ?? "serverless",
+    functionsDir: str(r["functionsDir"], "netlify/functions"),
+    runtime: str(r["runtime"], "nodejs20.x"),
+    nodeVersion: str(r["nodeVersion"], "20"),
+    envVars,
+    database,
+    guide: str(r["guide"], "BACKEND.md"),
+  };
+}
+
+// ---- deploy planning (pure; the deploy step's node IO consumes these) --------
+
+// One environment variable the deploy will set on the user's site, expressed as
+// NAME + policy only. There is deliberately no value field: the deploy reads the
+// value from its own process.env at run time and never carries it through here.
+export interface EnvVarPlanItem {
+  name: string;
+  scopes: string[]; // Netlify env scopes; functions need "functions", builds keep "builds"
+  isSecret: boolean; // mark connection strings / keys / tokens as secret in the Netlify API
+  context: string; // Netlify deploy context the value applies to
+}
+
+// Names that should be marked secret on Netlify (write-only there). A coarse,
+// safe-by-default match: anything that looks like a credential is secret.
+const SECRET_NAME_RE = /(KEY|SECRET|TOKEN|PASSWORD|PASSWD|DATABASE_URL|DB_URL|CONN|DSN|CREDENTIAL|PRIVATE|AUTH)/;
+
+function looksSecret(name: string): boolean {
+  return SECRET_NAME_RE.test(name.toUpperCase());
+}
+
+// Plan the env vars to push: the backend's declared names plus the database
+// connection var, deduped and sorted (via dedupeSortEnv, so names are sanitized
+// the same way the .env.example is). Names + policy only, never values.
+export function planEnvVars(backend: BackendMeta): EnvVarPlanItem[] {
+  const dbVar = backend.database?.envVar ? backend.database.envVar : "";
+  const merged: EnvVarSpec[] = [
+    ...(backend.envVars || []),
+    ...(dbVar ? [{ name: dbVar, required: true, description: "" }] : []),
+  ];
+  return dedupeSortEnv(merged).map((v) => ({
+    name: v.name,
+    scopes: ["builds", "functions"],
+    isSecret: looksSecret(v.name),
+    context: "all",
+  }));
+}
+
+export interface FunctionsPlan {
+  functionsDir: string; // relative dir from build.json (e.g. "netlify/functions")
+  runtime: string; // Netlify function runtime; serverless TypeScript bundles as "js"
+}
+
+export function planFunctions(backend: BackendMeta): FunctionsPlan {
+  return { functionsDir: backend.functionsDir || "netlify/functions", runtime: "js" };
+}