@hobocode/thought-layer 0.4.2 → 0.6.0

package/extensions/thought-layer.ts

@@ -214,20 +214,24 @@ export default function (pi: ExtensionAPI) {
     name: "deploy",
     label: "Thought Layer: deploy",
     description:
-      "Take the built site live to a user-owned URL. Reads build.json (publishDir/entry) next to the state file, then deploys to Netlify. " +
+      "Take the built site live to a user-owned URL. Reads build.json (publishDir/entry, and the backend block when present) next to the state file, then deploys to Netlify. " +
       "Two models, both BYOK with no lock-in: with NETLIFY_AUTH_TOKEN set (read from the environment only, never a parameter) it deploys into the user's OWN account via the file-digest API (no zip); with no token it uses the Netlify CLI's --allow-anonymous flow for an instant live URL plus a one-hour claim link. " +
-      "Run the build first (thought-layer-build / tl_scaffold). Use dryRun:true to preview the file plan with no network call. Static-first: if build.json has hasBackend:true it warns that only the front end ships this way.",
+      "When build.json declares a backend it ships automatically: the functions go up via the user's Netlify CLI and the declared env var names are set on the site (values read only from the environment, BYOK). DATABASE_URL is bring-your-own by default; provisionDb and applySchema are opt in. staticOnly ships just the front end. " +
+      "Run the build first (thought-layer-build / tl_scaffold). Use dryRun:true to preview the file and backend plan with no network call.",
     parameters: Type.Object({
       path: Type.Optional(Type.String({ description: "State file (or project dir) whose build.json to deploy. Defaults to ./.thought-layer/state.json; honors a named file." })),
-      dryRun: Type.Optional(Type.Boolean({ description: "Plan only: walk the publish dir and report the files + target, with no network call or CLI spawn." })),
+      dryRun: Type.Optional(Type.Boolean({ description: "Plan only: walk the publish dir and report the files, target, and backend plan, with no network call or CLI spawn." })),
       anonymous: Type.Optional(Type.Boolean({ description: "Force the no-account path (Netlify CLI --allow-anonymous) even if a token is set. Default: token path when NETLIFY_AUTH_TOKEN is set, else anonymous." })),
-      siteName: Type.Optional(Type.String({ description: "Token path only: create the site under this name (a-z0-9-). Omit to let Netlify assign a random subdomain." })),
-      siteId: Type.Optional(Type.String({ description: "Token path only: re-deploy to an existing site id instead of creating a new one." })),
+      siteName: Type.Optional(Type.String({ description: "Create the site under this name (a-z0-9-). Omit to let Netlify assign a random subdomain." })),
+      siteId: Type.Optional(Type.String({ description: "Re-deploy to an existing site id instead of creating a new one." })),
+      staticOnly: Type.Optional(Type.Boolean({ description: "Ship only the front end even when build.json declares a backend." })),
+      provisionDb: Type.Optional(Type.Boolean({ description: "Opt in: provision Neon with the user's own NEON_API_KEY (read from the environment). Default off, which uses a bring-your-own DATABASE_URL." })),
+      applySchema: Type.Optional(Type.Boolean({ description: "Opt in: apply schema.sql with psql once the database is reachable. Default off." })),
     }),
     async execute(_id, params): Promise<ToolResult> {
-      const p = params as { path?: string; dryRun?: boolean; anonymous?: boolean; siteName?: string; siteId?: string };
+      const p = params as { path?: string; dryRun?: boolean; anonymous?: boolean; siteName?: string; siteId?: string; staticOnly?: boolean; provisionDb?: boolean; applySchema?: boolean };
       const r = await runDeploy(
-        { path: p.path, dryRun: p.dryRun, anonymous: p.anonymous, siteName: p.siteName, siteId: p.siteId },
+        { path: p.path, dryRun: p.dryRun, anonymous: p.anonymous, siteName: p.siteName, siteId: p.siteId, staticOnly: p.staticOnly, provisionDb: p.provisionDb, applySchema: p.applySchema },
         { deployedAt: new Date().toISOString() },
       );
       return text(r.message, r.details);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hobocode/thought-layer",
-  "version": "0.4.2",
+  "version": "0.6.0",
   "description": "The Thought Layer: rigor for building. Validate an idea, grill it into a buildable spec, then build and deploy it, inside the agent you already use. BYOK, no telemetry.",
   "license": "MIT",
   "author": "Hobocode LLC <jerm@hobocode.net>",
@@ -64,6 +64,8 @@
     "core/stage-map.ts",
     "core/state-file.ts",
     "core/state-ops.ts",
+    "core/backend.ts",
+    "core/backend-io.ts",
     "core/scaffold.ts",
     "core/scaffold-io.ts",
     "core/deploy.ts",

package/prompts/tl-build.md

@@ -2,6 +2,6 @@ Apply the **thought-layer-build** skill. Build the hardened PRD into a static-fi
 
 If there is no hardened PRD in the state file (`state.prd.markdown` + requirements), say so and point me to `/tl` (or `/tl-prd` then `/tl-grill`) rather than building from a bare idea - only proceed cold if I tell you to.
 
-Honor the ubiquitous language (the glossary), R-ID traceability, the out-of-scope list, mobile+desktop, the brand if present, and the full SEO/discoverability layer. Default to static; escalate to a backend only when a requirement genuinely needs one, and flag loudly that the default deploy path is static. Verify the build runs, and leave `.thought-layer/build.json` + `DECISIONS.md` + `TRACEABILITY.md`.
+Honor the ubiquitous language (the glossary), R-ID traceability, the out-of-scope list, mobile+desktop, the brand if present, and the full SEO/discoverability layer. Default to static; when the three-question backend test shows a requirement genuinely needs a server, build the real backend too: serverless functions under `netlify/functions/` (one per backend R-ID, glossary-named), a `schema.sql`, a names-only `.env.example` (with `!.env.example` un-ignored in `.gitignore`), an updated `netlify.toml`, and a `BACKEND.md` guide, recorded in the manifest's `backend` block. Be honest that backend deploy automation is a follow-up, so `tl deploy` still ships the static front end for now. Verify the build runs, and leave `.thought-layer/build.json` + `DECISIONS.md` + `TRACEABILITY.md`.
 
 Read the spec from the state file (default `.thought-layer/state.json`; honor `--path` / `THOUGHT_LAYER_STATE` if a named file is in use). For the fastest deployable floor, or if a full build is too much, run the **tl_scaffold** tool to write a branded, SEO-complete static landing site and the same `build.json` manifest.

package/prompts/tl-deploy.md

@@ -2,6 +2,6 @@ Apply the **thought-layer-deploy** skill. Take the built site live to a URL I ow
 
 Read `.thought-layer/build.json` (next to the state file; honor `--path` / `THOUGHT_LAYER_STATE` if a named file is in use) for the publish directory and entry. If there is no `build.json`, say so and point me to `/tl-build` (or the `tl_scaffold` tool) rather than guessing - the build has to run first.
 
-Default to a dry run first so I can see exactly which files would ship and where. Then deploy: if `NETLIFY_AUTH_TOKEN` is set, deploy into my own Netlify account (owned immediately); otherwise delegate to my Netlify CLI - logged in it creates a site in my account, logged out it deploys anonymously with a one-hour claim link. Read the token only from the environment, never ask me to paste it. If `build.json` says `hasBackend: true`, warn me clearly that this static deploy publishes only the front end.
+Default to a dry run first so I can see exactly which files would ship and where. Then deploy: if `NETLIFY_AUTH_TOKEN` is set, deploy into my own Netlify account (owned immediately); otherwise delegate to my Netlify CLI - logged in it creates a site in my account, logged out it deploys anonymously with a one-hour claim link. Read the token only from the environment, never ask me to paste it. If `build.json` declares a backend, ship it automatically alongside the front end: the functions go up via my Netlify CLI and the declared env var names are set on the site (read the values from my environment, BYOK; tell me by name any that are missing). `DATABASE_URL` is bring-your-own by default; only provision Neon (`--provision-db`) or apply `schema.sql` (`--apply-schema`) if I ask. Use `--static-only` if I want just the front end. Tell me plainly which functions shipped and which env names were set versus missing, and never claim a backend is running when only the front end went live.
 
 After it is live, tell me the URL and (anonymous only) the claim link, and that a `.thought-layer/deploy.json` record was written.

package/skills/thought-layer-build/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: thought-layer-build
-description: "Turn the hardened PRD into a static-first, deploy-ready artifact, built directly by this agent. Reads the build brief from the shared state file (PRD, glossary, R-ID requirements, brand, business context, open to-dos), then builds a self-contained site or a Vite/static build that yields a predictable publish directory (dist/), honoring ubiquitous language, R-ID traceability, the out-of-scope list, mobile+desktop, brand, and the full SEO/discoverability layer. Escalates to a backend only when the spec genuinely requires server-side, and flags loudly that the default deploy path is static. Verifies the build runs and writes a .thought-layer/build.json manifest for the deploy step. Run it after the grill has hardened the PRD; it is a standalone build step, not a validation stage."
+description: "Turn the hardened PRD into a static-first, deploy-ready artifact, built directly by this agent. Reads the build brief from the shared state file (PRD, glossary, R-ID requirements, brand, business context, open to-dos), then builds a self-contained site or a Vite/static build that yields a predictable publish directory (dist/), honoring ubiquitous language, R-ID traceability, the out-of-scope list, mobile+desktop, brand, and the full SEO/discoverability layer. When the spec genuinely requires a server, it also emits a real backend (serverless functions under netlify/functions/, a schema.sql, a names-only .env.example, an updated netlify.toml, and a BACKEND.md guide) and records it in the manifest, while staying honest that backend deploy automation is a follow-up so the default deploy path stays static. Verifies the build runs and writes a .thought-layer/build.json manifest for the deploy step. Run it after the grill has hardened the PRD; it is a standalone build step, not a validation stage."
 ---
 
 # Build it: the hardened PRD becomes a deploy-ready artifact
@@ -49,7 +49,17 @@ Default to a self-contained static site or a Vite/Astro/static build whose outpu
 
 If all three are no, build **static, full stop.** localStorage, static data files, BYOK client-side AI calls, and third-party embeddable widgets do **not** count as a backend - many specs that sound like they need a server can ship a compelling static slice first.
 
-**When a backend is genuinely required:** build the static parts anyway, set `hasBackend: true` + `backendNote` in the manifest, and **warn loudly** in chat: the default deploy publishes a static `dist/` to Netlify, so the server part will not deploy that way and needs serverless functions or a separate host. Build the static shell with a clear seam for the backend.
+**When a backend is genuinely required, build it for real (do not just warn).** Build the static front end as above, set `hasBackend: true` and a one-line `backendNote`, and emit a coherent, buildable serverless backend alongside it:
+
+- **Serverless functions, one per backend R-ID**, under `netlify/functions/`. Name each file in the ubiquitous language (the glossary term for what it does, e.g. `netlify/functions/dispatch.ts`), open it with a comment naming the R-ID it implements, and keep it inside the out-of-scope boundary. Each function reads its inputs, talks to the database, and returns JSON. Give the front end a clear seam: it calls the function at `fetch('/.netlify/functions/<name>')`, never a hardcoded host.
+- **A `schema.sql`** at the project root, derived from the PRD data requirements and the domain entities. Name tables and columns in the glossary terms (no synonyms), and keep it idempotent where you can (`create table if not exists ...`).
+- **Neon Postgres by default.** The functions reach the database through the Neon serverless driver (`@neondatabase/serverless`, added to the product's `package.json`, never the kit's) and read the connection string from `DATABASE_URL` (Netlify sets `NETLIFY_DATABASE_URL` when you provision managed Neon, so read `DATABASE_URL` and map it). Neon is the single documented default and is overridable to any Postgres by pointing `DATABASE_URL` elsewhere; state that in `BACKEND.md` and do not invent a second provider.
+- **A names-only `.env.example`** at the project root listing every variable the backend reads (`DATABASE_URL` plus any others), each as a bare `NAME=` under a one-line comment. Never write a real value; real values live only in the host environment.
+- **An updated `netlify.toml`.** Extend the existing publish + redirect block (do not replace it) with a `[functions]` table declaring `directory = "netlify/functions"`. Keep the static publish dir and the SPA redirect intact.
+- **A `BACKEND.md` deploy guide** at the project root: what is in the repo, the honest status (automated backend deploy is a follow-up, so `tl deploy` ships only the front end today), how to provision Neon, the env-var table, the function-to-R-ID table, and the manual `netlify deploy` steps. The kit's `renderBackendGuide` and `renderEnvExample` helpers (in `core/backend.ts`) produce a dash-free skeleton if you have the core available; otherwise write the same content by hand.
+- **A project `.gitignore`** that ignores `.env` and `.env.*` but un-ignores the contract with `!.env.example`. Without that line the env contract is silently un-committable, and the deploy step cannot read it.
+
+Then record the backend in the manifest's `backend` block (shape below). Do **not** attempt to deploy in this step. The deploy is the next step: `tl deploy` (or the **thought-layer-deploy** skill) reads this `backend` block and ships the functions plus the declared env var names via the user's Netlify CLI into their own account, with `DATABASE_URL` bring-your-own by default. Point the user at `BACKEND.md` for the details and the opt-in database steps.
 
 ## SEO and discoverability (build all of it, do not skip it)
 
@@ -67,7 +77,8 @@ Do not declare victory - check:
 2. **Confirm the publish dir + entry load.** `dist/index.html` (or your entry) exists and is non-trivial. Where a preview or browser tool is available, load it and confirm it renders; otherwise inspect the built HTML for the expected title/nav/hero and that the mobile viewport meta is set.
 3. **R-ID coverage.** Walk TRACEABILITY.md: each R-ID is implemented (with a pointer) or explicitly deferred (with a reason). Put the counts in `build.json.requirements`.
 4. **SEO check.** Confirm the SEO files are actually in the publish dir; set `build.json.seo.*` from reality, not intent.
-5. **Report** what is built and what is deferred, plainly, in chat.
+5. **Backend check (only when `hasBackend`).** Each backend R-ID maps to a function in TRACEABILITY.md; `netlify/functions/` has one file per backend R-ID; `.env.example` is values-free (every variable line is a bare `NAME=`, never a value); `schema.sql` is non-empty; `netlify.toml` declares the functions directory; `.gitignore` has `!.env.example`. Do **not** try to run the backend or reach a database here (there is no `DATABASE_URL` in the build env); confirm the artifact is coherent and buildable, not live.
+6. **Report** what is built and what is deferred, plainly, in chat.
 
 ## Honest about being model-built
 
@@ -82,6 +93,7 @@ Write three files with your own file tools (the manifest is NOT a `tl_state` art
 { "app": "thought-layer", "kind": "build", "version": 1, "builtAt": "<ISO>",
   "producer": "agent", "publishDir": "dist", "entry": "index.html",
   "stack": "static|vite|astro|next-static|other", "hasBackend": false, "backendNote": null,
+  "backend": null,
   "buildCommand": null, "installCommand": null, "nodeVersion": "20",
   "provenance": { "stateFile": "<the file you read>", "prdTs": <state.prd.ts>, "grillDone": <bool>, "fromSpeedrun": <bool> },
   "requirements": { "total": 0, "built": 0, "deferred": 0, "deferredIds": [] },
@@ -90,6 +102,17 @@ Write three files with your own file tools (the manifest is NOT a `tl_state` art
   "verified": { "buildRan": true, "publishDirExists": true, "entryLoads": true, "notes": "..." } }
 ```
   `publishDir` + `entry` are load-bearing - the deploy step reads them. (The `tl_scaffold` tool writes this same manifest with `producer: "scaffold"`.)
+
+  When `hasBackend` is `true`, populate `backend` (leave it `null` for a static build). This is the forward-looking contract the backend deploy automation will consume:
+
+```jsonc
+"backend": {
+  "backendKind": "serverless", "functionsDir": "netlify/functions",
+  "runtime": "nodejs20.x", "nodeVersion": "20",
+  "envVars": [{ "name": "DATABASE_URL", "required": true, "description": "Neon Postgres connection string" }],
+  "database": { "provider": "neon", "schemaFile": "schema.sql", "envVar": "DATABASE_URL" },
+  "guide": "BACKEND.md" }
+```
 - `DECISIONS.md` - every choice the spec did not pin down, one line each, with the reason.
 - `TRACEABILITY.md` - the R-ID map. (`SEO.md` comes from the SEO step.)
 

package/skills/thought-layer-deploy/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: thought-layer-deploy
-description: "Take the built site live to a user-owned URL with no lock-in, the last step after the build. Reads .thought-layer/build.json (the publish dir + entry) next to the state file, then deploys to Netlify by one of two BYOK models: with NETLIFY_AUTH_TOKEN set it deploys into the user's OWN account via the file-digest API (owned immediately, no claim), and with no token it delegates to the user's Netlify CLI (logged in: a new site in their account; logged out: an anonymous, claimable URL with a one-hour claim link). Static-first: if build.json says hasBackend it warns that only the front end ships this way. Prefers the deploy tool (Pi) or the tl deploy CLI (any shell agent) so the deploy is one mechanical, honest step, never hand-rolled. Run it after thought-layer-build (or tl_scaffold) has produced build.json."
+description: "Take the built site live to a user-owned URL with no lock-in, the last step after the build. Reads .thought-layer/build.json (the publish dir + entry) next to the state file, then deploys to Netlify by one of two BYOK models: with NETLIFY_AUTH_TOKEN set it deploys into the user's OWN account via the file-digest API (owned immediately, no claim), and with no token it delegates to the user's Netlify CLI (logged in: a new site in their account; logged out: an anonymous, claimable URL with a one-hour claim link). When build.json declares a backend it ships automatically: the functions go up via the user's Netlify CLI and the declared env var names are set on the site (values read only from the environment, BYOK). DATABASE_URL is bring-your-own by default; provisionDb and applySchema (Neon provisioning and psql schema apply) are opt in; staticOnly ships just the front end. Prefers the deploy tool (Pi) or the tl deploy CLI (any shell agent) so the deploy is one mechanical, honest step, never hand-rolled. Run it after thought-layer-build (or tl_scaffold) has produced build.json."
 ---
 
 # Deploy it: the build goes live to a URL you own
@@ -33,9 +33,15 @@ The tool picks automatically; explain which one ran.
 
 If neither a token nor a usable CLI is available, relay the tool's guidance honestly (set a token, install/update the CLI, or drag the publish dir onto https://app.netlify.com/drop) instead of pretending it deployed. If the user explicitly asked for an anonymous deploy but the CLI is logged in, the tool says it went to their account instead and that `netlify logout` first would make it anonymous - pass that on.
 
-## Static-first honesty
+## Backend deploy (automatic; static stays the floor)
 
-The default deploy publishes a **static** publish directory. If `build.json.hasBackend` is `true`, the tool warns and you must repeat it plainly: only the front end goes live this way; the server part needs serverless functions or a separate host. Do not imply a backend is running when it is not.
+If `build.json.hasBackend` is `true` and the build declared a serverless backend, the tool ships it **automatically** alongside the static front end:
+- **Functions** go up via the user's **Netlify CLI** (it bundles the TypeScript). The CLI is required to ship functions; if it is missing but a token is set, the tool takes the front end live, sets the env vars, and tells the user plainly that functions need the CLI (it does not pretend they shipped).
+- **Env vars**: the declared names are set on the site. Values are read **only from the environment** (BYOK), set via the Netlify API when a token is present (secret-capable) or imported via the CLI otherwise. A declared name that is absent from the environment is reported by name so the user can set it and re-run. Never ask the user to paste a value into the chat.
+- **Database**: `DATABASE_URL` is **bring-your-own** by default (the tool also reads `NETLIFY_DATABASE_URL` / `NETLIFY_DATABASE_URL_UNPOOLED`). Two opt-in flags go further: `--provision-db` / `provisionDb` provisions Neon with the user's own `NEON_API_KEY`, and `--apply-schema` / `applySchema` applies `schema.sql` with `psql`. Both are off by default.
+- **`--static-only` / `staticOnly`** ships just the front end even when a backend is present; the tool then points at `BACKEND.md` for the manual steps.
+
+Relay what actually happened: which site got the backend, which functions shipped, which env names were set versus missing. Do not imply a backend is running when it is not (for example, when only the front end shipped because the CLI was absent).
 
 ## After it is live