@aotter/mantle 0.1.5-alpha.1 → 0.2.0-alpha.2

package/docs/handbook/start/overview.md

@@ -1,55 +1,63 @@
 ---
-description: Choose a Mantle integration, discover manifest capabilities, and find tutorials, task guides, concepts and field-level reference.
+description: What Mantle 0.2.0 is, how a service is put together, and where to read next.
 ---
 # Mantle handbook
 
-Mantle turns YAML manifests into a validated runtime plan, typed TypeScript
-bindings, and optional HTTP, MCP and Admin surfaces. Your application owns the
-host, storage and frontend. Four atoms describe the contract: **Schema** stores
-data, **View** reads it, **Procedure** acts on it, and **Trigger** binds an action
-to HTTP, MCP or lifecycle events.
-
-This handbook describes the SDK snapshot that carries it. For an installed
-project, read `node_modules/@aotter/mantle/docs/handbook/`; a website or Git
-branch can describe a different version. The [release index](../releases/index.md)
-links published releases. A prerelease capability is not a promise that the
-current npm `latest` contains it.
-
-## Start with your integration
-
-| Goal | Start here | Result |
-|---|---|---|
-| Understand what manifests can express | [Manifest feature reference](../reference/features.md) | A capability-to-field map across the four atoms. |
-| Validate manifests in an existing tool | [Spec-only adoption](../../spec-only-host-adoption.md) | Parse and link without Runtime, storage or a UI. |
-| Bring Mantle into an existing application | [Integration skill](../../skills/integrate/SKILL.md) | Inspect the app and choose embedding, incremental replacement or rebuilding with data migration. |
-| Embed Runtime in an existing host | [Runtime and adapters](../concepts/runtime-and-adapters.md), then [typed queries](../guides/typed-queries.md) | Bind your storage and call the generated API. |
-| Build a local Cloudflare API | [Minimal Worker tutorial](./quickstart-worker.md) | A running public View and a verified HTTP response. |
-| Add a staff console | [Local Admin tutorial](./quickstart-admin.md) | Email OTP, Admin assets and a local human workflow. |
-| Build on ChatGPT Sites | [Sites integration](../chatgpt-sites/index.md) | Host-owned sign-in and deployment with Mantle content. |
-| Work through a coding agent | [Skill installation and handoff](../guides/agent-setup.md) | Bootstrap skill, pinned package, then project-local instructions. |
-
-Human authors can follow these pages directly; installing an agent skill is
-optional. For a new app, install an exact SDK version and run
-`mantle generate --host cf` or `--host chatgpt-sites`. The CLI assembles a blank
-site by default; add your own Manifests after generation. Use explicit
-`--features` for a smaller composition, including host-free Spec-only.
-
-## Find the right kind of documentation
-
-- **Tutorials** walk through a running minimal service or local Admin.
-- **Task guides** explain typed queries, Admin customization and agent setup.
-- **Concepts** explain the four atoms, runtime, lifecycle, authorization and transports.
-- **Reference** lists accepted fields, defaults, restrictions and diagnostics.
-- **Host guides** cover Cloudflare and Sites wiring and operations.
-- **Examples** supply complete domain manifests and runnable host references.
-
-The [project and CLI guide](./project-and-cli.md) describes file ownership and
-the verification loop. The [examples hub](../examples/hub.md) helps select a
-domain model. Use the [field reference](../reference/manifest.md) when checking
-exact syntax; do not infer grammar from a UI screenshot.
-
-## Source
-
-- [Core package](../../../packages/mantle/README.md)
-- [Handbook navigation](../navigation.json)
-- [Consumer skills](../../../skills/README.md)
+Mantle is a manifest-driven service core. A service declares four atoms in
+YAML: a **Schema** stores rows, a **View** reads them, a **Procedure** writes
+them, and a **Trigger** binds a Procedure to HTTP, MCP, a lifecycle event or a
+schedule. Views and Procedures are SQL in PostgreSQL syntax.
+
+`mantle generate` compiles the manifests into a sealed plan and a typed module.
+The runtime serves the plan through one **Store**, which adds each caller's
+scope, TTL visibility and publishing state to every statement, and an
+optimistic lock where the caller passes one. Your service owns its entry, its users and its frontend; Mantle
+adds tables, surfaces and typed access to them.
+
+This handbook ships inside `@aotter/mantle` at
+`node_modules/@aotter/mantle/docs/handbook/`. The installed copy matches the
+installed package; prefer it over any online copy.
+
+## How a service fits together
+
+```
+manifests/*.yaml ──mantle generate──▶ .mantle/generated/plan.json + mantle.ts
+                                          │
+src/service.ts   createMantle(service, { plan, storage, schedules })
+  ├─ handlers    the `ref` Procedures, typed by mantle.ts
+  └─ fetch       your routes, then Mantle's surfaces behind withCaller(resolver, …):
+                 auth routes · Admin at /admin · MCP at /mcp · REST at /api
+src/index.ts     the host entry: Cloudflare's fetch and scheduled, or Bun.serve
+```
+
+The first `mantle generate` writes `src/service.ts`, `src/index.ts`,
+`src/handlers.ts`, `tsconfig.json` and, on Cloudflare, `wrangler.jsonc` once.
+After that they are your files.
+
+## Read next
+
+| You want to | Read |
+|---|---|
+| Start a service | [Quickstart](./quickstart-worker.md), then [Project layout and CLI](./project-and-cli.md) |
+| Learn the model | [The four atoms](../concepts/four-atoms.md), [Reads](../concepts/views.md), [Writes](../concepts/procedures-and-triggers.md), [Authorization](../concepts/authorization.md) |
+| Look up a field | [Feature table](../reference/features.md), then the [Schema](../reference/schema.md), [View](../reference/view.md), [Procedure](../reference/procedure.md) and [Trigger](../reference/trigger.md) references |
+| Wire sign-in, media or deploy | [Authentication](../cloudflare/authentication.md), [Media uploads with R2](../cloudflare/media-r2.md), [Deploy and operate](../cloudflare/deploy-and-operate.md) |
+| Run on ChatGPT Sites | [Mantle on ChatGPT Sites](../cloudflare/chatgpt-sites.md) |
+| See whole services | [Examples](../examples/hub.md) and the runnable reference service |
+| Move a 0.1.x project | `docs/upgrade-0.1-to-0.2.md` in the installed package |
+
+The decisions behind 0.2.0 are ADR-0032 to ADR-0038 in `docs/adr/`. The ADRs
+record decisions and may describe work not built yet; the installed code is the
+authority. When an ADR and this handbook disagree, follow this handbook and
+check the code.
+
+## What 0.2.0 does not have yet
+
+- **Mantle-rendered public pages.** `createWebSurface` is not ported. Render
+  pages from your own `fetch` or frontend over the REST surface.
+- **A preset for any host but Cloudflare and Bun.** With `--host none`,
+  `mantle generate` writes the plan and its types only, and your code calls
+  `createMantle`. The Bun preset has no scheduler.
+- **Site settings and media on PostgreSQL.** Only the `sqlite` dialect's
+  storage takes `SiteDefaults`; see [Site defaults](../reference/site-config.md).
+- **Deploying to Mantle Cloud.** See [Deploy and operate](../cloudflare/deploy-and-operate.md).

package/docs/handbook/start/project-and-cli.md

@@ -1,124 +1,151 @@
 ---
-description: Generate a full Mantle app or select smaller modules, then safely rerun the CLI as your manifests evolve.
+description: The files of a Mantle 0.2.0 project, who owns each, mantle.config.json, and every option of mantle generate.
 ---
 # Project layout and CLI
 
-`mantle generate` can assemble a new application or compile manifests in an
-existing one. A new hosted app defaults to Spec, Runtime, API, MCP, Admin and
-an editable blank home. You can select a smaller set explicitly. It never
-invents a business Schema or publishes a deployment.
+## Files
 
-## Start a new app
+| Path | Written by | Owner |
+|---|---|---|
+| `manifests/*.yaml` | you | you |
+| `mantle.config.json` | `mantle generate`, when the selection changes | you may edit it; generate keeps keys it does not know |
+| `.mantle/generated/plan.json` | every `mantle generate` | generated: never edit, commit it |
+| `.mantle/generated/mantle.ts` | every `mantle generate` | generated: never edit, commit it |
+| `src/service.ts`, `src/index.ts`, `src/handlers.ts`, `src/identity.ts` (identity `custom`) | the first `mantle generate` | yours from then on |
+| `wrangler.jsonc` (Cloudflare, unless a `wrangler.json` or `wrangler.toml` exists), `tsconfig.json`, `.dev.vars.example` (Cloudflare, identity `mantle`) or `.env.example` (Bun), `.gitignore` | the first `mantle generate`, each only if missing | yours |
+
+`mantle generate` writes each preset file only when it does not exist, and
+writes none once `src/service.ts` exists: from then on the application owns
+its composition. To get a fresh preset, move the old file aside and rerun.
+
+- `plan.json` is `{ "sourceHash": …, "plan": … }`: the sealed `RuntimePlan`
+  and the SHA-256 of the manifest sources. The plan's `fingerprint` is what
+  boot checks; reformatting a manifest changes `sourceHash` but not the
+  fingerprint.
+- `mantle.ts` exports `plan`, `sourceHash`, the `Mantle` namespace of row, input and output
+  types, `Store` and `CallerStore` typed over your Schemas and
+  Views, and `MantleHandlers`, which lists exactly the plan's `ref` handlers. A
+  missing or extra handler is a type error. See
+  [Query from TypeScript](../guides/typed-queries.md).
+
+## `mantle.config.json`
+
+```json
+{ "version": 2, "identity": "mantle", "features": ["mcp", "admin", "web"] }
+```
 
-Choose a host before running the command. The integrated hosts are `cf` and
-`chatgpt-sites`; `--features spec` works without a host. Install one exact
-published SDK version, then let the CLI declare its matching optional packages:
+| Key | Values | Meaning |
+|---|---|---|
+| `version` | `2` | `1` (a 0.1.x config) fails with exit 2 |
+| `identity` | `mantle`, `custom`, `none` | who the callers are. `mantle`: `@aotter/mantle/auth` (Better Auth sign-in, staff roles, OAuth for MCP). `custom`: your `src/identity.ts` maps your own sessions to callers. `none`: every caller is anonymous |
+| `features` | a subset of `mcp`, `admin`, `web`, in that order | `mcp`: the public MCP surface at `/mcp` (and staff MCP inside Admin). `admin`: the Admin console and its API at `/admin`, which needs an identity. `web`: reserved for public pages; REST at `/api` is always mounted |
+| `host` | `cloudflare`, `bun`, `none`, optional | where the service runs (ADR-0036); absent is `cloudflare`. `none` writes no preset: only the plan and its types, and your code calls `createMantle`. The host never enters the plan |
+| `dialect` | `sqlite` (alias `d1`), `postgres`, or a dialect package name, optional | the SQL engine; absent is `sqlite`, which runs on D1. `postgres` runs on PostgreSQL, through Hyperdrive on Cloudflare. The plan records the dialect (the SQLite dialect as `@aotter/mantle/d1`) |
+
+Without a config or flags, the selection is identity `mantle` and every
+feature. An explicit `--features` without `--identity` means identity `none`.
+A rerun keeps the saved identity, host and dialect; asking for another one is
+refused (exit 2), so switching never drops tables or orphans data. Change
+`mantle.config.json` deliberately once the data is moved.
+
+| | `cloudflare` | `bun` | `none` |
+|---|---|---|---|
+| `sqlite` | preset over D1 (the default) | native bun:sqlite | plan and types only |
+| `postgres` | preset over Hyperdrive and `pg` | native Bun.SQL PostgreSQL pool | plan and types only |
+| a dialect package | plan and types only; compose `src/service.ts` with its storage adapter | plan and types only | plan and types only |
+
+Each selection needs packages in the project. `mantle generate` checks them
+and stops before writing anything, naming the install command for your
+package manager. It never installs anything. The host and dialect rows, and the
+staff MCP App, are checked only while the preset is still to be written.
+
+| Selection | Packages |
+|---|---|
+| always | `@aotter/mantle` |
+| identity `mantle` | `better-auth`, `@better-auth/oauth-provider`, `@better-auth/mcp`, `@better-auth/cimd` |
+| feature `mcp` | `@modelcontextprotocol/server`, `@modelcontextprotocol/ext-apps` |
+| feature `admin` | `@aotter/mantle-ui` (its `dist/admin` is the console, bound as `ASSETS` in `wrangler.jsonc`) |
+| dialect `postgres` on `cloudflare` | `pg` |
+| host `bun` | `bun-types` |
+| feature `mcp` with an identity, without `admin` | `@aotter/mantle-ui` (the staff MCP App) |
+
+On Cloudflare the project also installs `wrangler`, `@cloudflare/workers-types` and
+`@types/node` itself. A TypeScript Cloudflare/PostgreSQL project also installs
+`@types/pg` as a dev dependency for the generated `pg` import.
+
+## `mantle generate`
 
 ```sh
-mkdir my-app && cd my-app
-npm init -y
-npm pkg set type=module
-MANTLE_VERSION=$(npm view @aotter/mantle@latest version)
-npm install --save-exact "@aotter/mantle@$MANTLE_VERSION"
-npx mantle generate --host cf
-npm install
-npx mantle generate
-npm run build
+mantle generate [--manifests <dir>] [--features <list>] [--identity <kind>] [--host <host>] [--dialect <name>]
+mantle generate --check [--database <file>]
 ```
 
-Confirm `MANTLE_VERSION` is a stable release, without a prerelease suffix.
-Use `--host chatgpt-sites` for a Site; [its guide](../chatgpt-sites/generated-app.md)
-continues with D1 migration, owner identity and local smoke. If you already
-use pnpm, keep that one package manager and run `pnpm install` /
-`pnpm exec mantle`. The first `generate` may exit asking you to install newly
-declared packages; the second completes generation. A README or lockfile from
-bootstrapping still counts as a new app. Once choices are saved, omit flags on
-rerun.
+| Option | Meaning |
+|---|---|
+| `--manifests <dir>` | the manifest directory, default `./manifests`. Every `.yaml` and `.yml` file directly in it is read |
+| `--features <list>` | comma-separated, a positive list. A missing dependency fails with `GENERATE_FEATURE_DEPENDENCY_MISSING`; an omitted feature is never added back |
+| `--identity <kind>` | `mantle`, `custom` or `none` |
+| `--host <host>` | `cloudflare`, `bun` or `none`, chosen on the first run. `bun` defaults the dialect to `postgres` |
+| `--dialect <name>` | `sqlite` (alias `d1`), `postgres` or a dialect package, chosen on the first run |
+| `--check` | writes nothing. Exits 1 when `plan.json`, `mantle.ts` or `mantle.config.json` differs from what generate would write, or a package is missing |
+| `--database <file>` | with `--check`: also read a local SQLite file (Wrangler's local D1 is under `.wrangler/state/v3/d1/`) read-only, and print the SQL storage convergence would run |
 
-For a smaller composition, list wanted modules positively:
+Exit codes: 0 success, 1 a diagnostic or a stale file, 2 a usage or file
+error. Diagnostics name the manifest file, the path in it and, for SQL, the
+position in the statement.
 
-```sh
-npx mantle generate --host cf --features spec,api
-npx mantle generate --features spec
-```
+`--check` is the gate for CI and for agents: run `mantle generate`, then
+`mantle generate --check`, then your typecheck.
 
-Dependencies are expanded in the saved selection: `api` includes Runtime and
-Spec; `admin` includes Runtime, API and MCP. Supported features are `spec`,
-`runtime`, `api`, `mcp`, `admin`, `web`. No host is needed for Spec-only. Use an
-explicit host in automation so a terminal prompt cannot stall. The selection
-lives in `mantle.config.json`; changing it later requires a deliberate migration,
-not an accidental CLI flag.
+### Schedules and Cloudflare crons
 
-## Files and ownership
+On host `cloudflare`, schedule Triggers use POSIX cron. `mantle generate` translates each enabled
+one with `toCloudflareCron` and refuses (exit 1, before writing) one that
+Cloudflare cannot run the same way. It writes `triggers.crons` into a new
+`wrangler.jsonc`; once the file is yours it only warns when the crons differ
+from the plan, and you update them.
 
-| File | Owner | What happens on rerun |
-|---|---|---|
-| `manifests/*.yaml`, `src/handlers.ts`, `src/home.ts` | You | Preserved. Add domain behavior and edit the blank home here. |
-| `package.json`, `mantle.config.json`, `wrangler.jsonc` | Shared | Required versions and scripts are added; existing user values are preserved or conflicts reported. |
-| `.mantle/generated/*`, `src/storage-fingerprint.json` | CLI | Rebuilt from the saved selection and manifests. |
-| `drizzle/*.sql` on Sites | Review and apply | New migrations append; applied migrations are never rewritten. |
-| `public/_mantle/admin/*` | CLI | Prebuilt Admin assets copied from the selected exact package. |
-| `.openai/hosting.json` on Sites | Sites | Binding metadata is created locally; existing project metadata is preserved. |
-
-A generated Worker imports the sealed `plan`, your handler map and the selected
-modules. Generated files carry a marker so the CLI can distinguish them from
-user-owned code. A rerun refuses to replace an unmarked file. Existing
-applications without a saved selection keep their direct-authoring compile
-behavior. For integration into an independently authored application, use the
-installed `docs/skills/integrate/SKILL.md` to choose an embedding, incremental
-replacement or rebuild strategy. Generation does not verify arbitrary host
-entry points or deployment wiring.
-
-`generate` accepts `--manifests <dir>` (default `./manifests`), `--output <dir>`
-(default `.mantle/generated`), `--namespace <name>`, `--host cf|chatgpt-sites`,
-`--features <comma-list>` and `--check`. Check mode exits nonzero
-on drift and does not write. Sites also accepts `--review-unique-indexes` for
-an explicit, source-bound unique tuple migration and duplicate preflight;
-[review the D1 procedure](../chatgpt-sites/generated-app.md) before applying
-it. `mantle validate` checks grammar and linked
-capabilities; `--phase deploy` adds deployment checks. `mantle skills` projects
-the installed SDK's project skills into `.agents/skills/` and `.claude/skills/`;
-`mantle skills --check` detects drift. `mantle emit-openapi` describes HTTP
-Triggers and View REST routes; `mantle-harness indexes` and `http` inspect
-queries and running routes. [Command reference](../reference/surface.md).
-
-## Verify the assembled app
+### Storage SQL
+
+Boot creates what the plan adds (tables, columns, indexes, check triggers) and
+refuses what it cannot change safely. `--check --database` prints the same
+diff without applying it, for the `sqlite` dialect; see [Deploy and operate](../cloudflare/deploy-and-operate.md).
+
+### PostgreSQL on Cloudflare
+
+`mantle generate --dialect postgres` writes the same preset over Hyperdrive:
+`src/service.ts` opens a `pg` client per operation from `env.HYPERDRIVE`, and
+`wrangler.jsonc` binds `HYPERDRIVE` with a placeholder id and a
+`localConnectionString` for `wrangler dev`. Before deploying, create the config
+with caching off and put its id in place of the placeholder; generate warns
+until you do:
 
 ```sh
-npx mantle generate --check
-npx mantle skills
-npx mantle skills --check
-npm run typecheck
-npm run build
-npm run dev
+wrangler hyperdrive create my-app --caching-disabled --connection-string="postgres://user:pass@host:5432/db"
 ```
 
-Run scripts that exist in the selected project; Spec-only has no server to
-start. Run `npx mantle validate` after adding the first Manifest: the validator
-reports `MANIFEST_ROOT_NOT_FOUND` on an intentionally empty app. A full
-Cloudflare app should serve the blank home and Admin assets.
-For local CF Admin, copy `.dev.vars.example` to `.dev.vars`, replace
-`ADMIN_EMAIL` with the intended owner's address and `BETTER_AUTH_SECRET` with
-a random 32-byte secret, and keep `PUBLIC_ORIGIN` equal to the loopback URL
-wrangler prints. Open `/admin/sign-in` with that email and read the OTP in
-wrangler logs. Check that `/_mantle/admin/assets/*` loads, an anonymous
-`/admin/api/site-settings` request is denied, and the signed-in owner can read
-it. Verify anonymous staff MCP access fails before claiming it works. A full
-Site must also apply its reviewed
-D1 migration before the Worker boots. The public MCP endpoint is discoverable
-without a staff session; the staff endpoint requires the host's verified
-identity and a Mantle role. See [MCP and agents](../concepts/mcp-and-agents.md).
-
-For an already installed app, confirm `node_modules/@aotter/mantle/package.json`
-and the lockfile agree. Use that package's CLI and `docs/` tree; the latest
-GitHub handbook may describe a newer release. Keep all selected
-`@aotter/mantle*` packages on one exact version. The prior direct-authoring
-contract remains valid for older installed SDKs; upgrade deliberately using
-[release notes](../releases/index.md).
-
-## Source
-
-- [Generate CLI](../../../packages/mantle/src/cli/generate.ts)
-- [Project selection and package pins](../../../packages/mantle/src/cli/generate-project.ts)
-- [Cloudflare assembly](../../../packages/mantle/src/cli/generate-cloudflare.ts)
-- [ChatGPT Sites assembly](../../../packages/mantle/src/cli/generate-sites.ts)
+## 0.1.x projects
+
+`apiVersion: cms.mantle.aotter.net/v1` manifests, a `version: 1` config, `mantle validate`,
+`mantle emit-openapi`, `mantle skills` and the `mantle-harness` bin are gone.
+`docs/upgrade-0.1-to-0.2.md` in the installed package is the guide.
+
+## Bun
+
+`mantle generate --host bun` selects PostgreSQL on first run. Install `bun-types`
+for the generated project's typecheck, the selected feature/identity packages,
+then set DATABASE_URL in `.env` and run `bun src/index.ts`. For SQLite pass
+`--dialect sqlite` and optionally DATABASE_FILE (default mantle.sqlite).
+The preset is written once, uses native drivers from `@aotter/mantle/bun`, and
+owns Bun.serve, trusted socket IP and shutdown/background work. Admin binds the
+installed UI bundle. Enabled schedule Triggers fail generation; use `host: none`
+with an explicit scheduler when schedules are required. Behind a reverse proxy,
+set TRUSTED_PROXIES to its addresses: otherwise every client shares the proxy's
+address, and with it one sign-in rate limit.
+
+PostgreSQL pools must set `prepare: false`: Bun otherwise re-encodes JSON strings,
+and the role needs TEMPORARY on the database (`GRANT TEMPORARY ON DATABASE app TO
+app_role`), which describing a result takes.
+Mantle refuses another setting. Native raw result metadata comes from PostgreSQL
+with three extra round trips per result, preserving microseconds and exact numeric
+scale. No pg dependency or PostgreSQL clone is added. See ADR-0038.