@aotter/mantle 0.1.0-alpha.9 → 0.1.2-alpha.2

package/docs/handbook/concepts/runtime-and-adapters.md

@@ -0,0 +1,177 @@
+---
+description: The sealed parse-link-compile-prepare-bind pipeline, Core versus optional products, and how Cloudflare, Bun, Vercel and IndexedDB differ.
+---
+# Runtime pipeline and adapters
+
+Mantle has exactly one path from YAML to a running service, and one owner for every rule along it. This page walks that pipeline, then shows what changes and what stays fixed when you swap the host underneath it.
+
+## The sealed pipeline
+
+```txt
+ManifestSourceSet
+  -> parse + normalize -> ParsedManifestSet
+  -> link               -> LinkedManifestSet
+  -> compile            -> RuntimePlan
+  -> prepare deployment -> PreparedRevision
+  -> bind runtime       -> MantleRuntime
+```
+
+Each stage's output can be constructed only by the stage that owns it, and a failed stage produces nothing the next stage can use. Parse, link and compile are pure and deterministic.
+
+| Stage | Owns | Rejects | Must not own |
+|---|---|---|---|
+| Parse + normalize | YAML syntax and alias limits, the closed four-atom shape, atom-local rules, behavior-affecting defaults, source metadata | Unknown keys, bad envelopes, unsupported JSON Schema keywords, invalid index and `uiSchema` shapes | Cross-atom references, handlers, storage, routes |
+| Link | Duplicate symbols, cross-atom references, guard graphs, translations, manifest-owned route and tool collisions | Unknown Schema or Procedure references, guard self-reference and chains, duplicate HTTP paths, MCP tool-name collisions | I/O, selected modules, handler availability |
+| Compile | Immutable lookup records, authorization plans, Trigger indices, Procedure descriptors, logical View plans, the semantic fingerprint | Nothing new; it projects an already valid graph | Connections, repositories, handlers, requests, templates, assets |
+| Prepare | Storage migrations, native Schema tables and indexes, prepared Views, handler availability, selected capability and reserved-route checks, the readiness revision | Missing handler refs, reserved-path conflicts, `View.spec.sql` on storage that does not declare the SQLite dialect | Re-interpreting YAML, executing requests |
+| Bind and invoke | Semantic ports, handler dispatch, parameter binding, centralized authorization, content, View, Procedure, Trigger and lifecycle operations | Invalid input, unauthorized callers, lifecycle transitions the state machine forbids | DDL, route mounting, assets, HTTP, session and cache policy |
+| Optional modules and adapters | Web and Admin composition; request, session, cache and platform translation | Whatever the platform itself rejects | Re-parsing, re-linking, or a second authorization stack |
+
+Nothing downstream may reinterpret raw manifests. That is what makes a rule verifiable exactly once, and why every rejection carries a diagnostic code instead of a guess. See [Diagnostic codes](../reference/diagnostics.md).
+
+## What `mantle generate` produces
+
+```sh
+pnpm exec mantle generate
+pnpm exec mantle generate --check
+```
+
+`generate` validates and compiles the manifests directory, then writes one typed module at `.mantle/generated/mantle.ts` containing the sealed `plan` with its fingerprint, the handler types, and two entry points:
+
+```ts
+import { bindMantle, createMantle, plan } from "../.mantle/generated/mantle.js";
+
+// Eager: one preparation attempt, no caching and no retry.
+const mantle = await createMantle({ storage, handlers, ports });
+const notes = await mantle.views.publishedNotes();
+await mantle.entries.orders.createDraft({ data, authorId: user.id });
+
+// Or bind a runtime the host already assembled and owns the lifecycle of.
+const bound = bindMantle(runtime);
+await bound.runtime.archive.execute({ id, ctx });
+```
+
+Generated property names are deterministic lower-camel identifiers; calls keep the authored wire names internally, and a collision is an error (`CODEGEN_IDENTIFIER_COLLISION`). Code generation is a pure projection: it never caches, retries, mounts routes or owns host lifecycle, and the typed API keeps its raw `runtime` so it hides nothing. Skipping generation is valid — call `runtime.executeView({ view: "published-notes" })` by name.
+
+## Core, optional products, adapters
+
+| Layer | Package | Responsibility |
+|---|---|---|
+| Core | `@aotter/mantle-spec` | Sources, parse, normalize, link, introspection, code generation. No runtime or platform dependency. |
+| Core | `@aotter/mantle-runtime` | `RuntimePlan`, preparation contracts, semantic storage ports, `MantleRuntime`. No Web, Admin or platform dependency. |
+| Optional product | `@aotter/mantle-web` | Public HTML, Markdown mirrors, `llms.txt`, sitemap, SEO, preview, templates, path composition. Owns no routes. |
+| Optional product | `@aotter/mantle-admin` | Admin API orchestration, OAuth surfaces, the asset contract. |
+| Optional product | `@aotter/mantle-admin-ui` | The pre-built React Admin SPA artifact. |
+| Adapter | `@aotter/mantle-cloudflare`, `-bun`, `-vercel`, `-indexeddb` | Bind platform storage, lifecycle, request, session, cache and asset concerns to Core ports. |
+
+The umbrella `@aotter/mantle` installs Spec and Runtime only; every other subpath is an optional peer you install when you select it. Core does not reserve Admin paths or serve a UI when the module is absent.
+
+## Capability matrix
+
+| | Cloudflare | Bun | Vercel Functions | IndexedDB |
+|---|---|---|---|---|
+| Storage | D1 through the SQLite chain | Caller-owned `bun:sqlite` `Database` | Any injected `MantleStorageAdapter`; optional `/libsql` Turso driver | One application-owned IndexedDB database |
+| Public View REST | Yes | Yes | Yes | Not mounted; call the runtime directly |
+| HTTP Triggers | Yes | Yes | Yes | Not mounted; `runtime.invokeTrigger` |
+| Admin, Auth, OAuth | Yes | Absent | Absent | Absent |
+| MCP `/mcp`, `/mcp/staff` | Yes | Absent | Absent | Absent (WebMCP is a separate browser binding) |
+| Public web pages | Opt-in via `mountPublicRoutes` plus templates and a resolver | Absent | Absent | Absent |
+| Cache policy | Owned, applied at the final boundary | Host-owned | Host-owned | n/a |
+| `View.spec.sql` | Supported | Supported | Supported on a SQLite-family driver | Rejected at preparation |
+| Optional capabilities | R2 media, Queues deferred hooks, KV catalog cache | None | Platform `waitUntil` | None |
+| Host still owns | Application routes and frontend | `Bun.serve`, auth, CSRF, database shutdown | Web handler, auth, CSRF, route composition | Database naming, persistence requests, UI invalidation, sync |
+
+The Manifest does not change across that row. What changes is which surfaces exist to reach it.
+
+## Storage ports
+
+A storage adapter prepares one `RuntimePlan` into semantic ports. Three are required:
+
+| Port | Role |
+|---|---|
+| `MantleStorageAdapter` / `PreparedMantleStorage` | Prepares one plan into a revision: migrations, native Schema tables and indexes, prepared Views |
+| `EntryRepository` and `EntryReader` | Entry writes and reads |
+| `ViewQueryExecutor` | Executes compiled logical View plans |
+
+Two are optional and only when a feature needs them: `MediaStorage` for upload flows, and `DeferredHookDispatcher` for at-least-once `after_*` delivery. `DatabaseDriver` is not a portability contract — it is the reusable SQLite/D1 seam. A PostgreSQL, MongoDB or application-owned-table adapter implements the semantic ports directly rather than emulating D1.
+
+Declarative Views compile to logical plans once, and preparation lowers those plans to native queries. `View.spec.sql` is the exception: it is explicitly SQLite-only in v0.1. Storage that does not declare that dialect rejects such a View at preparation with `VIEW_DIALECT_UNSUPPORTED`, before mutating any state. Mantle does not guess a translation and ships no universal query driver. A View that must run everywhere uses `from` with a filter AST; see [View](../reference/view.md).
+
+## Embedding each adapter
+
+Cloudflare, through the conventional facade:
+
+```ts
+import { createMantleWorker } from "@aotter/mantle/cloudflare";
+import { plan } from "../.mantle/generated/mantle.js";
+
+export default createMantleWorker({ plan });
+```
+
+Bun, with the server and SQLite handle staying yours:
+
+```ts
+import { Database } from "bun:sqlite";
+import { createBunMantle } from "@aotter/mantle-bun";
+
+const database = new Database("app.sqlite");
+const mantle = createBunMantle({ plan, database, handlers });
+
+Bun.serve({
+  async fetch(request) {
+    return (await mantle.handle(request)) ?? new Response("not found", { status: 404 });
+  },
+});
+```
+
+Vercel Functions, with storage injected:
+
+```ts
+import { SqliteMantleStorageAdapter } from "@aotter/mantle-runtime";
+import { createVercelMantle } from "@aotter/mantle-vercel";
+import { LibsqlDatabaseDriver } from "@aotter/mantle-vercel/libsql";
+
+const mantle = createVercelMantle({
+  plan,
+  handlers,
+  storage: new SqliteMantleStorageAdapter(new LibsqlDatabaseDriver(client)),
+});
+```
+
+Browser IndexedDB, with no HTTP transport at all:
+
+```ts
+import { bootMantleRuntime } from "@aotter/mantle-runtime";
+import { IndexedDbMantleStorageAdapter } from "@aotter/mantle-indexeddb";
+
+const storage = new IndexedDbMantleStorageAdapter({ databaseName: "my-app" });
+const runtime = await bootMantleRuntime({ plan, storage, handlers });
+await runtime.invokeTrigger({ trigger: "rename-board-mcp", input, ctx });
+```
+
+`handle()` returns `null` for a path Mantle does not own, so the host keeps its own routes. Never treat a Vercel Function's filesystem or `/tmp` as durable state.
+
+## The adapter-boundary rule
+
+Platform bindings belong at the composition root only: the Worker entry, `createMantleWorker` options, the `bindings` hook and `wrangler.jsonc`. Procedure handlers receive them through `ctx.env`.
+
+Application code does not bypass Mantle's storage ports to write Schema tables, `site_config`, media, or Auth tables. Each Schema is a native table, but Mantle still owns its metadata columns, lifecycle checks, and optimistic concurrency. Use Manifests, runtime use cases, `runtime.entries`, `runtime.siteConfig`, generated `bindMantle(runtime)`, and Views instead. An application may own separate tables behind its own repository; that is different from writing around Core's invariants.
+
+If a normal feature cannot be expressed through a purpose-shaped surface, treat that as a gap in the abstraction rather than teaching the project Mantle's internals. Internals change between versions; the ports do not.
+
+## Related
+
+- [The four atoms](./four-atoms.md) — what the pipeline is compiling.
+- [Bindings and primitives](../cloudflare/bindings.md) — the composition root in practice.
+- [Low-level composition](../cloudflare/low-level-composition.md) — assembling a Worker from the same public primitives.
+- [Project layout and the CLI loop](../start/project-and-cli.md) — where `generate` sits in the daily loop.
+
+## Source
+- [`docs/adr/0019-sealed-manifest-runtime-pipeline.md`](../../../docs/adr/0019-sealed-manifest-runtime-pipeline.md)
+- [`docs/adapter-guide.md`](../../../docs/adapter-guide.md)
+- [`packages/mantle/README.md`](../../../packages/mantle/README.md)
+- [`packages/mantle-runtime/README.md`](../../../packages/mantle-runtime/README.md)
+- [`packages/adapters/bun/README.md`](../../../packages/adapters/bun/README.md)
+- [`packages/adapters/vercel/README.md`](../../../packages/adapters/vercel/README.md)
+- [`packages/adapters/indexeddb/README.md`](../../../packages/adapters/indexeddb/README.md)
+- [`skills/develop/SKILL.md`](../../../skills/develop/SKILL.md)

package/docs/handbook/concepts/views.md

@@ -0,0 +1,191 @@
+---
+description: Views are the only public read surface — surfaces, declarative versus SQL form, params, identity filters, pagination, the MCP mirror and index planning.
+---
+# Reads: Views, REST and MCP
+
+Every read that leaves the runtime goes through a View. This page explains why, how a View is shaped, and what it costs at query time. Field-level rules are in the [View reference](../reference/view.md).
+
+## One read surface, not two
+
+A View is a named, read-only query over Schemas. Declaring one is enough: it mounts on its surface with no [Trigger](./procedures-and-triggers.md) involved. Schemas are never publicly readable on their own.
+
+Exposing collections directly — a `Schema.spec.expose.rest` flag, or a `GET /api/<collection>` shortcut — was considered and rejected. A Schema declares the storage shape. Its entries carry drafts, internal status, server-stamped fields and per-row data the author never intended to publish; `contact-messages` is the canonical example, where a direct collection route would be a privacy bug by default. A View already has the right semantics — a named query with explicit fields, filter, ordering and limit — so auto-exposing it only ratifies what the manifest already says.
+
+There is no internal-only View surface either. A query that should not be externally callable stays a TypeScript helper.
+
+## Surfaces
+
+`spec.surface` is required and closed to two values.
+
+| `surface` | REST | MCP | Extra |
+|---|---|---|---|
+| `public` | `GET /api/views/<name>` | `query_view_<segment>` on `/mcp` | Listed by `GET /api/views` |
+| `staff` | `GET /admin/api/views/<name>` | `query_view_<segment>` on `/mcp/staff` | `GET /admin/api/views/<name>/export`, Admin report sidebar |
+
+The adapter filters the View set before building each MCP dispatcher, so a guessed public tool call cannot reach a staff View. Surface decides transport visibility; `spec.requires` decides whether the verified caller may execute the View, on REST and MCP alike. See [Authorization](./authorization.md).
+
+## Declarative `from` versus one `sql` SELECT
+
+A View declares exactly one of `from` or `sql`; declaring both or neither is rejected.
+
+- **`from`** names a Schema and pairs with `fields`, `filter`, `orderBy` and `limit`. The filter is a closed AST: `eq`, `gt`, `gte`, `lt`, `lte`, combined with `and` and `or`. Field-to-field comparison and arithmetic are not expressible.
+- **`sql`** is a single `SELECT` with no semicolon. Every Schema is available as a logical table named after its `metadata.name`, with data properties projected as columns; quote names containing hyphens (`"post-translations"`). Combining `sql` with `fields`, `filter` or `orderBy` is rejected.
+
+**New Views should use one `SELECT`.** The declarative form remains accepted for existing manifests. SQL Views compile to native SQLite; a storage adapter that cannot run them fails at boot with `VIEW_DIALECT_UNSUPPORTED`, and static validation never executes the statement — run it against the selected adapter or through `mantle-harness indexes`.
+
+## Params
+
+`spec.params` is a JSON Schema with `type: object` and a `properties` map. The reserved names `page`, `show` and `cursor` are rejected (`VIEW_PARAMS_RESERVED_NAME`); the rest of the query-string namespace is yours.
+
+In the declarative form, a filter value may be the sentinel `{ $param: <name> }`. The name must resolve to a declared param and must appear in `params.required`. In a SQL View, params are named bindings written `:name`; they are bound, never interpolated, and every `:name` must be a required declared param.
+
+## Identity-bound Views
+
+A View can filter to the calling user's own rows with the closed sentinel `{ "$ctx.user": "id" }`. Three rules apply, each with its own diagnostic:
+
+1. It appears only under `eq`, as a single-key object whose value is the literal string `"id"` — otherwise `VIEW_FILTER_CTX_USER_REF_INVALID`.
+2. The View must require a signed-in user: `ctx.user` in `requires.auth.all` — otherwise `VIEW_FILTER_CTX_USER_REF_REQUIRES_AUTH`.
+3. The compared field must be the leftmost field of a declared index on the source Schema — otherwise `VIEW_FILTER_CTX_USER_REF_REQUIRES_INDEX`.
+
+This is a filter, not a row-level policy engine. `requires` authorizes the whole query; it does not inject per-row visibility predicates. Membership, payment and entitlement checks belong in a guard Procedure. See [Procurement approvals](../examples/procurement-approvals.md).
+
+## Shared response cache
+
+Caller-independent published data may declare `cache: { sharedMaxAge: <seconds> }`, with an integer from 1 through 86400. The Cloudflare adapter applies it only to anonymous REST responses when the Worker also has a stable `cacheScope`. Staff, guarded, identity-bound, SQL and operational-schema Views cannot opt in; MCP and WebMCP calls remain private. Publishing writes purge the deployment-scoped tag after the canonical write.
+
+## Pagination and the envelope
+
+REST callers pass `?page=<1-indexed>&show=<page size>`; MCP callers pass the same two names as tool arguments.
+
+| Knob | Behavior |
+|---|---|
+| `View.spec.limit` | The server-enforced cap. Missing or invalid becomes 50; otherwise the floor of the value, capped at 500. |
+| `show` | Missing, non-numeric or non-positive becomes the cap; otherwise the smaller of the request and the cap. |
+| `page` | Missing, non-numeric or below 1 becomes 1. |
+
+```json
+{ "ok": true, "data": { "rows": [], "page": 1, "show": 20, "hasMore": false } }
+```
+
+`hasMore` is lazy: it is true exactly when `rows.length === show`. There is no `COUNT(*)` and no `LIMIT n+1` probe, so a full final page reports `hasMore: true` and the next request returns an empty list. That is one false positive on the boundary in exchange for no extra round-trip on every request.
+
+Query strings arrive as strings, so the transport boundary coerces each declared param: `string` is identity, `integer` uses `parseInt` and rejects float-like input, `number` uses `Number()`, `boolean` accepts only `"true"` and `"false"`, and an `enum` is matched against its array. A missing required param or a coercion failure is `INPUT_VALIDATION_FAILED` (HTTP 400). Unknown query-string keys are ignored.
+
+## Staff report lists
+
+A staff View may opt into Admin's standard list chrome without changing its REST or MCP contract:
+
+```yaml
+spec:
+  uiSchema:
+    list:
+      columns: [orderNumber, customerName, orderStatus]
+      searchFields: [orderNumber, customerName, customerEmail]
+      filterFields: [orderStatus]
+```
+
+These names are output field names — SQL aliases for a `sql` View, Schema properties or reserved columns for a `from` View. Admin applies search and exact filters *before* pagination and carries them into `GET /admin/api/views/<name>/export`, whose CSV contains every matching row, not only the visible page. `uiSchema` is staff-only; a public View that declares it is `VIEW_UI_INVALID`.
+
+## The MCP mirror
+
+Every View is also a tool. The name is the View name lowercased with hyphens replaced by underscores, prefixed `query_view_`. Its input schema is `params.properties` plus `page` and `show`, and it is annotated `readOnlyHint: true`. One executor and one response shape serve REST and MCP, so an agent and a downstream service read exactly the same rows. See [MCP and agents](./mcp-and-agents.md).
+
+## Performance: declare the index the query needs
+
+On SQLite and D1, every Schema is a native table and each top-level property is
+a native column. Declared `indexes` and `uniqueIndexes` become B-tree indexes
+over those columns; Core-compiled projections, filters and ordering reference
+the same columns directly.
+
+Declare the **smallest ordered index justified by the measured path**, and respect SQLite's leftmost-prefix rule. An index on `[locale, publishedAt]` serves `WHERE locale = ?`, `WHERE locale = ? AND publishedAt > ?`, and `WHERE locale = ? ORDER BY publishedAt`. It does not serve `WHERE publishedAt > ?` alone. If a second hot path needs a different leading field, that is a second index — not a reason to enumerate every permutation, since each index costs storage and slows every write.
+
+```sh
+pnpm exec mantle-harness indexes --require-public --format text
+```
+
+The harness applies the real migrations and generated DDL, seeds skewed rows,
+compiles and executes the actual View SQL, and records `EXPLAIN QUERY PLAN`. A
+healthy indexed plan contains `SEARCH <schema-table> USING INDEX`; an indexed
+filter-and-order path should not contain `USE TEMP B-TREE FOR ORDER BY`.
+Findings are advisory unless you opt a View into the gate. Never change
+user-visible filter or ordering semantics just to make the gate pass.
+
+## Example: a public localized list
+
+```yaml
+apiVersion: cms.mantle.aotter.net/v1
+kind: View
+metadata:
+  name: published-announcements
+spec:
+  title: Published announcements
+  surface: public
+  from: announcements
+  cache: { sharedMaxAge: 3600 }
+  params:
+    type: object
+    additionalProperties: false
+    required: [locale]
+    properties:
+      locale: { type: string }
+  fields: [id, slug, locale, title, summary, publishedAt]
+  filter:
+    and:
+      - eq: { field: status, value: published }
+      - eq: { field: locale, value: { $param: locale } }
+      - gte: { field: publishedAt, value: 0 }
+  orderBy:
+    - { field: publishedAt, direction: desc }
+  limit: 50
+```
+
+The source Schema declares `indexes: [[locale, publishedAt]]`. The `gte publishedAt 0` clause is what keeps the ordered column inside the indexed range rather than forcing a sort. Omitting `locale` returns 400.
+
+## Example: a staff SQL report
+
+A `sql` View earns its keep when the answer is not one row per entry. Here an operational `requests` Schema stores a `tags` array, and the report needs one row per tag so staff can search and filter by tag:
+
+```yaml
+apiVersion: cms.mantle.aotter.net/v1
+kind: View
+metadata:
+  name: requests-by-tag
+spec:
+  title: Requests by tag
+  surface: staff
+  sql: |
+    SELECT r._mantle_id AS requestId,
+           r.subject AS subject,
+           r.requestStatus AS requestStatus,
+           tag.value AS tag,
+           r._mantle_created_at AS createdAt
+    FROM requests AS r
+    JOIN json_each(r.tags) AS tag
+    WHERE r.requestStatus = :requestStatus
+    ORDER BY r._mantle_created_at DESC
+  params:
+    type: object
+    required: [requestStatus]
+    properties:
+      requestStatus: { type: string, enum: [open, waiting, closed] }
+  uiSchema:
+    list:
+      columns: [subject, tag, requestStatus]
+      searchFields: [subject, tag]
+      filterFields: [tag]
+  limit: 200
+```
+
+`tags` is an array property, so `json_each` unnests it and one request appears once per tag. The runtime wraps the whole statement as a subquery before applying pagination, which is why Admin's search and filters attach to the SQL output aliases — `tag` is a real filterable column even though no Schema property is named `tag`. See [Commerce and settlement](../examples/commerce-transaction.md) for the same technique over order lines.
+
+## Source
+
+- [`docs/adr/0012-views-as-public-rest.md`](../../../docs/adr/0012-views-as-public-rest.md)
+- [`docs/schema-indexes.md`](../../../docs/schema-indexes.md)
+- [`packages/mantle-spec/src/domain/service/ManifestGraphValidator.ts`](../../../packages/mantle-spec/src/domain/service/ManifestGraphValidator.ts)
+- [`packages/mantle-runtime/src/domain/service/Pagination.ts`](../../../packages/mantle-runtime/src/domain/service/Pagination.ts)
+- [`packages/mantle-runtime/src/domain/service/ViewParamCoercer.ts`](../../../packages/mantle-runtime/src/domain/service/ViewParamCoercer.ts)
+- [`packages/mantle-runtime/src/infrastructure/storage/SqliteViewCompiler.ts`](../../../packages/mantle-runtime/src/infrastructure/storage/SqliteViewCompiler.ts)
+- [`packages/mantle-runtime/src/infrastructure/http/createMantleRequestHandler.ts`](../../../packages/mantle-runtime/src/infrastructure/http/createMantleRequestHandler.ts)
+- [`packages/adapters/cloudflare/src/mount/mountRuntimeEndpoints.ts`](../../../packages/adapters/cloudflare/src/mount/mountRuntimeEndpoints.ts)
+- [`skills/develop/SKILL.md`](../../../skills/develop/SKILL.md)