@aotter/mantle 0.2.0-alpha.6 → 0.2.0-alpha.8

package/docs/handbook/cloudflare/deploy-and-operate.md

@@ -1,124 +1,59 @@
 ---
-description: Deploy a Mantle 0.2.0 service to Cloudflare Workers, change Schemas safely with storage convergence, resolve a blocked change, and operate schedules, expiry and the boot handshake.
+description: Install schema explicitly, migrate before deploying, and operate a native Worker.
 ---
 # Deploy and operate
 
-## First deploy
+Run the same generation command with `--check`, typecheck the application and test its selected routes before deploying. Keep generated TypeScript, SQL and Wrangler edits committed; keep bundle `dist/` untracked. The app owns the database, auth secrets, trusted origin and assets.
 
-```sh
-pnpm exec mantle generate --check          # nothing stale
-pnpm exec tsc --noEmit
-pnpm exec wrangler d1 create my-service     # copy database_id into wrangler.jsonc's d1_databases[0]
-pnpm exec wrangler secret put BETTER_AUTH_SECRET   # identity mantle
-pnpm exec wrangler secret put ADMIN_EMAIL
-pnpm exec wrangler deploy
-```
-
-With dialect `postgres`, create the Hyperdrive config instead of the D1
-database (`pnpm exec wrangler hyperdrive create my-service --caching-disabled
---connection-string=…`) and copy its id into `hyperdrive[0].id`. Give the
-role a statement limit and UTC, which boot checks (`ALTER ROLE app SET
-statement_timeout = '10s'; ALTER ROLE app SET TimeZone = 'UTC'`); the boot
-error names any other setting to change.
-
-Set `PUBLIC_ORIGIN` in `wrangler.jsonc` `vars` to the deployed origin, and
-replace the console email sender first
-([Authentication](./authentication.md#production)). The first request boots
-the runtime, which creates every table the plan needs on the empty database.
-
-Deploying anywhere is your choice: Cloudflare and Bun have generated presets,
-`host: none` composes any other, and you may always self-host. Mantle Cloud is one option, never a
-requirement.
-
-## Changing Schemas
-
-There are no migration files. Edit the manifests, run `mantle generate`, and
-deploy. The first request after a plan change compares the database with the
-plan:
-
-- **Applied**: a new Schema table, a new field column, a new index, a new
-  unique index that existing rows satisfy, new `checks` and search triggers.
-- **Blocked** (`STORAGE_CHANGE_BLOCKED`, the service refuses to serve): a
-  unique index that existing rows break, a field whose type changed, an index
-  whose columns changed under the same name, an undeclared unique index.
-- **Kept, with a warning**: a column or non-unique index the plan no longer
-  declares. Nothing is dropped.
-
-Before deploying, see what boot will do against a local SQLite file, such as
-Wrangler's local D1 under `.wrangler/state/v3/d1/` after it has run the
-previous plan:
-
-```sh
-pnpm exec mantle generate --check --database .wrangler/state/v3/d1/<…>.sqlite
-```
+## Public fetch routing
 
-It prints the SQL boot would run, the undeclared differences as comments, or
-the blocked change (exit 1). It reads the file read-only. The dry run is the
-SQLite dialect's; PostgreSQL has none.
+Workers use `compatibility_flags: ["nodejs_compat", "global_fetch_strictly_public"]`.
+The public-fetch flag routes native `fetch` through Cloudflare's public Internet
+boundary. OAuth verification needs it to fetch JWKS on the service's own public
+hostname; CIMD also requires this boundary for client metadata. `nodejs_compat`
+alone does not establish it.
 
-### Resolving a blocked change
+`mantle generate cloudflare-*` adds these flags without replacing other flags,
+including in environments with their own `compatibility_flags` array. The same
+command with `--check` refuses missing flags without writing. JSON/JSONC configs
+are updated; TOML receives manual-edit instructions. Keep both flags in every
+TOML environment override. An explicit `global_fetch_private_origin` is preserved
+for applications without OAuth/CIMD, but conflicts with selected OAuth/CIMD auth
+and with the public-fetch flag: generation refuses that config without rewriting
+it. Such applications must use public routing for their OAuth/CIMD transport.
 
-Mantle verifies; you change. The diagnostic names the change and hints SQL
-that would reach the plan. For example, to make `email` unique when duplicates
-exist: remove the duplicates (with a Procedure run as the system caller, or
-`wrangler d1 execute`), then deploy again; boot creates the index. To rename a
-field: add the new field, copy the data with a Procedure or SQL, then drop the
-old field from the manifest (its column stays, unused). Never edit `_mantle_*`
-tables.
+After deployment, complete a real OAuth sign-in and use the resulting bearer
+for MCP `initialize` and `tools/list`. Local development and cookie-only Admin
+checks do not verify production public-hostname routing.
 
-### Hosts that refuse DDL from the Worker
+## Install and change schemas
 
-If a host applies schema changes only through its own migration mechanism,
-take the SQL `mantle generate --check --database` prints and deliver it
-through that mechanism; boot then finds the database converged.
+The runtime never creates, alters or checks tables. Boot and requests run no DDL, introspection or schema checks.
 
-## Schedules
+For the first install, apply generated `schema.sql`, including selected auth/site DDL, to an empty database. Apply selected seed SQL explicitly.
 
-`wrangler.jsonc` `triggers.crons` holds the Cloudflare spelling of every
-enabled schedule Trigger. `mantle generate` warns when it drifts from the plan;
-update it by hand (the file is yours). Locally:
+For every later schema change, the application's coding agent diffs the regenerated `schema.sql`, writes a migration with the host's native tool (`wrangler d1 migrations` or the app's PostgreSQL migration tool), rehearses it on a copy, and applies it before deploying the new code. Cloud executes migrations at deploy time.
 
-```sh
-pnpm exec wrangler dev --local --test-scheduled
-curl 'http://127.0.0.1:8787/__scheduled?cron=0+3+*+*+1'
-```
+Destructive changes and type changes need explicit data migration. Mantle drops nothing. Initial `schema.sql` is not an upgrade script.
 
-The cron in the URL is the Cloudflare spelling.
+For D1, create a migration with `pnpm exec wrangler d1 migrations create <database> <name>`, review its SQL, then rehearse `pnpm exec wrangler d1 migrations apply <database> --local` against copied data. Apply the reviewed migrations to the target before `wrangler deploy`. Initial DDL may be placed in the first migration for a new database.
 
-## Expired rows
+For PostgreSQL, use the app's migration tool and deployment credentials. Plan lock waits and long index builds there; request credentials need no DDL privileges. Configure session settings on the role/database as described in [native execution](../guides/native-execution.md#postgresql-settings). Hyperdrive query caching must be disabled for authority and mutation visibility.
 
-TTL hides expired rows at once. To delete them, call
-`ctx.store.sweepExpired({ collection, limit })` in a schedule Trigger's `ref`
-handler (the system caller's Store has it), or `runtime.store.sweepExpired` from
-your own maintenance route, page with `nextCursor`, and use
-`delete: false` to count first.
+## Operate
 
-## The boot handshake
+Enable Workers Logs and traces in production. The top-level `observability.enabled` alone does not enable traces:
 
-`runtime.bootReport()` returns `{ fingerprint, coreVersion }`. Pass
-`expectedFingerprint` to `createMantle` to refuse booting any plan but the one
-you built (`PLAN_FINGERPRINT_MISMATCH`). The fingerprint is in
-`.mantle/generated/plan.json`; reformatting a manifest does not change it.
-
-## Mantle Cloud
-
-Mantle Cloud accepts its pinned 0.2 service contract through host protocol 4. A verified source receipt is required before save.
-Connect Cloud MCP and follow the plugin's `mantle` skill and its
-`references/cloud.md` workflow. Call `cloud_host_contract` for the selected
-project before installing or packing: Cloud returns the exact Core version
-and revision. Do not substitute a moving npm tag or upload a self-hosted
-Worker's platform wiring. Cloud supplies identity and D1 storage; PostgreSQL
-plans are not accepted.
+```jsonc
+"observability": {
+  "enabled": true,
+  "logs": { "enabled": true, "head_sampling_rate": 1 },
+  "traces": { "enabled": true, "head_sampling_rate": 0.01 }
+}
+```
 
-The skill's `scripts/mantle-cloud.mjs` helper orchestrates save, paired
-preview and publish through the connected MCP tools. Follow its literal
-`nextAction`, keeping grants on stdin and preserving the project and operation
-IDs on retry. A saved or paired version is not a serving deployment; publish
-only under the user's authorization and verify Cloud's returned serving state.
-Its offline `check` reports `cloud: "not_checked"`, which proves local
-readiness only. Use Cloud's workflow, not `wrangler deploy`, for that target.
+Auth configuration and production email delivery are app-owned; follow [authentication](authentication.md). Keep secrets in the platform's secret store. With `secrets.required`, `wrangler deploy` refuses to upload while a required secret is unset: set them with `wrangler secret put` (or pass `--secrets-file` on the first deploy). Serve the selected Admin assets through the [native Admin](../guides/native-admin.md) shell contract.
 
-## Moving data from 0.1.x
+Scheduling is app-owned. Wire generated system-only TTL sweeps into `scheduled()` and configure Wrangler crons explicitly. Bound each invocation and retain a cursor if work remains; see [TTL maintenance](../guides/native-execution.md#ttl-maintenance). Expired rows remain stored and hold unique keys until removed.
 
-Give the 0.2.0 service a new database and import through
-`runtime.store`; `docs/upgrade-0.1-to-0.2.md` describes the steps.
+After deployment, verify public and protected reads, member/admin sign-in, a persisted write, stale OCC and selected MCP/Admin/web routes. An `OUTCOME_UNKNOWN` write may have committed; reconcile before deciding what to do next and never retry automatically.

package/docs/handbook/cloudflare/media-r2.md

@@ -1,144 +1,73 @@
----
-description: Media uploads in Mantle 0.2.0 on Cloudflare R2 — site media purposes, r2MediaStorage, wiring Admin and staff MCP media tools, the presigned upload flow, and what to store in a Schema.
----
-# Media uploads with R2
+# Optional site settings and media
 
-Media is optional. It needs three things in `src/service.ts`: site defaults
-with media purposes, an `r2MediaStorage`, and both passed where they belong.
-The bytes go from the uploader straight to R2; the Worker never proxies them.
+Declare a TypeScript `siteDefaults` object using the 0.2 `SiteDefaults` shape and
+pass it to generation, or use `mantle generate ... --site ./site-config.ts`.
+Apply the emitted `schema.sql` to a new database; apply idempotent `site-seed.sql`
+on subsequent deployments to seed only absent operator fields. Import `site` from the generated
+`site` module; call `site(db, verifiedCaller).read()`, `.updateSettings({brand,title,
+description})`, or `.media(storage)` to access the library. Construction runs no
+queries, migrations, seeds, or storage work. The library accepts verified admin
+Callers of any credential kind; the Admin handler must additionally keep its
+cookie-session-only gate and the shared session-mutation origin check. The optional module is absent from
+a Worker that does not import it.
 
-## 1. A public bucket and S3 credentials
-
-- An R2 bucket served publicly, from a custom domain or `pub-<hash>.r2.dev`.
-  That URL is `publicBase`.
-- An R2 API token with write access to the bucket, for presigning PUT URLs:
-  `R2_ACCESS_KEY_ID` and `R2_SECRET_ACCESS_KEY` as Worker secrets.
-- A CORS rule on the bucket that allows `PUT` with `Content-Type` from your
-  Admin's origin, if a browser uploads.
-
-```jsonc
-// wrangler.jsonc
-"r2_buckets": [{ "binding": "MEDIA", "bucket_name": "my-media" }],
-"vars": { "MEDIA_PUBLIC_BASE": "https://media.example.com", "R2_ENDPOINT": "https://my-media.<account>.r2.cloudflarestorage.com" }
-```
-
-## 2. Wire it
+Each media purpose in `siteDefaults.media.purposes` lists mime **slots** and a byte cap per full mime type:
 
 ```ts
-import { AwsClient } from "aws4fetch";
-import { d1Storage, r2MediaStorage } from "@aotter/mantle/cloudflare";
-
-// storage, with the purposes uploads may use
-export const mantle = createMantle(service, {
-  plan, schedules: true,
-  storage: (env) => d1Storage(env.DB, { site: {
-    title: "My site",
-    media: { purposes: [{ name: "cover", required: ["image/webp", "image/jpeg,image/png"], maxBytes: { "image/webp": 2_000_000, "image/jpeg": 2_000_000, "image/png": 2_000_000 } }] },
-  } }),
-});
-
-// in mount(): one media storage, shared by Admin and staff MCP
-const media = r2MediaStorage({
-  bucket: env.MEDIA,
-  signer: new AwsClient({ accessKeyId: env.R2_ACCESS_KEY_ID, secretAccessKey: env.R2_SECRET_ACCESS_KEY, region: "auto", service: "s3" }),
-  endpoint: env.R2_ENDPOINT,
-  publicBase: env.MEDIA_PUBLIC_BASE,
-});
-const admin = guard(createAdminSurface(runtime, { basePath: "/admin", media, /* …the generated options… */ }));
-const staffMcp = guard(createMcpSurface(runtime, {
-  basePath: "/mcp/staff", surface: "staff", media, resourceMetadata,
-}), { resourceMetadata });
-```
-
-`aws4fetch` is the application's dependency; Mantle only calls its `sign`.
-Without both `media` and site defaults, every Admin media route answers 501
-`MEDIA_NOT_CONFIGURED`, and MCP advertises no media tools. Public MCP never
-advertises media tools. The generated preset does not invent bucket credentials;
-add the same explicit adapter to its application-owned service composition.
-
-## 3. The upload flow
-
-All through Admin's API, `editor` role or higher:
-
-1. `POST /admin/api/media/uploads` with
-   `{ filename, purpose, variants: [{ mimeType, byteSize, role }], alt?, caption? }`.
-   The purpose's MIME slots and per-type byte caps are checked here, and exactly one variant must be `role: "primary"`. The answer is
-   `{ uploadGroupId, capabilities: [{ role, mimeType, method: "PUT", uploadUrl, requiredHeaders }], expiresAt }`.
-2. `PUT` each variant's bytes to its `uploadUrl` with its `requiredHeaders`.
-3. `POST /admin/api/media/uploads/{uploadGroupId}/commit` (optionally with
-   `{ alt, caption }`). Commit checks every stored object's type and size,
-   then copies each to a public key no upload URL can reach. A rejected group
-   publishes nothing and its upload objects are deleted. An expired or unknown
-   group is 410 `MEDIA_UPLOAD_EXPIRED`.
-
-The committed asset is `{ id, variants: [{ role, mimeType, publicUrl, storageKey, byteSize }], alt?, caption?, createdAt }`.
-`GET /admin/api/media` lists assets (`search`, `limit`, `cursor`); `GET`,
-`PATCH` (`alt`, `caption`) and `DELETE /admin/api/media/{id}` manage one.
-Delete removes the objects first, then the row, so a partial failure can be
-retried.
-
-Optimize images where they are made (in the browser or the agent), and upload
-the variants; Mantle does not transform images.
-
-## Staff MCP upload flow
-
-Connect the tenant's `/mcp/staff` using its own identity, with the `mcp` scope
-and `editor` or `owner` role. Cloud project membership is not tenant staff.
-The verified role is checked again on every call, as in Admin.
-
-1. Call `get_media_upload_policy` to read the declared purpose MIME slots and
-   byte caps. Obtain the chat attachment bytes in the agent environment and
-   optimize/encode the required variants there; do not ask the user to run a
-   terminal command.
-2. Call `create_media_upload` with the same body as Admin's create endpoint.
-3. PUT every variant's actual bytes to its capability URL using all returned
-   `requiredHeaders`, before `expiresAt`. Bytes never travel as base64 MCP JSON.
-4. Call `commit_media_upload` with `{ uploadGroupId, alt?, caption? }`.
-   Verify success before using the returned permanent asset `id`.
-5. Pass that ID to the application's content Procedure. Images are content
-   operations, not a new Cloud source build or deploy.
-
-`list_media_assets`, `get_media_asset`, `update_media_asset` (`id`, `alt?`,
-`caption?`) and `delete_media_asset` (`id`) share the same library and editor
-permission as Admin. A create refusal publishes nothing. Expired groups need
-new capabilities; partial delete failures can retry the same ID. Keep upload
-capabilities out of logs and persisted content.
-
-## Storing media in a Schema
-
-Store the committed asset ID, not its temporary capability URL:
-
-```yaml
-coverAssetId:
-  type: string
-  x-mantle-ref: media_assets
-  x-mcp-hint: media-image
+{ name: "cover", required: ["image/jpeg,image/png,image/webp"], maxBytes: { "image/jpeg": 5_000_000, "image/png": 5_000_000, "image/webp": 5_000_000 } }
 ```
 
-The string reference defaults to the native library's `id` and enables Admin's
-existing MediaPicker. The hint alone does not enable the picker. Do not author
-a replacement `media_assets` Schema: the site's library owns that table.
+`required` entries use `<input accept>` grammar (full mime, comma list, or a subtype shorthand such as `webp`);
+`maxBytes` must cover every mime the slots allow, keyed by the full mime type.
 
-An asset ID is not an image URL. Resolve it with the existing site library in
-an application-owned frontend handler, then use a variant's `publicUrl`:
+OSS applications own their R2 binding, public bucket URL and S3 credentials.
+Mantle Cloud supplies that wiring by default; the OSS package has no Cloud API
+or credential management. Import `r2MediaStorage` from `@aotter/mantle/media/r2`:
 
 ```ts
-// The same media adapter used by Admin and staff MCP.
-const asset = await runtime.site!.media(media).get(post.coverAssetId);
-const primary = asset.variants.find((variant) => variant.role === "primary")!;
-// Render primary.publicUrl as the image src; asset.alt as its alt text.
+const storage = r2MediaStorage({
+  bucket: env.MEDIA,
+  endpoint: "https://bucket.account.r2.cloudflarestorage.com",
+  publicBase: "https://media.example.com",
+  signer, // app-owned SigV4 signer: sign(url, {method:"PUT",aws:{signQuery:true,service:"s3"}})
+});
+const media = site(db, caller).media(storage);
+const upload = await media.createUpload({
+  filename: "cover.jpg", purpose: "cover",
+  variants: [{mimeType:"image/jpeg",byteSize:1234,role:"primary"}]
+});
+// Client PUTs bytes directly to upload.capabilities[i].uploadUrl,
+// with its requiredHeaders. Never send the bytes through Mantle.
+const asset = await media.commitUpload(upload.uploadGroupId);
 ```
 
-A public frontend route must enforce the content's publication and visibility
-before resolving its media. Return only public URLs/alt/variant metadata it
-needs; never expose upload capabilities or a staff MCP credential. Native site
-product tables are not manifest Store tables: do not query `media_assets` in
-View/Procedure SQL or widen the SQL grammar to resolve them. A custom handler
-can use the existing library capability while its content reads remain
-caller-scoped.
-
-## Stale uploads
-
-A group created but never committed leaves objects under `uploads/` until its
-URLs expire. The package skill `docs/skills/media-gc/SKILL.md` audits and
-removes them.
+An existing S3 signer (for example an application already using aws4fetch) fits
+this port. This package adds no AWS SDK. The signer must return a SigV4
+query-signed PUT URL and preserve `X-Amz-Expires`; never give its credentials to
+the client. Configure bucket CORS to allow PUT from the application's origins
+and the returned Content-Type header. Serve the `committed/` prefix on the public domain; these keys differ from 0.2's `{purpose}/{group}/{role}` keys. Keep `uploads/` private/unserved on the
+public domain and add a bucket lifecycle rule for abandoned upload keys.
+
+Commit checks every object's MIME and size, then rechecks the exact GET
+response used for the copy. Public keys cannot be reached by an upload URL. Each commit attempt has a separate
+public-key UUID, while the asset id remains the upload-group UUID. A concurrent
+loser can neither overwrite nor remove the winner's objects. After SQL selects
+the durable asset, the losing attempt removes only its own unselected copies.
+Upload sources remain until the database commit succeeds, so an uncertain DB
+write can be reconciled/retried with the same UUID. Failed object deletion keeps
+the media row; retry the same id. An upload group expires after 15 minutes. Asset `createdAt` is the upload-group creation time (epoch milliseconds), not the later commit time.
+No R2 upload or deployment is performed by generation.
+
+A positively identified native rollback deletes only that attempt's copied
+objects. Plain/network failures lack rollback evidence, retain copies, and return
+`OUTCOME_UNKNOWN`, including a lost D1 batch response.
+An uncertain native write preserves copies, because they may belong to the
+durable winning row. If SQL durability or unselected-copy cleanup returns `OUTCOME_UNKNOWN`, reconcile
+storage with the durable `media_assets.variants` keys. Attempt-isolated copies
+may remain after an uncertain outcome; never delete the winning asset's keys or
+expire all `committed/` objects indiscriminately. The group UUID remains the
+reconciliation and retry id.
+
+Admin's `media` option takes a `MediaStorage` value, so build it where `env` is available (for example once per
+Worker in the fetch handler). Uploads use presigned S3 PUT URLs: `wrangler dev --local` has an R2 binding but no S3
+endpoint, so test the upload flow against a real bucket; local tests can cover everything up to the signed URL.