@aotter/mantle 0.1.0-alpha.9 → 0.1.2-alpha.2

package/docs/media-uploads.md

@@ -1,186 +1,5 @@
-# Cloudflare R2 Media Uploads
+# Media uploads
 
-Mantle supports staff media uploads through Staff MCP tools:
-`create_media_upload` and `commit_media_upload`. This is an optional
-post-launch feature for sites that use the Cloudflare adapter and need
-staff-managed images or files.
+This guide moved into the handbook: [`docs/handbook/cloudflare/media-r2.md`](handbook/cloudflare/media-r2.md).
 
-Do not make R2 part of the Day 1 launch path. Cloudflare R2 setup may
-require billing or a credit card, so landing and the blank starter do not
-provision it by default.
-
-## When To Enable
-
-Enable this after the site already has:
-
-- a working Cloudflare Worker deploy;
-- staff auth configured;
-- a real need for staff or an agent to upload images/files.
-
-Use Claude Code, Codex, Cursor, or another local/non-sandboxed coding
-agent for media maintenance. Claude Cowork often cannot complete this
-flow because the final upload is a direct HTTP PUT to
-`*.r2.cloudflarestorage.com`.
-
-## How It Works
-
-The upload flow is deliberately split:
-
-1. The agent calls Staff MCP `create_media_upload` with purpose, variant
-   metadata, byte sizes, and mime types.
-2. Mantle returns signed upload URLs and required headers.
-3. The agent reads the local/chat attachment bytes and PUTs them directly
-   to R2.
-4. The agent calls `commit_media_upload`.
-
-Do not pass image bytes or base64 payloads through MCP tool arguments.
-The Worker validates policy; the agent runtime performs file processing
-and upload.
-
-## Cloudflare Setup
-
-Create a bucket and public read URL:
-
-```bash
-wrangler r2 bucket create <project>-media
-wrangler r2 bucket dev-url enable <project>-media
-```
-
-In the Cloudflare dashboard, create an R2 S3 API token:
-
-1. Open **R2**.
-2. Open **Manage R2 API Tokens**.
-3. Create an Object Read & Write token.
-4. Copy the Access Key ID and Secret Access Key.
-
-The R2 binding alone cannot issue presigned PUT URLs; Mantle also needs
-these S3-compatible credentials.
-
-## `wrangler.toml`
-
-```toml
-[vars]
-R2_ACCOUNT_ID = "<account-id>"
-MEDIA_PUBLIC_URL_BASE = "https://pub-<hash>.r2.dev"
-
-[[r2_buckets]]
-binding = "MEDIA_BUCKET"
-bucket_name = "<project>-media"
-```
-
-Set secrets:
-
-```bash
-wrangler secret put R2_ACCESS_KEY_ID
-wrangler secret put R2_SECRET_ACCESS_KEY
-```
-
-## `src/mantle/config.ts`
-
-```ts
-import { R2MediaStorage, type MantleCloudflareConfig } from "@aotter/mantle/cloudflare";
-import { AwsClient } from "aws4fetch";
-
-export interface Env {
-  readonly MEDIA_BUCKET?: R2Bucket;
-  readonly R2_ACCOUNT_ID?: string;
-  readonly R2_ACCESS_KEY_ID?: string;
-  readonly R2_SECRET_ACCESS_KEY?: string;
-  readonly MEDIA_PUBLIC_URL_BASE?: string;
-}
-
-function buildMediaStorage(env: Env): MantleCloudflareConfig["bindings"]["mediaStorage"] {
-  if (
-    !env.MEDIA_BUCKET ||
-    !env.R2_ACCOUNT_ID ||
-    !env.R2_ACCESS_KEY_ID ||
-    !env.R2_SECRET_ACCESS_KEY ||
-    !env.MEDIA_PUBLIC_URL_BASE
-  ) {
-    return undefined;
-  }
-
-  const s3 = new AwsClient({
-    accessKeyId: env.R2_ACCESS_KEY_ID,
-    secretAccessKey: env.R2_SECRET_ACCESS_KEY,
-    region: "auto",
-    service: "s3",
-  });
-
-  return new R2MediaStorage(
-    env.MEDIA_BUCKET,
-    s3,
-    `https://<project>-media.${env.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
-    env.MEDIA_PUBLIC_URL_BASE,
-  );
-}
-```
-
-Then wire it into the CMS config:
-
-```ts
-siteDefaults: {
-  media: {
-    purposes: [
-      {
-        name: "page-image",
-        required: ["image/jpeg,image/png", "image/webp", "image/avif"],
-        maxBytes: {
-          "image/jpeg": 5_000_000,
-          "image/png": 5_000_000,
-          "image/webp": 3_000_000,
-          "image/avif": 2_000_000,
-        },
-      },
-    ],
-  },
-},
-bindings: {
-  mediaStorage: buildMediaStorage(env),
-},
-```
-
-`required` is slot-based. The example above declares three slots:
-
-- slot 0: choose exactly one fallback mime, preserving the source
-  semantics (`image/jpeg` for opaque photos, `image/png` when alpha must
-  be preserved);
-- slot 1: include a WebP variant;
-- slot 2: include an AVIF variant.
-
-This models the ADR-0017 multi-variant `<picture>` shape. If a site wants
-a simpler single-variant upload for a narrow purpose, it may declare one
-slot such as `["image/jpeg,image/png,image/webp,image/gif"]`; in that
-case, the agent chooses exactly one mime from the comma-separated list and
-must not upload one variant per listed mime.
-
-## Tool Visibility
-
-The Staff MCP media tools are registered only when both are true:
-
-- `bindings.mediaStorage` is set;
-- `siteDefaults.media.purposes` contains at least one purpose.
-
-If either side is missing, `create_media_upload` and
-`commit_media_upload` will not appear in `tools/list`.
-
-## Agent Guidance
-
-For image maintenance, prefer Claude Code or another coding agent that can
-read local files, process images, and make outbound PUT requests to R2.
-Avoid Claude Cowork for this workflow; use a non-sandboxed agent instead.
-
-Preserve source semantics:
-
-- photos may use JPEG/WebP variants;
-- transparent logos must keep alpha;
-- animated GIFs must stay animated;
-- do not silently flatten, resize, or recompress user assets without
-  asking.
-
-## Cleanup
-
-Run the version-matched `media-gc` skill when an upload reached R2 but was
-never committed. It audits first and removes only stale objects without
-`committedAt` metadata after explicit operator confirmation. Do not use an R2
-lifecycle rule: committed and uncommitted media share the same purpose prefix.
+The handbook ships inside the `@aotter/mantle` package under `node_modules/@aotter/mantle/docs/handbook/` and describes the installed release.

package/docs/migration-0.1.2.md

@@ -1,4 +1,60 @@
-# Migrating from 0.1.0-alpha.7 to 0.1.2
+# Leaving the legacy alpha.17 launch line
+
+0.1.0-alpha.17 remains immutable and is the final version for existing Landing
+and Starters. Those repositories/tags/bundle URLs remain available. Staying on
+that version requires no migration. No stable 0.1.0 release is planned.
+
+The new line removes `mantle create`, the Starter bundle `mantle update`
+command, and `@aotter/mantle/provision`. There are no aliases or replacement
+scaffold commands. `generate`, `validate`, `emit-openapi` and `skills` remain.
+Generation and runtime Web rendering retain their existing responsibilities.
+
+To upgrade an existing application:
+
+1. Read its installed version, lockfile, entry, scripts and provider bindings;
+   keep unrelated work safe in the normal git workflow.
+2. Pin the selected SDK packages to the intended exact new release and update
+   the lockfile through the package manager. Review required peer upgrades.
+3. Remove scripts invoking the retired scaffolder/bundle updater. If application
+   code imports the provision renderer, remain on alpha.17 until that host's
+   provisioning design is migrated explicitly; do not replace it with a
+   handwritten remote-code loader.
+4. Retain application source and all Worker/D1/KV identity, origins, auth mode,
+   secrets and legacy `.mantle` metadata. Those files are not templates to
+   replace or evidence that new Starter tags must exist.
+5. Run the installed `mantle generate`, `generate --check`, `skills`,
+   `skills --check`, `validate`, and the project's TypeScript/tests. Test local
+   routes and configured authorization before considering deployment.
+
+## Native Schema-table storage reset
+
+The 0.1.2 pre-beta line replaces the generic `entries` JSON table with one
+native SQLite/D1 table per Manifest Schema. Mantle's row envelope uses
+`_mantle_id`, `_mantle_status`, `_mantle_version`, `_mantle_author_id`,
+`_mantle_created_at`, and `_mantle_updated_at`; authored fields keep their exact
+names as native columns. The old generated columns, projection views and
+compatibility repository were removed.
+
+This is intentionally a storage-format break before beta. Reset and
+re-bootstrap development or internal-alpha content databases that contain the
+old `entries` layout. Move required data manually outside Mantle and Control;
+there is no product migration workflow for this unreleased format. Automatic
+artifacts cover initial and additive changes only. Removed columns and tables
+remain physically present so the previous Worker can still run. Renames, type
+changes and data transforms require the same manual rebuild. The pre-beta Cloud
+path does not accept or execute destructive SQL.
+
+Row APIs are now Schema-qualified. `EntryRepository.get` and
+`EntryReader.readById` accept `{ collection, id }`; Admin entry detail and
+mutation routes require `?collection=<schema>`, and generic MCP entry tools
+require `collection`. Generated `entries.<schema>` bindings supply it for you.
+
+A new project follows [direct authoring](direct-authoring.md). Templates and
+provider setup are not hidden inside `generate`. Future Builder/landing-next
+provisioning is a separate decision; this change does not migrate those hosts.
+
+## Earlier alpha.7 compatibility changes
+
 
 0.1.2 removes the temporary full-site compatibility stack. Mantle Core is now
 an embeddable parse → link → compile → prepare → bind pipeline; Web, Admin,
@@ -41,5 +97,6 @@ Intentional behavior changes:
 
 This alpha changes the Better Auth D1 schema, including required account
 issuer identity and OAuth resource/client tables. Reset and re-bootstrap a
-pre-1.7 alpha auth database; do not guess an issuer backfill. Content tables
-remain portable through the normal application migration/export path.
+pre-1.7 alpha auth database; do not guess an issuer backfill. Reset old generic
+content storage as described above and move required data manually outside
+Mantle.

package/docs/performance-harness.md

@@ -31,7 +31,7 @@ must not query Mantle-owned tables outside the selected storage adapter.
   the Worker. Cache keys are version-local, so a deploy starts with no stale
   response from the previous Worker version.
 - Successful publishing-content and site-setting writes purge the shared
-  `mantle-public` Cache-Tag through Cloudflare's native cache API. Operational
+  deployment-scoped public Cache-Tag through Cloudflare's native cache API. Operational
   records and immutable assets stay outside that invalidation boundary.
 - Do not cache every repository read. Cross-isolate correctness for editable
   data wins unless a read has a measured hot-path contract and explicit
@@ -50,7 +50,7 @@ pnpm exec mantle-harness indexes --require account-members --format json
 ```
 
 Without `--require-public` or `--require`, findings are advisory. A required
-path fails on an `entries` table scan, a temporary ORDER BY B-tree, or a
+path fails on a required Schema-table scan, a temporary ORDER BY B-tree, or a
 data-field predicate/order that does not use a declared Schema index.
 Projection alone does not require an index. `mantle validate` remains a pure
 correctness check; no performance grammar or manifest atom was added.
@@ -75,12 +75,14 @@ Timing always reports p50/p95/max. A test-only Worker wrapper may also return
 `x-mantle-query-count` and `x-mantle-rows-read`; those become distributions in
 the same report. Do not expose these diagnostic headers in production.
 
-Core CI runs `pnpm bench:wrangler` against real Wrangler-local D1, Worker HTTP
-routing, View execution, and origin page rendering. It compares 100 and 10,000
-row fixtures and gates row-read scaling plus endpoint query budgets, not
-absolute milliseconds. Wrangler-local does not emulate the new entrypoint
-Workers Cache, so cache hits are a deployment-level smoke check rather than a
-fabricated local metric.
+The path-scoped Cloudflare benchmark workflow runs `pnpm bench:wrangler`
+against real Wrangler-local D1, Worker HTTP routing, View execution, and origin
+page rendering. It compares 100 and 10,000 row fixtures and gates row-read
+scaling plus endpoint query budgets, not absolute milliseconds. It is separate
+from the required repository checks so an unrelated dependency or docs PR does
+not fail on the platform harness. Wrangler-local does not emulate the new
+entrypoint Workers Cache, so cache hits are a deployment-level smoke check
+rather than a fabricated local metric.
 
 ## Seven findings: measured disposition
 
@@ -90,8 +92,8 @@ diagnostic, while query/row counts are the stable assertions.
 | Finding | Disposition |
 |---|---|
 | Public cache hits read D1 first | Removed from Worker code. Cloudflare's entrypoint Workers Cache runs before the Worker; Core has no inner render cache. |
-| Slug/locale reads bypass generated indexes | Fixed by the shared schema-aware entry-read boundary. A 10,000-row page MISS measured 2 queries / 5 rows read. |
-| OFFSET pagination | Accepted for the v0.1 bounded-result surfaces: every response is capped at 500 rows and public hot paths must stay shallow. Deep/export workloads require a purpose-shaped cursor API before they are declared hot. |
+| Slug/locale reads bypass Schema indexes | Fixed by the shared schema-aware entry-read boundary over native columns. A 10,000-row page MISS measured 2 queries / 5 rows read. |
+| OFFSET pagination | Retained only where the View/Admin contract explicitly uses it, with a 500-row response cap. Public content lists and discovery now use forward keyset pages; they are not covered by the old 500-row claim. |
 | Admin substring search scans | Accepted only for the authenticated Admin collection browser, with a 500-row response cap. Large/search-heavy sites should add a purpose-shaped indexed View or dedicated search service; do not expose this scan publicly. |
 | Published list/sitemap/llms paths lack system indexes | Fixed with measured partial indexes for published global, locale, collection, and collection+locale ordering. The 100-row and 10,000-row API runs both measured 1 query / 20 rows read. |
 | Page MISS waits for cache write-back | Removed. Origin rendering returns directly; Workers Cache owns response storage outside the Worker. |
@@ -102,3 +104,118 @@ not patterns for new public APIs. Re-measure before widening either scope.
 
 See also [Schema indexes](./schema-indexes.md) and the official Cloudflare
 [D1 index guidance](https://developers.cloudflare.com/d1/best-practices/use-indexes/).
+
+### Prepared database, new Worker state
+
+The Wrangler fixture resets its in-isolate runtime after seeding while retaining
+D1. The first public page has a four-statement budget (fingerprint, lazy locale,
+site settings, entry); subsequent origin pages have a two-statement budget.
+This is a new application state in the same workerd isolate, not a measurement
+of module startup CPU or an entrypoint cache HIT. Locale caching is enabled by
+a successful preparation fingerprint; editable settings and media policy still
+read the canonical database on every call.
+
+### HTTP Trigger routing
+
+The portable request handler indexes Trigger paths by method and segments once.
+Literal and wildcard branches retain sealed-plan order, including encoded literal
+collisions that an outer router may select differently. Each request decodes its
+segments once and invokes the original Trigger identity through the same runtime.
+The 1/10/100/1,000-route regression uses four segment lookups at every size;
+overlapping wildcard shapes can visit multiple branches, pruned by route rank.
+The HTTP microbench and workerd harness include the same route-count axis.
+Workerd wall times include I/O and are not CPU measurements or a fixed-ms CI gate.
+
+### Public content pages and discovery (#809)
+
+| Surface | Canonical read / continuation |
+|---|---|
+| Collection HTML and collection Markdown | 50 entries by default, forward `cursor`; visible Next link plus HTTP `Link: rel="next"`. |
+| Locale and root llms.txt | One canonical page, default 50 entries; root expands that page across configured locales in memory. Shared entries are not reread once per locale. Follow the body/HTTP continuation link. |
+| Sitemap part | Up to 2,000 entries, with only declared path fields (built-in resolver: `slug`). A small site returns a urlset directly; a larger site returns a sitemap index linking every part. |
+| Sitemap index | Walks metadata pages to derive exact part cursors; O(N) metadata work on an index MISS, with one page resident at a time. It is not a constant-work list endpoint. |
+
+`EntryReader.readPublishedPage` caps returned data JSON at 1 MiB and 2,000 rows.
+One oversized entry is returned alone to make progress. SQLite applies the byte
+budget before transferring/parsing JSON in the Worker; one extra candidate
+identifies continuation. A localized + shared page merges two indexed ranges
+inside the same statement. The original `readPublished` remains an explicit
+unbounded read unless its caller supplies a limit.
+
+Translation lists resolve at most one newest published parent per join value,
+then resolve media once for the bounded list. Parent payloads and media metadata
+are additional input; the 1 MiB budget describes the canonical child page, not
+arbitrary template output or total Worker heap. Custom renderers own their output
+size. IndexedDB keeps identical page semantics but currently scans its local
+collection; this is not a claim of bounded IndexedDB storage I/O.
+
+The real SQLite fixture matrix covers 100/10,000/50,000 published rows, 64 B/4 KiB
+bodies, and 1/3/10 locales. At limit 20, every case transfers 21 candidate rows;
+4 KiB body data occupies 87,003–87,129 bytes, independent of collection size.
+Complete llms traversal uses 2/200/1,000 statements at the default 50-row page
+size, independent of locale count. Sitemap and llms URL sets match, including
+275,000 URLs for 50,000 mixed localized/shared rows across 10 locales.
+
+Run `pnpm --filter @aotter/mantle-cloudflare exec vitest run
+test/public-content-scaling.test.ts` to emit JSON transfer sizes and traversal
+CPU/wall/RSS diagnostics. These are Node + SQLite + assertions, including the
+fixture database and URL-validation set; RSS is a process high-water mark, not
+per-request Worker peak memory. Worker CPU, true cache HIT/MISS and placement
+measurements belong to the matched native/full-stack harness (#812).
+
+The workerd smoke also measures public list, llms and sitemap. The first two
+use two warm statements (settings + page) and bounded D1 work. Sitemap index
+queries and rows-read scale with the number of metadata parts; this explicit
+cost preserves complete discovery instead of silently dropping URLs.
+
+### Request diagnostics (version 1, test/performance only)
+
+Import `runWithRequestDiagnostics`, `instrumentD1`, `instrumentKv` and
+`instrumentR2` from `@aotter/mantle-cloudflare/testing`. Wrap native bindings once
+before handing the same D1 object to Auth and Runtime. Open the request context
+outside the complete facade fetch, and pass binding-presence flags matching the
+instrumented fixture. The observer receives one response-time record; it never
+receives request headers, tokens, proofs, user IDs, SQL, parameters, tool arguments,
+object keys or response content. Sync/async observer failure cannot change the
+original response or exception. The library emits no diagnostic headers or logs.
+
+Records distinguish HTTP outcome from JSON-RPC result/error/tool-error. They
+include actual in-isolate arrivals, inclusive OAuth/DPoP, role, runtime, catalog,
+dispatcher construction and dispatch wall spans. Unreached phases are null.
+Shared KV loads charge native I/O once to the initiating request; waiters record
+wait duration and the same hit/miss/repair/error source. Boot publication has a
+separate counter. The original rejected shared load remains retryable.
+
+D1 statements and binding calls are separate: a batch is one call and N attempted
+statements. Failed attempts count. `exec` uses the provider's count; unavailable
+counts stay in `unknownStatementCalls`, never a semicolon parser. `first(column)`
+and `raw` preserve native behavior and do not silently execute `all` to manufacture
+metadata. Their absent metadata is null. Rows/duration are sums of available
+metadata, and `metadataStatements` identifies coverage; incomplete coverage is
+not a full-workload total. Serialized binding results are measured bytes, not a
+claim about bytes on the provider's wire. Existing D1DatabaseDriver observers can
+request metadata for first-row reads when a fixture explicitly chooses that mode.
+
+KV bytes identify UTF-8, buffer or reserialized JSON sources. R2 payload bytes
+remain unknown for an unconsumed/partly consumed GET. A successful PUT of that exact
+native GET stream confirms the transferred body size on both operations. Streams
+are never wrapped or buffered for diagnostics, preserving R2's native known-length
+contract. `byteSamples` distinguishes known payload samples from the operation
+count. Metadata/list response serialization is labeled separately from object
+payload. R2 coverage is head/get/put/delete/list, not multipart-upload instrumentation.
+
+Snapshots freeze at response creation; outstanding/deferred operations remain
+visible through `inFlight` and metadata coverage and cannot rewrite a published
+record. `totalMs` excludes the subsequent test-only JSON-RPC response inspection,
+delivery and deferred work. CPU, TTFB, full-body duration and heap must be measured
+separately. Worker wall clocks advance on I/O and are not a CPU timer; use the
+[official CPU profiler](https://developers.cloudflare.com/workers/observability/dev-tools/cpu-usage/).
+Measure diagnostics off/on overhead with the same workload before interpreting
+small latency differences.
+
+### Matched native facade controls (#812)
+
+`pnpm bench:wrangler` now also runs the native parity smoke. The full
+`pnpm bench:parity` matrix adds real Auth/MCP, cold workerd processes, R2,
+TTFB/full-body timing and CPU/heap evidence. See
+[the controls, gates and reproducible commands](./adr/adr-lite-812-native-parity.md).