@jimhoyd/urlcode 0.4.0-alpha.1 → 0.4.0-alpha.3

package/docs/EXTENSIONS.md

@@ -1,10 +1,17 @@
 # Operator-installed extensions
 
-Extensions are trusted operator modules, separate from application WASM. Auth
+Extensions are trusted operator modules, separate from a project's own
+`function`/`middleware` code. Auth
 and admin implementations live in `urlcode-auth` and `urlcode-admin`; the runtime
 supplies only the generic integration contract. No project file can import a host
 extension or choose its npm package.
 
+Stored short links moved out of core this way too: a `urlcode-dynamic-link`
+package (mount-based, like `auth`/`admin`) owned the durable link store, its CLI
+and management API. That package has since been retired and unpublished, so no
+supported stored-link extension ships today. Core no longer has a native `link`
+handler or a `dynamicLinks` project flag.
+
 A project declares versioned configuration and exclusive route mounts:
 
 ```yaml
@@ -52,9 +59,9 @@ fails, naming the route, when `auth` appears without an `extensions.auth`
 declaration, next to `policies.extensions.auth`, or next to
 `policies.extensions: false`.
 
-The same shape is reserved for the cache policy: a future `cache: {strategy,
-maxAge}` route key may expand to `policies.cache` in the same pass. It is not
-implemented; declare `policies.cache` today (see [policies](POLICIES.md)).
+The same shape is used for the cache policy: a route-level `cache: {strategy,
+maxAge, ...}` expands to `policies.cache` in the same pass (see
+[policies](POLICIES.md)).
 
 The configuration and requirement objects above are validated by the installed
 extension's schemas. They are examples of extension-owned fields, not built-in
@@ -71,17 +78,86 @@ project and place the exact returned SHA-256 in each registration's
 in the revision. Changing them requires an explicit operator reapproval.
 
 Registrations provide a name, contract version, target list, JSON configuration
-schema, optional policy schema and activation factory. Activation receives the
+schema, optional policy schema, an optional declared `cacheSensitive` (below)
+and activation factory. Activation receives the
 canonical operator origin, target, revision and mount bases. Its instance handles
-bounded requests and, when used in policies, authorizes requests. Missing
+bounded requests and, when named in a route's policies, gates the request via
+`authorize`, wraps the rest of the pipeline via `middleware`, or both (see
+[Wrapping a route](#wrapping-a-route-extension-middleware) above). Missing
 registrations, stale grants, invalid configuration and unsupported targets fail
 activation. Multiple mounts cannot overlap other declared routes.
 
 For extension-protected routes, agents/throttle run before authorization and
-cache access happens only after authorization. Extension routes and protected
-routes reject cache strategies other than no-store; every resulting response is
-forced to no-store after host response hooks. Compression is disabled on these
-responses.
+cache access happens only after authorization. This part is unconditional:
+naming any extension in `policies.extensions` always runs its `authorize()`
+(when it implements one) before the route's own handler, whatever this
+section says next.
+
+An `extension:` mount is always confidential: its route rejects cache
+strategies other than no-store, and every response is forced to no-store
+after host response hooks, with compression disabled. A `policies.extensions`
+route (no mount, `authorize`/`middleware` only) gets the same treatment
+**unless every extension it names explicitly declares
+`cacheSensitive: false`** on its `RuntimeExtension` registration. That field
+defaults to sensitive (unset or `true`): the safe default is unchanged, and
+relaxing it is an explicit, reviewed operator opt-in an extension author
+makes once, in host code, never inferred from a route or from a response the
+extension happens to return. It exists for a generic, cache-transparent
+extension whose `middleware()` is pure request/response wrapping with no
+access-control semantics of its own (a logging or header-rewriting
+extension, for example) — declared this way, its wrapped route keeps
+whatever `Cache-Control` its own handler sets, exactly like the native
+`middleware:` array already does, and compression is not disabled either. A
+route naming more than one extension stays confidential if any one of them
+is sensitive (or leaves the field unset); one `cacheSensitive: false`
+extension cannot relax a route that also names a sensitive one. This can
+only relax the no-store floor a generic extension would otherwise inherit —
+it has no effect on `authorize()`, which runs the same way regardless, and
+`auth`/`admin`-style extensions gating real access must leave it at the
+default.
+
+## Wrapping a route: extension middleware
+
+`authorize` is a gate: it runs once, before the route's handler, and can only
+either let the request through unchanged or answer instead of it. It cannot
+see or change what the handler itself returns.
+
+`middleware` is a wrap. An extension instance may implement it alongside or
+instead of `authorize`, attached the same way, via
+`policies.extensions.<name>` on a route (no `extension:` mount required); its
+`config` is exactly the same per-route value `authorize`'s `requirement`
+receives, validated once against the extension's `policySchema`:
+
+```ts
+middleware?(config: Readonly<Record<string, unknown>>, request: ExtensionRequest,
+  next: () => Promise<HandlerResult>): HandlerResult | Promise<HandlerResult>;
+```
+
+`next()` invokes the rest of the pipeline for that route: any other extension
+`middleware()` also declared on the route (see below), then the route's own
+native `middleware:` chain and handler, dispatched through the sandboxed or
+trusted engine exactly as it is today. Calling it lets the hook run code
+before and after the rest of the pipeline, inspecting or mutating the
+`HandlerResult` it resolves to — the same "add a header to whatever the
+handler returns" shape as the native `middleware/headers.mjs` cookbook
+recipe, but declared by an operator-installed extension instead of project
+code. Skipping it short-circuits everything after that point, the same
+capability `authorize` already has, just usable from either side of the
+handler now. `next()` may be called at most once; calling it again throws.
+
+A route naming more than one extension in `policies.extensions` chains every
+one that implements `middleware`, in the order the keys are declared, each
+one's `next()` reaching the next one and the innermost `next()` reaching the
+native pipeline — the first declared name is outermost. This is purely
+additive at the `policies.extensions` layer and never touches the native
+`middleware:` array, its schema, or its dispatch, all of which are unchanged.
+
+`authorize` and `middleware` compose on the same route, from the same or
+different extensions, without special-casing: `authorize` always runs first
+(unchanged), and any declared `middleware()` wraps everything after that
+point, including the rest of the authorize-gated pipeline. A route naming an
+extension via `policies.extensions` only requires that extension to
+implement `authorize`, `middleware`, or both — never both unconditionally.
 
 One exception exists for content-hashed assets. A registration may declare
 `immutableAssets: {prefix: '/static'}`, a normalized literal path under each
@@ -105,6 +181,112 @@ HTML/JS on an authentication origin must be trusted by that site's operator.
 Cloudflare refuses extensions until its artifact format supports their execution.
 Node adapter conformance is not a live-provider deployment claim.
 
+## Project-level lifecycle hooks
+
+`authorize` and `middleware` let core call *into* an extension. They do not
+let a project hand its own code *to* an extension to run at a defined point.
+That gap matters once an extension has meaningful lifecycle events —
+registration, deletion, an administrative action, a link resolution, and so
+on. Presentation already has a standard layering mechanism for this: a
+project customizes an extension's *look* through `urlcode-ui`'s
+`copy`/`extra.css`/`templates` without forking it. Behavior needs the same
+standard, or every extension author (and every project depending on one)
+either reinvents it or forks the extension. `urlcode-auth` and
+`urlcode-admin` independently hit this gap (auth/#35, admin/#32); auth's own
+[SPIKE-AUTH.md](https://github.com/jimhoyd-com/urlcode-auth/blob/main/docs/SPIKE-AUTH.md)
+already scoped a shape for `onSignUp`/`beforeRegister`/`onDelete`.
+
+**The pattern.** An extension with lifecycle events an author judges worth
+exposing should let the project name its own function in the extension's own
+`config`, using the same source shape `function`/`middleware` routes already
+use (a string path, or `{source, export, args}` — `schemas/urlcode.schema.json`),
+and add its own `sandbox` boolean next to it (below). The extension's own
+`activate()` reads that config, and its own runtime dispatch — not a new core
+primitive, an ordinary call the extension package makes with the request
+context it already has — invokes the named function at the lifecycle point
+it defines, with a typed input and a typed verdict the extension's own
+schema documents. For example, an auth-style extension might declare:
+
+```yaml
+extensions:
+  auth:
+    version: "1"
+    config:
+      hooks:
+        beforeRegister:
+          source: ./hooks/registration-rule.mjs
+          export: default
+        onSignUp:
+          source: ./hooks/on-signup.mjs
+```
+
+with `beforeRegister` called before an account is created, given a typed
+`{email, traits}`-shaped input and returning a typed verdict (`{allow: true}`
+or `{allow: false, reason}`), and `onSignUp` called after, for side effects
+such as provisioning a workspace. The exact hook names, input/verdict shapes
+and invocation points are the extension's own design — `hooks` is not a core
+schema key — but the source/export/sandbox shape, and the fact that this is
+project code the extension calls rather than an operator callback in
+`host.mjs`, should be consistent across extensions so an author who has
+learned one has learned the pattern.
+
+**Trust and execution mode.** Project-level lifecycle hooks are first-party
+project code, the same category as any `function`/`middleware` route, and
+follow the same trusted-by-default rule with no special case
+(docs/SPIKE-DEFAULT-TRUST-MODEL.md, [FUNCTION-SECURITY.md](FUNCTION-SECURITY.md)):
+trusted, in-process execution by default; a project sets `sandbox: true` on
+a given hook to opt that hook into isolated QuickJS/WASM execution, exactly
+the mechanism `function`/`middleware` routes already use and for the same
+reason — the hook's own code, not the trustworthiness of whatever triggered
+it, is what calls for isolation (docs/AI-AUTHORING.md's "Deciding when a
+route needs `sandbox: true`"). This was raised as an open question — whether
+a lifecycle hook should always run sandboxed because it makes a
+security-relevant decision — and settled the other way: uniformity with the
+rest of the trust model was chosen over hardwiring isolation for lifecycle
+hooks specifically, the same "no special case" call already made between
+`function` and `middleware` trust. A `beforeRegister` hook enforcing "only
+`@acme.com` may register" is the project's own governance rule over its own
+signup flow; it is not more dangerous than any other route the project
+wrote, and does not get a different default.
+
+Core's own trusted/sandboxed dispatch (`TrustedFunctions`/`FunctionPool`,
+`src/runtime.ts`) is wired to route dispatch, not exposed to extensions — but
+each half of a hook's `sandbox: true` opt-in has its own answer:
+
+- **Trusted (the default, no `sandbox: true`).** No core primitive is needed
+  or provided: this is ordinary first-party project code, and the
+  extension's own `activate()` already has `ExtensionActivation.root` to
+  resolve the hook's `source` against and can `import()` it directly, the
+  same way any trusted `function`/`middleware` route does. Do that import
+  with a per-activation cache-busting query, the way core's own trusted
+  route activation does (`src/trusted-functions.ts`): Node's ESM loader
+  caches a resolved module forever by URL, so a plain `import()` of the
+  unchanged file URL makes a second activation in the same process keep
+  serving the hook code that was on disk at the first one
+  (jimhoyd-com/urlcode#198). Only the hook's **entry** module is refreshed
+  this way — modules the hook itself imports stay on Node's module cache,
+  the same limitation the trusted route path has, so a change to a hook's
+  own dependency still needs a process restart.
+- **Sandboxed (`sandbox: true`).** `@jimhoyd/urlcode/sandbox` exports
+  `SandboxPool`, the same QuickJS/worker-thread engine that backs a
+  sandboxed `function`/`middleware` route — the identical module-allowlist
+  walk, memory/stack limits, two-layer deadline enforcement, `maxBytes` and
+  response-shape validation, with no separate or weaker engine for
+  extensions. It takes an explicit list of `{source, export}` entries
+  (resolve a hook's `source` string with the re-exported `functionFile()`,
+  the same resolution/validation a native route's `source` gets) instead of
+  anything route/YAML-shaped, and `execute({entry, chain}, request, context)`
+  in place of a `FunctionRoute`. There is no "trusted" mode exported
+  alongside it — `SandboxPool` is only ever the isolated path; see
+  [FUNCTION-SECURITY.md](FUNCTION-SECURITY.md) and
+  [TYPESCRIPT.md](TYPESCRIPT.md) for the full contract.
+
+An extension honoring a hook's `sandbox: true` is expected to actually
+isolate that invocation through `SandboxPool` now that the primitive exists
+(or document plainly that it does not yet, rather than accepting the field
+and silently running it trusted) — say which, in the extension's own docs,
+so an author reading them is not misled about what opt-in exists.
+
 ## Discovering schemas
 
 Each registration carries the JSON Schemas that validate its `config` block and
@@ -161,7 +343,7 @@ and each extension's own operator files. Core never bundles or imports the
 extension packages at build time; at run time it resolves
 `@jimhoyd/urlcode-<name>` for each name with Node's package resolution from
 the invoking directory (so `npm install @jimhoyd/urlcode-auth` in that
-directory, from npm where the packages are published as `0.1.0-alpha.1`
+directory, from npm where the packages are published as `0.1.0-alpha.x`
 prereleases, is the normal path and what makes `--with auth` work), imports
 the package and calls its
 `scaffold` export with this request:

package/docs/FRAMEWORK.md

@@ -9,14 +9,14 @@ claim here is implemented in the linked repository; nothing is roadmap.
 
 | Package | Repository | What it adds | How a project declares it |
 |---|---|---|---|
-| `@jimhoyd/urlcode` | [urlcode](https://github.com/jimhoyd-com/urlcode) | The runtime: YAML routes, isolated functions and middleware, pages and assets, live short links, policies, site conventions, CLI, provider adapters, the extension contract | `urlcode.yaml` with `version: "1"` |
+| `@jimhoyd/urlcode` | [urlcode](https://github.com/jimhoyd-com/urlcode) | The runtime: YAML routes, functions and middleware (trusted by default, `sandbox: true` opt-in), pages and assets, policies, site conventions, CLI, provider adapters, the extension contract | `urlcode.yaml` with `version: "1"` |
 | `@jimhoyd/urlcode-ui` | [urlcode-ui](https://github.com/jimhoyd-com/urlcode-ui) | Shared presentation: escaped templates, shadcn/ui partials, one stylesheet with light and dark, themes, translations, the `ui` extension that serves the kit's assets | `extensions.ui` plus an asset mount route |
 | `@jimhoyd/urlcode-auth` | [urlcode-auth](https://github.com/jimhoyd-com/urlcode-auth) | Accounts: password, passkeys, OpenID Connect, email codes, TOTP, recovery, sessions, roles, registration modes, account page, operator CLI | `extensions.auth` plus an `/account/*` mount and `policies.extensions.auth` on protected routes |
 | `@jimhoyd/urlcode-admin` | [urlcode-admin](https://github.com/jimhoyd-com/urlcode-admin) | Administration: users, sessions, roles, audit, registration approval, two-person cases, support impersonation, health | `extensions.admin` plus an `/admin/*` mount |
 
 The core is Apache-2.0 and released. The three extension packages are
-Apache-2.0 and published to npm as alphas (`@jimhoyd/urlcode-ui@0.1.0-alpha.1`,
-`@jimhoyd/urlcode-auth@0.1.0-alpha.1`, `@jimhoyd/urlcode-admin@0.1.0-alpha.1`,
+Apache-2.0 and published to npm as alphas (`@jimhoyd/urlcode-ui@0.1.0-alpha.4`,
+`@jimhoyd/urlcode-auth@0.1.0-alpha.2`, `@jimhoyd/urlcode-admin@0.1.0-alpha.2`,
 on core `0.4.0-alpha.1`). An alpha on npm is a distribution channel, not an
 endorsement: the source is complete, but independent review, deployment
 evidence and an accessibility assessment are still pending
@@ -24,6 +24,9 @@ evidence and an accessibility assessment are still pending
 files say exactly what is built: [auth](https://github.com/jimhoyd-com/urlcode-auth/blob/main/IMPLEMENTATION-STATUS.md),
 [admin](https://github.com/jimhoyd-com/urlcode-admin/blob/main/IMPLEMENTATION-STATUS.md),
 [ui](https://github.com/jimhoyd-com/urlcode-ui/blob/main/IMPLEMENTATION-STATUS.md).
+Which core version each package supports, how it declares that, and the order
+in which a core change reaches the downstream repositories are recorded in
+[core version alignment](VERSION-ALIGNMENT.md).
 
 ## The ladder
 
@@ -38,22 +41,27 @@ Each rung's YAML is valid on every rung above it.
    llms.txt) and `policies` (throttle, agents, security headers, compression,
    cache). Still no code.
 3. **Functions and middleware.** `function` routes and ordered `middleware`
-   in isolated JavaScript (QuickJS inside WebAssembly, fresh heap per call, no
-   Node, filesystem or network). Secrets reach a function only through an
-   operator grant pinned to the project revision.
-4. **Live short links.** A `link` route backed by an optional SQLite store,
-   with CLI and a private management API. Records change without reloads.
-5. **Accounts.** The `auth` extension: sign-in, registration, MFA, account
+   in JavaScript, trusted and in-process by default; a route declaring
+   `sandbox: true` runs isolated instead (QuickJS inside WebAssembly, fresh
+   heap per call, no Node, filesystem or network). Secrets reach a function
+   only through an operator grant pinned to the project revision.
+4. **Accounts.** The `auth` extension: sign-in, registration, MFA, account
    page and protected routes. The operator installs it in a host file outside
    the project; YAML only declares the mount and configuration.
-6. **Administration.** The `admin` extension on the same service: manage the
+5. **Administration.** The `admin` extension on the same service: manage the
    people who signed up, their sessions and roles, review the audit trail.
-7. **Your own look.** A shared `presentation` (catalogue and theme variables)
+
+6. **Your own look.** A shared `presentation` (catalogue and theme variables)
    restyles auth and admin together; the `ui` extension adds the template kit,
    project copy, template and stylesheet overrides for kit-rendered pages.
 
-Rungs 1 to 4 need only the core package. Rungs 5 to 7 need the extension
-packages, installed from npm as `0.1.0-alpha.1` prereleases, and a Node host
+Stored short links previously sat here as a native `link` route; that handler
+was removed from core. A `urlcode-dynamic-link` package owned them the same way
+`auth`/`admin` own their mounts, but it has been retired and unpublished; no
+package occupies this rung today.
+
+Rungs 1 to 3 need only the core package. Rungs 4 to 6 need the extension
+packages, installed from npm as `0.1.0-alpha.x` prereleases, and a Node host
 with a patched SQLite build; see each repository's README for the exact
 requirement.
 
@@ -66,9 +74,9 @@ npm install @jimhoyd/urlcode @jimhoyd/urlcode-ui @jimhoyd/urlcode-auth @jimhoyd/
 urlcode init my-site --with auth,admin
 ```
 
-Installing from npm is the normal path; the extension repositories'
-`pack-sources.mjs` still builds local tarballs from a reviewed checkout for
-operators who install only source they have read. Three files make an extended
+Installing from npm is the normal path; `scripts/pack-sources.mjs` still builds
+local tarballs from a reviewed checkout for operators who install only source
+they have read — one revision now covers core and every extension. Three files make an extended
 project. Nothing else is discovered by convention.
 
 ```
@@ -100,7 +108,8 @@ routes:
         auth: {}
 ```
 
-The operator host wires the packages, once, in code the project cannot reach:
+The operator host explicitly registers the packages. Registration is an
+activation boundary; it does not isolate trusted application code from the host:
 
 ```js
 import { createUiExtension } from '@jimhoyd/urlcode-ui/host';
@@ -120,12 +129,12 @@ export default {
 };
 ```
 
-Auth and admin pages currently render through the shared primitives and a
-`presentation` (catalogue and `--ui-*` theme variables). The `ui` extension
-serves the template kit and its assets for pages an extension renders through
-`ui.kit`; moving the auth and admin screens onto the kit is a listed remaining
-item in the ui repository, so today one `presentation` restyles them and the
-`ui` block is optional.
+Auth and admin already render their screens through `ui.kit` when the host
+supplies the UI extension; their package-owned templates and catalogues must be
+registered with that kit. Without it, both retain shared primitive rendering
+through `presentation`. The example above uses that primitive fallback. The UI
+block is optional, and a migration to the kit is not unfinished framework work.
+See each package README for its complete kit registration example.
 
 ```sh
 urlcode serve --project /absolute/site --host-file /absolute/operator/host.mjs --origin https://site.example
@@ -147,26 +156,37 @@ or mounts changes the revision and needs an explicit operator reapproval.
 These are the facts that keep generated projects valid. The full matrix is in
 [AI authoring](AI-AUTHORING.md); this is the short list.
 
-- **YAML never names code, packages, databases or credentials.** Extensions are
+- **Extension YAML names logical extensions, not host packages or credentials.**
+  Function and middleware `source` fields do name project modules. Extensions are
   logical names; the host file chooses the implementation. There is no
   `--extension` flag, no `import` in YAML, no interpolation.
 - **One handler per route.** `redirect`, `respond`, `page`, `static`, `download`,
-  `function`, `link`, `proxy`, `conditional` or `extension`, plus optional
+  `function`, `proxy`, `conditional` or `extension`, plus optional
   `middleware`. Paths are exact or single-segment `{param}`; `/*` only on
   `static` and `extension` mounts. No regex.
-- **Guest code is sandboxed.** Functions see a text/JSON `Request`/`Response`
-  subset, validated `args` and granted `env`. No `fetch`, Node, filesystem or
-  timers. Do not write code that needs them; declare a `proxy` or a binding.
+- **`function`/`middleware` code is trusted by default, sandboxed opt-in.**
+  It runs in-process with full Node access unless the route declares
+  `sandbox: true`, which isolates it to a text/JSON `Request`/`Response`
+  subset, validated `args` and granted `env`, with no `fetch`, Node,
+  filesystem or general network access; bounded timers are available. Either way, `args`/`env`/`secrets` are exactly what
+  the route declares and an operator grants — trust changes where code runs,
+  not what it is handed. See docs/SPIKE-DEFAULT-TRUST-MODEL.md and
+  docs/FUNCTION-SECURITY.md.
 - **Authentication is host processing.** Do not build login forms, session
-  cookies or password checks in functions. Declare `policies.extensions.auth`
-  on the route; the runtime withholds `Cookie` and `Authorization` from guests.
+  cookies or password checks in functions. With the auth extension declared,
+  prefer `auth: true` or `auth: {role: admin}`; these expand to
+  `policies.extensions.auth`. The runtime filters credential headers passed to
+  application handlers. This is not a security boundary against trusted Node code.
 - **Everything is validated before it runs.** `urlcode validate --local`,
   `urlcode test`, `urlcode audit --expect-routes N`. Unsupported features fail
   with the route named; nothing degrades silently.
 - **Provider targets refuse what they cannot enforce.** Cloudflare runs
   redirects and declared responses only. Serverless adapters refuse functions,
-  links, proxy, signals and extensions. Check
-  `urlcode capabilities --target NAME` before promising a deployment.
+  proxy, signals and extensions. The `static` target (S3 + CloudFront,
+  no server) refuses everything that needs request-time logic, keeping only
+  `redirect`/`respond`/`page`/`static`/`download` — see [static
+  hosting](STATIC.md). Check `urlcode capabilities --target NAME` before
+  promising a deployment.
 - **Report evidence, not hope.** The commands above are the evidence. Local
   tests are not deployment, soak or independent security review.
 
@@ -178,5 +198,5 @@ These are the facts that keep generated projects valid. The full matrix is in
 | Add accounts | [auth README](https://github.com/jimhoyd-com/urlcode-auth#readme), [auth security](https://github.com/jimhoyd-com/urlcode-auth/blob/main/SECURITY.md) |
 | Add administration | [admin README](https://github.com/jimhoyd-com/urlcode-admin#readme) |
 | Restyle every page | [ui README](https://github.com/jimhoyd-com/urlcode-ui#readme), [ui contract](https://github.com/jimhoyd-com/urlcode-ui/blob/main/CONTRACT.md) |
-| Write an extension | [extensions](EXTENSIONS.md), [extension model review](SPIKE-EXTENSION-MODEL.md) |
+| Write an extension | [extensions](EXTENSIONS.md), [extension model review](archive/2026-09-19/SPIKE-EXTENSION-MODEL.md) |
 | Run it | [operations](OPERATIONS.md), [install](INSTALL.md), [deployment checks](DEPLOYMENT-CHECKS.md) |