@jimhoyd/urlcode 0.4.0-alpha.2 → 0.4.0-alpha.3

package/docs/DEVELOPMENT-PIPELINE.md

@@ -0,0 +1,181 @@
+# Development and release pipeline
+
+The repository uses npm workspaces, independent package versions, and Changesets
+for UI, auth and admin. Core remains at the repository root and is explicitly
+included in the shared release inventory. Moving it is not required to use the
+same release checks and coordinator.
+
+## Pull requests
+
+Every PR starts `verify`. A complete Git diff selects one of two lanes:
+
+- **Prose:** root project Markdown, `docs/**/*.md`, `llms.txt` and
+  `llms-full.txt` changes run guidance/generated-resource checks, runtime audit,
+  and the required container job. CodeQL retains its repository policy.
+- **Full:** all other changes, mixed changes, empty/unavailable diffs and main
+  pushes run static checks once and core and workspace suites separately. Both
+  suites retain Linux on Node 22/24/26. Main adds Windows/macOS on Node 24.
+  PRs add those platform legs for runtime, CLI, SQLite, fixture, dependency,
+  workflow and unknown changes; known UI presentation-only changes omit them. Package, action,
+  cookbook, reproducibility and operational checks retain their coverage.
+
+The prose allowlist is deliberately narrow. Package documentation, skills,
+starters, examples, schemas, manifests and workflow changes select full checks.
+A rename from source into docs also selects full checks. No required workflow
+uses `paths-ignore`.
+
+`verify-complete` accepts only the results specified by the successful plan.
+Failed, canceled, missing or unexpectedly skipped work fails the gate. Required
+check names (`verify-complete`, `container`) and CodeQL enforcement are preserved.
+The repository ruleset currently does not require a branch to be up to date;
+release publication separately requires verification of the exact main commit.
+
+```sh
+npm run check:docs                 # prose checks without the runtime suite
+npm run ci:plan -- BASE_SHA HEAD_SHA
+npm run ci:report -- RUN_ID         # read GitHub job/step durations
+npm run ci:history -- 100 2026-09-19 # group historical timing samples
+npm run verify                    # full local validation remains available
+npm run test:package              # builds and installs a real archive
+```
+
+CI uses `test:package:built` only after building in that same job. Core tests and
+workspace tests run in separate jobs to shorten their serial critical path;
+this increases job setup overhead and needs monitoring for runner queue pressure.
+After building all three extensions, the workspace job also runs the real
+`init --with auth,admin,ui` scaffold integration. Missing workspace outputs fail
+instead of silently skipping an absent external checkout.
+The [audit](CI-RELEASE-AUDIT-2026-09-19.md) records the previous timings.
+The [follow-up measurements](CI-FOLLOWUP-2026-09-19.md) record the first compact
+main result and explain why the new lanes still need 20 organic runs each.
+
+## Version and release ownership
+
+A feature PR records release intent in a Changeset for a changed workspace
+package. Review dependency/peer changes explicitly. Keep pre-mode enabled until
+an explicit decision to leave alpha. Do not force all packages to one version.
+Core version bumps remain explicit in the release PR, including its CLI banner.
+`release:check` verifies every manifest against its lockfile entry and checks the
+alpha-mode policy. Existing CLI tests catch core banner/version disagreement.
+
+A release PR collects version/changelog and lockfile changes together. Ordinary
+unreleased development does not move existing version tags or npm versions.
+Use the exact release commit after its full platform checks have passed. Routine
+main builds use five OS/Node combinations per suite (ten jobs total), rather
+than the full nine per suite (eighteen). Nightly runs at 07:17 UTC and manual
+runs retain all three operating systems on all three Node versions. Before a
+release, run `gh workflow run ci.yml --ref main` and wait for that exact commit's
+full run to succeed. A successful compact main run alone cannot authorize a
+release. Main pushes do not cancel scheduled/manual verification.
+
+Inspect release state:
+
+```sh
+npm run release:status   # registry channels, peer compatibility, remote tag SHAs
+npm run release:plan     # read-only JSON inventory, including root core
+npm run release:run      # read-only ordered proposal at HEAD
+```
+
+All three commands require network access; none publish. `release:run` selects
+unpublished manifest versions and releases already tagged at HEAD (for resuming
+partial completion). Its order is core, UI, auth, admin, skipping other already
+published versions. Plan fields come from package manifests, not copied versions
+in another config file.
+
+Once the release itself is authorized, from a clean checkout of that exact SHA:
+
+```sh
+npm run release:run -- --execute
+```
+
+This opt-in command creates missing tags through GitHub, refuses an existing tag
+at a different commit, and waits for each package's tag-triggered release before
+starting the next. It stops on failure or missing npm publication. Do not push
+all release tags at once: publication workflows share one concurrency group and
+GitHub may replace pending runs. The coordinator intentionally starts one at a
+time. It never merges a PR, bypasses checks, force-pushes or publishes locally.
+
+The coordinator needs `gh` authentication with repository Contents write and
+Actions read; reading checks also needs Checks read. It is intended for an
+authorized maintainer session or a repository-scoped GitHub App. A fine-grained
+PAT can serve a short-lived maintainer script with those permissions, but must
+have no ruleset bypass. The workflow `GITHUB_TOKEN` should not be used to create
+these trigger tags: its push events do not start another ordinary push workflow.
+For long-lived automation prefer a GitHub App; do not add a bypass to `main`.
+[GitHub App guidance](https://docs.github.com/en/apps/creating-github-apps/about-creating-github-apps/deciding-when-to-build-a-github-app)
+
+## Publication and recovery
+
+The four workflow filenames remain unchanged because npm trusted publishing
+names them. They call shared helpers for identity, preflight, peer installation,
+retry handling and publication. npm authentication remains OIDC; no npm token
+is introduced. Core candidate and release share `prepare-core-release.sh`.
+For an authorized release, first dispatch the manual `candidate.yml` workflow on
+the selected main commit. It extends the core candidate with UI/auth/admin
+archives and verifies all four together in an isolated temporary consumer:
+peer compatibility, installed versions, public imports and real scaffold
+composition. Its signed `train.json` records the proposed archives and integrity.
+A candidate does not publish, validate live providers or prove registry OIDC;
+release workflows still prepare and retain their own immutable retry artifacts.
+Extensions share `prepare-extension-release.sh` and test published peer floors.
+Auth/admin build in the workspace for packaging, then build and run their suites
+in a temporary copy outside the monorepo against exact registry peer floors.
+This preserves #184’s isolation fix; npm `--prefix` is not an isolation boundary.
+
+Preflight checks the checkout SHA, main ancestry, a successful exact-SHA full
+`ci.yml` nightly or explicit manual run, CodeQL, remote tag SHA, npm
+channel monotonicity and published peer floors. When a full run was canceled,
+run `verify` manually at the selected tag/ref, then rerun the failed release;
+never substitute another commit's passing run or move the tag.
+
+Prepared artifacts are retained for 90 days before publication. A rerun of the
+same workflow run restores those original bytes and skips preparation. npm
+versions already present must have identical SHA-512 integrity; existing GitHub
+assets must match byte for byte. Different bytes stop the release. An absent or
+expired artifact requires reconstruction that still passes these comparisons;
+if it cannot, diagnose and create a new version rather than overwrite history.
+
+GitHub release classification follows the manifest's prerelease status. New
+GitHub releases use `--latest=false`; stable latest promotion is a separate
+maintainer decision, avoiding accidental promotion by a package-level release.
+GHCR updates the derived channel (`alpha` for alphas, `latest` for stable) and
+preserves existing version images only when their source label matches. An
+existing image/channel without the required labels fails closed and needs a
+reviewed migration; this change does not silently relabel old images.
+
+Partial npm/GitHub/GHCR success is possible; those systems cannot be updated
+atomically. Rerun the original failed run, check its summary and then rerun the
+coordinator. A changed source commit requires a new version and tag. Failed
+OIDC configuration needs correction on npm, not tag deletion. A green dry run
+cannot prove registry-side OIDC trust; each package's first authorized publish
+must verify it.
+
+Historical tags, GitHub release flags and npm channels are not retroactively
+rewritten by these scripts. Use `release:status` to inspect them. The active [Immutable release tags rule](https://github.com/jimhoyd-com/urlcode/rules/23712319)
+prohibits update/deletion of `v*` and `@jimhoyd/urlcode-*@*` tags, permits
+creation, and has no bypass actors. Its reviewed configuration is tracked in
+`.github/rulesets/release-tags.json`. Main protection is unchanged.
+
+During the September 19 alignment, GitHub releases `v0.4.0-alpha.1` and
+`v0.4.0-alpha.2` were explicitly marked as prereleases and GitHub latest was
+restored to `v0.3.0`, matching npm. Their tags and artifact bytes were unchanged.
+Further performance and release-train validation is tracked in
+[issue #185](https://github.com/jimhoyd-com/urlcode/issues/185).
+
+### Windows fixture cleanup
+
+Auth/admin tests register resources with their package-local `test/cleanup.ts`.
+Cleanup runs in reverse acquisition order: close servers and SQLite services
+before deleting temporary directories, including services reopened by a test.
+Every registered callback is attempted even if another closer throws, and the
+combined error fails the test. Register each closer as soon as its resource opens.
+The suites use a five-minute test-file timeout so a stuck fixture is diagnosed
+before the CI job limit. Node applies this limit to whole test files too; the
+large auth-core file legitimately exceeds two minutes on Windows Node 22.
+Windows regression coverage runs on Node 24 for platform-sensitive PRs and
+main; nightly/manual runs cover Node 22/24/26.
+
+Failed auth service initialization also waits for its SQLite worker to terminate
+before rejecting. Callers can clean up or retry after a rejected open without
+racing a remaining database handle; configuration identity failures still fail
+closed with the same error code.

package/docs/EXTENSIONS.md

@@ -6,10 +6,11 @@ and admin implementations live in `urlcode-auth` and `urlcode-admin`; the runtim
 supplies only the generic integration contract. No project file can import a host
 extension or choose its npm package.
 
-Stored short links are moving out of core this way too: a future
-`urlcode-dynamic-link` package (mount-based, like `auth`/`admin`, not yet
-published) will own the durable link store, its CLI and management API. Core
-no longer has a native `link` handler or a `dynamicLinks` project flag.
+Stored short links moved out of core this way too: a `urlcode-dynamic-link`
+package (mount-based, like `auth`/`admin`) owned the durable link store, its CLI
+and management API. That package has since been retired and unpublished, so no
+supported stored-link extension ships today. Core no longer has a native `link`
+handler or a `dynamicLinks` project flag.
 
 A project declares versioned configuration and exclusive route mounts:
 
@@ -256,7 +257,16 @@ each half of a hook's `sandbox: true` opt-in has its own answer:
   or provided: this is ordinary first-party project code, and the
   extension's own `activate()` already has `ExtensionActivation.root` to
   resolve the hook's `source` against and can `import()` it directly, the
-  same way any trusted `function`/`middleware` route does.
+  same way any trusted `function`/`middleware` route does. Do that import
+  with a per-activation cache-busting query, the way core's own trusted
+  route activation does (`src/trusted-functions.ts`): Node's ESM loader
+  caches a resolved module forever by URL, so a plain `import()` of the
+  unchanged file URL makes a second activation in the same process keep
+  serving the hook code that was on disk at the first one
+  (jimhoyd-com/urlcode#198). Only the hook's **entry** module is refreshed
+  this way — modules the hook itself imports stay on Node's module cache,
+  the same limitation the trusted route path has, so a change to a hook's
+  own dependency still needs a process restart.
 - **Sandboxed (`sandbox: true`).** `@jimhoyd/urlcode/sandbox` exports
   `SandboxPool`, the same QuickJS/worker-thread engine that backs a
   sandboxed `function`/`middleware` route — the identical module-allowlist

package/docs/FRAMEWORK.md

@@ -56,9 +56,9 @@ Each rung's YAML is valid on every rung above it.
    project copy, template and stylesheet overrides for kit-rendered pages.
 
 Stored short links previously sat here as a native `link` route; that handler
-was removed from core. A future `urlcode-dynamic-link` package (not yet
-published, so this one line is the exception to "nothing is roadmap" above)
-is expected to own them the same way `auth`/`admin` own their mounts.
+was removed from core. A `urlcode-dynamic-link` package owned them the same way
+`auth`/`admin` own their mounts, but it has been retired and unpublished; no
+package occupies this rung today.
 
 Rungs 1 to 3 need only the core package. Rungs 4 to 6 need the extension
 packages, installed from npm as `0.1.0-alpha.x` prereleases, and a Node host
@@ -74,9 +74,9 @@ npm install @jimhoyd/urlcode @jimhoyd/urlcode-ui @jimhoyd/urlcode-auth @jimhoyd/
 urlcode init my-site --with auth,admin
 ```
 
-Installing from npm is the normal path; the extension repositories'
-`pack-sources.mjs` still builds local tarballs from a reviewed checkout for
-operators who install only source they have read. Three files make an extended
+Installing from npm is the normal path; `scripts/pack-sources.mjs` still builds
+local tarballs from a reviewed checkout for operators who install only source
+they have read — one revision now covers core and every extension. Three files make an extended
 project. Nothing else is discovered by convention.
 
 ```
@@ -108,7 +108,8 @@ routes:
         auth: {}
 ```
 
-The operator host wires the packages, once, in code the project cannot reach:
+The operator host explicitly registers the packages. Registration is an
+activation boundary; it does not isolate trusted application code from the host:
 
 ```js
 import { createUiExtension } from '@jimhoyd/urlcode-ui/host';
@@ -128,12 +129,12 @@ export default {
 };
 ```
 
-Auth and admin pages currently render through the shared primitives and a
-`presentation` (catalogue and `--ui-*` theme variables). The `ui` extension
-serves the template kit and its assets for pages an extension renders through
-`ui.kit`; moving the auth and admin screens onto the kit is a listed remaining
-item in the ui repository, so today one `presentation` restyles them and the
-`ui` block is optional.
+Auth and admin already render their screens through `ui.kit` when the host
+supplies the UI extension; their package-owned templates and catalogues must be
+registered with that kit. Without it, both retain shared primitive rendering
+through `presentation`. The example above uses that primitive fallback. The UI
+block is optional, and a migration to the kit is not unfinished framework work.
+See each package README for its complete kit registration example.
 
 ```sh
 urlcode serve --project /absolute/site --host-file /absolute/operator/host.mjs --origin https://site.example
@@ -155,7 +156,8 @@ or mounts changes the revision and needs an explicit operator reapproval.
 These are the facts that keep generated projects valid. The full matrix is in
 [AI authoring](AI-AUTHORING.md); this is the short list.
 
-- **YAML never names code, packages, databases or credentials.** Extensions are
+- **Extension YAML names logical extensions, not host packages or credentials.**
+  Function and middleware `source` fields do name project modules. Extensions are
   logical names; the host file chooses the implementation. There is no
   `--extension` flag, no `import` in YAML, no interpolation.
 - **One handler per route.** `redirect`, `respond`, `page`, `static`, `download`,
@@ -166,13 +168,15 @@ These are the facts that keep generated projects valid. The full matrix is in
   It runs in-process with full Node access unless the route declares
   `sandbox: true`, which isolates it to a text/JSON `Request`/`Response`
   subset, validated `args` and granted `env`, with no `fetch`, Node,
-  filesystem or timers. Either way, `args`/`env`/`secrets` are exactly what
+  filesystem or general network access; bounded timers are available. Either way, `args`/`env`/`secrets` are exactly what
   the route declares and an operator grants — trust changes where code runs,
   not what it is handed. See docs/SPIKE-DEFAULT-TRUST-MODEL.md and
   docs/FUNCTION-SECURITY.md.
 - **Authentication is host processing.** Do not build login forms, session
-  cookies or password checks in functions. Declare `policies.extensions.auth`
-  on the route; the runtime withholds `Cookie` and `Authorization` from guests.
+  cookies or password checks in functions. With the auth extension declared,
+  prefer `auth: true` or `auth: {role: admin}`; these expand to
+  `policies.extensions.auth`. The runtime filters credential headers passed to
+  application handlers. This is not a security boundary against trusted Node code.
 - **Everything is validated before it runs.** `urlcode validate --local`,
   `urlcode test`, `urlcode audit --expect-routes N`. Unsupported features fail
   with the route named; nothing degrades silently.
@@ -194,5 +198,5 @@ These are the facts that keep generated projects valid. The full matrix is in
 | Add accounts | [auth README](https://github.com/jimhoyd-com/urlcode-auth#readme), [auth security](https://github.com/jimhoyd-com/urlcode-auth/blob/main/SECURITY.md) |
 | Add administration | [admin README](https://github.com/jimhoyd-com/urlcode-admin#readme) |
 | Restyle every page | [ui README](https://github.com/jimhoyd-com/urlcode-ui#readme), [ui contract](https://github.com/jimhoyd-com/urlcode-ui/blob/main/CONTRACT.md) |
-| Write an extension | [extensions](EXTENSIONS.md), [extension model review](SPIKE-EXTENSION-MODEL.md) |
+| Write an extension | [extensions](EXTENSIONS.md), [extension model review](archive/2026-09-19/SPIKE-EXTENSION-MODEL.md) |
 | Run it | [operations](OPERATIONS.md), [install](INSTALL.md), [deployment checks](DEPLOYMENT-CHECKS.md) |

package/docs/FUNCTION-SECURITY.md

@@ -40,6 +40,35 @@ API is all it has, so its binding grant *is* effectively its whole reach into
 the environment. Trusted code's reach is not bounded that way; treat the
 grant as scoping `context`, not as scoping the process.
 
+## Migrating to the trusted default
+
+If you are upgrading a project from a release before this change shipped:
+**every existing `function` and `middleware` route silently changes execution
+mode**, from sandboxed to trusted, unless it already has (or you add)
+`sandbox: true`. This is a real behavior change on upgrade, not a
+documentation update — a route that used to run with no filesystem or network
+access will, after the upgrade, run with full Node access unless you opt it
+back into the sandbox.
+
+Before upgrading:
+
+- List every `function` and `middleware` route in the project.
+- For each one, decide whether you fully trust that code to run in-process
+  with full Node/filesystem/network access — the same trust you would extend
+  to any other code you deploy to that server.
+- Add `sandbox: true` explicitly to any route whose code you do not fully
+  trust, that processes input from a source you don't control, or that handles
+  a secret binding you want isolated — before you upgrade, not after.
+- Routes you do want running trusted need no change; that is now the default.
+
+The change moves the sandbox from an unconditional guarantee to an explicit,
+per-route choice, mainly for performance: the previous blanket sandbox capped
+concurrency at two workers with no queue shared across every function route on
+the server, which does not scale to real concurrent traffic. It also brings
+first-party code in line with how the rest of the Node ecosystem treats
+deployed application code. The sandbox itself is unchanged for routes that opt
+into it; only the default for routes that declare neither option has changed.
+
 ## What "sandboxed" (`sandbox: true`) still guarantees
 
 - Function sources are parsed/snapshotted without importing them into Node.
@@ -116,6 +145,21 @@ it is handed *through `context`*. It does not change what the code can go get
 on its own once it is running; see "binding grants are unaffected" above for
 that distinction.
 
+## Trusted code, not trusted requests
+
+"Trusted" describes the code's authorship — first-party project code you
+reviewed and deployed — not the requests it handles. Every request, in either
+mode, still carries client-controlled path, query, header and body data that is
+exactly as adversarial as it always was. Running trusted means that code
+executes with full Node access if it mishandles that input; it does not mean
+the input itself became safe to trust. Declare `parameters` and `request.body`
+validation in YAML, check `args` and any other request data again inside
+function/middleware code, and implement your own authentication and
+authorization — no route, sandboxed or trusted, adds automatic auth.
+`sandbox: true` narrows what a bug or an unreviewed dependency in the *code*
+can do with that same request data; it is not a substitute for validating or
+authenticating the request itself.
+
 ## Granting selected bindings
 
 An application may request a named binding in YAML, but only an operator can

package/docs/LOCAL-DEVELOPMENT.md

@@ -10,6 +10,9 @@ ship, together with the declarations; `dist` is never committed. Make is an opti
 shortcut layer; npm and the CLI work on Windows, macOS and Linux. No global
 package install, hosting account, database or Docker is needed for the local loop.
 
+For the repository's prose-only checks, CI selection and release helper commands,
+see [the development pipeline](DEVELOPMENT-PIPELINE.md).
+
 ## Try the runtime
 
 From the runtime checkout, `make dev` installs locked dependencies if needed and