@voidbase-cloud/voidbase 0.2.2 → 0.3.0

package/docs/ci.md

@@ -0,0 +1,195 @@
+# CI and release: one flow, Cloudflare runs it
+
+`scripts/ci.sh` is the whole CI, and its last step runs `scripts/release.sh`, the whole release flow, when the
+commits call for it. Both run the same way on a dev machine and on Cloudflare Workers Builds; GitHub Actions only
+starts the builds (`.github/workflows/cloudflare.yml`).
+Every step is recorded (`scripts/ci-lib.sh`) and `scripts/ci-status.ts` renders the record into `ci/public`:
+`index.html`, `status.json`, `badge.svg`, the suite logs and the screenshots. That directory is the status Worker a
+build deploys.
+
+## The CI steps (`scripts/ci.sh`)
+
+| step | what |
+| --- | --- |
+| install | `bun install --frozen-lockfile` |
+| commitlint | the commits the push or pull request introduces (`--last` when there is nothing to compare with) |
+| oracles | the starter (`scripts/ci-oracles.sh`: `STARTER_DIR`, else the sibling checkout, else a shallow clone in the cache), the panel (`panel:sync`), the starter's frontend build next to it (`app:sync`), `void prepare` |
+| plan | `scripts/ci-plan.ts`: which of the following steps and suites this run needs (below); hot mode trims the list to a time budget |
+| typecheck, unit | `tsc --noEmit`, `bun test` |
+| browser | a Chrome for the panel and starter suites (`scripts/ci-browser.sh`, below); `CI_BROWSER=0` skips them |
+| boot | the run's `.env`, `void db migrate`, the dev server on 5180 (`CI_PORT`), the app user |
+| reference | PocketBase 0.39.11 on 8090 (`CI_PB_PORT`) freshly seeded from the starter (again before the Bun pass: the reference keeps state the suites cannot undo, such as a stored S3 secret), the SMTP sink, the OIDC, S3 and Cloudflare API mocks, awaited before a first mail warms the SMTP transport; whatever already listens on a port is reused |
+| suites | `scripts/ci-suites.sh`: every differential suite, the SDK suite, the panel suites |
+| suites-bun | the same suites against `voidbase serve` on 8093 (Bun, SQLite, local files), without the browser suites |
+| deploy-cf, adapter, fresh-db, mail-http, exe-smoke | the deploy dry run against the API mock, a Void app converted and run through the adapter, the fresh-database boot and the HTTP mail transport of the production build, the prebuilt executable and its update flow |
+| starter | the unmodified starter frontend against voidbase |
+| release | on master, with `GH_TOKEN`: the release flow (docs/releasing.md) when a releasable commit was pushed, the release PR was merged, a `Release: dry-run` trailer asks for a rehearsal, or a release still needs npm or its executables |
+
+The script stops at the first failed step, prints the relevant logs, renders the status page and stops the servers
+it started; a dev machine's `.env` is put back. Steps the plan does not select are recorded as skipped with the reason. Ports, oracles and Chrome come from the environment, so a dev machine
+runs the same flow with `bun run ci`.
+
+## Incremental runs (`scripts/ci-plan.ts`)
+
+A full run takes about nine minutes on the build image, most of it the two suite passes, so a run only repeats the
+checks its changes reach. Every check has a set of files: the import closure of its test entry point plus the runtime
+it exercises, resolved through the same import graph (`#platform/*` follows the `workerd` condition for the Workers
+server and the default one for the Bun runtime), so `src/node/deploy-cf.ts` reaches the deploy dry run, the
+executable smoke, typecheck and the unit tests and nothing else, while a file of the server reaches every suite on
+both runtimes. File hashes come from git blob ids, so they are exact and cost nothing; the combined hash of a check's
+files is compared with the one the last green run recorded, and the check runs only when they differ. The record is
+the deployed status page of master (`CI_STATUS_URL`, https://release.voidbase.cloud/status.json, whose `verified`
+map holds the hash each check last passed on); on a dev machine it is `ci/public/status.json` from the previous run. What passed
+gets this run's hashes, what was skipped keeps the previous record's, so a chain of partial runs stays sound. The
+oracle sync, the servers, the reference and Chrome happen only when a selected check needs them.
+
+| change | what runs |
+| --- | --- |
+| docs, README, surface, `ci/`, `.github/` | commit messages and the plan: about half a minute |
+| one suite's file | that suite on both runtimes, with the servers it needs |
+| `src/node/deploy-cf.ts`, `src/cloud` | typecheck, unit, the deploy dry run, the executable smoke, cloud-rest: under two minutes |
+| `src/node/serve.ts`, `bin/voidbase.ts` | the Bun pass and the executable smoke |
+| `src/server`, `routes`, the app config, the harness | everything |
+
+`bun scripts/ci-plan.ts affected <file>` prints the checks a file reaches; `explain` prints every check with its
+file count and hash. Uncommitted changes never match a record, so a dirty working tree reruns what it touches.
+`CI_PLAN=full` (or `--full`) runs everything regardless, and so does a commit whose message carries `Tests: all`; the
+same happens when the record cannot be fetched. Only the repository's own files are hashed: a new commit of the
+starter oracle is picked up by the next full run.
+
+## Hot mode
+
+The suites test the server as a black box, so a server change reaches all of them and a full run is the honest
+answer. During a development phase that is too slow, so hot mode keeps every run within a time budget:
+`bun scripts/cf-builds.ts hot on` (`--budget 60` to change the default of sixty seconds; `hot off` to return to
+full runs). With hot mode on, a run does typecheck and the unit tests always, then whatever the commits name, then
+the suites of the commits' scopes, then the cheapest of the remaining selected checks until the budget is spent,
+using the durations the last run recorded (`status.json`, `suites[].seconds`). The Bun pass, the browser suites and
+the starter smoke wait for a normal run. Deferred checks are listed on the status page and are never marked verified,
+so the first run after `hot off` does them all. Releasing in hot mode publishes to npm only, so voidbase-site can
+pick the version up at once; the executables of that release are built by the first normal run on master.
+
+The commits steer it: a Conventional Commit scope (`fix(records): ...`) puts that area's suites first
+(`SCOPE_KEYS` in the planner maps every scope of `commitlint.config.js` to suites), a `Tests:` trailer names checks
+that are never deferred (`Tests: thumbs s3`, `Tests: bun` for the Bun pass, `Tests: browser`), and `Tests: all`
+forces a full run. A commit that edits a suite's file always runs that suite. The messages of every commit since the
+last green run count, not only the last one: a Cloudflare build checks out a single commit, so the planner deepens
+the history until the last green run's commit is reachable before it reads them.
+
+## What is kept between runs
+
+`CI_CACHE_DIR` holds the downloads: Playwright's headless shell and the unpacked libraries, the Ubuntu packages,
+the starter clone with its `node_modules` and its frontend build (reused while the starter's commit is the same), the
+panel tarball (through `XDG_CACHE_HOME`) and the PocketBase archive of the reference. On a dev machine it is
+`~/.cache/voidbase-ci` and simply stays there. Workers Builds keeps nothing between builds but the package manager's
+cache, and that one only until the lockfile changes, so on Cloudflare `scripts/ci-cache.sh` restores the directory from
+an R2 bucket at the start of a build and saves it at the end, one archive per component, uploaded only when its content
+changed. `setup` creates the bucket (`voidbase-ci-cache`) when `CI_CACHE_TOKEN` is in the environment (an API token with
+Workers R2 Storage edit; the deploy token of the site is accepted) and stores it on every trigger as a build secret,
+with `CI_CACHE_ACCOUNT` and `CI_CACHE_BUCKET`. Without those the builds fetch everything each time, about forty
+seconds of a full run.
+
+## Chrome (`scripts/ci-browser.sh`)
+
+The browser suites launch whatever `CHROME_PATH` points at, else the Chrome on the PATH (a dev machine). Where there
+is no Chrome, the script downloads Playwright's chromium-headless-shell under `.void/browsers` and, when the machine also
+lacks the shared libraries Chrome needs, unpacks them from Ubuntu's packages into `.void/chrome-libs` without root (a
+private apt root, `dpkg-deb -x`, `LD_LIBRARY_PATH`): the Workers Builds image has neither Chrome nor sudo nor those
+libraries. The unpacking path is exercised on a dev machine with `CI_BROWSER_DOWNLOAD=1 CI_BROWSER_LIBS=always`.
+
+## GitHub Actions only starts builds
+
+`.github/workflows/cloudflare.yml` is the only workflow. It never runs the flows: it asks Cloudflare Workers Builds
+for a build of the commit at hand through the Builds API (a few seconds of Actions time) and exits.
+
+| event | build started |
+| --- | --- |
+| push to master | the master trigger; the release step of the build then refreshes the release PR for releasable commits or publishes a merged release PR |
+| pull request from a branch of this repository | the branches trigger: a preview URL of the results, no release work, no secrets |
+| release published | the master trigger for the tagged commit, whose release step publishes it |
+| Actions > cloudflare > Run workflow | the ref's trigger |
+
+The job is skipped until the repository variables exist (`CF_ACCOUNT_ID`, `CF_CI_TRIGGER_MASTER`,
+`CF_CI_TRIGGER_BRANCHES`) together with the secret `CLOUDFLARE_BUILDS_TOKEN`; `bun scripts/cf-builds.ts setup
+--github` stores all four. With the repository variable
+`CF_BUILDS_WAIT=1` the job also waits for the builds it started and fails when one fails, so the pull request check
+reflects the result; without it the check only means "started", and the result lives in the dashboard, in
+`cf-builds.ts logs`, and on the status page. Pushes never build on their own: every trigger's watch paths exclude every
+path, and the API is not subject to them. Standard GitHub-hosted runners are free for public repositories anyway; the
+trigger job spends seconds.
+
+Two things stop existing with this layout because they need the OIDC token only a GitHub Actions run can mint:
+`npm publish --provenance` and the build attestations of the release archives. Releases carry checksums only.
+
+## Cloudflare Workers Builds
+
+[Workers Builds](https://developers.cloudflare.com/workers/ci-cd/builds/) is Cloudflare's build system for Workers: the
+"Cloudflare Workers and Pages" GitHub App starts a build on every push to a connected repository, runs a build command
+and a deploy command in Cloudflare's build image, and posts a check run (and a preview URL on pull requests) back to
+GitHub. One project, `voidbase-ci`, runs voidbase's flows there: a Worker whose deploy publishes the status page of
+the build (`ci/wrangler.jsonc`, assets only). A project has at most two triggers with one command each, which is
+why the release flow runs inside the CI build rather than as a project of its own:
+
+| trigger | build command | deploy command |
+| --- | --- | --- |
+| master | `bash scripts/ci.sh` | `wrangler deploy -c ci/wrangler.jsonc` |
+| branches | `bash scripts/ci.sh` | `wrangler versions upload -c ci/wrangler.jsonc`: a preview URL of the results on the pull request |
+
+A failed build command means no deploy, so the status Worker shows the last build that ran to the end; the log of a
+failed build is in the dashboard and in `cf-builds.ts logs`. The release secrets (`GH_TOKEN`, `NPM_TOKEN`, optionally
+`GH_PACKAGES_TOKEN`) are build secrets of the master trigger only, so builds of other branches never carry them.
+
+### Limits and cost
+
+| | Free plan | Paid plan |
+| --- | --- | --- |
+| build minutes | 3,000 a month | 6,000 a month, then $0.005 a minute |
+| concurrent builds | 1 | 6 |
+| build timeout | 20 minutes | 20 minutes |
+| CPU, memory | 2 vCPU, 8 GB | 4 vCPU, 8 GB |
+
+The image is Ubuntu 24.04 x86_64 with Node 22, Bun 1.2.15 (the projects set `BUN_VERSION=1.3.14`, the version the
+workflows pin), git, curl, unzip and build-essential; no Chrome, no lsof, no jq, no gh. The scripts need none of them:
+`scripts/gh-release.ts` talks to GitHub's API directly. The 20-minute timeout is the constraint to watch: the CI run
+takes 7.5 minutes on GitHub's 4 vCPU and gets 2 on the Free plan, and the two projects build one after the other there.
+
+### Setup, once
+
+1. Install the [Cloudflare Workers and Pages GitHub App](https://github.com/apps/cloudflare-workers-and-pages) for
+   `voidbase-cloud/voidbase` (an organization owner does this on GitHub; limit it to that repository).
+2. Create a user API token at dash.cloudflare.com/profile/api-tokens with **Workers Builds Configuration: Edit** and
+   **Workers Scripts: Edit**, and export it as `CLOUDFLARE_BUILDS_TOKEN`. The Builds API takes user tokens only; the
+   account-owned token `voidbase deploy` uses is rejected.
+3. `GH_TOKEN=... NPM_TOKEN=... bun scripts/cf-builds.ts setup --github` connects the repository, creates the Worker
+   and its two triggers with push builds off, sets `BUN_VERSION`, stores the release secrets it finds in the
+   environment on the master trigger, and writes the workflow's variables and secret into the GitHub repository
+   (`gh variable set`, `gh secret set`). The release secrets: `GH_TOKEN` (a fine-grained PAT with contents and pull requests write on the repository, for release-please
+   and the release assets), `NPM_TOKEN` (the npm granular token), optionally `GH_PACKAGES_TOKEN` (a classic PAT with
+   `write:packages`; fine-grained tokens cannot publish packages, and without it the GitHub Packages copy is skipped).
+   The first run stops when the account has no build token yet: open the `voidbase-ci` Worker in the dashboard,
+   Settings > Builds > API token > Create new token, and run setup again.
+4. `bun scripts/cf-builds.ts build --branch master --follow` runs the first build and streams its log; `status`,
+   `builds`, `logs <uuid>`, `cancel <uuid>` and `env` cover the rest (the header of the script lists them).
+5. From then on every push and pull request goes through the workflow, one build each. `gh variable set
+   CF_BUILDS_WAIT --body 1` makes the workflow wait for the build it started.
+
+`test/cf-builds.ts` runs the CLI against `test/cf-mock.ts`, whose Builds endpoints follow the request and response
+shapes of Cloudflare's API reference; the live API is exercised the first time the App and the user token exist.
+
+### What to expect
+
+- No provenance and no attestations (OIDC): `npm publish` runs without `--provenance`, the release archives carry
+  checksums only.
+- GitHub Packages only with `GH_PACKAGES_TOKEN`.
+- Logs live in the dashboard and in `cf-builds.ts logs`; the deployed page is the last build that ran to the end.
+- release-please and the release assets use `GH_TOKEN`, a PAT, so a release it creates fires the `release` event and
+  the workflow starts one more release build for the tagged commit, which finds everything published and stops.
+
+## The status page
+
+`ci/public/index.html` lists the steps with their durations and logs, every suite with its result and last line, the
+screenshots of the panel and starter suites, and links `status.json` (the same, as data) and `badge.svg`
+(`ci: passing`). The page's canonical address is https://release.voidbase.cloud (the custom domain
+`ci/wrangler.jsonc` declares; the workers.dev address stays on for the preview URLs): `badge.svg` there is the badge
+in the README and `status.json` the record of the last green run and the feed for anything else; a pull request's
+build uploads a version, so its preview URL shows the same page for that commit.

package/docs/deploy.md

@@ -43,7 +43,26 @@ deploy just skips the jobs queue and says so.
 
 ```bash
 export VOIDBASE_DEPLOY_CF_API_KEY=...        # or put it in .env next to pb_hooks, or a CI secret
-voidbase deploy --public-dir ../sk/build     # --name <worker>, --account <id> when the token reaches several accounts
+voidbase deploy                              # --name <worker>, --account <id> when the token reaches several accounts
+```
+
+A static site is served at `/` by the same Worker when there is one: `./pb_public` (PocketBase's convention, picked up
+automatically when the directory exists, exactly like `voidbase serve`), `--public-dir <dir>` or
+`VOIDBASE_DEPLOY_PUBLIC_DIR`. Build it first: the directory needs an `index.html`. Unknown paths get its `404.html`
+(a copy of `index.html` unless the build made one) with status 404, `/api` and `/_/` are untouched. A `_redirects`
+file in that directory (Netlify/Pages syntax, `source destination [status]`) is split in two: path-only lines are
+uploaded with the assets as Cloudflare's own `_redirects` (evaluated before the Worker), and lines whose source names a
+host (`https://api.example.com/  /_/  302`) become zone Redirect Rules after the upload, tagged with the Worker's name so
+a redeploy replaces exactly its own rules. That is how one Worker behind several custom domains answers differently per
+hostname: the site on the apex, the API hostname's root sent to the admin panel, `www` sent to the apex. Writing them needs
+the zone permission Single Redirect > Edit on the deploy token (the API's permission listing calls it Dynamic URL
+Redirects Write; it sits with the zone permissions, like DNS Edit, scoped to the zone). Without it the deploy prints the
+rules to create by hand and carries on.
+(Void's own `routing.redirects` are not used: they are applied by the Void platform's dispatch worker, which a
+self-hosted deploy does not have.)
+
+```bash
+voidbase deploy --public-dir ../sk/build     # a build that lives elsewhere
 ```
 
 What it does, in order: resolves the account through the token, creates `<name>-db` (D1), `<name>-storage`
@@ -51,7 +70,8 @@ What it does, in order: resolves the account through the token, creates `<name>-
 (`node_modules/voidbase/.cloud/<name>`, nothing appears in your tree) with a `wrangler.jsonc` carrying the real
 ids and, when the directory has a `main.ts` exporting `register(app)`, composes it into the Worker; stores the
 superuser as worker secrets (from `VOIDBASE_SUPERUSER_*` / `PB_SUPERUSER_*`, or a generated
-password saved in `pb_data/.superuser-credentials`; the local dev default `changeme123` never goes live), syncs
+password saved in `pb_data/.superuser-credentials`; the local dev default `changeme123` never goes live) together
+with the secrets `pb_secrets/` declares (below), syncs
 the admin panel and your frontend build into that project, and runs `void deploy --backend cloudflare`,
 which builds, applies the D1 migrations and uploads the Worker with its cron trigger. It ends with the
 `https://<name>.<your-subdomain>.workers.dev` URL and a health check. Re-running is idempotent: existing resources
@@ -66,8 +86,9 @@ https://dash.cloudflare.com/?to=/:account/api-tokens).
 
 ### A custom domain
 
-`voidbase deploy --domain api.example.com` (or `VOIDBASE_DEPLOY_DOMAIN`) turns workers.dev off for the Worker and attaches
-the hostname through the Workers Custom Domains API after the upload: Cloudflare creates the DNS record and the
+`voidbase deploy --domain api.example.com` (or `VOIDBASE_DEPLOY_DOMAIN`; several hostnames comma separated, the first is
+the URL the deploy reports) turns workers.dev off for the Worker and attaches each hostname through the Workers Custom
+Domains API after the upload: Cloudflare creates the DNS record and the
 certificate (a minute or two), and the token needs nothing beyond Workers Scripts edit, provided the zone is on the same
 account. Cloudflare still requires the account to have a workers.dev subdomain before it accepts any upload (error
 10063): open Workers & Pages once, or `PUT /accounts/<id>/workers/subdomain {"subdomain": "<name>"}`.
@@ -83,6 +104,40 @@ account. Cloudflare still requires the account to have a workers.dev subdomain b
 | `HUB` (Durable Object `VoidbaseHub`, SQLite-backed, in this Worker) | the realtime hub: every SSE connection holds one hibernatable socket to it, writes publish to it, so events arrive in tens of milliseconds instead of the D1 poll's second, and idle apps cost nothing (the object sleeps). Free plan included | `--no-hub` / `VOIDBASE_DEPLOY_HUB=0` keeps the D1 poll |
 | Smart Placement | the Worker runs next to its D1 database | always on |
 
+### Secrets: `pb_secrets/`
+
+The app's own secrets (an SMTP password, OAuth client secrets, `VOIDBASE_ENCRYPTION_KEY`) have a directory, in
+PocketBase's naming:
+
+```
+pb_secrets/main.pb.js      secrets({ SMTP_PASSWORD: "the mail provider's password", ... })   committed
+pb_secrets/secrets.json    { "SMTP_PASSWORD": "..." }                                       git-ignored
+```
+
+`main.pb.js` names the secrets and is read, never run (`voidbase init` writes an empty one and the `.gitignore`
+lines). `secrets.json` holds the values on your machine; the shell outranks it, and it outranks the `.env` files,
+so a dev placeholder such as `VOIDBASE_SUPERUSER_PASSWORD=changeme123` never shadows it. `voidbase serve` loads them into the environment, so
+`$os.getenv("SMTP_PASSWORD")` and the app's own code see the same names locally as on Cloudflare. `voidbase deploy`
+stores the declared values the Worker does not have yet as the Worker's secrets (encrypted, per Worker: two
+instances never share one) next to the superuser, and refuses to deploy while a declared secret has neither a local
+value nor one already on the Worker. A value the Worker already holds is left alone by a deploy: a deploy ships
+code, and a checkout whose `secrets.json` carries dev values (another OAuth client, the placeholder password) must
+not overwrite production by deploying. Replacing is explicit:
+
+```bash
+voidbase secrets              # each declared name: local value or not, on the Worker or not
+voidbase secrets push         # store the local values on the Worker (replacing), without redeploying
+```
+
+That is also what makes CI simple: a checkout without `secrets.json` deploys with nothing but the deploy token,
+because the values were pushed once from a machine that has them. The superuser follows the same rule: a checkout
+without credentials of its own keeps the superuser the Worker has.
+
+A value in `secrets.json` that `main.pb.js` does not declare is never deployed (the list says so). `VOIDBASE_DEPLOY_SECRETS=A,B`
+still stores environment variables as secrets for a deploy driven purely by the shell. Cloudflare's account-level
+Secrets Store is deliberately not used: one store is shared by every Worker of the account, and its bindings are
+read asynchronously, which `$os.getenv` is not.
+
 ### Every instance is isolated
 
 Two voidbase instances on one account never share a resource. Everything the deploy creates is named or derived from
@@ -128,7 +183,7 @@ release: `provisionInstance(cf, { account, name, release, superuser })` creates
 assets through an upload session and the script with its bindings, DO migration, cron trigger and workers.dev
 subdomain, tagged `voidbase` + `voidbase-release:<version>`; `destroyInstance` removes all of it (worker first,
 bucket last, emptied before); `listVoidbaseWorkers` finds instances by tag. The token is the user's OAuth access token
-(`cloudflare` OAuth2 provider, see `voidbase-site/vb/cloud`) or an API token with the same permissions.
+(`cloudflare` OAuth2 provider, see `voidbase-site/cloud`) or an API token with the same permissions.
 `test/cloud-rest.ts` exercises it against `test/cf-mock.ts`. The hub and the queue are decided when the release
 is bundled (`voidbase bundle --no-hub` / `--no-queue`), not per instance: an instance can leave them out at
 provisioning, but cannot add what the release does not carry. Tokens a control plane keeps go to rest sealed

package/docs/releasing.md

@@ -36,52 +36,60 @@ ci(release): compile release notes with release-please
 ```
 
 `.husky/commit-msg` runs commitlint on every commit; `.husky/pre-commit` runs `bun run check` and `bun test`.
-`bun install` installs the hooks (`prepare`); `git commit --no-verify` skips them, and CI (`ci.yml`, job
-`commitlint`) checks the pushed or proposed commits regardless.
+`bun install` installs the hooks (`prepare`); `git commit --no-verify` skips them, and the CI build
+(`scripts/ci.sh`, step `commitlint`) checks the pushed or proposed commits regardless.
 
 ## The release
 
-1. Push or merge conventional commits to `master`. `release.yml` runs release-please, which opens or updates the
-   pull request "chore(master): release X.Y.Z": the next version from the commit types, the `CHANGELOG.md`
-   section compiled from the commits, the `package.json` bump. Keep merging work; the PR follows.
-2. Merge the PR. release-please tags `vX.Y.Z` and creates the GitHub release with that section as notes.
-3. The `publish` job of the same run then installs, typechecks, runs the unit and cloud-rest tests, packs, smoke-
-   installs the tarball and runs the CLI from it, publishes to npm (`--provenance` when the repository is public)
-   and to GitHub Packages, and attaches the tarball to the release.
-4. The `executables` job builds the prebuilt executables for every platform (`scripts/build-exe.ts`: Bun
-   cross-compiles from one runner; the panel, the system migrations and the hooks typings are embedded), smokes the
-   runner's own build (`test/exe-smoke.ts`: serve with pb_hooks, the panel from the embedded zip, a thumbnail through
-   the wasm, then `voidbase update` against a mock GitHub API), attaches `voidbase_<version>_<os>_<arch>.zip` for
-   linux/darwin/windows × amd64/arm64 (plus musl builds) and `checksums.txt` to the release, attests each archive
-   with `actions/attest-build-provenance`, and puts the release notes in PocketBase's shape: the
-   `./voidbase update` hint first, then the compiled notes.
+`scripts/release.sh` is the whole flow. It runs as the last step of the CI build on master (docs/ci.md) when the
+pushed commits are releasable (`feat`, `fix`, `perf`, `revert`, a breaking change), when the release PR was merged,
+when a commit carries a `Release: dry-run` trailer, or when a release still needs npm or its executables (one cut
+by hand, or one published in hot mode). It is idempotent: a re-run after a partial failure does only what is still
+missing. In hot mode it publishes to npm and leaves the executables to the first normal run.
+
+1. Push or merge conventional commits to `master`. `release-pr` (release-please) opens or updates the pull request
+   "chore(master): release X.Y.Z": the next version from the commit types, the `CHANGELOG.md` section compiled from
+   the commits, the `package.json` bump. Keep merging work; the PR follows.
+2. Merge the PR. `github-release` tags `vX.Y.Z` and creates the GitHub release with that section as notes.
+3. `publish`, when release `v<package.json version>` exists and npm lacks the version: install, typecheck, the unit
+   and cloud-rest tests, `npm pack`, a smoke install of the tarball that runs the CLI from it, `npm publish`, the
+   GitHub Packages copy (with `GH_PACKAGES_TOKEN`), and the tarball attached to the release (`scripts/gh-release.ts`).
+4. `executables`, when that release lacks `checksums.txt`: the prebuilt executables for every platform
+   (`scripts/build-exe.ts`: Bun cross-compiles from one machine; the panel, the system migrations and the hooks
+   typings are embedded), the smoke of the machine's own build (`test/exe-smoke.ts`: serve with pb_hooks, the panel
+   from the embedded zip, a thumbnail through the wasm, then `voidbase update` against a mock GitHub API),
+   `voidbase_<version>_<os>_<arch>.zip` for linux/darwin/windows × amd64/arm64 (plus musl builds) and `checksums.txt`
+   attached to the release, and the release notes in PocketBase's shape: the `./voidbase update` hint first, then the
+   compiled notes.
 
 The layout mirrors PocketBase's releases: the zip holds the executable, `CHANGELOG.md` and `LICENSE`;
 `checksums.txt` is goreleaser's format (`<sha256>  <file>`), which `voidbase update` checks before replacing the
-executable. Verify an archive's provenance with `gh attestation verify voidbase_X.Y.Z_linux_amd64.zip --owner
-voidbase-cloud`. For the "Immutable" badge and the release attestation GitHub adds itself, enable immutable releases
-once in the repository settings (Settings > General > Releases); it is a setting, not something the workflow can
-turn on.
+executable. Builds on Cloudflare cannot attest the archives or publish with npm provenance (both need the OIDC token
+of a GitHub Actions run), so the checksums are the integrity check. For the "Immutable" badge and the release
+attestation GitHub adds itself, enable immutable releases once in the repository settings (Settings > General >
+Releases); it is a setting, not something a workflow can turn on.
 
 `.release-please-manifest.json` holds the released version (0.1.0 was cut by hand and its notes written by hand;
 everything after it is compiled). `release-please-config.json` maps commit types to changelog sections.
 
 ## Rehearsals and manual paths
 
-- Actions > release > Run workflow with `dry_run` on (or `gh workflow run release.yml -f dry_run=true`): the
-  publish job with `npm publish --dry-run`, nothing published, no release touched.
-- A release cut by hand also publishes: `gh release create vX.Y.Z --notes-file notes.md` after bumping
-  `package.json` to X.Y.Z on `master`; the `release: published` event runs the publish job against that tag.
+- A dry run: a commit on master whose message carries a `Release: dry-run` trailer makes the build rehearse the
+  flow (release-please in dry-run mode, `npm publish --dry-run`, the executables, nothing published, no release
+  touched), or locally `bun run release -- --dry-run` (needs `GH_TOKEN` with read access and `NPM_TOKEN`).
+- A release cut by hand also publishes: `gh release create vX.Y.Z --notes-file notes.md` after bumping `package.json`
+  to X.Y.Z on `master`; the `release` event starts a build of the tagged commit, whose release step publishes it.
 - Publishing from a machine: `bun run check && bun test`, then
   `NPM_CONFIG_//registry.npmjs.org/:_authToken=$VOIDBASE_NPM_TOKEN npm publish --access public`.
 
-Secrets and permissions: `NPM_TOKEN` (repository secret, an npm granular token with publish rights on the
-`@voidbase-cloud` scope). release-please opens the release PR with the workflow's own token only if the
-organization allows it (voidbase-cloud > Settings > Actions > General > "Allow GitHub Actions to create and
-approve pull requests", then the same switch on the repository); otherwise add `RELEASE_PLEASE_TOKEN`, a
-fine-grained PAT with contents and pull requests write on this repository, which the workflow prefers when
-present and which also makes CI run on the release PR (the workflow's own token cannot trigger other workflows).
-GitHub Packages and the release assets use the workflow's `GITHUB_TOKEN`.
+Secrets and permissions. The master trigger on Cloudflare holds `GH_TOKEN` (a fine-grained PAT with contents, pull
+requests and issues write on this repository: release-please opens and labels the release PR and creates the release
+with it, the assets are uploaded with it), `NPM_TOKEN` (an npm granular token with publish rights on the
+`@voidbase-cloud` scope) and optionally `GH_PACKAGES_TOKEN` (a classic PAT with `write:packages`; without it the
+GitHub Packages copy is skipped). `bun scripts/cf-builds.ts setup` stores them from the environment; builds of other
+branches never carry them. GitHub itself holds only what the workflow needs to start builds: the secret
+`CLOUDFLARE_BUILDS_TOKEN` and the trigger variables. release-please's PR needs no organization setting for Actions,
+since a PAT opens it.
 
 Consumers: `bun add @voidbase-cloud/voidbase`; from GitHub Packages instead, `.npmrc` with
 `@voidbase-cloud:registry=https://npm.pkg.github.com` and a token with `read:packages`.

package/hooks-plugin.ts

@@ -21,24 +21,25 @@ const ASYNC_PROPS = new Set([
   "findAuthRecordByEmail", "findAuthRecordByToken", "expandRecord", "expandRecords",
   "fileFromURL", "fileFromBytes", "fileFromPath", "bindBody", "requestInfo",
   "importCollections",
+  "queueJob", // $jobs.queueJob hands work to the jobs queue
 ]);
 
 export const HOOK_GLOBALS = [
-  "$app", "$apis", "$http", "$os", "$filesystem", "$security", "$mails", "$template", "$dbx",
+  "$app", "$apis", "$http", "$os", "$filesystem", "$security", "$mails", "$template", "$dbx", "$env", "$jobs",
   "routerAdd", "routerUse", "cronAdd", "cronRemove", "migrate",
   "Record", "Collection", "RecordUpsertForm", "MailerMessage", "DateTime", "RequestInfo",
   "Field", "TextField", "EditorField", "NumberField", "BoolField", "EmailField", "URLField", "DateField", "AutodateField", "SelectField", "FileField", "RelationField", "JSONField", "GeoPointField", "PasswordField",
   "ApiError", "NotFoundError", "BadRequestError", "ForbiddenError", "UnauthorizedError", "InternalServerError", "ValidationError",
   "__hooks", "require", "module", "exports", "console", "toString", "sleep", "arrayOf", "unmarshal",
 ];
-const EVENT_HOOKS = ["Bootstrap", "Serve", "Terminate", "BackupCreate", "BackupRestore",
+export const EVENT_HOOKS = ["Bootstrap", "Serve", "Terminate", "BackupCreate", "BackupRestore",
   "ModelValidate", "ModelCreate", "ModelCreateExecute", "ModelAfterCreateSuccess", "ModelAfterCreateError", "ModelUpdate", "ModelUpdateExecute", "ModelAfterUpdateSuccess", "ModelAfterUpdateError", "ModelDelete", "ModelDeleteExecute", "ModelAfterDeleteSuccess", "ModelAfterDeleteError",
   "RecordEnrich", "RecordValidate", "RecordCreate", "RecordCreateExecute", "RecordAfterCreateSuccess", "RecordAfterCreateError", "RecordUpdate", "RecordUpdateExecute", "RecordAfterUpdateSuccess", "RecordAfterUpdateError", "RecordDelete", "RecordDeleteExecute", "RecordAfterDeleteSuccess", "RecordAfterDeleteError",
   "CollectionValidate", "CollectionCreate", "CollectionCreateExecute", "CollectionAfterCreateSuccess", "CollectionAfterCreateError", "CollectionUpdate", "CollectionUpdateExecute", "CollectionAfterUpdateSuccess", "CollectionAfterUpdateError", "CollectionDelete", "CollectionDeleteExecute", "CollectionAfterDeleteSuccess", "CollectionAfterDeleteError",
   "MailerSend", "MailerRecordAuthAlertSend", "MailerRecordPasswordResetSend", "MailerRecordVerificationSend", "MailerRecordEmailChangeSend", "MailerRecordOTPSend",
   "RealtimeConnectRequest", "RealtimeMessageSend", "RealtimeSubscribeRequest",
   "SettingsListRequest", "SettingsUpdateRequest", "SettingsReload", "FileDownloadRequest", "FileTokenRequest",
-  "RecordAuthRequest", "RecordAuthWithPasswordRequest", "RecordAuthRefreshRequest", "RecordRequestPasswordResetRequest", "RecordConfirmPasswordResetRequest", "RecordRequestVerificationRequest", "RecordConfirmVerificationRequest", "RecordRequestEmailChangeRequest", "RecordConfirmEmailChangeRequest", "RecordRequestOTPRequest", "RecordAuthWithOTPRequest",
+  "RecordAuthRequest", "RecordAuthWithPasswordRequest", "RecordAuthWithOAuth2Request", "RecordAuthRefreshRequest", "RecordRequestPasswordResetRequest", "RecordConfirmPasswordResetRequest", "RecordRequestVerificationRequest", "RecordConfirmVerificationRequest", "RecordRequestEmailChangeRequest", "RecordConfirmEmailChangeRequest", "RecordRequestOTPRequest", "RecordAuthWithOTPRequest",
   "RecordsListRequest", "RecordViewRequest", "RecordCreateRequest", "RecordUpdateRequest", "RecordDeleteRequest",
   "CollectionsListRequest", "CollectionViewRequest", "CollectionCreateRequest", "CollectionUpdateRequest", "CollectionDeleteRequest", "CollectionsImportRequest", "BatchRequest",
 ].map((n) => "on" + n);
@@ -126,10 +127,15 @@ function transform(sf: ts.SourceFile, asyncNames: Set<string>, asyncFns: Set<ts.
   return out;
 }
 
+// Generated bundles in pb_hooks must skip the await insertion below: it rewrites calls by method name (`delete`,
+// `next`, `send` ...), which in ordinary bundled code turns the wrong functions async and breaks their callers. A
+// file whose first line is `// voidbase:raw` is emitted as it stands, with only module/exports/require in scope.
+const isRaw = (code: string) => /^\s*\/\/\s*voidbase:raw\b/.test(code);
+
 export function compileHooksDir(dir: string): string {
   const files = readDir(dir);
   const sources = new Map<string, ts.SourceFile>();
-  for (const f of files) if (f.kind !== "file") sources.set(f.name, ts.createSourceFile(f.name, f.code, ts.ScriptTarget.ES2022, true, ts.ScriptKind.JS));
+  for (const f of files) if (f.kind !== "file" && !isRaw(f.code)) sources.set(f.name, ts.createSourceFile(f.name, f.code, ts.ScriptTarget.ES2022, true, ts.ScriptKind.JS));
   const asyncNames = new Set<string>();
   const asyncFns = new Set<ts.Node>();
   for (let i = 0; i < 20; i++) {
@@ -143,6 +149,11 @@ export function compileHooksDir(dir: string): string {
   const raw: string[] = [];
   for (const f of files) {
     if (f.kind === "file") { raw.push(`${JSON.stringify(f.name)}: ${JSON.stringify(f.code)}`); continue; }
+    if (isRaw(f.code)) {
+      // no destructure of every global: a bundle declares its own names and would collide with them
+      modules.push(`${JSON.stringify(basename(f.name, ".js"))}: async function (__g) { const { module, exports, require } = __g;\n${f.code}\nreturn module.exports; }`);
+      continue;
+    }
     const body = transform(sources.get(f.name)!, asyncNames, asyncFns);
     if (f.kind === "hook") hooks.push(`{ name: ${JSON.stringify(f.name)}, run: async function (__g) { ${destructure}\n${body}\n} }`);
     else modules.push(`${JSON.stringify(basename(f.name, ".js"))}: async function (__g) { ${destructure}\n${body}\nreturn module.exports; }`);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@voidbase-cloud/voidbase",
-  "version": "0.2.2",
+  "version": "0.3.0",
   "description": "PocketBase-compatible backend on Cloudflare Workers (D1, R2, Queues, Durable Objects) via Void, or a single Bun process",
   "license": "MIT",
   "repository": {
@@ -35,7 +35,9 @@
     "./api": "./src/server/api.ts",
     "./passkeys": "./src/server/webauthn.ts",
     "./cloud": "./src/cloud/rest.ts",
-    "./bundle": "./src/node/bundle.ts"
+    "./bundle": "./src/node/bundle.ts",
+    "./adapter": "./src/adapter/runtime.ts",
+    "./adapter/plugin": "./src/adapter/index.ts"
   },
   "imports": {
     "#platform/env": {
@@ -110,11 +112,14 @@
     "surface": "bun surface/render.ts",
     "test": "bun test",
     "app:sync": "bun scripts/sync-app.ts",
-    "check": "void prepare && tsc -p tsconfig.json --noEmit && tsc -p tsconfig.node.json --noEmit",
+    "check": "void prepare && tsc -p tsconfig.json --noEmit && tsc -p tsconfig.node.json --noEmit && tsc -p tsconfig.scripts.json --noEmit",
     "pack:check": "npm pack --dry-run",
     "prepublishOnly": "bun run check && bun test",
     "prepare": "husky || true",
-    "build:exe": "bun scripts/build-exe.ts"
+    "build:exe": "bun scripts/build-exe.ts",
+    "ci": "bash scripts/ci.sh",
+    "release": "bash scripts/release.sh",
+    "cf:builds": "bun scripts/cf-builds.ts"
   },
   "dependencies": {
     "@cf-wasm/photon": "^0.4.0",

package/routes/api/[...path].ts

@@ -1,11 +1,6 @@
 // Every /api/* request is handled by the voidbase Hono app (PocketBase wire protocol).
 import { defineHandler } from "void";
 import { app } from "../../src/server/app";
-import { appApi } from "../../src/server/api";
-import { mountWebAuthn } from "../../src/server/webauthn";
-
-// this checkout serves the pocketbase-sveltekit-starter, whose backend registers passkey routes (pb/webauthn)
-mountWebAuthn(appApi().router);
 
 const handle = defineHandler((c) =>
   app.fetch(c.req.raw, c.env, (c as unknown as { executionCtx?: ExecutionContext }).executionCtx),