toiljs 0.0.119 → 0.0.121

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,33 +1,33 @@
1
1
  export const TOIL_DOCS = {
2
2
  "README.md": "# toiljs\n\ntoiljs is a full-stack web framework. You write your **frontend in React** and your **backend in\nTypeScript**, and toiljs turns the backend into a tiny, fast **WebAssembly** program that runs at the edge\n(on servers close to your users, all over the world). One language, one project, one deploy.\n\nIf you have used Next.js, this will feel familiar: file-based routes, server code next to client code, a dev\nserver with hot reload. The difference is what happens underneath. Your server code is compiled by\n**toilscript** (a TypeScript-to-WebAssembly compiler) into a sandboxed `.wasm` module, and it runs on the\n**Dacely edge** with a built-in worldwide database (**ToilDB**), streaming, background jobs, and auth all\nincluded.\n\n## The mental model\n\n```mermaid\nflowchart LR\n A[\"Your project<br/>(TypeScript + React)\"] -->|toiljs build| B[\"client bundle<br/>(React, runs in the browser)\"]\n A -->|toilscript compile| C[\"server.wasm<br/>(your backend, sandboxed)\"]\n B --> U[\"User's browser\"]\n C --> E[\"Dacely edge<br/>(worldwide)\"]\n E --> D[(\"ToilDB<br/>global database\")]\n U <-->|HTTP / WebTransport| E\n```\n\n- **client/** is your React app (pages, components, styles). It runs in the browser.\n- **server/** is your backend (routes, database, auth). It compiles to WebAssembly and runs on the edge.\n- **shared/** is the typed bridge: toiljs generates a client here so the browser calls your server with full\n type safety.\n\n## Hello, toiljs\n\n```ts\n// server/routes/Hello.ts (a backend HTTP route)\nimport { Response } from 'toiljs/server/runtime';\n\n@rest('hello')\nclass Hello {\n @get('/')\n public hi(): Response {\n return Response.text('Hello from the edge!\\n');\n }\n}\n```\n\n```tsx\n// client/routes/index.tsx (a frontend page)\nexport default function Home() {\n return <main><h1>Welcome</h1></main>;\n}\n```\n\nRun `toiljs dev`, open the browser, and both are live with hot reload. That is the whole loop.\n\n## Learn toiljs\n\n**Understand toil first**\n- [Understanding toil](./introduction/README.md): what toil is and the one big idea, then\n [why toil and who it is for](./introduction/why-toil.md),\n [the modern stack you get](./introduction/modern-stack.md),\n [how it works](./introduction/how-it-works.md),\n [what makes it hyper-scalable](./introduction/hyperscale.md),\n [how it is distributed](./introduction/distributed.md),\n [toil versus other frameworks](./introduction/vs-other-frameworks.md), and\n [why it is built this way](./introduction/design-principles.md).\n\n**Start here**\n- [Getting started](./getting-started/README.md): install, create a project, project structure, your first\n app, and migrating an existing React app.\n- [The CLI](./cli/README.md): `dev`, `build`, `create`, `doctor`, and every flag.\n- [Deploy](./getting-started/deploy.md): build for production, self-host it, and how the managed edge fits in.\n\n**Build the frontend**\n- [Frontend overview](./frontend/README.md), [Routing](./frontend/routing.md),\n [Navigation](./frontend/navigation.md), [Components](./frontend/components.md),\n [Rendering and SSR](./frontend/rendering.md), [Styling](./frontend/styling.md),\n [Images](./frontend/images.md), [Metadata and SEO](./frontend/metadata.md),\n [Fetching data](./frontend/data-fetching.md), [Scripts](./frontend/scripts.md),\n [Search](./frontend/search.md), [The Toil global (reference)](./frontend/toil-global.md).\n\n**Build the backend**\n- [Backend overview](./backend/README.md), [HTTP routes (`@rest`)](./backend/rest.md),\n [Typed RPC (`@service`/`@remote`)](./backend/rpc.md), [Data types (`@data`)](./backend/data.md).\n\n**The database (ToilDB)**\n- [Database overview and choosing a family](./database/README.md), [Setup (`@database`)](./database/setup.md),\n [Documents](./database/documents.md), [Unique](./database/unique.md), [Counters](./database/counters.md),\n [Events](./database/events.md), [Views and `@derive`](./database/views.md),\n [Membership](./database/membership.md), [Capacity](./database/capacity.md).\n\n**Auth**: [the full auth guide](./auth/README.md) covers post-quantum login, sessions, and `ToilUserId`;\n[customizing the auth emails](./auth/emails.md) shows how to brand the verification, reset, and 2FA mail.\n\n**Realtime and background**\n- [Streams](./realtime/README.md) and [channels](./realtime/channels.md), [Daemons and scheduled\n jobs](./background/daemons.md), [Derived views (`@derive`)](./background/derive.md).\n\n**Platform services**\n- [Caching](./services/caching.md), [Rate limiting](./services/ratelimit.md),\n [Environment and secrets](./services/environment.md), [Email and 2FA](./services/email.md),\n [Analytics](./services/analytics.md), [Crypto](./services/crypto.md), [Cookies](./services/cookies.md),\n [Time](./services/time.md).\n\n**Concepts and reference**\n- [Writing toiljs correctly (AI + human quick rules)](./concepts/ai-guide.md),\n [Compute tiers (L1 to L4)](./concepts/tiers.md), [Types (u64, u256, and friends)](./concepts/types.md),\n [Every decorator](./concepts/decorators.md), [Configuration](./concepts/config.md),\n [Security and SRI](./concepts/security.md).\n",
3
- "auth/configuration.md": "# Configuring auth for production\n\nBuilt-in auth runs locally with **zero configuration**, it falls back to published, insecure DEV secrets\nso `toiljs dev` Just Works. **A deployment MUST replace all three secrets and pin its KEM key.** This page\nis the checklist.\n\n## The secrets\n\nAuth reads these from the tenant environment store (locally, `.env.secrets`; on the edge, the per-host\nsecure env). They resolve **lazily** the first time auth runs, so no startup wiring is needed.\n\n| Key | What it is | Dev fallback |\n| --- | --- | --- |\n| `AUTH_SESSION_SECRET` | HMAC-SHA256 key that signs the session cookie. Must be identical on every edge instance (a cookie minted anywhere must verify everywhere). | a public constant, **anyone can forge a session** |\n| `AUTH_OPRF_SEED` | Master seed for the per-user OPRF salt key. Rotating it invalidates every password (users must re-register). | a hashed public constant |\n| `AUTH_KEM_SK` | The server's ML-KEM-768 **secret** key (hex). Its public half is what the client encapsulates to. | a pinned dev key pair |\n\n```bash\n# .env.secrets (gitignored; mode 0600 on the edge, NEVER under hosts/, NEVER in the .wasm)\nAUTH_SESSION_SECRET=…64 hex chars (32 bytes)…\nAUTH_OPRF_SEED=…64 hex chars (32 bytes)…\nAUTH_KEM_SK=…hex of an ML-KEM-768 secret key…\n```\n\n### Generating them\n\n`AUTH_SESSION_SECRET` and `AUTH_OPRF_SEED` are just 32 random bytes each:\n\n```bash\nnode -e \"console.log(require('crypto').randomBytes(32).toString('hex'))\"\n```\n\n`AUTH_KEM_SK` is an ML-KEM-768 key pair. Generate it and keep BOTH halves, the secret goes in the env, the\npublic half is pinned in the client (next section):\n\n```ts\nimport { ml_kem768 } from '@dacely/noble-post-quantum/ml-kem';\nconst { secretKey, publicKey } = ml_kem768.keygen();\nconsole.log('AUTH_KEM_SK =', Buffer.from(secretKey).toString('hex'));\nconsole.log('client serverKemPublicKey =', Buffer.from(publicKey).toString('hex'));\n```\n\n## Pin the client's KEM public key\n\nThe browser must know the server's genuine KEM public key to run the mutual-auth handshake, this is the\nanti-phishing anchor. The `toiljs/client` `Auth` helper ships with the **dev** key pinned, so a deployment\nMUST pass its own:\n\n```ts\nimport { Auth } from 'toiljs/client';\n\nconst SERVER_KEM_PUBLIC_KEY = /* the publicKey bytes from AUTH_KEM_SK */;\n\nawait Auth.login(username, password, { serverKemPublicKey: SERVER_KEM_PUBLIC_KEY });\nawait Auth.register(username, password, { serverKemPublicKey: SERVER_KEM_PUBLIC_KEY });\n```\n\nShip the public key with your client bundle (it's public, safe to embed). If it doesn't match the server's\n`AUTH_KEM_SK`, login's `serverConfirm` check fails and the client aborts.\n\n## Optional: audience & domain\n\nBoth are optional and have sensible defaults; set them for stability across host aliases:\n\n| Key | Meaning | Default |\n| --- | --- | --- |\n| `TOIL_AUTH_AUDIENCE` | The service audience bound into the signed login message. | `\"toil\"` |\n| `TOIL_AUTH_DOMAIN` | The `domain` input of the stable `ToilUserId` (`sha256(pubkey ‖ username ‖ domain)`). | the request `Host` header, else `localhost` |\n\nSet `TOIL_AUTH_DOMAIN` explicitly if your site answers on multiple hostnames, otherwise the same user could\nget different `ToilUserId`s from different aliases. Once users exist, changing it changes everyone's id, so\npick it before launch.\n\n## Email confirmation\n\nTwo plain env vars (tenant-readable, like `PUBLIC_BASE_URL`) control the built-in email-verification flow. They are separate on purpose: sending a verify email at registration is one decision, blocking login until confirmed is another.\n\n| Key | What it does |\n| --- | --- |\n| `AUTH_EMAIL_CONFIRMATION` | Registration emails a one-time confirm link (`/confirm#token=…`) and stores the account **unconfirmed**. Login is **not** blocked: the user can sign in, and you nudge them to verify. Use this for \"verify at signup, but let people in\". |\n| `AUTH_REQUIRE_EMAIL_CONFIRMATION` | The stricter form: does everything above AND **refuses login** until the email is confirmed (a distinguishable status the client surfaces as `EmailNotConfirmedError`, so the UI can prompt \"confirm your email / resend\"). Implies `AUTH_EMAIL_CONFIRMATION`. On the managed edge this is force-set from the per-domain Dacely setting. |\n\nWith neither set, accounts are auto-confirmed at registration and no confirm email is sent. The one-time link is delivered by email; in `toiljs dev` it is drawn in the console (no mailer needed). Both use the standard non-enumerating responses (a generic 200 that never reveals whether an address exists).\n\n## Argon2id strength (known limitation)\n\nThe built-in controller currently uses **demo-light** Argon2id params (32 MiB, 2 iterations, 1 lane) so it\nstays responsive in a browser tab. These are baked into the shipped controller today; **config-driven\ntuning is a planned follow-up.** For a high-value production deployment you should either wait for the\nconfig knob or hand-write your own controller with `≥ 256 MiB / ≥ 3 iterations`. The OPRF still provides the\nprimary offline-attack resistance regardless, but raise these before protecting anything sensitive.\n\nThe client always derives against whatever params the server returns in `/login/start`, so when the config\nknob lands you can raise them server-side with **no client change**.\n\n## Deploy checklist\n\n- [ ] `AUTH_SESSION_SECRET` set (32 random bytes), identical on every edge instance.\n- [ ] `AUTH_OPRF_SEED` set (32 random bytes).\n- [ ] `AUTH_KEM_SK` set (an ML-KEM-768 secret key), and its **public** half pinned in the client via\n `serverKemPublicKey`.\n- [ ] `TOIL_AUTH_DOMAIN` set if you serve multiple hostnames (stable `ToilUserId`).\n- [ ] (Recommended) Argon2id params reviewed for your threat model.\n\nThe CLI doctor warns when `server.auth` is on and the secrets are missing, run it before you ship.\n\n## Upgrading to 0.0.118\n\nLogin now requires `AuthOptions.serverKemPublicKey`, the 1184-byte public key matching\n`AUTH_KEM_SK`. There is no implicit fallback to the published development key. Keep\nexisting OPRF and KEM secrets when upgrading; replacing the OPRF seed invalidates\nexisting password credentials. Project-root `VITE_*` variables are loaded by default.\n\nSessions use a versioned signed payload with a random session identifier and a user\nrevocation generation. Pre-upgrade session cookies and pending login challenges are\nrejected, so users must sign in again. Single-session logout revokes the signed session;\n`POST /auth/logout-all`, password reset, and confirmed two-factor changes revoke all\nsessions. Two-factor changes require a fresh login after success. Existing account IDs\nremain stable after password reset. The compiler includes the new revocation and factor\nsettings database declarations automatically.\n\nAuthenticated factor operations accept `headers` and `onRequestFailed(status)` options\nfor application account guards. HTTP failures include their status in the client error.\n",
3
+ "auth/configuration.md": "# Configuring auth for production\n\nBuilt-in auth runs locally with **zero configuration**, it falls back to published, insecure DEV secrets\nso `toiljs dev` Just Works. **A deployment MUST replace all three secrets and pin its KEM key.** This page\nis the checklist.\n\n## The secrets\n\nAuth reads these from the tenant environment store (locally, `.env.secrets`; on the edge, the per-host\nsecure env). They resolve **lazily** the first time auth runs, so no startup wiring is needed.\n\n| Key | What it is | Dev fallback |\n| --- | --- | --- |\n| `AUTH_SESSION_SECRET` | HMAC-SHA256 key that signs the session cookie. Must be identical on every edge instance (a cookie minted anywhere must verify everywhere). | a public constant, **anyone can forge a session** |\n| `AUTH_OPRF_SEED` | Master seed for the per-user OPRF salt key. Rotating it invalidates every password (users must re-register). | a hashed public constant |\n| `AUTH_KEM_SK` | The server's ML-KEM-768 **secret** key (hex). Its public half is what the client encapsulates to. | a pinned dev key pair |\n\n```bash\n# .env.secrets (gitignored; mode 0600 on the edge, NEVER under hosts/, NEVER in the .wasm)\nAUTH_SESSION_SECRET=…64 hex chars (32 bytes)…\nAUTH_OPRF_SEED=…64 hex chars (32 bytes)…\nAUTH_KEM_SK=…hex of an ML-KEM-768 secret key…\n```\n\n### Generating them\n\nRun this from your project root (or pass `--root <dir>`):\n\n```bash\ntoiljs generate auth\n```\n\nThe command saves `AUTH_SESSION_SECRET`, `AUTH_OPRF_SEED`, and `AUTH_KEM_SK` in\n`.env.secrets` with mode `0600`, and the matching `VITE_AUTH_KEM_PUBLIC_KEY` in `.env`.\nIt adds both files to `.gitignore`, preserves unrelated settings, and does not print credentials.\nRerunning preserves existing keys and restores a missing public key from `AUTH_KEM_SK`.\nInvalid, empty, or mismatched existing credentials cause an error without changing the env files.\nThere is no automatic rotation: changing the OPRF seed invalidates passwords, and changing the\nsession secret invalidates sessions. Back up the private file securely.\n\nFor toil-backend, merge the private settings into `/run/toil/env/<host>.env.secrets`\n(or the directory selected by `TOIL_ENV_DIR`). If using database-backed env settings, run\n`toil env sync <host>` with the production configuration after editing the files.\nThe **public** key must also be present in the project `.env` on the machine running the frontend\nbuild. Backend runtime env files do not supply Vite's build-time variables. Rebuild and deploy\nthe client with its matching key; the command does not edit your client code or deploy anything.\n\nTo generate keys manually instead:\n\n`AUTH_SESSION_SECRET` and `AUTH_OPRF_SEED` are just 32 random bytes each:\n\n```bash\nnode -e \"console.log(require('crypto').randomBytes(32).toString('hex'))\"\n```\n\n`AUTH_KEM_SK` is an ML-KEM-768 key pair. Generate it and keep BOTH halves, the secret goes in the env, the\npublic half is pinned in the client (next section):\n\n```ts\nimport { ml_kem768 } from '@dacely/noble-post-quantum/ml-kem';\nconst { secretKey, publicKey } = ml_kem768.keygen();\nconsole.log('AUTH_KEM_SK =', Buffer.from(secretKey).toString('hex'));\nconsole.log('client serverKemPublicKey =', Buffer.from(publicKey).toString('hex'));\n```\n\n## Pin the client's KEM public key\n\nThe browser must know the server's genuine KEM public key to run the mutual-auth handshake, this is the\nanti-phishing anchor. The `toiljs/client` `Auth` helper ships with the **dev** key pinned, so a deployment\nMUST pass its own:\n\n```ts\nimport { Auth } from 'toiljs/client';\n\n// Set by `toiljs generate auth` in the project .env, embedded during the client build.\nconst hex = import.meta.env.VITE_AUTH_KEM_PUBLIC_KEY;\nif (!hex || !/^[0-9a-f]{2368}$/.test(hex)) throw new Error('Missing auth public key');\nconst SERVER_KEM_PUBLIC_KEY = Uint8Array.from(hex.match(/../g)!, byte => parseInt(byte, 16));\n\nawait Auth.login(username, password, { serverKemPublicKey: SERVER_KEM_PUBLIC_KEY });\nawait Auth.register(username, password, { serverKemPublicKey: SERVER_KEM_PUBLIC_KEY });\n```\n\nShip the public key with your client bundle (it's public, safe to embed). If it doesn't match the server's\n`AUTH_KEM_SK`, login's `serverConfirm` check fails and the client aborts.\n\n## Optional: audience & domain\n\nBoth are optional and have sensible defaults; set them for stability across host aliases:\n\n| Key | Meaning | Default |\n| --- | --- | --- |\n| `TOIL_AUTH_AUDIENCE` | The service audience bound into the signed login message. | `\"toil\"` |\n| `TOIL_AUTH_DOMAIN` | The `domain` input of the stable `ToilUserId` (`sha256(pubkey ‖ username ‖ domain)`). | the request `Host` header, else `localhost` |\n\nSet `TOIL_AUTH_DOMAIN` explicitly if your site answers on multiple hostnames, otherwise the same user could\nget different `ToilUserId`s from different aliases. Once users exist, changing it changes everyone's id, so\npick it before launch.\n\n## Email confirmation\n\nTwo plain env vars (tenant-readable, like `PUBLIC_BASE_URL`) control the built-in email-verification flow. They are separate on purpose: sending a verify email at registration is one decision, blocking login until confirmed is another.\n\n| Key | What it does |\n| --- | --- |\n| `AUTH_EMAIL_CONFIRMATION` | Registration emails a one-time confirm link (`/confirm#token=…`) and stores the account **unconfirmed**. Login is **not** blocked: the user can sign in, and you nudge them to verify. Use this for \"verify at signup, but let people in\". |\n| `AUTH_REQUIRE_EMAIL_CONFIRMATION` | The stricter form: does everything above AND **refuses login** until the email is confirmed (a distinguishable status the client surfaces as `EmailNotConfirmedError`, so the UI can prompt \"confirm your email / resend\"). Implies `AUTH_EMAIL_CONFIRMATION`. On the managed edge this is force-set from the per-domain Dacely setting. |\n\nWith neither set, accounts are auto-confirmed at registration and no confirm email is sent. The one-time link is delivered by email; in `toiljs dev` it is drawn in the console (no mailer needed). Both use the standard non-enumerating responses (a generic 200 that never reveals whether an address exists).\n\n## Argon2id strength (known limitation)\n\nThe built-in controller currently uses **demo-light** Argon2id params (32 MiB, 2 iterations, 1 lane) so it\nstays responsive in a browser tab. These are baked into the shipped controller today; **config-driven\ntuning is a planned follow-up.** For a high-value production deployment you should either wait for the\nconfig knob or hand-write your own controller with `≥ 256 MiB / ≥ 3 iterations`. The OPRF still provides the\nprimary offline-attack resistance regardless, but raise these before protecting anything sensitive.\n\nThe client always derives against whatever params the server returns in `/login/start`, so when the config\nknob lands you can raise them server-side with **no client change**.\n\n## Deploy checklist\n\n- [ ] `AUTH_SESSION_SECRET` set (32 random bytes), identical on every edge instance.\n- [ ] `AUTH_OPRF_SEED` set (32 random bytes).\n- [ ] `AUTH_KEM_SK` set (an ML-KEM-768 secret key), and its **public** half pinned in the client via\n `serverKemPublicKey`.\n- [ ] `TOIL_AUTH_DOMAIN` set if you serve multiple hostnames (stable `ToilUserId`).\n- [ ] (Recommended) Argon2id params reviewed for your threat model.\n\nThe CLI doctor warns when `server.auth` is on and the secrets are missing, run it before you ship.\n\n## Upgrading to 0.0.118\n\nLogin now requires `AuthOptions.serverKemPublicKey`, the 1184-byte public key matching\n`AUTH_KEM_SK`. There is no implicit fallback to the published development key. Keep\nexisting OPRF and KEM secrets when upgrading; replacing the OPRF seed invalidates\nexisting password credentials. Project-root `VITE_*` variables are loaded by default.\n\nSessions use a versioned signed payload with a random session identifier and a user\nrevocation generation. Pre-upgrade session cookies and pending login challenges are\nrejected, so users must sign in again. Single-session logout revokes the signed session;\n`POST /auth/logout-all`, password reset, and confirmed two-factor changes revoke all\nsessions. Two-factor changes require a fresh login after success. Existing account IDs\nremain stable after password reset. The compiler includes the new revocation and factor\nsettings database declarations automatically.\n\nAuthenticated factor operations accept `headers` and `onRequestFailed(status)` options\nfor application account guards. HTTP failures include their status in the client error.\n\n## Session lifetime\n\nSuccessful password and two-factor sign-ins issue sessions valid for seven days\n(604800 seconds). The signed expiry and both session/display cookie lifetimes\nmatch. This is a fixed lifetime from sign-in, not an idle timeout. Logout, password\nreset, and account revocation can end a session earlier. Existing cookies retain\nthe expiry they were issued with; sign in again to receive the longer lifetime.\n",
4
4
  "auth/emails.md": "# Customizing the auth emails\n\nBuilt-in auth sends three transactional emails: the email-verification link, the password-reset link, and the two-factor code. Each ships with a plain, working default. You can replace any of them with your own branded template, and you never touch the auth code to do it.\n\n## The three emails\n\n| Email | When it sends | The value it gives your template |\n| --- | --- | --- |\n| Email verification | On register, when email confirmation is on | `link` (the confirm URL) |\n| Password reset | On `POST /auth/reset/request` | `link` (the reset URL) |\n| Two-factor code | On login or 2FA setup, for the email method | `code` (the numeric code) and `action` (the flow phrase) |\n\nThe defaults are intentionally minimal so a fresh project sends real, correct mail with no setup. To match your brand, override them.\n\n## How to override\n\nDrop a React template in your project's `emails/` folder with the reserved name for that email. If the file exists, the build uses it; if not, the default stands. This is the same `emails/*.tsx` system you use for your own mail (see [Email](../services/email.md)), so you get full markup and inline CSS.\n\n| Reserved file | Overrides | Prop it must read |\n| --- | --- | --- |\n| `emails/auth-confirm.tsx` | Email verification | `link` |\n| `emails/auth-reset.tsx` | Password reset | `link` |\n| `emails/auth-2fa.tsx` | Two-factor code | `code`, `action` |\n\nEach template is a default-exported React component. Read the one prop it is given, and it becomes a `{{token}}` hole the edge fills per send. Reading any other prop renders empty, so the build warns you.\n\n```tsx\n// emails/auth-confirm.tsx -> overrides the email-verification message.\nexport const subject = 'Verify your Acme account';\n\nexport default function AuthConfirm({ link }: { link: string }) {\n return (\n <div style={{ fontFamily: 'system-ui', padding: 24 }}>\n <h1 style={{ color: '#cb9820' }}>Welcome to Acme</h1>\n <p>Tap below to verify your email and finish signing up.</p>\n <p>\n <a\n href={link}\n style={{ background: '#cb9820', color: '#fff', padding: '10px 18px', borderRadius: 8, textDecoration: 'none' }}\n >\n Verify my email\n </a>\n </p>\n <p style={{ color: '#666', fontSize: 13 }}>Or paste this link: {link}</p>\n </div>\n );\n}\n```\n\nThe reset template works the same way with the `link` prop.\n\n## The subject line\n\nFor verification and reset, set the subject with `export const subject = '...'`. Leave it out and the default subject is used (`Confirm your account`, `Reset your password`).\n\nThe two-factor email is different. It serves three flows (signing in, enabling 2FA, and turning 2FA off), so besides the `code` it also gets an `action` prop: the framework fills it with the flow phrase, `sign in`, `enable two-factor authentication`, or `turn off two-factor authentication`. Write it into a sentence like \"Enter this code to {action}.\" and the one template reads correctly for every flow. The subject is likewise set by the framework and stays contextual, so a `subject` export in this file is ignored.\n\n```tsx\n// emails/auth-2fa.tsx -> overrides the 2FA code message (subject is contextual).\nexport default function Auth2fa({ code, action }: { code: string; action: string }) {\n return (\n <div style={{ fontFamily: 'system-ui', padding: 24, textAlign: 'center' }}>\n <p>Your Acme verification code:</p>\n <p style={{ fontSize: 32, fontWeight: 700, letterSpacing: 6 }}>{code}</p>\n <p>Enter it to {action}.</p>\n <p style={{ color: '#666' }}>It expires in a few minutes.</p>\n </div>\n );\n}\n```\n\n## What you can and cannot put in\n\nEach template is rendered once, at build time, into a static string of HTML with your styles inlined, and the prop becomes a fill hole. So a fixed layout and inline styles work, but per-send logic does not: a `{items.map(...)}` or a conditional runs at build, not per email. For these three messages that is all you need, one link or one code.\n\nThe only values auth hands each template are the ones in the table above (`link`, or `code` plus `action` for 2FA). There is no user name, no app name, no expiry value. Write those into the template text directly.\n\nStyling rules are the same as any toiljs email: write inline `style={{ ... }}`, or import a stylesheet and its rules are inlined for you. A bare CSS import with no inlining has no effect in an email client. See [Email](../services/email.md) for the details.\n\n## Delivery stays safe\n\nYou cannot weaken the security of these flows by overriding a template. Every auth email is sent detached (fire-and-forget), so the response time never reveals whether an address maps to an account. That holds whether the message is the default or your override. Overriding only changes what the email looks like, never how it is sent.\n\n## Where the templates go\n\nThere is nothing to import and nothing to register. The build reads `emails/auth-*.tsx`, bakes the effective templates into the compiled server, and the auth controller sends them. Changing a template and rebuilding (or saving under `toiljs dev`) is all it takes.\n\n## Related\n\n- [Email](../services/email.md): the full `emails/*.tsx` system, `EmailService`, and `EmailTemplate`.\n- [Configuration](./configuration.md): the auth secrets and the email provider a deployment must set.\n- [Usage](./usage.md): enabling auth, the client API, and guarding routes.\n",
5
5
  "auth/extending.md": "# Extending & integrating auth\n\nBuilt-in auth is deliberately opinionated so the common case is one line. This page covers the identity\nyou build ON, `ToilUserId`, and how to go beyond the defaults: keying your own data on a user, a custom\nuser shape, and hand-writing auth from the same primitives.\n\n## `ToilUserId`\n\nThe stable, tenant-scoped user identity: `sha256(mldsaPublicKey ‖ identifier ‖ domain)`, a 256-bit value.\nIt's a **global** (no import), like `crypto`.\n\n```ts\n// Read the current user's id in any handler (gate on hasSession(), see the null note below).\nconst id: ToilUserId = AuthService.userId()!;\n\n// Or derive one yourself.\nconst id2 = ToilUserId.derive(mldsaPublicKey, 'alice@example.com', 'acme.dacely.com');\n```\n\n| Member | Description |\n| --- | --- |\n| `ToilUserId.derive(pk, identifier, domain)` | Derive from an ML-DSA public key + email/username + tenant domain. Deterministic. |\n| `ToilUserId.fromBytes(b)` | Rebuild from a 32-byte digest (from `toBytes()` or storage). |\n| `toBytes(): Uint8Array` | The 32 identity bytes. |\n| `toHex(): string` | Lowercase 64-char hex, a convenient string key. |\n| `isZero(): bool` | True for the unset / anonymous id. |\n| `equals(other): bool` | Value equality. |\n| `a == b` / `a != b` | Overloaded value comparison, **O(1)** (four `u64` word compares, no byte loop, no allocation). |\n\n```ts\nconst a = ToilUserId.derive(pk, 'alice', 'acme.com');\nconst b = ToilUserId.derive(pk, 'alice', 'acme.com');\nconst c = ToilUserId.derive(pk, 'bob', 'acme.com');\na == b; // true, same inputs, same id\na != c; // true, different user\n```\n\n> **Null-check gotcha:** because `ToilUserId` overloads `==`, `AuthService.userId() == null` does NOT\n> type-check (`==` expects a `ToilUserId`). Gate with `AuthService.hasSession()` and then `userId()!`, or\n> compare with `getUser()` (a plain nullable). `===` is reference identity in AssemblyScript and is not\n> overloadable, use `==` for value equality.\n\n## Keying your own data on the user\n\n`toilUserId` is the right key for per-user data, it's stable across sessions/devices and opaque. Use the\nhex as a string key, or the bytes in a `@data` key class:\n\n```ts\n@data\nclass UserKey {\n id: Uint8Array = new Uint8Array(0); // toilUserId bytes\n constructor(id: Uint8Array = new Uint8Array(0)) { this.id = id; }\n}\n\n@data class Profile { displayName: string = ''; bio: string = ''; }\n\n@database\nclass AppDb {\n @collection static profiles: Documents<UserKey, Profile>;\n}\n\n@rest('profile')\nclass ProfileApi {\n @auth\n @post('/')\n public save(ctx: RouteContext): Response {\n const key = new UserKey(AuthService.userId()!.toBytes());\n const p = Profile.decode(ctx.request.body);\n // Save this user's profile. create is insert-only, so the first save creates\n // the record and later saves overwrite the existing one with enqueue.\n if (!AppDb.profiles.create(key, p)) {\n AppDb.profiles.enqueue(key, p);\n }\n return Response.text('saved\\n');\n }\n}\n```\n\n## Extending the user: add your own fields\n\nBuilt-in auth reserves exactly two fields on the authenticated user: `toilUserId` and `username`. To carry\nmore (a role, a display name, a tenant), just declare your OWN `@user` in your server code while\n`server.auth` is on. The build detects it and **extends** it: it injects the reserved `toilUserId` +\n`username` as the first two fields and mints sessions for your shape automatically. You do not opt out of\nanything, and there is still exactly one `@user` per program.\n\n```ts\n@user\nclass Account {\n admin: bool = false;\n displayName: string = '';\n tenant: string = '';\n // toilUserId + username are INJECTED by the build; do not declare them here.\n}\n```\n\nRules:\n\n- Do **not** declare `toilUserId` or `username` yourself. They are reserved and injected; declaring either is\n a compile error (`'username' is reserved by built-in auth`).\n- Your `@user` must be default-constructible (give every field an initializer), like any `@data` class.\n\n`AuthService.getUser()` is now typed to YOUR shape:\n\n```ts\nconst user = AuthService.getUser(); // { toilUserId, username, admin, displayName, tenant } | null\n```\n\n**Populating your fields.** Login fills `toilUserId` + `username`; your extra fields start at their declared\ndefaults (`admin = false`, and so on). The built-in controller cannot know your business fields, so you set\nthem yourself on one of your own `@auth` routes by reading the user, updating it, and re-minting the session:\n\n```ts\n@rest('account')\nclass AccountApi {\n @auth @post('/promote')\n public promote(): Response {\n const user = AuthService.getUser()!;\n user.admin = true;\n const resp = Response.text('promoted\\n');\n resp.setCookie(AuthService.mintSession(user.encode())); // re-sign the session with the new fields\n resp.setCookie(AuthService.userCookie(user.encode())); // update the readable companion cookie\n return resp;\n }\n}\n```\n\nIf you would rather hand-write the whole controller instead, you still can: do **not** enable `server.auth`,\nand build your own `@user` + routes from the [AuthService primitives](#the-authservice-primitive-reference)\nbelow. But for most apps, extending is all you need.\n\n## Adding email verification / 2FA\n\nLayer a second factor on top of the session with `TwoFactor` (stateless email codes, no DB; see\n[email](../services/email.md)). Typical flow: after login, require a verified email before granting access to\nsensitive routes.\n\n```ts\n@rest('2fa')\nclass TwoFactorApi {\n // Step 1: email a code to the logged-in user, hand back the signed token.\n @auth @post('/send')\n public send(): Response {\n const email = /* the user's email, e.g. their username, or a stored profile field */;\n const ch = TwoFactor.send(email, 'login'); // emails the code, returns { token, status }\n return Response.bytes(new DataWriter().writeString(ch.token).toBytes());\n }\n\n // Step 2: verify the code the user typed against the token.\n @auth @post('/verify')\n public verify(ctx: RouteContext): Response {\n const r = new DataReader(ctx.request.body);\n const token = r.readString(); const email = r.readString(); const code = r.readString();\n if (!TwoFactor.verify(token, email, code)) return Response.text('bad code\\n', 401);\n // Mark this session 2FA-verified: re-mint the session with a flag in your own @user, or store a\n // per-user \"verified\" record keyed on AuthService.userId().\n return Response.text('verified\\n');\n }\n}\n```\n\n`TwoFactor` gives integrity + expiry but not single-use (a code re-verifies within its TTL); keep the TTL\nshort. For a branded email, use `TwoFactor.issue(...)` (returns the code without sending) + your own\n`Emails.*` template. Call `TwoFactor.setSecret(...)` once at startup in production.\n\n## The `AuthService` primitive reference\n\nEverything the built-in controller is built from, available for hand-written auth. All are ambient globals\n(no import).\n\n**Sessions & cookies**\n- `mintSession(userData: Uint8Array, ttlSecs?: u64): Cookie`: the signed `__Host-toil_sess` cookie.\n- `userCookie(userData, ttlSecs?): Cookie`: the readable `__Secure-toil_user` companion.\n- `clearSession(): Cookie` / `clearUserCookie(): Cookie`.\n- `hasSession(): bool`: the `@auth` predicate.\n- `getSessionBytes(): Uint8Array | null`: the verified `@user` payload bytes.\n- `getUser(): <your @user> | null`: decoded, typed to your `@user`.\n- `userId(): ToilUserId | null`: the stable id (built-in `@user` layout).\n- `setSecret(secret: Uint8Array)`: override the session HMAC key programmatically.\n\n**Post-quantum login crypto**\n- `oprfEvaluate(username, blinded): Uint8Array`: server-keyed OPRF eval.\n- `buildRegisterMessage(username, pk)` / `verifyRegister(pk, msg, sig): bool`: proof-of-possession.\n- `buildLoginMessage(sub, aud, cid, nonce, iat, exp, ct, memKiB, iterations, parallelism, serverKemKeyId)`\n / `verifyLogin(pk, msg, sig): bool`.\n- `mlkemDecapsulate(ct): Uint8Array`, `serverKemKeyId(): Uint8Array`.\n- `deriveSessionKey(sharedSecret, transcriptHash)`, `serverConfirmTag(sessionKey, transcriptHash)`: the\n mutual-auth confirmation.\n- `sha256(data): Uint8Array`.\n- `setOprfSeed(seed)`, `setServerKemSecretKey(sk)`, `setServerKemPublicKey(pk)`: override the seeds/keys.\n- Sizes: `PUBLIC_KEY_LEN`, `SIGNATURE_LEN`, `OPRF_ELEMENT_LEN`, `KEM_CIPHERTEXT_LEN`, `SHARED_SECRET_LEN`, …\n\n**Related globals**, `TwoFactor` (email codes; see [email](../services/email.md)), `RateLimitService` (used by\n`@ratelimit`), `Environment` (the secret store).\n\n## Two ways in, one behavior\n\nThe config flag and the import are identical at build time, both make the build append the shipped\n`@user` + `@rest('auth')` controller to the toilscript **entry** set, where their decorators weave and the\n`@rest` class self-mounts. (A framework decorator source only weaves as an entry; that's why a plain\n`import` of the controller isn't enough on its own, the marker import is detected by the build, which does\nthe entry injection.) This is why built-in auth needs no runtime registration call.\n",
6
6
  "auth/how-it-works.md": "# How Toil auth works\n\nToil auth is a **post-quantum, password-based, mutually-authenticated** login. The design goal: the user\ntypes a password, but the server never sees it, never stores it, and can't be phished into leaking a\nverifier, and every primitive is quantum-resistant.\n\nThis page explains the protocol. You do not need to understand it to use auth (see [Usage](./usage.md)),\nbut you should understand the guarantees before you ship.\n\n## The building blocks\n\n| Primitive | Role |\n| --- | --- |\n| **OPRF** (RFC 9497, ristretto255-SHA512) | A *server-keyed* salt. The client blinds its password, the server evaluates it under a per-user key, the client unblinds. The result can't be computed without the server, so a stolen database can't be brute-forced offline. |\n| **Argon2id** | Stretches the OPRF output into key material (memory-hard, GPU/ASIC-resistant). Runs in the browser. |\n| **ML-DSA-44** (FIPS 204) | The user's identity key pair is *derived* from the Argon2id output. The server stores only the **public** key. Login is proved by an ML-DSA signature. |\n| **ML-KEM-768** (FIPS 203) | Key encapsulation on login: the server proves it holds its KEM secret by returning a confirmation tag only derivable from the decapsulated shared secret → **mutual** auth (anti-phishing). |\n| **HMAC-SHA256** | Signs the session cookie (`AUTH_SESSION_SECRET`). |\n\n> The whole thing is sometimes called an *aPAKE* (asymmetric password-authenticated key exchange). Do not\n> call it OPAQUE, this is Toil's own OPRF + KEM + signature construction.\n\n## What the server stores\n\nPer account, in ToilDB (`@database AuthDb`):\n\n- `username`, a deterministic `salt`, the **ML-DSA public key**, and the Argon2id params.\n\nThat's it. **No password, no password hash, no verifier that can be brute-forced without the OPRF key.**\nLogin challenges are a second collection, consumed exactly once (atomic `getDelete`).\n\n## Registration\n\n```\n Browser (client) Toil edge (server)\n ───────────────── ──────────────────\n blind(password) OPRF key = f(seed, username)\n │ username, blinded │\n │ ───────── POST /auth/register/start ───────────► │\n │ │ evaluated = OPRF(blinded)\n │ ◄──── {mem,iters,par, salt, evaluated} ───────── │ (salt is deterministic per user)\n │ │\n unblind → oprfOut │\n seed = Argon2id(oprfOut, salt, params) │\n (pk, sk) = ML-DSA-44.keygen(seed) │\n proof = ML-DSA.sign(sk, \"register|username|pk\") │\n │ username, pk, proof │\n │ ───────── POST /auth/register/finish ──────────► │\n │ │ verifyRegister(pk, msg, proof) ✓ proof-of-possession\n │ ◄──────────── {status: 0 ok | 1 taken} ──────── │ store AuthAccount{username, salt, pk, params}\n```\n\nThe server never learns the password or the secret key `sk`, only the public key `pk` and a proof the\nclient holds the matching `sk`. A duplicate username returns a **distinguishable** `status = 1` (so the UI\ncan say \"taken, log in instead\"); everything else fails generically.\n\n## Login (with mutual auth)\n\n```\n Browser (client) Toil edge (server)\n ───────────────── ──────────────────\n blind(password) │\n │ username, blinded │\n │ ───────── POST /auth/login/start ──────────────► │ evaluated = OPRF(blinded) (ALWAYS, even unknown user)\n │ │ if known: store Challenge{cid, nonce, iat, exp}\n │ ◄─ {cid, aud, params, salt, nonce, iat, exp, ── │ (identical response whether the user exists or not)\n │ evaluated} │\n unblind → Argon2id → (pk, sk) = ML-DSA.keygen │\n (ct, ssС) = ML-KEM-768.encapsulate(serverKemPublicKey) │\n msg = \"login|username|aud|cid|nonce|iat|exp|ct|params|kid\" │\n sig = ML-DSA.sign(sk, msg) │\n │ cid, ct, sig │\n │ ───────── POST /auth/login/finish ─────────────► │ ch = challenges.getDelete(cid) (consume once; check exp)\n │ │ rebuild msg from OUR stored values + ct\n │ │ verifyLogin(acct.pk, msg, sig) ✓ it's really this user\n │ │ ssS = ML-KEM.decapsulate(ct) ✓ we hold the KEM key\n │ │ K = deriveSessionKey(ssS, H(msg))\n │ ◄──── {0, sessionToken, serverConfirm} ──────── │ serverConfirm = tag(K, H(msg))\n │ + Set-Cookie: __Host-toil_sess=… │ mint session cookie\n verify serverConfirm using ssC ✓ the server is genuine │\n```\n\nTwo verifications, both required:\n1. **The server verifies the client**: the ML-DSA signature over a message bound to the challenge, the\n Argon2id params, and the server's KEM key id. Replays fail (the challenge is consumed).\n2. **The client verifies the server**: the `serverConfirm` tag is derivable only from the ML-KEM shared\n secret, which only the holder of the KEM secret key can decapsulate. A phishing site can't forge it.\n\n## Anti-enumeration\n\n`/login/start` behaves identically for a known and an unknown user: it always runs the OPRF (a decoy key\nfor unknown users), returns a **deterministic** per-user salt and constant params, and a fresh challenge.\nThe challenge is persisted only for a real account, and `/login/finish` fails generically at consume for an\nunknown user. So an attacker can't probe which usernames exist. Every failure path returns the same\n`401 auth: request failed`.\n\n## Sessions & cookies\n\nOn successful login the server mints **two** cookies:\n\n- **`__Host-toil_sess`**: the authoritative session. HMAC-SHA256 signed with `AUTH_SESSION_SECRET`,\n `HttpOnly`, `Secure`, `SameSite=Lax`. It carries the `@user` codec payload. `@auth` and\n `AuthService.getUser()` open + verify this cookie server-side; a forged or tampered cookie fails.\n- **`__Secure-toil_user`**: a **readable** companion carrying the same payload, so the browser can show\n \"logged in as …\" via the client's `getUser()`. The server **never trusts it**, it is display-only.\n\nEach request runs in a fresh wasm instance, but the signed cookie is self-contained, so no server-side\nsession store is needed. `AUTH_SESSION_SECRET` must be identical across every edge instance (it is, via the\nenv store) so a cookie minted anywhere verifies everywhere.\n\n## The stable user identity: `ToilUserId`\n\nAt login the server derives a stable, tenant-scoped id and stores it in the session (the first field of the\nbuilt-in `@user`):\n\n```\ntoilUserId = SHA-256( mldsaPublicKey ‖ username ‖ domain ) // 256 bits\n```\n\n- **Stable:** same login key + username on the same tenant `domain` → same id, forever, across sessions\n and devices. Key your own data on it.\n- **Opaque + one-way:** it's a hash: safe to store, log, or expose without leaking the key or the address.\n- Read it anywhere with `AuthService.userId()`. See [Extending](./extending.md#toiluserid) for the\n `ToilUserId` API (O(1) `==` / `!=`, `toHex()`, …).\n\n## Threat-model summary\n\n- **Server database stolen** → attacker gets public keys + salts, not passwords. Brute-forcing needs the\n OPRF key (server-side) *and* Argon2id work per guess.\n- **Server compromised / malicious** → still can't recover passwords (only public keys) and can't forge a\n past session without `AUTH_SESSION_SECRET`.\n- **Phishing site** → can't produce the `serverConfirm` tag (no KEM secret), so a correct client aborts.\n- **Quantum adversary** → ML-DSA + ML-KEM are post-quantum; the OPRF/Argon2id/HMAC pieces are classical but\n not the long-term identity or key-exchange.\n- **Replay** → challenges are single-use (`getDelete`) with a short TTL.\n\nResidual responsibilities are yours: set the [secrets](./configuration.md), pin your deployment's KEM\npublic key in the client, and raise the Argon2id params for production.\n",
7
- "auth/README.md": "# Authentication\n\nToil ships a complete, **post-quantum password login** you turn on with one line. No passwords on your\nserver, no third-party identity provider, no hand-written crypto: enable it and you get a `/auth/*` API,\nsigned sessions, `@auth`-guarded routes, and a stable per-user identity.\n\n```ts\n// toil.config.ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n server: { auth: true }, // ← mounts the full /auth/* API + sessions\n});\n```\n\nThat is the whole setup. The build appends the framework's auth controller to your server, so `/auth/*`\nis live and `@auth` works everywhere.\n\n## What you get\n\n| Endpoint | Purpose |\n| --- | --- |\n| `POST /auth/register/start`, `/register/finish` | Create an account (password never leaves the browser) |\n| `POST /auth/login/start`, `/login/finish` | Log in; sets a signed session cookie |\n| `GET /auth/me` *(`@auth`)* | The current user (`toilUserId` + `username`) |\n| `POST /auth/logout` *(`@auth`)* | Clear the session |\n\nPlus, in your own code:\n\n- **`@auth`** on any route (or a whole `@rest` class) → 401 unless there's a valid session.\n- **`AuthService.getUser()`** → the typed logged-in user.\n- **`AuthService.userId()`** → the stable [`ToilUserId`](./extending.md#toiluserid) (a 256-bit id you can\n key your data on).\n- **The client**: `import { Auth } from 'toiljs/client'`, then `Auth.register(username, password)` /\n `Auth.login(username, password)`. It does all the browser-side crypto and talks to `/auth/*` for you.\n\n## A login page in full\n\n```tsx\n// client/routes/login.tsx\nimport { useState } from 'react';\nimport { Auth } from 'toiljs/client';\n\nexport default function Login() {\n const [u, setU] = useState('');\n const [p, setP] = useState('');\n const [msg, setMsg] = useState('');\n\n const register = async () => {\n try { await Auth.register(u, p); setMsg('registered, now log in'); }\n catch (e) { setMsg(parseError(e)); }\n };\n const login = async () => {\n try { await Auth.login(u, p); setMsg('logged in'); } // sets the session cookie\n catch (e) { setMsg(parseError(e)); }\n };\n\n return (\n <main>\n <input value={u} onChange={(e) => setU(e.currentTarget.value)} placeholder=\"username\" />\n <input value={p} type=\"password\" onChange={(e) => setP(e.currentTarget.value)} placeholder=\"password\" />\n <button onClick={register}>Register</button>\n <button onClick={login}>Log in</button>\n <p>{msg}</p>\n </main>\n );\n}\n```\n\n```ts\n// server/routes/Secret.ts, a route only a logged-in user can reach\nimport { Response } from 'toiljs/server/runtime';\n\n@rest('secret')\nclass Secret {\n @auth // 401 without a valid session\n @get('/')\n public secret(): Response {\n const user = AuthService.getUser()!; // typed: { toilUserId, username }\n return Response.text('hello ' + user.username + '\\n');\n }\n}\n```\n\nThat's a real, production-grade auth system, the password is stretched with Argon2id in the browser into\nan ML-DSA-44 key pair, your server only ever stores a public key, and login is a mutually-authenticated\nML-KEM-768 challenge. You didn't write any of it.\n\n## Where to go next\n\n- **[How it works](./how-it-works.md)**: the protocol (OPRF + Argon2id + ML-DSA + ML-KEM), sessions,\n cookies, and the `ToilUserId`, with sequence diagrams.\n- **[Usage](./usage.md)**: enabling it, the client API, guarding routes, reading the user, the full wire\n contract of each endpoint.\n- **[Configuration](./configuration.md)**: the secrets a deployment MUST set, the audience/domain, tuning\n Argon2id, and the deploy checklist.\n- **[Extending & integrating](./extending.md)**: `ToilUserId`, keying your own data on a user, a custom\n user shape / opting out, and the `AuthService` primitive reference.\n- **[Customizing the auth emails](./emails.md)**: replace the verification, password-reset, and 2FA\n emails with your own branded React templates by dropping `emails/auth-*.tsx` files.\n\n> **One rule before you ship:** built-in auth runs with **insecure DEV fallback secrets** so it Just Works\n> locally. A deployment MUST set `AUTH_SESSION_SECRET`, `AUTH_OPRF_SEED`, and `AUTH_KEM_SK`. See\n> [Configuration](./configuration.md).\n",
7
+ "auth/README.md": "# Authentication\n\nToil ships a complete, **post-quantum password login** you turn on with one line. No passwords on your\nserver, no third-party identity provider, no hand-written crypto: enable it and you get a `/auth/*` API,\nsigned sessions, `@auth`-guarded routes, and a stable per-user identity.\n\n```ts\n// toil.config.ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n server: { auth: true }, // ← mounts the full /auth/* API + sessions\n});\n```\n\nThat is the whole setup. The build appends the framework's auth controller to your server, so `/auth/*`\nis live and `@auth` works everywhere.\n\n## Storage and request lifetime\n\nAuthentication uses collections in two databases: `AuthDb` for accounts and\nchallenges, and `AuthStateDb` for revoked sessions, user generations, and two-factor\nsettings. Verification is cached only on the current request object. Later requests\ncheck revocation state again; local revocation and signing-secret changes invalidate\nthe cached result. Temporary failures reading revocations, generations, or\ntwo-factor state abort the request; they never mean that a restriction is absent.\n\nThe `AuthStateDb` layout replaces the separate session and factor database\nnamespaces. It requires a fresh store or an explicit migration of existing security\nstate; it does not read the previous namespaces. The Nerva rollout uses a database\nreset alongside the application schema change.\n\n## What you get\n\n| Endpoint | Purpose |\n| --- | --- |\n| `POST /auth/register/start`, `/register/finish` | Create an account (password never leaves the browser) |\n| `POST /auth/login/start`, `/login/finish` | Log in; sets a signed session cookie |\n| `GET /auth/me` *(`@auth`)* | The current user (`toilUserId` + `username`) |\n| `POST /auth/logout` *(`@auth`)* | Clear the session |\n\nPlus, in your own code:\n\n- **`@auth`** on any route (or a whole `@rest` class) → 401 unless there's a valid session.\n- **`AuthService.getUser()`** → the typed logged-in user.\n- **`AuthService.userId()`** → the stable [`ToilUserId`](./extending.md#toiluserid) (a 256-bit id you can\n key your data on).\n- **The client**: `import { Auth } from 'toiljs/client'`, then `Auth.register(username, password)` /\n `Auth.login(username, password)`. It does all the browser-side crypto and talks to `/auth/*` for you.\n\n## A login page in full\n\n```tsx\n// client/routes/login.tsx\nimport { useState } from 'react';\nimport { Auth } from 'toiljs/client';\n\nexport default function Login() {\n const [u, setU] = useState('');\n const [p, setP] = useState('');\n const [msg, setMsg] = useState('');\n\n const register = async () => {\n try { await Auth.register(u, p); setMsg('registered, now log in'); }\n catch (e) { setMsg(parseError(e)); }\n };\n const login = async () => {\n try { await Auth.login(u, p); setMsg('logged in'); } // sets the session cookie\n catch (e) { setMsg(parseError(e)); }\n };\n\n return (\n <main>\n <input value={u} onChange={(e) => setU(e.currentTarget.value)} placeholder=\"username\" />\n <input value={p} type=\"password\" onChange={(e) => setP(e.currentTarget.value)} placeholder=\"password\" />\n <button onClick={register}>Register</button>\n <button onClick={login}>Log in</button>\n <p>{msg}</p>\n </main>\n );\n}\n```\n\n```ts\n// server/routes/Secret.ts, a route only a logged-in user can reach\nimport { Response } from 'toiljs/server/runtime';\n\n@rest('secret')\nclass Secret {\n @auth // 401 without a valid session\n @get('/')\n public secret(): Response {\n const user = AuthService.getUser()!; // typed: { toilUserId, username }\n return Response.text('hello ' + user.username + '\\n');\n }\n}\n```\n\nThat's a real, production-grade auth system, the password is stretched with Argon2id in the browser into\nan ML-DSA-44 key pair, your server only ever stores a public key, and login is a mutually-authenticated\nML-KEM-768 challenge. You didn't write any of it.\n\n## Where to go next\n\n- **[How it works](./how-it-works.md)**: the protocol (OPRF + Argon2id + ML-DSA + ML-KEM), sessions,\n cookies, and the `ToilUserId`, with sequence diagrams.\n- **[Usage](./usage.md)**: enabling it, the client API, guarding routes, reading the user, the full wire\n contract of each endpoint.\n- **[Configuration](./configuration.md)**: the secrets a deployment MUST set, the audience/domain, tuning\n Argon2id, and the deploy checklist.\n- **[Extending & integrating](./extending.md)**: `ToilUserId`, keying your own data on a user, a custom\n user shape / opting out, and the `AuthService` primitive reference.\n- **[Customizing the auth emails](./emails.md)**: replace the verification, password-reset, and 2FA\n emails with your own branded React templates by dropping `emails/auth-*.tsx` files.\n\n> **One rule before you ship:** built-in auth runs with **insecure DEV fallback secrets** so it Just Works\n> locally. A deployment MUST set `AUTH_SESSION_SECRET`, `AUTH_OPRF_SEED`, and `AUTH_KEM_SK`. See\n> [Configuration](./configuration.md).\n",
8
8
  "auth/usage.md": "# Using auth\n\n## 1. Enable it\n\nEither the config flag (canonical) or a one-line import, both do the same thing (the build appends the\nframework's auth controller + user shape to your server as compiled entries, so `/auth/*` self-mounts):\n\n```ts\n// toil.config.ts, canonical\nexport default defineConfig({ server: { auth: true } });\n```\n\n```ts\n// server/main.ts, escape hatch (equivalent)\nimport 'toiljs/server/auth';\n```\n\nThere is also an `AuthService.enable()` no-op you can call for discoverability, but enabling is done by the\nflag/import above (it happens at build time). Turn it off by removing the flag/import, with neither, no\n`/auth` routes and no accounts collection are generated. It is strictly opt-in.\n\n## 2. The client\n\n`toiljs/client` ships the browser half, it runs the OPRF blinding, Argon2id, ML-DSA keygen, and ML-KEM\nencapsulation, and talks to `/auth/*`. You never touch the crypto.\n\n```ts\nimport { Auth } from 'toiljs/client';\n\n// Create an account. Throws on a taken username or a transport error.\nawait Auth.register(username, password);\n\n// Log in. On success the browser holds the signed session cookie; subsequent\n// requests to @auth routes are authorized automatically.\nawait Auth.login(username, password);\n```\n\nOptions (second argument, `AuthOptions`):\n\n```ts\nawait Auth.login(username, password, {\n baseUrl: '/auth', // default; change if you mount elsewhere\n serverKemPublicKey: MY_KEM_PUB, // REQUIRED in production, pin your deployment's KEM key\n});\n```\n\n> **Production:** the client ships with the DEV KEM public key pinned. A real deployment MUST pass its own\n> `serverKemPublicKey` (derived from `AUTH_KEM_SK`). See [Configuration](./configuration.md).\n\n## 2b. Client-side: who's logged in, and protecting pages\n\n`register`/`login` set the session cookies; to *render* login state on the client, use the generated\n`getUser()` (emitted from the built-in `@user`). It reads the **readable** `__Secure-toil_user` companion\ncookie and returns the typed user or `null`, instant, no network call. It is **display-only**; the server\nstill enforces access via `@auth`, so never trust it for authorization.\n\n```tsx\nimport { getUser } from 'shared/server'; // generated; typed to the built-in @user\n\nfunction Nav() {\n const user = getUser(); // { toilUserId, username } | null\n return user\n ? <span>Hi {user.username} <button onClick={logout}>Log out</button></span>\n : <a href=\"/login\">Log in</a>;\n}\n\nasync function logout() {\n await fetch('/auth/logout', { method: 'POST' });\n location.href = '/login'; // cookies are cleared; bounce to login\n}\n```\n\nGate a whole client route by redirecting when there's no session:\n\n```tsx\nexport default function Dashboard() {\n const user = getUser();\n if (user == null) { location.href = '/login'; return null; } // not logged in -> to /login\n return <main>Welcome, {user.username}</main>;\n}\n```\n\nThe authoritative check is still the server: any data this page fetches should come from an `@auth` route,\nso a user who forged/deleted the readable cookie sees the redirect OR a `401`, never real data.\n\n## 2c. Handling errors\n\n`Auth.register` / `Auth.login` reject with a message you can show. The important distinguishable cases:\n\n```ts\ntry {\n await Auth.register(username, password);\n} catch (e) {\n const m = String(e);\n if (m.includes('already registered')) setError('That username is taken, log in instead.');\n else setError('Could not register, try again.');\n}\n\ntry {\n await Auth.login(username, password);\n} catch {\n // Wrong password OR unknown user both fail generically (anti-enumeration), one message.\n setError('Incorrect username or password.');\n}\n```\n\n- **Username taken** → `register` throws `auth: username already registered (log in instead)` (a\n distinguishable case so you can guide the user).\n- **Wrong password / unknown user** → `login` throws generically (`auth: request failed`): by design,\n the two are indistinguishable, so use ONE \"incorrect username or password\" message.\n- **Rate limited** → after 5 attempts / 60s a `429` surfaces as the same generic throw; back off and tell\n the user to wait.\n\n## 3. Guard your routes: `@auth`\n\nPut `@auth` on a route method or a whole `@rest` class. The generated dispatcher checks for a valid signed\nsession **before** your handler runs and returns `401` otherwise. `@auth` is unchanged by built-in auth,\nit's the same decorator you'd use with hand-written auth.\n\n```ts\n@rest('account')\nclass AccountApi {\n @auth // this route needs a session\n @get('/settings')\n public settings(): Response { /* … */ }\n\n @get('/public') // this one is open\n public open(): Response { /* … */ }\n}\n\n@auth // …or guard the ENTIRE class\n@rest('admin')\nclass AdminApi { /* every route requires a session */ }\n```\n\n## 4. Read the current user\n\nInside any handler:\n\n```ts\n// The typed logged-in user (the built-in `@user`: toilUserId + username), or null.\nconst user = AuthService.getUser();\nif (user != null) {\n user.username; // string\n user.toilUserId; // Uint8Array(32), the stable id bytes\n}\n\n// The stable identity as a ToilUserId (gate on hasSession() first, see note).\nif (AuthService.hasSession()) {\n const id = AuthService.userId()!; // ToilUserId\n // key your own data on id (see Extending)\n}\n```\n\n> `ToilUserId` overloads `==`, so `AuthService.userId() == null` does not type-check. Gate with\n> `AuthService.hasSession()` and then use `userId()!`, or use `getUser()` and null-check that.\n\n## 5. The endpoint wire contract\n\nYou normally use the `Auth` client, but the raw endpoints are binary (`DataWriter`/`DataReader`, never\nJSON):\n\n| Route | Request body | Response |\n| --- | --- | --- |\n| `POST /auth/register/start` | `str(username) bytes(blinded)` | `u8(0) u32(mem) u32(iters) u32(par) bytes(salt) bytes(evaluated)` |\n| `POST /auth/register/finish` | `str(username) bytes(pubkey) bytes(proof)` | `u8(status)`, `0` ok, `1` username taken |\n| `POST /auth/login/start` | `str(username) bytes(blinded)` | `bytes(cid) str(aud) u32(mem) u32(iters) u32(par) bytes(salt) bytes(nonce) u64(iat) u64(exp) bytes(evaluated)` |\n| `POST /auth/login/finish` | `bytes(cid) bytes(ct) bytes(sig)` | `u8(0) bytes(sessionToken) bytes(serverConfirm)` + `Set-Cookie`, or `u8(≠0)` on failure |\n| `GET /auth/me` *(`@auth`)* | (none) | `bytes(toilUserId) str(username)` |\n| `POST /auth/logout` *(`@auth`)* | (none) | `200` + cookie-clearing `Set-Cookie` |\n\nRate limiting: every register/login POST carries `@ratelimit(SlidingWindow, 5, 60)` (5 requests / 60s per\nclient) out of the box, so brute-force is throttled before it reaches the crypto.\n\n## 6. Under `toiljs dev`\n\nEverything runs locally with **zero setup**: the dev server emulates the ToilDB account/challenge storage\nand the ML-DSA/ML-KEM/OPRF host functions in process, and the auth secrets fall back to insecure DEV\nvalues. Register and login span requests (the accounts persist for the dev session). You'll see a warning\nthat `AUTH_SESSION_SECRET` is unset, that's expected in dev; set it before you deploy (see\n[Configuration](./configuration.md)).\n\n## 7. `Server.REST.auth.*`\n\nBecause the controller is a normal `@rest('auth')` class, a typed `Server.REST.auth.*` fetch client is\ngenerated for free (`me`, `logout`, and the register/login methods). Use it for `/me` and `/logout`; but\n**drive register/login through `toiljs/client` `Auth`**, not the generated client, only the `Auth` helper\nruns the required browser-side OPRF/Argon2id/ML-DSA/ML-KEM crypto.\n",
9
9
  "backend/data.md": "# Data types (`@data`)\n\n`@data` turns a plain class into a typed value that can travel safely between your frontend and your WASM backend, and in and out of the database, with both a binary and a JSON codec generated for you.\n\n## What `@data` is\n\nA **codec** is a pair of functions: one that turns a value into bytes (encode), and one that turns those bytes back into a value (decode). Any time data crosses a boundary (browser to server, server to database, one WASM call to another) it has to become bytes and back. Writing that by hand is tedious and easy to get wrong.\n\n`@data` writes it for you. You declare a class with typed fields, tag it `@data`, and the compiler synthesizes a deterministic binary codec and a JSON codec on the class. The exact same class is also generated into your `shared/server.ts`, so the browser and the WASM backend agree on the format down to the byte.\n\n```ts\n@data\nclass Player {\n username: string = '';\n admin: bool = false;\n score: u64 = 0;\n}\n```\n\nThat is a complete, serializable type. Note every field has a **default value**; that is required (the generated decoder and the client constructor use it).\n\n## Why and when\n\n`@data` is the backbone of almost everything that moves in a toiljs app:\n\n- **Request and response bodies** for [`@rest`](./rest.md) routes.\n- **Arguments and return values** for [`@service` / `@remote`](./rpc.md) calls.\n- **Values stored in [ToilDB](../database/README.md)**.\n- **Sessions**, [stream](../realtime/README.md) messages, and any custom payload you design.\n\nWhenever a route, an RPC method, or the database needs a structured value, that value is a `@data` class. You will define these constantly.\n\n```mermaid\nflowchart LR\n D[\"@data class<br/>(one definition)\"] --> R[\"REST route body\"]\n D --> P[\"RPC argument / result\"]\n D --> B[(\"ToilDB value\")]\n D --> S[\"Stream message\"]\n```\n\n## What the compiler generates\n\nFrom your `@data` class, the compiler adds these members:\n\n| Member | What it does |\n| --- | --- |\n| `encode(): Uint8Array` | Serialize to bytes, with a 4-byte type-id prefix. |\n| `static decode(buf): T` | Rebuild a value from bytes produced by `encode()`. |\n| `encodeInto(w: DataWriter)` | Serialize without the type-id frame, for nesting inside another value. |\n| `static decodeFrom(r: DataReader): T` | The matching read, for nested values. |\n| `toJSON()` / `static fromJSON(v)` | The JSON codec (large integers become decimal strings, so `JSON.parse` keeps them exact). |\n| `static dataId(): u32` | A stable hash (FNV-1a) of the class name, written as the type-id prefix. |\n\nYou mostly do not call these directly; routes, RPC, and the database call them for you. But they are there when you need them (for example `value.toJSON().toString()` to build a `Response.json`).\n\n## Supported field types\n\nA `@data` field may be:\n\n- a scalar: `u8` through `u256`, `i8` through `i256`, `f32`, `f64`, `bool`;\n- a `string`;\n- a `Uint8Array` (a raw byte buffer);\n- a nested `@data` class;\n- an array `T[]` of any of the above.\n\nFor the number types (why `u64` is a `bigint` on the client, when to reach for `u256`, and so on), see [Types](../concepts/types.md).\n\nGive every field a default. The layout is exactly the field declaration order (this matters, see [gotchas](#gotchas)).\n\n## Nested `@data`, arrays, and bytes\n\n`@data` classes compose. A field can be another `@data` class, an array, or a byte buffer, and it all encodes and decodes as one value:\n\n```ts\n@data\nclass Tag {\n label: string = '';\n weight: f32 = 0;\n}\n\n@data\nclass Document {\n id: u64 = 0;\n title: string = '';\n tags: Tag[] = []; // array of nested @data\n authors: string[] = []; // array of strings\n thumbnail: Uint8Array = new Uint8Array(0); // raw bytes\n}\n```\n\n`Document.encode()` walks the whole tree: it writes `id`, `title`, each `Tag` (via `encodeInto`), each author string, and the raw `thumbnail` bytes, in field order. `Document.decode(bytes)` reads them back in the same order. On the JSON side, a `Uint8Array` field becomes a JSON array of byte numbers, and nested `@data` fields become nested JSON objects.\n\n## Using `@data` in a route\n\nA route's body parameter and its return value are `@data` values. Which codec runs depends on the route's stream mode (see [REST bodies](./rest.md#request-and-response-bodies)):\n\n```ts\n// JSON route (the default): body from JSON.parse, result via toJSON()\n@post('/')\npublic create(input: NewPlayer): Player { /* ... */ }\n\n// Binary route: body via decode(), result via encode()\n@route({ method: Methods.POST, path: '/blob', stream: DataStream.Binary })\npublic blob(input: FileData): FileResult { /* ... */ }\n```\n\n- In a **JSON** route, the incoming body is `JSON.parse`d and revived with the type's `fromJSON`, and the returned value is serialized with `toJSON()`.\n- In a **Binary** route, the incoming body is `decode`d and the returned value is `encode`d.\n\nYou do not call the codec yourself in either case; you just declare the types.\n\n## The raw codec: `DataWriter` and `DataReader`\n\nSometimes you want to lay out bytes by hand: a custom body, a session token, a challenge message, a wire format someone else defined. For that, use the codec directly. It lives in the `data` module:\n\n```ts\nimport { DataWriter, DataReader } from 'data';\n```\n\nThis is the same codec `@data` classes are built from, and it has a byte-for-byte identical TypeScript version in `toiljs/io` (`src/io/codec.ts`), so your browser code can read and write the exact same bytes the WASM backend does.\n\n### `DataWriter`\n\nEvery write method returns the writer, so calls chain.\n\n| Method | Signature | Wire format |\n| --- | --- | --- |\n| `writeU8` / `writeI8` | `(v): DataWriter` | 1 byte |\n| `writeU16` / `writeI16` | `(v): DataWriter` | 2 bytes, little-endian |\n| `writeU32` / `writeI32` | `(v): DataWriter` | 4 bytes, LE |\n| `writeU64` / `writeI64` | `(v): DataWriter` | 8 bytes, LE |\n| `writeF32` / `writeF64` | `(v): DataWriter` | 4 / 8 bytes, IEEE-754 LE |\n| `writeBool` | `(v): DataWriter` | 1 byte (`1` / `0`) |\n| `writeBytes` | `(b: Uint8Array): DataWriter` | `u32` length (LE) + the raw bytes |\n| `writeString` | `(s: string): DataWriter` | `u32` length (LE) + UTF-8 bytes |\n| `writeU128` / `writeI128` | `(v): DataWriter` | two `u64` limbs (lo, hi) |\n| `writeU256` / `writeI256` | `(v): DataWriter` | four `u64` limbs |\n| `length` | `(): i32` | bytes written so far |\n| `toBytes` | `(): Uint8Array` | an exact-length copy of the buffer |\n\n### `DataReader`\n\nReads are **bounds-safe**: reading past the end of the buffer never crashes. Instead the read returns a zero or empty default and flips the reader's public `ok` flag to `false`. Check `ok` after a sequence of reads to catch a truncated or malformed buffer.\n\n| Method | Signature | On over-read |\n| --- | --- | --- |\n| `readU8` / `readI8` | `(): integer` | `0` |\n| `readU16`..`readU64`, `readI16`..`readI64` | `(): integer` | `0` |\n| `readF32` / `readF64` | `(): float` | `0` |\n| `readBool` | `(): bool` | `false` |\n| `readBytes` | `(): Uint8Array` | empty array |\n| `readString` | `(): string` | `\"\"` |\n| `readU128` / `readI128` / `readU256` / `readI256` | `(): bignum` | `0` |\n| `remaining` | `(): i32` | bytes left unread |\n| `ok` | `bool` (field) | `false` once any read over-ran |\n\n### Encode and decode, both directions\n\n```ts\nimport { DataWriter, DataReader } from 'data';\n\n// Encode: a version byte, a name, a score, then a blob.\nconst out = new DataWriter()\n .writeU8(1)\n .writeString('alice')\n .writeU64(1234)\n .writeBytes(payload)\n .toBytes();\n\n// Decode: read the fields back in the exact same order.\nconst r = new DataReader(out);\nconst version = r.readU8();\nconst name = r.readString();\nconst score = r.readU64();\nconst blob = r.readBytes();\nif (!r.ok) return Response.badRequest('truncated');\n```\n\nThe order of reads must match the order of writes exactly. The layout is the format.\n\n## JSON vs binary: which to use\n\nYou usually pick this at the route level (see [REST bodies](./rest.md#request-and-response-bodies)), but the trade-off is the same everywhere:\n\n- **JSON** is human-readable and understood by every tool. Pick it for endpoints a browser or a third party calls directly. Large integers ride as decimal strings so they stay exact.\n- **Binary** is smaller, faster, and lossless for big numbers. Pick it for app-to-app traffic and anything performance sensitive. RPC always uses binary under the hood.\n\n## Evolving a `@data` format: `@migrate`\n\nOnce records are stored, changing a `@data` **value** type is a problem: as the [gotchas](#gotchas) explain, the binary layout *is* the field order, so adding or changing a field means the bytes already on disk no longer match the new shape and would fail to decode. `@migrate` is how you evolve a stored type without a backfill and without downtime.\n\n**What it is.** `@migrate` marks a plain (free) function that upgrades one record written under an **old** version of a value type into the **current** shape. The compiler weaves that function into the value type's decoder as a version dispatch: every stored record carries the schema version it was written under, so when a read hits an old record, the decoder decodes it as its old shape and runs your `@migrate` forward to today's shape. This happens **lazily**, per row, only when a record is actually read (nothing rewrites your whole collection up front).\n\n**When to use it.** Any time you change a `@data` type that is already stored in ToilDB: you added a field, renamed one, changed a type. New writes use the new layout; old rows keep working because the migration upgrades them on the way out.\n\nHere is the whole story. You shipped this value type:\n\n```ts\n// server/models/GuestEntry.ts (version 1: what you shipped first)\n@data\nexport class GuestEntry {\n author: string = '';\n message: string = '';\n}\n```\n\nLater you add an `at` timestamp:\n\n```ts\n// server/models/GuestEntry.ts (version 2: gained an `at` field)\n@data\nexport class GuestEntry {\n author: string = '';\n message: string = '';\n at: u64 = 0; // NEW field\n}\n```\n\nEvery entry already stored was written without `at`, so its bytes cannot decode as the new `GuestEntry`. You add a migration file to bridge them:\n\n```ts\n// server/migrations/GuestEntry.migration.ts\nimport { GuestEntry } from '../models/GuestEntry';\n\n// Keep the ORIGINAL layout as its own class so old rows still decode.\n// One kept class per past version.\n@data\nexport class GuestEntryV1 {\n author: string = '';\n message: string = '';\n}\n\n// Upgrade a v1 row to the current GuestEntry. This is the DELTA form:\n// (old, into). The compiler pre-copies the fields the two layouts share\n// (author, message), so your body fills only what is new.\n@migrate\nexport function up(old: GuestEntryV1, into: GuestEntry): void {\n into.at = 0; // unknown for entries written before the timestamp existed\n}\n```\n\nThat is it. Old entries now surface as fully-formed `GuestEntry` values with `at = 0`; new entries carry a real timestamp; the same read path serves both.\n\nA `@migrate` function comes in two shapes, and you pick whichever reads better:\n\n- **Delta form** `up(old: OldType, into: NewType): void` (used above). The compiler copies every field the two layouts share by name and type, and your body fills only the changed or new fields. Least to write when most fields carry over.\n- **Full form** `up(old: OldType): NewType`. Your body builds and returns the whole new value itself. Use it when the transform is not a simple field-for-field copy.\n\n```ts\n// The same migration written in the full form.\n@migrate\nexport function up(old: GuestEntryV1): GuestEntry {\n const e = new GuestEntry();\n e.author = old.author;\n e.message = old.message;\n e.at = 0;\n return e;\n}\n```\n\nGotchas specific to `@migrate`:\n\n- **Location is enforced.** Every `@migrate` must live in a `migrations/<Type>.migration.ts` file (a `*.migration.ts` file under a `migrations/` folder). The build auto-discovers it (nothing imports it), and a `@migrate` placed anywhere else is a hard compile error, because it would silently never run.\n- **A migration is a pure value transform.** It may not touch the database or any host service; it only turns old fields into new ones. Trying to read or write ToilDB from inside a `@migrate` is a compile error.\n- **Migrations chain.** If you evolve a type more than once, keep one old class and one `@migrate` per step (`V0 -> V1`, `V1 -> V2`). The compiler walks the chain, so a row written under the oldest layout is carried all the way forward to the current shape, shortest path first.\n- **When it actually runs.** On any read that decodes an old row (`get`, `getMany`, a view read, events `latest`). It also runs when a [`@derive`](../background/derive.md) rebuilds a view on box load and re-reads old stored events (see [Views](../database/views.md)).\n\n## Dynamic JSON on the server: the `JSON` value tree\n\n`@data` is for shapes you know ahead of time. Sometimes you do not: a webhook whose body varies, a third-party payload with optional fields, or a response you assemble on the fly. For those, toiljs gives you a `JSON` **value tree**: an in-memory value that can be a null, a bool, a number, a string, an array, or an object, which you read and build at runtime.\n\n**What it is.** `JSON` is an ambient global class (no import needed). It represents one dynamic JSON value. `JSON.parse(text)` turns JSON text into one of these trees, and a `@data` class's `toJSON()` actually returns one too. It is the **untyped** counterpart to `@data`'s typed, fixed-shape codec: reach for `@data` when the shape is known and you want type safety, and for `JSON` when the shape is dynamic.\n\nThe statics that make a value:\n\n| Static | What it does |\n| --- | --- |\n| `JSON.parse(text: string): JSON` | Parse JSON text into a value tree (returns an error value on malformed input). |\n| `JSON.obj(): JSON` | A new empty object; fill it with `.set(key, value)`. |\n| `JSON.arr(): JSON` | A new empty array; fill it with `.push(value)`. |\n| `JSON.of<T>(value: T): JSON` | Wrap a scalar, string, bool, or array as a JSON value. |\n| `JSON.nul(): JSON` | A JSON null. |\n| `JSON.stringify<T>(value: T): string` | Serialize a scalar / string / bool / array value straight to a JSON string. |\n\nThe instance methods that read and build a value:\n\n| Method | What it does |\n| --- | --- |\n| `.isObject()` / `.isArray()` / `.isString()` / `.isNumber()` / `.isBool()` / `.isNull(): bool` | Test the value's type before you read it. |\n| `.has(key: string): bool` | Whether an object has `key`. |\n| `.get(key: string): JSON` | The value for `key` on an object. |\n| `.objectKeys(): Array<string>` | The keys of an object. |\n| `.at(index: i32): JSON` | The element at `index` of an array. |\n| `.length(): i32` | The element count of an array (0 otherwise). |\n| `.asString(): string` | Read the value as a string. |\n| `.asF64(): f64` / `.asI64(): i64` / `.asU64(): u64` | Read the value as a number. |\n| `.asBool(): bool` | Read the value as a bool. |\n| `.set(key: string, value: JSON): JSON` | Set a key on an object; returns `this`, so calls chain. |\n| `.push(value: JSON): JSON` | Append to an array; returns `this`, so calls chain. |\n| `.toString(): string` | Serialize this tree back to a JSON string. |\n\nReading an untyped body and building a reply, together:\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('echo')\nclass Echo {\n // POST /echo with an arbitrary JSON body we do not model as a @data class.\n @post('/')\n public handle(ctx: RouteContext): Response {\n const body = JSON.parse(ctx.text()); // ctx.text() is the raw body as text\n if (!body.isObject() || !body.has('name')) {\n return Response.badRequest('expected an object with a \"name\" field');\n }\n\n // Read fields out of the tree, guarding types and optional keys.\n const name = body.get('name').asString();\n const age = body.has('age') ? body.get('age').asI64() : 0;\n\n // Build a fresh JSON object to send back (chaining .set).\n const out = JSON.obj()\n .set('greeting', JSON.of<string>('hello, ' + name))\n .set('age', JSON.of<i64>(age));\n return Response.json(out.toString());\n }\n}\n```\n\nGotchas specific to `JSON`:\n\n- **Check the type before you read.** `JSON.parse` never throws; a malformed input or a wrong-typed field yields an error or default value rather than a crash. Use `.isObject()` / `.has(...)` / `.isString()` and friends to validate untrusted input before you trust it.\n- **It is a value tree, not a typed struct.** You get no compile-time field checking. When a shape is stable, prefer a `@data` class so the compiler catches typos and the client gets a typed type for free.\n- **Big-integer care still applies.** As with any JSON, integers above 2^53 are best carried as strings; read them with `.asString()` when exactness matters.\n\n## Gotchas\n\n- **Field order is the format.** The binary layout is exactly your field declaration order. Reordering fields, or changing a field's type, is a breaking change: old bytes will decode wrong. To evolve a format safely, add new fields at the **end** (and, for hand-rolled payloads, bump a leading version byte).\n- **Every field needs a default.** The generated decoder and the client constructor rely on it. A field with no default will not compile as `@data`.\n- **`encode()` carries a type id; `encodeInto` does not.** The 4-byte `dataId()` prefix lets a decoder confirm it is reading the type it expected. When nesting one `@data` inside another, `encodeInto` / `decodeFrom` skip that frame (the outer type already identifies the whole value).\n- **Endianness.** The WASM codec is little-endian. The `toiljs/io` codec defaults to little-endian too, and also accepts a per-call big-endian flag for network formats. Keep both ends on the same setting.\n- **Plain JSON numbers lose precision above 2^53.** That is why `@data` sends 64-bit-and-larger integers as decimal strings over JSON. If you hand-build JSON, do the same, or use the binary codec.\n- **`DataReader` never throws; check `ok`.** An over-read returns a default and sets `ok = false`. Always check `ok` after decoding untrusted bytes.\n\n## Related\n\n- [Types](../concepts/types.md): `u64`, `u256`, `f64`, and how each maps to `number` or `bigint`.\n- [HTTP routes (`@rest`)](./rest.md): where `@data` bodies and return values are used, and the JSON vs binary route modes.\n- [Typed RPC](./rpc.md): `@data` as RPC arguments and results, and the generated client classes.\n- [The database](../database/README.md): storing `@data` values in ToilDB.\n- [Backend overview](./README.md): where `@data` fits in the request lifecycle.\n",
10
10
  "backend/README.md": "# Backend overview\n\nYour backend is TypeScript that toilscript compiles into a small, sandboxed WebAssembly program, which runs on the Dacely edge and answers every request.\n\n## What the backend is\n\nIn a toiljs project, everything under `server/` is your backend. You write it in TypeScript, the same language as your frontend. But it does not run in a browser and it does not run in Node. Instead, a compiler called **toilscript** turns it into **WebAssembly** (often shortened to **WASM**): a compact, fast, portable binary format that many kinds of servers can run safely.\n\nThat compiled program is then deployed to the **Dacely edge**. Two pieces of jargon to unpack there:\n\n- **The edge** means a fleet of servers spread across many cities around the world. When a user makes a request, it is served by the edge node physically closest to them. Close means fast: less distance for the data to travel, so lower latency. You do not pick a region or manage servers; your one compiled backend runs everywhere at once.\n- **Sandboxed** means your WASM program runs inside a locked box. It cannot open files, reach the operating system, or make raw network connections on its own. The only way it can touch the outside world is through a small, fixed set of **host functions** that toiljs provides (read the request, build a response, query the database, send an email, and so on). Every one of those calls is metered and bounded, so a buggy or hostile backend cannot crash the node or read another app's data. This is what makes it safe to run thousands of different apps on the same shared edge.\n\nYou never call the WASM boundary by hand. You write normal TypeScript classes and functions, tag them with decorators (like `@rest` or `@service`), and the compiler wires everything up.\n\n> New to decorators? A **decorator** is the `@name` you write just above a class or method. It attaches meaning to that code without changing what the code does line by line. toiljs uses decorators to say \"this class is an HTTP controller\" or \"this method is callable from the browser.\" See [Decorators](../concepts/decorators.md).\n\n## The request lifecycle\n\nHere is what happens, end to end, when a browser talks to your backend.\n\n```mermaid\nflowchart TD\n U[\"User's browser\"] -->|\"HTTP request\"| E[\"Nearest Dacely edge node\"]\n E --> S{\"Static file<br/>for this path?\"}\n S -->|\"Yes (GET/HEAD)\"| F[\"Serve the file<br/>(HTML, JS, images)\"]\n S -->|\"No\"| W[\"Your compiled backend<br/>(server.wasm handle)\"]\n W --> H[\"Your handler runs<br/>and returns a Response\"]\n H --> E\n F --> E\n E -->|\"HTTP response\"| U\n```\n\nStep by step:\n\n1. The request lands on the closest edge node.\n2. The edge first checks whether the path is a **static file** it can serve directly (your built frontend: HTML pages, JavaScript bundles, images). If so, it serves the file and never wakes your code. This is fast and free.\n3. Otherwise the edge hands the request to your compiled backend by calling its single WASM export, `handle`.\n4. Inside, toiljs decodes the raw bytes into a friendly [`Request`](./rest.md#the-request-object) object and calls your handler's `handle(req)` method.\n5. Your handler returns a [`Response`](./rest.md#building-a-response). toiljs encodes it back into bytes.\n6. The edge sends that response to the browser.\n\nThe key mental model: your backend is a pure function of the request. Bytes in, bytes out, one request at a time.\n\n## Stateless by default\n\nA **fresh copy** of your handler serves each request. Any fields you set on a controller do not survive to the next request, and the request might even be served by a different edge node on the other side of the world. This is called being **stateless**.\n\nThat is a feature, not a limitation: it is what lets your backend scale to the whole planet with no coordination. When you need data to persist between requests (a user account, a counter, a list of posts), you store it in the built-in global database, **ToilDB**. See [the database section](../database/README.md).\n\n```mermaid\nflowchart LR\n R1[\"Request 1\"] --> B1[\"Handler copy A<br/>(fields reset)\"]\n R2[\"Request 2\"] --> B2[\"Handler copy B<br/>(fields reset)\"]\n B1 --> DB[(\"ToilDB<br/>(shared, persistent)\")]\n B2 --> DB\n```\n\n## The three surfaces\n\nYour backend can expose three different kinds of endpoint. Each is opted into with a decorator, and each has its own page:\n\n| Surface | Decorator | What it is | When to use it |\n| --- | --- | --- | --- |\n| **HTTP REST** | `@rest` + `@get`/`@post`/... | Plain HTTP routes with paths, methods, and status codes. | A public API, webhooks, anything a browser, `curl`, or a third party calls directly. See [REST](./rest.md). |\n| **Typed RPC** | `@service` / `@remote` | Server functions your own frontend calls like local async functions, fully type-checked end to end. | Talking from your own React app to your own backend. See [RPC](./rpc.md). |\n| **Realtime** | `@stream` | A long-lived connection where the server keeps state per connected client. | Chat, live cursors, notifications, anything push-based. See [Realtime](../realtime/README.md). |\n\nREST and RPC are the everyday tools. Most apps use both: REST for anything the outside world calls, RPC for your own frontend. They are not exclusive; you can use all three in one project.\n\nAll of these share the same building block for their data: **`@data` classes**, which are typed structs that travel safely between the browser and your WASM backend. See [Data types](./data.md).\n\n## Where your handler lives\n\nREST, RPC, and streams self-register: you tag a class and the compiler adds it to the right dispatcher. A tiny amount of glue lives in `server/main.ts`, which imports your route files and names your top-level handler class. In a typical project you rarely touch `main.ts`; you add route and service files and they are discovered automatically.\n\nYour handler class extends `ToilHandler` and overrides `handle`. The common pattern is to try the REST dispatcher first, then fall back to your own logic:\n\n```ts\nimport { Method, Request, Response, Rest, ToilHandler } from 'toiljs/server/runtime';\n\nexport class AppHandler extends ToilHandler {\n public handle(req: Request): Response {\n // Try every @rest controller. Returns the first match, or null.\n const hit = Rest.dispatch(req);\n if (hit != null) return hit;\n\n // Your own hand-written endpoints can go here.\n if (req.path == '/health') return Response.text('ok\\n');\n\n // \"I have no answer for this path\": let the edge serve it (a static\n // file, the client app) instead of returning a hard 404.\n return Response.unhandled();\n }\n}\n```\n\nRPC calls (to the reserved path `/__toil_rpc`) are handled by the framework before your `handle` runs, so you do not dispatch them yourself.\n\n### Per-request hooks: `onRequestStarted` and `onRequestCompleted`\n\n`ToilHandler` gives you two optional hooks that run around every request, so you can add cross-cutting logic (logging, timing, metrics) once instead of repeating it in every route, and without re-implementing `handle`:\n\n- **`onRequestStarted(req: Request): void`** runs just **before** `handle(req)`. Override it for per-request setup: start a timer, log the incoming method and path, read a header you need everywhere.\n- **`onRequestCompleted(req: Request, resp: Response): void`** runs just **after** `handle` returns, and it is handed the `Response` that is about to go out. It **also runs when your handler throws**, after the runtime has turned that throw into a `500`, so it is the right place for teardown you always want to happen (record the outcome, stop the timer, emit a metric).\n\nBoth are empty by default, so override only the one you need. The framework calls them around every request; you never call them yourself.\n\n```ts\nimport { Request, Response, Rest, ToilHandler } from 'toiljs/server/runtime';\n\nexport class AppHandler extends ToilHandler {\n public handle(req: Request): Response {\n const hit = Rest.dispatch(req);\n return hit != null ? hit : Response.unhandled();\n }\n\n // Runs before every handle().\n public onRequestStarted(req: Request): void {\n // per-request setup, e.g. note the path you are about to serve\n }\n\n // Runs after every handle(), including after a throw became a 500.\n public onRequestCompleted(req: Request, resp: Response): void {\n // always-run teardown: record the outcome, stop a timer, emit a metric\n }\n}\n```\n\nIf your project is REST-only, you do not even need a custom handler; toiljs ships a ready-made one. See [REST](./rest.md#dispatch-and-the-404-fallback).\n\n## Compute tiers\n\nThe request/response backend described here is the default and most common tier, called **L1**. toiljs also has higher tiers for long-lived connections (streams) and scheduled background work (daemons), each compiled into its own WASM artifact from the same project. You opt into a tier just by adding its entry file and surface decorator. For the full picture, see [Compute tiers](../concepts/tiers.md).\n\n## Related\n\n- [HTTP routes (`@rest`)](./rest.md): paths, methods, params, and responses.\n- [Typed RPC (`@service`/`@remote`)](./rpc.md): calling the server from your frontend with end-to-end types.\n- [Data types (`@data`)](./data.md): the serializable structs everything uses.\n- [The database (ToilDB)](../database/README.md): where persistent state lives.\n- [Compute tiers](../concepts/tiers.md): L1 request, L2/L3 stream, L4 daemon.\n- [Realtime streams](../realtime/README.md): the `@stream` surface.\n",
11
- "backend/rest.md": "# HTTP routes (`@rest`)\n\nExpose a real HTTP API by tagging a class `@rest` and its methods with an HTTP verb; toiljs generates the router for you.\n\n## Why and when\n\nUse `@rest` when something outside your own frontend needs to call your backend over plain HTTP: a webhook from another service, a public API, a mobile app, a `curl` script, or a browser hitting a URL directly. REST endpoints are ordinary URLs with ordinary HTTP methods and status codes, so any HTTP client understands them.\n\nIf instead you are calling your own backend from your own React frontend, [typed RPC](./rpc.md) is usually nicer (you get end-to-end types and skip URL wrangling). The two live happily side by side.\n\n## The shape of a route\n\nA REST route is a method on a class. The class decorator says where the class is mounted; the method decorator says which HTTP method and path it answers.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('players') // this controller is mounted at /players\nclass Players {\n @get('/:id') // answers GET /players/:id\n public get(ctx: RouteContext): Response {\n const id = ctx.param('id');\n return Response.json(`{\"id\":\"${id}\"}`);\n }\n}\n```\n\nSave that file, import it once (see [How routes are discovered](#how-routes-are-discovered)), and `GET /players/42` returns `{\"id\":\"42\"}`. That is a working endpoint.\n\n## `@rest`: mounting a controller\n\n`@rest` marks a class as a route **controller** (a group of related routes) and mounts it at a URL prefix.\n\n```ts\n@rest('api') // mounted at /api\n@rest('/') // or @rest('') : mounted at the root\n@rest({ stream: DataStream.Binary }) // root mount, binary body codec by default\n```\n\n- The string is the mount prefix. `\"api\"`, `\"/api\"`, and `\"api/\"` all normalize to `/api`. `\"\"` and `\"/\"` mean the root.\n- The object form sets class-wide defaults. `stream: DataStream.Binary` makes every route in the class use the binary body codec instead of JSON (more on that in [Request and response bodies](#request-and-response-bodies)).\n\nThe full URL of a route is the controller prefix joined with the method path. With `@rest('api')` and `@get('/todos/:id')`, the route is `GET /api/todos/:id`.\n\n## Verb decorators\n\nEach HTTP method has a decorator that takes the route path:\n\n```ts\n@get('/path') @post('/path') @put('/path') @patch('/path')\n@del('/path') @head('/path') @options('/path')\n```\n\nNote the delete decorator is `@del`, not `@delete` (the word `delete` is reserved in TypeScript, so it cannot be a decorator name).\n\n### `@route`: the explicit form\n\n`@route` is the general form. Reach for it when you want to set the body codec per route, or you just prefer an object:\n\n```ts\n@route({ method: Methods.POST, path: '/upload', stream: DataStream.Binary })\npublic upload(body: FileData): FileResult { /* ... */ }\n```\n\n`method` (from the `Methods` enum) and `path` are required; `stream` is optional and overrides the controller default. `Methods` and `DataStream` are global enums (like the decorators themselves, they need no import).\n\n## Path parameters\n\nA path segment written as `:name` is a **path parameter**: it captures whatever the request has in that position. Read it with `ctx.param('name')`.\n\n```ts\n@get('/todos/:id/items/:itemId')\npublic getItem(ctx: RouteContext): Response {\n const id = ctx.param('id'); // \"42\" for /todos/42/items/9\n const itemId = ctx.param('itemId'); // \"9\"\n return Response.json(`{\"todo\":\"${id}\",\"item\":\"${itemId}\"}`);\n}\n```\n\nMatching is **segment-exact**: the request path must have the same number of `/`-separated segments, the static segments must match literally, and each `:param` captures one segment. The query string is removed before matching. A captured param is always a string; convert it yourself if you need a number (for example `u64.parse(ctx.param('id'))`).\n\n## Method parameters: reading the request\n\nA route method may declare zero, one, or two parameters. toiljs looks at their types to decide what to pass:\n\n- a parameter typed `RouteContext` receives the [request context](#the-routecontext-object) (path params, query, headers, raw body);\n- any other type is treated as the **request body**, decoded into that [`@data`](./data.md) type.\n\n```ts\n@get('/status')\npublic status(): StatusResponse { /* no body, no context */ }\n\n@get('/user/:id')\npublic getUser(ctx: RouteContext): User { /* context only */ }\n\n@post('/create')\npublic create(input: NewTodo): Todo { /* body only */ }\n\n@post('/user/:id/score')\npublic addScore(input: ScoreDelta, ctx: RouteContext): Player {\n const id = ctx.param('id'); // body AND context\n /* ... */\n}\n```\n\nThe order of the two parameters does not matter; toiljs classifies them by type.\n\n## Return types\n\nYou return one of two things from a route, and toiljs encodes it for you:\n\n| You return | toiljs sends |\n| --- | --- |\n| a [`@data`](./data.md) value | The value serialized (JSON by default, binary if the route is in binary mode). Status `200`. |\n| a `Response` | Exactly that response: your status, headers, cookies, and body, untouched. |\n| `void` (nothing) | `204 No Content`. |\n\nReturning a `@data` value is the short path when you just want to send the data. Returning a `Response` gives you full control: a custom status, extra headers, cookies, caching. Use whichever fits.\n\n```ts\n// Short path: return the data, let toiljs serialize it as JSON.\n@get('/me')\npublic me(): Player { return currentPlayer(); }\n\n// Full control: custom 404, a header, and the body serialized by hand.\n@get('/:id')\npublic get(ctx: RouteContext): Response {\n const id = u64.parse(ctx.param('id'));\n if (!store.has(id)) return Response.notFound();\n return Response.json(store.get(id).toJSON().toString())\n .setHeader('cache-control', 'no-store');\n}\n```\n\n## Request and response bodies\n\nEvery route is either a **JSON route** (the default) or a **Binary route**. This decides how request bodies are decoded and response values are encoded:\n\n- **JSON**: the request body is parsed as JSON and revived into your `@data` type; the response value is serialized to JSON. Best for endpoints a browser or a third party calls directly, because JSON is universally readable.\n- **Binary**: the request body and response use toiljs's compact binary codec (`DataWriter`/`DataReader`). Smaller and faster, and exact for very large integers. Best for app-to-app traffic and anything performance sensitive.\n\nSet the mode on the whole controller with `@rest({ stream: DataStream.Binary })`, or per route with `@route({ ..., stream: DataStream.Binary })`. Full detail on both codecs is in [Data types](./data.md).\n\n> Large integers and JSON: numbers of 64 bits or more (`u64`, `i64`, `u128`, `u256`, and friends) are sent over JSON as decimal strings, so they stay exact at any size (plain JSON numbers would lose precision). The generated client turns them back into `bigint`. See [Types](../concepts/types.md).\n\n## The `RouteContext` object\n\n`RouteContext` is your window into the incoming request. toiljs builds one and passes it to any route method that declares a `RouteContext` parameter.\n\n| Member | Signature | What it gives you |\n| --- | --- | --- |\n| `request` | `Request` | The raw request (method, path, headers, body). |\n| `param` | `param(name: string): string` | A captured path param, or `\"\"` if absent. |\n| `query` | `query(name: string): string` | A query-string value (`?q=hi`), or `\"\"` if absent. Not URL-decoded in v1. |\n| `header` | `header(name: string): string \\| null` | A request header, case-insensitive, or `null`. |\n| `text` | `text(): string` | The raw request body decoded as UTF-8 text. |\n| `clientIp` | `clientIp(): string` | The connecting client's IP, or `\"\"` if unavailable. |\n\n`clientIp()` is the real socket address the edge observed, not a forgeable header like `X-Forwarded-For`, so it is safe to key rate limits, geo lookups, or audit logs on it.\n\n## The `Request` object\n\nThe raw request. You get it as `ctx.request`, or directly as the argument to a hand-written `handle(req)`.\n\n**Fields:**\n\n| Field | Type | Notes |\n| --- | --- | --- |\n| `method` | `Method` | `GET`, `POST`, `PUT`, `DELETE`, `PATCH`, `HEAD`, `OPTIONS`, or `UNKNOWN`. |\n| `path` | `string` | The path, including the query string. |\n| `headers` | `Array<Header>` | Ordered list; a `Header` is `{ name, value }`. |\n| `body` | `Uint8Array` | The raw request body bytes. |\n\n**Methods:**\n\n| Method | Signature | Notes |\n| --- | --- | --- |\n| `header` | `header(name: string): string \\| null` | Case-insensitive lookup, `null` if absent. |\n| `cookies` | `cookies(): CookieMap` | Parses the `Cookie` header (values percent-decoded); cached per request. |\n| `cookie` | `cookie(name: string): string \\| null` | A single cookie value, or `null`. |\n\nThe `Method` enum and the `Header` class are exported from `toiljs/server/runtime`.\n\n## Building a response\n\n`Response` is what you return. Create one with a static factory, then chain instance methods to add headers, cookies, and caching. Every instance method returns the same `Response`, so calls chain.\n\n### Static factories\n\n| Factory | Signature | Status | Content-Type |\n| --- | --- | --- | --- |\n| `Response.text` | `text(body: string, status = 200)` | 200 | `text/plain; charset=utf-8` |\n| `Response.html` | `html(body: string, status = 200)` | 200 | `text/html; charset=utf-8` |\n| `Response.json` | `json(body: string, status = 200)` | 200 | `application/json; charset=utf-8` |\n| `Response.bytes` | `bytes(body: Uint8Array, status = 200)` | 200 | `application/octet-stream` |\n| `Response.empty` | `empty(status)` | custom | (none) |\n| `Response.notFound` | `notFound()` | 404 | text |\n| `Response.badRequest` | `badRequest(msg = 'bad request')` | 400 | text |\n| `Response.internalError` | `internalError(msg = 'internal error')` | 500 | text |\n| `Response.unhandled` | `unhandled()` | 404 | text, plus a marker header (see below) |\n\n`Response.json` takes an **already-serialized** string, not an object. Build it from a `@data` value with `value.toJSON().toString()`, or return the `@data` value directly and let toiljs serialize it (usually simpler).\n\n### Instance methods\n\n| Method | Signature | What it does |\n| --- | --- | --- |\n| `setHeader` | `setHeader(name, value): Response` | Appends a header (call again to add more). |\n| `setCookie` | `setCookie(cookie: Cookie): Response` | Appends a `Set-Cookie`. |\n| `setCookieKV` | `setCookieKV(name, value): Response` | Shorthand for a cookie with no attributes. |\n| `clearCookie` | `clearCookie(name, path = '/', domain = ''): Response` | Emits a deletion cookie (empty value, `Max-Age=0`). |\n| `cache` | `cache(edgeTtlMinutes, browserTtlSeconds = 0, privateScope = false, allowAuth = false): Response` | Marks the response cacheable. See [Caching](../services/caching.md). |\n| `cacheFor` | `cacheFor(minutes): Response` | Shorthand for edge-caching for N minutes. |\n\n```ts\nreturn Response.json('{\"id\":42}')\n .setHeader('x-trace', traceId)\n .setCookieKV('seen', '1')\n .cacheFor(5);\n```\n\nSee [Cookies](../services/cookies.md) for the full cookie builder and [Caching](../services/caching.md) for the caching rules.\n\n## How dispatch works\n\nEvery `@rest` controller registers itself into a global `Rest` registry when its module loads. At request time, `Rest.dispatch(req)` walks the controllers and tries their routes.\n\n```mermaid\nflowchart TD\n A[\"Request arrives\"] --> B[\"Rest.dispatch(req)\"]\n B --> C{\"Any controller<br/>route match<br/>the method + path?\"}\n C -->|\"Yes\"| D[\"Run that method\"]\n D --> E[\"Return its Response\"]\n C -->|\"No\"| F[\"dispatch returns null\"]\n F --> G[\"Your fallback logic<br/>(or Response.unhandled)\"]\n```\n\nMatching order is deterministic: controllers are tried in the order their modules load, and routes within a controller in declaration order. The **first match wins**, so put more specific routes before catch-all ones.\n\n## How routes are discovered\n\n`toiljs build` scans every file under `server/` and finds your decorated classes on its own, so a route file is picked up as soon as it exists. To keep a plain `toilscript` run (and your editor) seeing the same set, projects also `import` each route file once in `server/main.ts`:\n\n```ts\n// server/main.ts (excerpt)\nimport './routes/Players';\nimport './routes/Todos';\n```\n\nAdding a new controller is two steps: write the file, add the one-line import.\n\n## Dispatch and the 404 fallback\n\nInside your handler, the usual pattern is: try REST first, then fall through to anything else.\n\n```ts\nconst hit = Rest.dispatch(req); // Response, or null if nothing matched\nif (hit != null) return hit;\nreturn Response.unhandled(); // no route matched here\n```\n\nThere are two different 404s, and the difference matters:\n\n- **`Response.notFound()`** means \"I looked, and that resource does not exist.\" It is sent to the client as a plain `404`.\n- **`Response.unhandled()`** means \"this server has no route for that path.\" It is a `404` carrying a marker header (`x-toil-unhandled`). The edge (and the dev server) reads that marker and tries to serve the path another way: a static file, or the client-side app. The marker is stripped before anything reaches the browser.\n\nRule of thumb: return `unhandled()` when a path is simply not yours to handle, and `notFound()` when the path is yours but the specific thing is missing.\n\nIf your project is REST-only, you do not need a custom handler at all. toiljs ships `RestHandler`, which does exactly the dispatch-then-`unhandled` dance:\n\n```ts\nimport { Server, RestHandler } from 'toiljs/server/runtime';\nServer.handler = () => new RestHandler();\n```\n\n## Guards: auth, rate limits, caching\n\nYou can stack extra decorators on a route (or a whole controller) to protect or cache it. They compose with the verb decorators:\n\n```ts\n@rest('admin')\nclass Admin {\n @get('/stats')\n @auth // reject with 401 if there is no valid session\n @ratelimit(RateLimit.SlidingWindow, 30, 60) // at most 30 requests per 60 seconds\n public stats(): Stats { /* ... */ }\n}\n```\n\n- `@auth` requires a valid signed session, else the request is rejected with `401` before your method runs. See the [Auth guide](../auth/README.md).\n- `@ratelimit` caps how often a caller may hit the route. See [Rate limiting](../services/ratelimit.md).\n- Response caching is opt-in per response with `.cache(...)` / `.cacheFor(...)`. See [Caching](../services/caching.md).\n\n## A complete CRUD example\n\nA small in-memory players API showing create, read, update, and delete. In a real app you would store players in [ToilDB](../database/README.md) instead of a local map, because each request gets a fresh handler (see [statelessness](./README.md#stateless-by-default)); this example keeps it in memory to stay focused on routing.\n\n```ts\n// server/models/NewPlayer.ts\n@data\nexport class NewPlayer {\n name: string = '';\n}\n```\n\n```ts\n// server/models/Player.ts\n@data\nexport class Player {\n id: u64 = 0;\n name: string = '';\n score: i64 = 0;\n}\n```\n\n```ts\n// server/routes/Players.ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\nimport { NewPlayer } from '../models/NewPlayer';\nimport { Player } from '../models/Player';\n\n// A stand-in store. Real apps use ToilDB (see the database section).\nconst store = new Map<u64, Player>();\nlet nextId: u64 = 1;\n\n@rest('players')\nclass Players {\n // CREATE: POST /players (JSON body -> Player)\n @post('/')\n public create(input: NewPlayer): Player {\n const p = new Player();\n p.id = nextId++;\n p.name = input.name;\n p.score = 0;\n store.set(p.id, p);\n return p; // 200 with the player as JSON\n }\n\n // READ: GET /players/:id\n @get('/:id')\n public get(ctx: RouteContext): Response {\n const id = u64.parse(ctx.param('id'));\n if (!store.has(id)) return Response.notFound();\n return Response.json(store.get(id).toJSON().toString());\n }\n\n // UPDATE: PUT /players/:id (JSON body -> Player)\n @put('/:id')\n public update(input: NewPlayer, ctx: RouteContext): Response {\n const id = u64.parse(ctx.param('id'));\n if (!store.has(id)) return Response.notFound();\n const p = store.get(id);\n p.name = input.name;\n return Response.json(p.toJSON().toString());\n }\n\n // DELETE: DELETE /players/:id\n @del('/:id')\n public remove(ctx: RouteContext): Response {\n const id = u64.parse(ctx.param('id'));\n if (!store.has(id)) return Response.notFound();\n store.delete(id);\n return Response.empty(204); // 204 No Content\n }\n}\n```\n\nCalling it from anywhere:\n\n```sh\ncurl -X POST localhost:5173/players -d '{\"name\":\"Ada\"}'\ncurl localhost:5173/players/1\ncurl -X PUT localhost:5173/players/1 -d '{\"name\":\"Ada Lovelace\"}'\ncurl -X DELETE localhost:5173/players/1\n```\n\nBecause `@rest` also generates a typed fetch client, your React frontend can call the same routes without writing URLs:\n\n```ts\nawait Server.REST.players.create({ body: new NewPlayer('Ada') });\nawait Server.REST.players.get({ params: { id: 1 } });\n```\n\nSee [RPC and the generated client](./rpc.md#the-rest-fetch-client) for that client.\n\n## Gotchas\n\n- **Fields do not persist.** A fresh controller instance serves each request, so instance fields reset every time and are never shared between requests or edge nodes. Persist to [ToilDB](../database/README.md).\n- **`Response.json` wants a string.** Pass an already-serialized JSON string (or return the `@data` value directly). Passing an object will not do what you expect.\n- **`@del`, not `@delete`.** `delete` is a reserved word, so the decorator is `@del`.\n- **Query values are not URL-decoded in v1.** `ctx.query('q')` returns the raw value; decode it yourself if it may contain percent-encoding.\n- **Path matching is exact on segment count.** `/todos/:id` does not match `/todos/1/extra`. Add a route for the longer path.\n- **`notFound()` vs `unhandled()`.** Returning `notFound()` from your top handler stops the edge from falling through to static files or the client app. Use `unhandled()` for \"not my path.\"\n\n## Related\n\n- [Data types (`@data`)](./data.md): the request and response body structs, and the JSON vs binary codecs.\n- [Typed RPC](./rpc.md): call your backend from your frontend with end-to-end types (and the generated REST fetch client).\n- [Backend overview](./README.md): the request lifecycle and handler model.\n- [Cookies](../services/cookies.md), [Caching](../services/caching.md), [Rate limiting](../services/ratelimit.md): response helpers and guards.\n- [Auth](../auth/README.md): protecting routes with `@auth`.\n- [The database](../database/README.md): persisting data between requests.\n",
11
+ "backend/rest.md": "# HTTP routes (`@rest`)\n\nExpose a real HTTP API by tagging a class `@rest` and its methods with an HTTP verb; toiljs generates the router for you.\n\n## Why and when\n\nUse `@rest` when something outside your own frontend needs to call your backend over plain HTTP: a webhook from another service, a public API, a mobile app, a `curl` script, or a browser hitting a URL directly. REST endpoints are ordinary URLs with ordinary HTTP methods and status codes, so any HTTP client understands them.\n\nIf instead you are calling your own backend from your own React frontend, [typed RPC](./rpc.md) is usually nicer (you get end-to-end types and skip URL wrangling). The two live happily side by side.\n\n## The shape of a route\n\nA REST route is a method on a class. The class decorator says where the class is mounted; the method decorator says which HTTP method and path it answers.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('players') // this controller is mounted at /players\nclass Players {\n @get('/:id') // answers GET /players/:id\n public get(ctx: RouteContext): Response {\n const id = ctx.param('id');\n return Response.json(`{\"id\":\"${id}\"}`);\n }\n}\n```\n\nSave that file, import it once (see [How routes are discovered](#how-routes-are-discovered)), and `GET /players/42` returns `{\"id\":\"42\"}`. That is a working endpoint.\n\n## `@rest`: mounting a controller\n\n`@rest` marks a class as a route **controller** (a group of related routes) and mounts it at a URL prefix.\n\n```ts\n@rest('api') // mounted at /api\n@rest('/') // or @rest('') : mounted at the root\n@rest({ stream: DataStream.Binary }) // root mount, binary body codec by default\n```\n\n- The string is the mount prefix. `\"api\"`, `\"/api\"`, and `\"api/\"` all normalize to `/api`. `\"\"` and `\"/\"` mean the root.\n- The object form sets class-wide defaults. `stream: DataStream.Binary` makes every route in the class use the binary body codec instead of JSON (more on that in [Request and response bodies](#request-and-response-bodies)).\n\nThe full URL of a route is the controller prefix joined with the method path. With `@rest('api')` and `@get('/todos/:id')`, the route is `GET /api/todos/:id`.\n\n## Verb decorators\n\nEach HTTP method has a decorator that takes the route path:\n\n```ts\n@get('/path') @post('/path') @put('/path') @patch('/path')\n@del('/path') @head('/path') @options('/path')\n```\n\nNote the delete decorator is `@del`, not `@delete` (the word `delete` is reserved in TypeScript, so it cannot be a decorator name).\n\n### `@route`: the explicit form\n\n`@route` is the general form. Reach for it when you want to set the body codec per route, or you just prefer an object:\n\n```ts\n@route({ method: Methods.POST, path: '/upload', stream: DataStream.Binary })\npublic upload(body: FileData): FileResult { /* ... */ }\n```\n\n`method` (from the `Methods` enum) and `path` are required; `stream` is optional and overrides the controller default. `Methods` and `DataStream` are global enums (like the decorators themselves, they need no import).\n\n## Path parameters\n\nA path segment written as `:name` is a **path parameter**: it captures whatever the request has in that position. Read it with `ctx.param('name')`.\n\n```ts\n@get('/todos/:id/items/:itemId')\npublic getItem(ctx: RouteContext): Response {\n const id = ctx.param('id'); // \"42\" for /todos/42/items/9\n const itemId = ctx.param('itemId'); // \"9\"\n return Response.json(`{\"todo\":\"${id}\",\"item\":\"${itemId}\"}`);\n}\n```\n\nMatching is **segment-exact**: the request path must have the same number of `/`-separated segments, the static segments must match literally, and each `:param` captures one segment. The query string is removed before matching. A captured param is always a string; convert it yourself if you need a number (for example `u64.parse(ctx.param('id'))`).\n\n## Method parameters: reading the request\n\nA route method may declare zero, one, or two parameters. toiljs looks at their types to decide what to pass:\n\n- a parameter typed `RouteContext` receives the [request context](#the-routecontext-object) (path params, query, headers, raw body);\n- any other type is treated as the **request body**, decoded into that [`@data`](./data.md) type.\n\n```ts\n@get('/status')\npublic status(): StatusResponse { /* no body, no context */ }\n\n@get('/user/:id')\npublic getUser(ctx: RouteContext): User { /* context only */ }\n\n@post('/create')\npublic create(input: NewTodo): Todo { /* body only */ }\n\n@post('/user/:id/score')\npublic addScore(input: ScoreDelta, ctx: RouteContext): Player {\n const id = ctx.param('id'); // body AND context\n /* ... */\n}\n```\n\nThe order of the two parameters does not matter; toiljs classifies them by type.\n\n## Return types\n\nYou return one of two things from a route, and toiljs encodes it for you:\n\n| You return | toiljs sends |\n| --- | --- |\n| a [`@data`](./data.md) value | The value serialized (JSON by default, binary if the route is in binary mode). Status `200`. |\n| a `Response` | Exactly that response: your status, headers, cookies, and body, untouched. |\n| `void` (nothing) | `204 No Content`. |\n\nReturning a `@data` value is the short path when you just want to send the data. Returning a `Response` gives you full control: a custom status, extra headers, cookies, caching. Use whichever fits.\n\n```ts\n// Short path: return the data, let toiljs serialize it as JSON.\n@get('/me')\npublic me(): Player { return currentPlayer(); }\n\n// Full control: custom 404, a header, and the body serialized by hand.\n@get('/:id')\npublic get(ctx: RouteContext): Response {\n const id = u64.parse(ctx.param('id'));\n if (!store.has(id)) return Response.notFound();\n return Response.json(store.get(id).toJSON().toString())\n .setHeader('cache-control', 'no-store');\n}\n```\n\n## Request and response bodies\n\nEvery route is either a **JSON route** (the default) or a **Binary route**. This decides how request bodies are decoded and response values are encoded:\n\n- **JSON**: the request body is parsed as JSON and revived into your `@data` type; the response value is serialized to JSON. Best for endpoints a browser or a third party calls directly, because JSON is universally readable.\n- **Binary**: the request body and response use toiljs's compact binary codec (`DataWriter`/`DataReader`). Smaller and faster, and exact for very large integers. Best for app-to-app traffic and anything performance sensitive.\n\nSet the mode on the whole controller with `@rest({ stream: DataStream.Binary })`, or per route with `@route({ ..., stream: DataStream.Binary })`. Full detail on both codecs is in [Data types](./data.md).\n\n> Large integers and JSON: numbers of 64 bits or more (`u64`, `i64`, `u128`, `u256`, and friends) are sent over JSON as decimal strings, so they stay exact at any size (plain JSON numbers would lose precision). The generated client turns them back into `bigint`. See [Types](../concepts/types.md).\n\n## The `RouteContext` object\n\n`RouteContext` is your window into the incoming request. toiljs builds one and passes it to any route method that declares a `RouteContext` parameter.\n\n| Member | Signature | What it gives you |\n| --- | --- | --- |\n| `request` | `Request` | The raw request (method, path, headers, body). |\n| `param` | `param(name: string): string` | A captured path param, or `\"\"` if absent. |\n| `query` | `query(name: string): string` | A query-string value (`?q=hi`), or `\"\"` if absent. Not URL-decoded in v1. |\n| `header` | `header(name: string): string \\| null` | A request header, case-insensitive, or `null`. |\n| `text` | `text(): string` | The raw request body decoded as UTF-8 text. |\n| `clientIp` | `clientIp(): string` | The connecting client's IP, or `\"\"` if unavailable. |\n\n`clientIp()` is the real socket address the edge observed, not a forgeable header like `X-Forwarded-For`, so it is safe to key rate limits, geo lookups, or audit logs on it.\n\n## The `Request` object\n\nThe raw request. You get it as `ctx.request`, or directly as the argument to a hand-written `handle(req)`.\n\n**Fields:**\n\n| Field | Type | Notes |\n| --- | --- | --- |\n| `method` | `Method` | `GET`, `POST`, `PUT`, `DELETE`, `PATCH`, `HEAD`, `OPTIONS`, or `UNKNOWN`. |\n| `path` | `string` | The path, including the query string. |\n| `headers` | `Array<Header>` | Ordered list; a `Header` is `{ name, value }`. |\n| `body` | `Uint8Array` | The raw request body bytes. |\n\n**Methods:**\n\n| Method | Signature | Notes |\n| --- | --- | --- |\n| `header` | `header(name: string): string \\| null` | Case-insensitive lookup, `null` if absent. |\n| `cookies` | `cookies(): CookieMap` | Parses the `Cookie` header (values percent-decoded); cached per request. |\n| `cookie` | `cookie(name: string): string \\| null` | A single cookie value, or `null`. |\n\nThe `Method` enum and the `Header` class are exported from `toiljs/server/runtime`.\n\n## Building a response\n\n`Response` is what you return. Create one with a static factory, then chain instance methods to add headers, cookies, and caching. Every instance method returns the same `Response`, so calls chain.\n\n### Static factories\n\n| Factory | Signature | Status | Content-Type |\n| --- | --- | --- | --- |\n| `Response.text` | `text(body: string, status = 200)` | 200 | `text/plain; charset=utf-8` |\n| `Response.html` | `html(body: string, status = 200)` | 200 | `text/html; charset=utf-8` |\n| `Response.json` | `json(body: string, status = 200)` | 200 | `application/json; charset=utf-8` |\n| `Response.bytes` | `bytes(body: Uint8Array, status = 200)` | 200 | `application/octet-stream` |\n| `Response.empty` | `empty(status)` | custom | (none) |\n| `Response.notFound` | `notFound()` | 404 | text |\n| `Response.badRequest` | `badRequest(msg = 'bad request')` | 400 | text |\n| `Response.internalError` | `internalError(msg = 'internal error')` | 500 | text |\n| `Response.unhandled` | `unhandled()` | 404 | text, plus a marker header (see below) |\n\n`Response.json` takes an **already-serialized** string, not an object. Build it from a `@data` value with `value.toJSON().toString()`, or return the `@data` value directly and let toiljs serialize it (usually simpler).\n\n### Instance methods\n\n| Method | Signature | What it does |\n| --- | --- | --- |\n| `setHeader` | `setHeader(name, value): Response` | Appends a header (call again to add more). |\n| `setCookie` | `setCookie(cookie: Cookie): Response` | Appends a `Set-Cookie`. |\n| `setCookieKV` | `setCookieKV(name, value): Response` | Shorthand for a cookie with no attributes. |\n| `clearCookie` | `clearCookie(name, path = '/', domain = ''): Response` | Emits a deletion cookie (empty value, `Max-Age=0`). |\n| `cache` | `cache(edgeTtlMinutes, browserTtlSeconds = 0, privateScope = false, allowAuth = false): Response` | Marks the response cacheable. See [Caching](../services/caching.md). |\n| `cacheFor` | `cacheFor(minutes): Response` | Shorthand for edge-caching for N minutes. |\n\n```ts\nreturn Response.json('{\"id\":42}')\n .setHeader('x-trace', traceId)\n .setCookieKV('seen', '1')\n .cacheFor(5);\n```\n\nSee [Cookies](../services/cookies.md) for the full cookie builder and [Caching](../services/caching.md) for the caching rules.\n\n## How dispatch works\n\nEvery `@rest` controller registers itself into a global `Rest` registry when its module loads. At request time, `Rest.dispatch(req)` walks the controllers and tries their routes.\n\n```mermaid\nflowchart TD\n A[\"Request arrives\"] --> B[\"Rest.dispatch(req)\"]\n B --> C{\"Any controller<br/>route match<br/>the method + path?\"}\n C -->|\"Yes\"| D[\"Run that method\"]\n D --> E[\"Return its Response\"]\n C -->|\"No\"| F[\"dispatch returns null\"]\n F --> G[\"Your fallback logic<br/>(or Response.unhandled)\"]\n```\n\nMatching order is deterministic: controllers are tried in the order their modules load, and routes within a controller in declaration order. The **first match wins**, so put more specific routes before catch-all ones.\n\n## How routes are discovered\n\n`toiljs build` scans every file under `server/` and finds your decorated classes on its own, so a route file is picked up as soon as it exists. To keep a plain `toilscript` run (and your editor) seeing the same set, projects also `import` each route file once in `server/main.ts`:\n\n```ts\n// server/main.ts (excerpt)\nimport './routes/Players';\nimport './routes/Todos';\n```\n\nAdding a new controller is two steps: write the file, add the one-line import.\n\n## Dispatch and the 404 fallback\n\nInside your handler, the usual pattern is: try REST first, then fall through to anything else.\n\n```ts\nconst hit = Rest.dispatch(req); // Response, or null if nothing matched\nif (hit != null) return hit;\nreturn Response.unhandled(); // no route matched here\n```\n\nThere are two different 404s, and the difference matters:\n\n- **`Response.notFound()`** means \"I looked, and that resource does not exist.\" It is sent to the client as a plain `404`.\n- **`Response.unhandled()`** means \"this server has no route for that path.\" It is a `404` carrying a marker header (`x-toil-unhandled`). The edge (and the dev server) reads that marker and tries to serve the path another way: a static file, or the client-side app. The marker is stripped before anything reaches the browser.\n\nRule of thumb: return `unhandled()` when a path is simply not yours to handle, and `notFound()` when the path is yours but the specific thing is missing.\n\nIf your project is REST-only, you do not need a custom handler at all. toiljs ships `RestHandler`, which does exactly the dispatch-then-`unhandled` dance:\n\n```ts\nimport { Server, RestHandler } from 'toiljs/server/runtime';\nServer.handler = () => new RestHandler();\n```\n\n## Guards: auth, rate limits, caching\n\nYou can stack extra decorators on a route (or a whole controller) to protect or cache it. They compose with the verb decorators:\n\n```ts\n@rest('admin')\nclass Admin {\n @get('/stats')\n @auth // reject with 401 if there is no valid session\n @ratelimit(RateLimit.SlidingWindow, 30, 60) // at most 30 requests per 60 seconds\n public stats(): Stats { /* ... */ }\n}\n```\n\n- `@auth` requires a valid signed session, else the request is rejected with `401` before your method runs. See the [Auth guide](../auth/README.md).\n- `@ratelimit` caps how often a caller may hit the route. See [Rate limiting](../services/ratelimit.md).\n- Response caching is opt-in per response with `.cache(...)` / `.cacheFor(...)`. See [Caching](../services/caching.md).\n\n## A complete CRUD example\n\nA small in-memory players API showing create, read, update, and delete. In a real app you would store players in [ToilDB](../database/README.md) instead of a local map, because each request gets a fresh handler (see [statelessness](./README.md#stateless-by-default)); this example keeps it in memory to stay focused on routing.\n\n```ts\n// server/models/NewPlayer.ts\n@data\nexport class NewPlayer {\n name: string = '';\n}\n```\n\n```ts\n// server/models/Player.ts\n@data\nexport class Player {\n id: u64 = 0;\n name: string = '';\n score: i64 = 0;\n}\n```\n\n```ts\n// server/routes/Players.ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\nimport { NewPlayer } from '../models/NewPlayer';\nimport { Player } from '../models/Player';\n\n// A stand-in store. Real apps use ToilDB (see the database section).\nconst store = new Map<u64, Player>();\nlet nextId: u64 = 1;\n\n@rest('players')\nclass Players {\n // CREATE: POST /players (JSON body -> Player)\n @post('/')\n public create(input: NewPlayer): Player {\n const p = new Player();\n p.id = nextId++;\n p.name = input.name;\n p.score = 0;\n store.set(p.id, p);\n return p; // 200 with the player as JSON\n }\n\n // READ: GET /players/:id\n @get('/:id')\n public get(ctx: RouteContext): Response {\n const id = u64.parse(ctx.param('id'));\n if (!store.has(id)) return Response.notFound();\n return Response.json(store.get(id).toJSON().toString());\n }\n\n // UPDATE: PUT /players/:id (JSON body -> Player)\n @put('/:id')\n public update(input: NewPlayer, ctx: RouteContext): Response {\n const id = u64.parse(ctx.param('id'));\n if (!store.has(id)) return Response.notFound();\n const p = store.get(id);\n p.name = input.name;\n return Response.json(p.toJSON().toString());\n }\n\n // DELETE: DELETE /players/:id\n @del('/:id')\n public remove(ctx: RouteContext): Response {\n const id = u64.parse(ctx.param('id'));\n if (!store.has(id)) return Response.notFound();\n store.delete(id);\n return Response.empty(204); // 204 No Content\n }\n}\n```\n\nCalling it from anywhere:\n\n```sh\ncurl -X POST localhost:5173/players -d '{\"name\":\"Ada\"}'\ncurl localhost:5173/players/1\ncurl -X PUT localhost:5173/players/1 -d '{\"name\":\"Ada Lovelace\"}'\ncurl -X DELETE localhost:5173/players/1\n```\n\nBecause `@rest` also generates a typed fetch client, your React frontend can call the same routes without writing URLs:\n\n```ts\nawait Server.REST.players.create({ body: new NewPlayer('Ada') });\nawait Server.REST.players.get({ params: { id: 1 } });\n```\n\nSee [RPC and the generated client](./rpc.md#the-rest-fetch-client) for that client.\n\n## Gotchas\n\n- **Fields do not persist.** A fresh controller instance serves each request, so instance fields reset every time and are never shared between requests or edge nodes. Persist to [ToilDB](../database/README.md).\n- **`Response.json` wants a string.** Pass an already-serialized JSON string (or return the `@data` value directly). Passing an object will not do what you expect.\n- **`@del`, not `@delete`.** `delete` is a reserved word, so the decorator is `@del`.\n- **Query values are not URL-decoded in v1.** `ctx.query('q')` returns the raw value; decode it yourself if it may contain percent-encoding.\n- **Path matching is exact on segment count.** `/todos/:id` does not match `/todos/1/extra`. Add a route for the longer path.\n- **`notFound()` vs `unhandled()`.** Returning `notFound()` from your top handler stops the edge from falling through to static files or the client app. Use `unhandled()` for \"not my path.\"\n\n## Related\n\n- [Data types (`@data`)](./data.md): the request and response body structs, and the JSON vs binary codecs.\n- [Typed RPC](./rpc.md): call your backend from your frontend with end-to-end types (and the generated REST fetch client).\n- [Backend overview](./README.md): the request lifecycle and handler model.\n- [Route execution and transport limits](../concepts/config.md#route-execution-and-transport-limits): handler budgets, HTTP timeouts, and upload size limits.\n- [Cookies](../services/cookies.md), [Caching](../services/caching.md), [Rate limiting](../services/ratelimit.md): response helpers and guards.\n- [Auth](../auth/README.md): protecting routes with `@auth`.\n- [The database](../database/README.md): persisting data between requests.\n",
12
12
  "backend/rpc.md": "# Typed RPC (`@service` / `@remote`)\n\nWrite a server function, tag it `@remote`, and call it from your React frontend like a local async function, with the argument and return types checked end to end.\n\n## What RPC is\n\n**RPC** stands for Remote Procedure Call. The idea is old and simple: call a function that actually runs somewhere else (here, on the edge) as if it were a normal function in your own code. You write a method on the server, and on the client you `await` it. No URLs, no `fetch`, no manual JSON. Just a function call.\n\ntoiljs makes that call **fully typed**. When you build the server, it generates a TypeScript file (`shared/server.ts`) describing every callable function, its arguments, and its return type. Your frontend imports nothing extra: a global object called `Server` is available with all of it typed. If you change a server function's signature and rebuild, your frontend code stops type-checking until you fix the call. The client can never drift from the server.\n\n## Why and when\n\nUse RPC when **your own frontend** talks to **your own backend**. It is the most ergonomic and safest way to do that:\n\n- End-to-end types: rename a field on the server, and the client sees it immediately.\n- No plumbing: no route paths to invent, no request or response shapes to hand-write.\n- Exact numbers: 64-bit-and-larger integers arrive as `bigint`, never rounded.\n\nUse [`@rest`](./rest.md) instead when the caller is **not** your frontend: a webhook, a third-party integration, a public API, a mobile client, or anything that speaks in plain URLs and HTTP methods. RPC uses one internal endpoint and a binary wire format, so it is not meant to be called by hand.\n\nYou can use both in the same project. A common split: RPC for your app's own screens, REST for everything public.\n\n## Declaring callable functions\n\nTwo decorators expose server code to the client:\n\n- **`@remote`** on a top-level function makes it directly callable.\n- **`@service`** on a class groups related `@remote` methods under a namespace.\n\n```ts\n// server/services/Stats.ts\nimport { store } from '../core/store';\n\n@service\nclass Stats {\n @remote\n public playerCount(): i32 {\n return store.size;\n }\n}\n```\n\n```ts\n// server/services/remotes.ts\n@remote\nfunction ping(n: i32): i32 {\n return n + 1;\n}\n```\n\nThat is the whole server side. Build the server and the client can call them.\n\n## Calling from the frontend\n\nThe generated client surfaces everything on a global `Server`. A `@service` becomes a namespace keyed by the class name with a lowercase first letter (`Stats` becomes `stats`). A free `@remote` sits directly on `Server`. Every call returns a `Promise`.\n\n```ts\n// anywhere in your React app, no import needed\nconst count = await Server.stats.playerCount(); // number\nconst next = await Server.ping(41); // number -> 42\n```\n\nAutocomplete, argument checking, and the return type all come from `shared/server.ts`, which the build regenerates every time (and `toiljs dev` regenerates on save), so it is always in sync.\n\n## The round trip\n\nHere is what actually happens under that innocent-looking `await`.\n\n```mermaid\nsequenceDiagram\n participant C as Browser (Server.stats.playerCount)\n participant E as Dacely edge\n participant W as Your backend (WASM)\n C->>C: encode args with DataWriter\n C->>E: POST /__toil_rpc<br/>header dacely-rpc = method id<br/>body = encoded args\n E->>W: dispatch to the matching @remote\n W->>W: run playerCount(), encode the result\n W->>E: encoded result bytes\n E->>C: 200 application/octet-stream\n C->>C: decode into the typed value\n```\n\nA few facts worth knowing:\n\n- Every callable has a stable numeric **method id** (a hash of `\"Service.method\"` or the function name). The client sends it in the `dacely-rpc` header; the server dispatches on it.\n- Arguments and results travel in the compact **binary `@data` codec** (see [Data types](./data.md)), so large integers are exact and payloads are small.\n- The endpoint is a single reserved path, `/__toil_rpc`. You never route it yourself; the framework handles it before your `handle` runs.\n\n## RPC is stateless too\n\nJust like a REST controller, a fresh service instance serves each call. Fields you set on a `@service` class do not survive between calls. If two calls need to share data, that data lives in [ToilDB](../database/README.md), not in an instance field. See [statelessness](./README.md#stateless-by-default).\n\n## Argument and return types\n\nArguments and return values may be scalars, arrays, or [`@data`](./data.md) classes, in both directions. Here is how toilscript's types map to what you see on the TypeScript client:\n\n| ToilScript type | TypeScript type |\n| --- | --- |\n| `u8`, `u16`, `u32`, `i8`, `i16`, `i32`, `f32`, `f64` | `number` |\n| `u64`, `i64`, `u128`, `i128`, `u256`, `i256` | `bigint` |\n| `bool` | `boolean` |\n| `string` | `string` |\n| a `@data` class `T` | `T` (the generated class) |\n| `T[]` | `T[]` |\n\nIntegers of 64 bits or more become `bigint` on the client, so they are exact at any magnitude. See [Types](../concepts/types.md) for the full number story.\n\nPassing a `@data` value is just as easy as a scalar. Construct it on the client and pass it in:\n\n```ts\nimport { NewPlayer } from './shared/server';\n\nconst created = await Server.roster.add(new NewPlayer('Ada')); // returns a typed Player\n```\n\nThe `@data` classes in `shared/server.ts` share a byte-for-byte identical codec with the server, so values round-trip exactly.\n\n## Reading and writing the database from a `@remote`\n\nA `@remote` can use the database, but with a safety default: **a plain `@remote` is read-only.** If it tries to write to ToilDB, the compiler rejects it. To let a `@remote` write, add `@action`:\n\n```ts\n@service\nclass Roster {\n @remote\n public count(): i32 { // read-only: fine to just read\n return db.players.count();\n }\n\n @remote\n @action // opts into writes\n public add(input: NewPlayer): Player {\n return db.players.create(/* ... */);\n }\n}\n```\n\n`@query` is the explicit opposite of `@action`: it marks a function read-only on purpose (the default for a `@remote`, so you rarely need to write it). These are ToilDB **function kinds**; the full rules, including what each kind may and may not do, are in the [database docs](../database/README.md). The takeaway for RPC: reads work out of the box, and a write needs `@action`, so a read-only endpoint can never silently mutate your data.\n\n## Guarding a `@remote`\n\nGuards stack on a `@remote` exactly as they do on a REST route:\n\n```ts\n@service\nclass Stats {\n @remote\n @auth // reject with 401 when there is no valid session\n public secretCount(): i32 {\n return store.size;\n }\n}\n```\n\nThe RPC dispatcher enforces `@auth` (and `@ratelimit`) the same way the REST router does: the guard runs first, and an unauthenticated call gets a `401` before your method body executes. See the [Auth guide](../auth/README.md) and [Rate limiting](../services/ratelimit.md).\n\n## The generated `Server` surface\n\n`shared/server.ts` declares `Server` as a global with a shape like this (schematic):\n\n```ts\ndeclare global {\n const Server: {\n // free @remote functions\n ping(n: number): Promise<number>;\n\n // @service classes, keyed by lowercased name\n readonly stats: {\n playerCount(): Promise<number>;\n secretCount(): Promise<number>;\n };\n\n // @rest controllers get a fetch client under REST (see below)\n readonly REST: { /* ... */ };\n\n // @stream classes get a client under Stream (see the realtime docs)\n readonly Stream: { /* ... */ };\n };\n}\n```\n\nYou never edit this file; the build regenerates it. If a `Server` method throws that it is \"unavailable,\" the generated client has not loaded yet: run the server build (or `toiljs dev`, which does it on save).\n\n## The REST fetch client\n\n`@rest` controllers also get a typed client, under `Server.REST.<controller>.<route>`. It is real `fetch` code (because REST is just HTTP), and it is handy when you want your frontend to call a route you also expose publicly:\n\n```ts\n// controller @rest('players') with a create route taking a NewPlayer body\nconst player = await Server.REST.players.create({\n body: new NewPlayer('Ada'), // present only if the route takes a body\n // params: { id: 7 }, // present only if the path has :params\n query: { ref: 'home' }, // optional\n headers: { 'x-trace': id }, // optional\n});\n```\n\nThe wrapper builds the URL, substitutes `:params`, appends `query`, sends the request, throws on a non-2xx status, and decodes the response into the route's return type. A route declared to return `Response` resolves to the raw `fetch` `Response` so you can inspect headers or stream it yourself. See [REST](./rest.md) for the routes themselves.\n\n## RPC vs REST at a glance\n\n| | RPC (`@service` / `@remote`) | REST (`@rest`) |\n| --- | --- | --- |\n| Caller | Your own frontend | Anyone (browser, webhook, third party, `curl`) |\n| Client | `Server.svc.method(args)` | `Server.REST.ctrl.route(args)`, or plain `fetch` |\n| URL shape | One internal endpoint | Real paths and HTTP methods you design |\n| Wire format | Compact binary `@data` | JSON (or binary), your choice |\n| Types | End to end, automatic | End to end for the generated client |\n| Best for | App-internal calls | Public APIs, integrations, webhooks |\n\n## Gotchas\n\n- **RPC is not a public API.** It uses one reserved endpoint and a binary format meant for the generated client. If an outside system needs to call in, expose a [`@rest`](./rest.md) route.\n- **Instance fields do not persist.** A fresh service instance serves every call. Shared state belongs in [ToilDB](../database/README.md).\n- **Writes need `@action`.** A plain `@remote` is read-only; the compiler rejects a database write unless the method is `@action`.\n- **Rebuild after signature changes.** `shared/server.ts` is generated. If autocomplete looks stale, rebuild the server (`toiljs dev` does this on save).\n- **`bigint`, not `number`, for 64-bit values.** A `u64`/`i64`/`u256` argument or return is a `bigint` on the client. Pass `10n`, not `10`.\n\n## Related\n\n- [Data types (`@data`)](./data.md): the structs your RPC arguments and results are made of, and the binary codec they travel in.\n- [HTTP routes (`@rest`)](./rest.md): the public-facing alternative, and the `Server.REST` fetch client.\n- [Types](../concepts/types.md): `u64`, `u256`, and how they map to `number` / `bigint`.\n- [The database](../database/README.md): `@action` vs `@query`, and persisting state.\n- [Fetching data on the frontend](../frontend/data-fetching.md): using `Server.*` from your React components.\n- [Auth](../auth/README.md): guarding a `@remote` with `@auth`.\n",
13
13
  "background/daemons.md": "# Daemons (`@daemon` / `@scheduled`)\n\nA `@daemon` is a single, long-lived background worker for your whole app. You mark a class `@daemon`, add `@scheduled` methods that fire on a timer, and the Dacely edge keeps exactly **one** copy of it running worldwide, restarting it elsewhere if the machine it is on fails.\n\n## What a daemon is\n\nThe word \"daemon\" (say \"DEE-mon\") is an old computing term for a program that runs quietly in the background, not tied to any single user. That is exactly what this is.\n\nCompare the three ways your server code can run:\n\n| Kind | How many run | Lives for |\n| ------------------------------- | ------------------------------------ | ----------------------------- |\n| [Request handler](../backend/rest.md) (`@rest`) | a fresh one per request | one request |\n| [Stream box](../realtime/streams.md) (`@stream`) | one per open connection | one connection |\n| **Daemon** (`@daemon`) | **exactly one for the whole app** | as long as it holds the lease |\n\nBecause it is a single, resident instance, its fields persist across scheduled runs (a request handler forgets everything after each request; a daemon does not). It is the right home for work that must happen **once globally on a cadence**, not once per user and not once per server.\n\n```ts\n@daemon\nclass Jobs {\n @scheduled('1h')\n hourly(): void {\n // Runs once an hour, on the one elected worker. Put recurring background\n // work here: rollups, cleanup, polling an upstream, and so on.\n }\n}\n```\n\n## `@scheduled`: run on a cadence\n\nA `@scheduled` method fires on a schedule. The single string argument is the cadence, and it comes in two flavours.\n\n### Interval schedules\n\nAn **interval** fires every fixed span of time. Write a number followed by a unit: `s` (seconds), `m` (minutes), `h` (hours), or `d` (days). The number must be at least 1, and the span may not exceed 7 days.\n\n```ts\n@scheduled('30s') everyHalfMinute(): void { /* ... */ }\n@scheduled('5m') everyFiveMinutes(): void { /* ... */ }\n@scheduled('1h') hourly(): void { /* ... */ }\n@scheduled('1d') daily(): void { /* ... */ }\n```\n\n### Cron schedules\n\nA **cron** expression fires at wall-clock times (\"every weekday at 9:15\", \"midnight on the first of the month\"). Use it when you care about the actual time of day, not just a repeating gap. A cron spec is five fields separated by spaces, in this order:\n\n```\nminute hour day-of-month month day-of-week\n```\n\ntoiljs recognises a cron spec by the spaces in it (an interval has none).\n\n```ts\n@scheduled('15 9 * * 1-5') // 09:15, Monday to Friday\nweekdayMorning(): void { /* ... */ }\n\n@scheduled('0 0 1 * *') // 00:00 on the 1st of every month\nmonthlyReset(): void { /* ... */ }\n```\n\nCron times are evaluated in **UTC** and are **minute-granular** (the smallest cron step is one minute). A `*` means \"every value\" for that field.\n\n### Rules\n\n- A `@scheduled` method takes **no arguments and returns `void`**.\n- A daemon class may have **several** `@scheduled` methods, each on its own cadence.\n- Because only the one elected worker fires them, a task runs **once per tick for the whole app**, never once per server.\n\n### `onStart`: run once at boot\n\nA daemon may also declare a plain `onStart(): void` method (not decorated). It runs **once**, when the daemon box first starts on the elected worker. Use it to set up state or kick off a long-running loop.\n\n```ts\n@daemon\nclass Jobs {\n onStart(): void {\n // one-time setup when this daemon becomes active\n }\n\n @scheduled('1h')\n hourly(): void { /* ... */ }\n}\n```\n\n## No backfill: missed runs are skipped, not replayed\n\nThis is the single most important thing to understand about scheduling.\n\nIf the daemon is down when a tick was due (say the leader failed and a standby is still taking over), or if the clock jumps forward, toiljs does **not** go back and run all the ticks you missed. It simply fires the **next** due run and moves on. This is called a **no-backfill** policy.\n\nTwo practical consequences:\n\n1. **Design tasks to be safe to skip.** \"Recompute the summary\" is fine to miss (the next run fixes it). \"Charge every user once\" is not, unless you make it idempotent.\n2. **Make tasks idempotent where a missed run matters.** Idempotent means running it twice (or catching up later) has the same effect as running it once. For example, \"set yesterday's total to X\" is idempotent; \"add 1 to a counter\" is not.\n\n## One global worker, with safe failover\n\nThere is exactly **one** daemon running for your app at any moment. A second machine keeps a **warm standby** ready but idle. If the active worker's hold on the job expires (it crashed, lost the network, or was shut down), the standby takes over and fires the following runs.\n\nThe mechanism is a **lease**. Think of the lease as a \"who is in charge\" token that only one worker can hold at a time, and that has to be renewed to keep. Only the worker holding the lease (the **leader**) runs the schedule.\n\n```mermaid\nsequenceDiagram\n participant A as Worker A (leader)\n participant L as Lease\n participant B as Worker B (standby)\n A->>L: hold the lease, renew it\n Note over A: A fires @scheduled ticks\n Note over B: B stays idle, watching\n A--xL: A crashes, stops renewing\n Note over L: lease expires\n B->>L: acquire the lease\n Note over B: B is now leader, fires the next ticks\n```\n\nThe important guarantee: **two workers never run the same tick at the same time.** This is called **at-most-once** scheduling. The trade-off is the no-backfill behaviour above: to be sure a tick is never run twice, the edge would rather skip the in-flight tick when a leader is lost than risk running it on two machines. You never start, stop, or place the daemon yourself; the edge elects the leader and drives it.\n\n## Leadership fencing: side effects only run on the leader\n\nA subtle risk with a warm standby is a \"split brain\": for a brief moment, two workers might both think they are the leader. To make that harmless, toiljs **fences** every side effect behind a leadership check. A **side effect** is anything that changes the outside world:\n\n- **Database writes** (creating, patching, deleting rows, adding to counters, appending events, publishing views).\n- **Outbound HTTP calls** (`http_call`, described below).\n\nThese run **only** on the confirmed leader. If code that is not the leader tries one, the edge refuses it (a \"not leader\" error) rather than let it happen twice. Plain **reads** and computation are not fenced (they are safe to do anywhere). So even if two workers briefly overlap, only one of them can actually write or call out. You do not write the fencing yourself; it is automatic. The upshot: put your writes and outbound calls in `@scheduled` methods freely, and trust that they happen once.\n\n## `daemon.*` host calls\n\nA daemon has a small set of host abilities beyond ordinary computation:\n\n- **Database access.** A daemon reads and writes [ToilDB](../database/README.md) with the same collection handles you use in a route or a derive (`.get`, `.add`, `.append`, `.publish`, and so on). Writes are leader-fenced as described above.\n- **Outbound HTTP (`http_call`).** A daemon can call an external service (to poll an API, post to a webhook). This is the one place your server code reaches out to the internet, so it is deliberately restricted:\n - It is **leader-only** (fenced, like any side effect).\n - It is **SSRF-bounded**. SSRF (server-side request forgery) is an attack where code is tricked into calling internal addresses it should not reach. The edge resolves the target host and blocks private or internal addresses, so a daemon cannot use `http_call` to poke around inside the network.\n - It is **metered**: making many calls or pulling huge responses costs budget, which caps abuse.\n- **Leadership info** (`is_leader`, `current_epoch`). A daemon can check whether it is currently the leader, which is useful for guarding a long `onStart` loop.\n\n> **Note:** In `toiljs dev` (the single-process local emulator), the daemon is always the leader (there is nothing to fail over to), and `http_call` is stubbed to return a \"call failed\" result rather than make real network requests. Everything else, including the schedule and your database writes, runs exactly as it does on the edge.\n\n### The `Daemon` global\n\nInside a `@daemon` class, `Daemon` is an ambient global (no import), like `Analytics`. It needs `toilscript` 0.1.60 or newer.\n\n| Member | Description |\n| --- | --- |\n| `Daemon.isLeader(): bool` | Whether this worker currently holds the lease. A snapshot: re-check it inside a long task rather than trusting the value it started with. |\n| `Daemon.epoch(): i64` | The monotonic fencing token, bumped on every (re)acquire, or `-1` when this worker is not the leader. Stamp it into work that must not outlive the lease that authorized it. |\n| `Daemon.taskCount(): i32` | How many `@scheduled` tasks are registered. |\n| `Daemon.nextFireMs(taskId: i32): i64` | The next fire time for a task, in epoch milliseconds, or `-1` when the id is unknown or the cron never fires again. |\n| `Daemon.yieldNow(): DaemonError` | Give the edge a chance to observe a lost lease. `DaemonError.None` means you are still the leader. |\n| `Daemon.sleep(ms: i64): DaemonError` | Park the task. The edge clamps this to 3 seconds. `LeaseLost` means the lease went away while you slept, and the task should stop. |\n| `Daemon.httpCall(request, responseCap?)` | Make an outbound HTTP request. Returns a `DaemonHttpResponse`, or `null` on failure. |\n| `Daemon.lastError(): DaemonError` | The failure recorded by the most recent `Daemon` call. |\n\n### Outbound HTTP\n\n`Daemon.httpCall` performs the leader-fenced, SSRF-bounded, metered call described above. Its usage shows up in the `Analytics` counters `daemonHttpCallAttempts` / `daemonHttpCallFailures`.\n\n```ts\n@daemon\nclass Jobs {\n @scheduled('5m')\n poll(): void {\n const req = new DaemonHttpRequest('POST', 'https://api.example.com/events');\n req.header('content-type', 'application/json');\n req.body = Uint8Array.wrap(String.UTF8.encode('{\"ping\":true}'));\n\n const res = Daemon.httpCall(req);\n if (res == null) {\n // Daemon.lastError(): NotLeader, CallFailed, ResponseTooLarge, BadEnvelope\n return;\n }\n if (res.status == 200) App.events.add(new Event(res.text()));\n }\n}\n```\n\nA `null` result is never ambiguous: read `Daemon.lastError()`.\n\n| `DaemonError` | Meaning |\n| --- | --- |\n| `NotLeader` | This worker does not hold the lease. The edge refused without touching the network. |\n| `LeaseLost` | The lease went away mid-call. |\n| `CallFailed` | The request did not complete: blocked by the SSRF guard, timed out, or the transport failed. |\n| `ResponseTooLarge` | The response did not fit your buffer. **The call already happened**, so do not blindly retry it: pass a larger `responseCap` instead. |\n| `BadEnvelope` | Your request broke a host cap (nothing was sent), or the response could not be parsed. |\n\nThe caps the edge enforces, which `httpCall` checks in the guest so an over-cap request never costs you a round trip: the method is 1 to 16 bytes, the URL at most 8 KiB, at most 64 headers of at most 8 KiB each, and a request body of at most 256 KiB. Responses are truncated at 1 MiB; `responseCap` defaults to 64 KiB.\n\nIn `toiljs dev` the emulator returns `CallFailed` for every `httpCall` rather than reaching the network, so a daemon that polls an external service does nothing locally. Everything else behaves as it does on the edge.\n\n## The `main.daemon.ts` file (a separate tier)\n\nLike streams, daemons live in their **own entry file**, `server/main.daemon.ts`, and compile into their **own artifact**, `build/server/release-cold.wasm`. Importing your `@daemon` module there pulls it into that artifact.\n\n```ts\n// server/main.daemon.ts\nimport { revertOnError } from 'toiljs/server/runtime/abort/abort';\n\nimport './daemon/Jobs'; // add each @daemon module here\n\n// NOTE: unlike main.ts / main.stream.ts, the daemon entry does NOT re-export the\n// request runtime. A daemon artifact exposes its schedule hooks, not the request handler.\nexport function abort(message: string, fileName: string, line: u32, column: u32): void {\n revertOnError(message, fileName, line, column);\n}\n```\n\n`toiljs build` produces `release-cold.wasm` automatically when your project declares a `@daemon` surface:\n\n```sh\n$ ls build/server/*.wasm\nbuild/server/release.wasm # L1 request (@rest / @service)\nbuild/server/release-stream.wasm # L2/L3 stream (@stream)\nbuild/server/release-cold.wasm # L4 daemon (@daemon)\n```\n\nThere is **at most one** `@daemon` class per project, and it is a compile error to put a `@daemon` in the request build. See [Compute tiers](../concepts/tiers.md) for how one source tree becomes three artifacts.\n\n## Worked example: a periodic rollup\n\nA common daemon job: every hour, read a running total and store a small summary so pages can show it with one cheap read. This reads a [counter](../database/counters.md) and writes a summary [document](../database/documents.md); the write only takes effect on the leader.\n\n```ts\n// server/models/StatKey.ts and Summary.ts (@data types)\n@data\nclass StatKey {\n name: string = 'signups';\n constructor(name: string = 'signups') { this.name = name; }\n}\n\n@data\nclass Summary {\n total: u64 = 0;\n updatedAt: u64 = 0;\n}\n\n// server/data/StatsDb.ts\n@database\nclass StatsDb {\n @collection static signups: Counter<StatKey>;\n @collection static summary: Documents<StatKey, Summary>;\n}\n\n// server/daemon/Jobs.ts\n@daemon\nclass Jobs {\n @scheduled('1h')\n rollup(): void {\n const key = new StatKey('signups');\n const total = StatsDb.signups.get(key); // a read (allowed anywhere)\n\n const s = new Summary();\n s.total = total;\n s.updatedAt = <u64>(Date.now() / 1000);\n StatsDb.summary.patch(key, s); // a write: runs only on the leader\n }\n}\n```\n\nNow any request handler can serve the summary with a single keyed read, and it is never more than an hour stale. Because the write is fenced to the leader and the rollup is idempotent (it *sets* the total rather than adding to it), a missed or failed-over tick is harmless: the next hourly run simply refreshes it.\n\n## When not to use a daemon\n\n- **When a user is waiting for the result.** Do that work in a [route](../backend/rest.md) or [RPC](../backend/rpc.md), on the request path.\n- **When you need a fast read-view kept in sync with data changes.** That is a [`@derive`](./derive.md), which runs on every write rather than on a timer.\n- **When you need exactly-once, never-skipped execution.** Scheduling is at-most-once with no backfill, so make tasks idempotent or safe to skip. A daemon is not a guaranteed job queue.\n\n## Related\n\n- [Background overview](./README.md): daemons versus `@derive`, and which to reach for.\n- [Derived views (`@derive`)](./derive.md): keep a read-view in sync on every write.\n- [Compute tiers (L1 to L4)](../concepts/tiers.md): the daemon runs on the L4 global tier.\n- [Counters](../database/counters.md), [Documents](../database/documents.md), [Events](../database/events.md): the data a daemon reads and writes.\n",
14
14
  "background/derive.md": "# Derived views (`@derive`)\n\nA `@derive` keeps a fast, precomputed **View** of your data in sync automatically. You write a method that reads your source data and publishes a summary; toiljs re-runs it whenever the source data changes, so your pages can read the summary with one cheap lookup instead of doing the work on every request.\n\n## The problem it solves\n\nSome reads are expensive. \"Show the 10 newest comments\" means scanning an [events](../database/events.md) log. \"Show the leaderboard\" means totalling up scores. A **scan** (walking many rows to build a result) can fan out across an unbounded amount of data, and that is too slow and too unpredictable to do while a user waits.\n\nSo toiljs **bars scans on the request path**. A [route](../backend/rest.md) handler runs under a restricted mode:\n\n- A `@get` runs as a **query** (reads only, no scans).\n- A `@post` / `@put` / `@patch` / `@del` runs as an **action** (keyed reads and writes, still no scans).\n\nIf you cannot scan in a route, how do you show \"the latest 10\"? You precompute it. A `@derive` does the scan **off** the request path, folds the result into a [View](../database/views.md) (a read-optimized snapshot stored by key), and your route reads that View with a single keyed lookup, which is not a scan and so is allowed.\n\n```mermaid\nflowchart LR\n W[\"POST /guestbook<br/>(action: append + count)\"] --> S[(Events + Counter<br/>source data)]\n S -->|write triggers| D[\"@derive recompute()<br/>scans, builds, publishes\"]\n D --> V[(View<br/>precomputed snapshot)]\n G[\"GET /guestbook<br/>(query: one keyed read)\"] --> V\n```\n\n## A worked example: a guestbook\n\nHere is the whole pattern in one database class: an [events](../database/events.md) log of signatures, a [counter](../database/counters.md) of how many there are, and a [View](../database/views.md) that holds the ready-to-serve page.\n\n```ts\n@data\nclass GuestKey {\n room: string = 'main';\n constructor(room: string = 'main') { this.room = room; }\n}\n\n@database\nclass GuestbookDb {\n @collection static entries: Events<GuestKey, GuestEntry>; // a source: the log\n @collection static totals: Counter<GuestKey>; // a source: the count\n @collection static book: View<GuestKey, GuestbookView>; // the view we publish\n\n // Recompute the view from the sources. It MAY scan and publish; a route may not.\n @derive\n recompute(): void {\n const key = new GuestKey('main');\n const view = new GuestbookView();\n view.total = GuestbookDb.totals.get(key); // a keyed read\n view.entries = GuestbookDb.entries.latest(key, 10); // a scan: allowed here\n GuestbookDb.book.publish(key, view); // publish the snapshot\n }\n}\n```\n\nThe route then writes to the sources and reads the view:\n\n```ts\n@rest('guestbook')\nclass Guestbook {\n @get('/')\n list(): GuestbookView {\n const key = new GuestKey('main');\n const view = GuestbookDb.book.get(key); // one keyed read, not a scan\n return view == null ? new GuestbookView() : view; // empty until first publish\n }\n\n @post('/')\n sign(input: NewMessage): GuestbookView {\n const key = new GuestKey('main');\n GuestbookDb.entries.append(key, new GuestEntry(input.author, input.message, 0));\n GuestbookDb.totals.add(key, 1);\n // The @derive republishes `book` right after this action returns, so GET\n // serves the new entry. The action just acks with the new total (a counter\n // read is allowed here; scanning the entries list is not).\n const view = new GuestbookView();\n view.total = GuestbookDb.totals.get(key);\n return view;\n }\n}\n```\n\nSign the guestbook twice and the total climbs across requests, because the data lives in the database (and its view), not in module memory.\n\n## What a `@derive` may do\n\nA `@derive` runs under a special **derive** mode that is more powerful than a route:\n\n| Ability | Query (`@get`) | Action (`@post` ...) | Derive |\n| -------------------------------------------- | :------------: | :------------------: | :----------: |\n| Keyed reads (`.get`, counter total) | yes | yes | yes |\n| Writes (`create`, `patch`, `add`, `append`) | no | yes | yes |\n| **Scans** (`events.latest`, membership list) | **no** | **no** | **yes** |\n| `view.publish` / `view.append` | no | no | yes |\n\nSo a derive is exactly the place to do the reads and scans a route cannot, and to publish the result.\n\n## Declaring a derive\n\nA `@derive` is a method on your [`@database`](../database/setup.md) class, next to the collections it reads and the View it writes.\n\n```ts\n@database\nclass MyDb {\n @collection static events: Events<Key, Fact>; // a source\n @collection static home: View<Key, HomePage>; // the materialized view\n\n @derive\n rebuild(): void {\n // read sources, build the value, publish it\n }\n}\n```\n\nRules:\n\n- A `@derive` method takes **no arguments and returns `void`**.\n- A database may declare **multiple** `@derive` methods; each runs independently.\n- The View value and its key are ordinary [`@data`](../backend/data.md) types, so they round-trip through the codec like any other stored value.\n\n## When a derive runs\n\nYou never call a derive yourself. The runtime runs it for you at two moments:\n\n1. **After a write to a source.** When a request writes one of the database's source collections (an `events.append`, a `counter.add`, a document `create` or `patch`), that database's derives run **right after the response is produced**, so the view reflects the new data on the next read. Many writes to one database in a single request are **coalesced** into one recompute (it does not run once per write).\n\n2. **On box load.** When a server box starts, hot-reloads, or notices the underlying data changed out of band, the views are rebuilt from their sources **before the first read is served**. This is also where a value type's [`@migrate`](../backend/data.md) runs against old stored events, as the derive re-reads and republishes them.\n\nA derive's own `view.publish` never re-triggers it, so there is no infinite loop.\n\nThe same code runs under `toiljs dev` (the in-process emulator) and on the production edge, with no flags or wiring to change.\n\n## Folding a growing log incrementally\n\nThe guestbook above uses `events.latest` and **recomputes** the view from scratch on every change. That is simple, correct, and the right default for a **bounded** read: the latest N, a counter total, a small set.\n\nSome views instead fold an **unbounded** log: a running total over every event ever, an activity rollup, an audit summary. Rescanning the whole log on every change gets slower as it grows. For those, read the source with [`events.since`](../database/events.md) instead of `latest`. `since` hands you only the events you have **not folded yet**, so the derive folds forward incrementally.\n\n```ts\n@derive\nrollup(): void {\n const key = new StatsKey('all');\n const view = StatsDb.summary.get(key) ?? new Summary(); // the running view so far\n let batch = StatsDb.events.since(key, 500);\n while (batch.length > 0) { // drain the new events in bounded batches\n for (let i = 0; i < batch.length; i++) view.apply(batch[i]);\n batch = StatsDb.events.since(key, 500);\n }\n StatsDb.summary.publish(key, view);\n}\n```\n\nThe host owns the cursor, you never manage it: it seeds `since` from a durable checkpoint, advances it as it hands you events, and saves it **only after** your `publish` lands (so a crash re-folds the batch instead of skipping it). On the next trigger the derive resumes exactly where it left off.\n\n**One rule: the fold must be idempotent per event.** A rare crash-recovery case (an event that \"heals\" at an older position after the derive already passed it) makes the host re-read the whole log that one run, so applying an event twice must not change the result. Use set-style updates keyed by the event's id, not blind `count += 1` accumulation. If your fold cannot be idempotent, stay on the simple `latest` recompute path.\n\n## Guarantees and limitations\n\n**Guarantees**\n\n- **It converges to a correct snapshot.** Publishes are last-writer-wins (the host versions each publish so a later one always supersedes an earlier one), and a derive recomputes from the source of truth, so the view always ends up matching its sources.\n- **Reads stay cheap.** A route serves the view with a single keyed lookup, never a scan.\n\n**Limitations (read these)**\n\n- **By default it recomputes from scratch each time.** A derive re-reads its sources and republishes on every trigger. That is the simple path and a great fit for a **bounded** read: the latest N, a counter total, a small set. To fold an **unbounded, ever-growing** log efficiently, read it with [`events.since`](../database/events.md) instead of `latest` (see [Folding a growing log incrementally](#folding-a-growing-log-incrementally) above); it folds only the new events since a checkpoint.\n- **It is eventually consistent, by a moment.** The view is republished right after the writing request finishes, so there is a tiny window where a reader could see the pre-write view. For most pages (a feed, a leaderboard) that is invisible and fine.\n- **`view.publish` is derive-only.** Routes cannot publish; they can only read the view. That is the whole point: the expensive build happens off the request path.\n\n## When not to use a derive\n\n- **When the read is already cheap.** If a route can answer with a plain keyed `.get`, you do not need a view at all.\n- **When you need it on a timer, not on a data change.** Recomputing \"yesterday's report\" at 2am is a [daemon](./daemons.md) job, not a derive (a derive is triggered by writes, not by the clock).\n- **When the source is an unbounded full-history fold.** See the limitation above; keep derives to bounded reads.\n\n## Related\n\n- [Views](../database/views.md): the `View<K, V>` family a derive publishes into, and how to read it.\n- [Events](../database/events.md): the append-only log a derive commonly folds into a view.\n- [Counters](../database/counters.md): running totals a derive can read.\n- [Background overview](./README.md): `@derive` versus `@daemon`, and which to reach for.\n- [Data types (`@data`)](../backend/data.md): the value and key types a view stores.\n",
15
15
  "background/README.md": "# Background work\n\nBackground work is code that runs **without a user waiting for it**: on a timer, once globally, or automatically after your data changes. toiljs gives you two tools for it, `@daemon` and `@derive`, and this page helps you pick the right one.\n\n## Why background work\n\nMost of your server code runs **because a user asked**: a browser hits a [route](../backend/rest.md) or calls an [RPC](../backend/rpc.md), your handler runs, and a response goes back. But some work does not belong on that path:\n\n- It should happen **on a schedule** (every hour, every night at 2am), not when a request happens to arrive.\n- It would be **too slow** to do inside a request, so you want it precomputed and ready.\n- It must happen **exactly once across the whole world**, not once per user and not once per server.\n\nThat is background work. In toiljs there are two kinds, and they solve two different problems.\n\n## The two tools\n\n```mermaid\nflowchart TD\n Q{What do you need?}\n Q -->|Run on a timer, or once globally| D[\"@daemon + @scheduled<br/>a single background worker\"]\n Q -->|Keep a fast read-view in sync with your data| V[\"@derive<br/>maintains a materialized View\"]\n D --> DD[\"e.g. nightly cleanup, hourly poll,<br/>periodic rollup\"]\n V --> VV[\"e.g. a leaderboard, a 'latest 10' feed,<br/>a home-page summary\"]\n```\n\n### `@daemon`: a scheduled, global worker\n\nA [daemon](./daemons.md) is one long-lived background worker for your whole app. It runs on a **schedule** you set (an interval like every 5 minutes, or a cron time like \"9:15 on weekdays\"), and there is exactly **one** of it worldwide at any moment (a \"singleton\"). Use it for work that is driven by **time** or that must run **once globally**:\n\n- clean up stale rows every night;\n- poll an external API every few minutes;\n- send a daily digest email;\n- roll up yesterday's numbers into a summary.\n\n### `@derive`: keep a read-view up to date\n\nA [derive](./derive.md) is not on a timer. It runs **automatically whenever the data it depends on changes**, and its job is to keep a precomputed **View** (a read-optimized copy of your data) fresh. Use it when a page needs data that is **expensive to compute on every read**, like a leaderboard or a \"latest 10 comments\" list, so the read itself stays a single cheap lookup:\n\n- fold an [events](../database/events.md) log into a \"latest N\" list;\n- total up [counters](../database/counters.md) into a scoreboard;\n- assemble a home-page summary from several sources.\n\n## Which one do I reach for?\n\n| Question | Use |\n| ----------------------------------------------------- | ------------ |\n| \"Run this every hour / at midnight.\" | `@daemon` |\n| \"Do this once for the whole app, not per server.\" | `@daemon` |\n| \"Poll or call an outside service on a schedule.\" | `@daemon` |\n| \"This page's data is too slow to compute per request.\"| `@derive` |\n| \"Keep a leaderboard / feed in sync as data changes.\" | `@derive` |\n\nA simple rule of thumb: if the trigger is **the clock**, use a `@daemon`. If the trigger is **a change to your data** and the goal is a fast read, use a `@derive`.\n\nThey also combine well. A `@daemon` might do a heavy nightly aggregation and write a summary row, while a `@derive` keeps a small live view fresh on every write. They are different tiers of the edge and are covered on their own pages.\n\n## `@job`: the widest database surface for background work\n\n`@daemon` and `@derive` decide **where and when** background code runs. `@job` answers a different question: **what a function is allowed to do to the database.** It is one of the ToilDB **function kinds** (the same family as `@query`, `@action`, and `@derive`), and it grants the **widest** data surface of them all.\n\nA **function kind** is a label the compiler puts on a backend function to gate which database operations it may issue. This is a safety rail: a read-only endpoint physically cannot write, and an expensive scan cannot run on the hot request path. The kinds line up from narrowest to widest:\n\n| Kind | Typical trigger | Point reads | **Scan** (`latest`, membership `list`) | Writes | `publish` a View |\n| ---------- | ------------------------- | ----------- | -------------------------------------- | ----------------- | ---------------- |\n| `@query` | a `@get` / plain `@remote`| yes | no | no | no |\n| `@action` | a `@post` / `@action` | yes | no | yes (bounded) | no |\n| `@derive` | your data changed | yes | yes | append / counter add only | yes |\n| `@job` | you drive it (background) | yes | yes | yes (all) | yes |\n\nA **scan** is a read that can fan out across many rows (like \"the newest 50 events\" via `events.latest`, or \"every member of this set\" via `membership.list`). Scans are **barred from request handlers** because a request must stay fast and bounded. `@derive` and `@job` run **off the request path**, so they are the only kinds allowed to scan.\n\n### When to reach for `@job`\n\nUse `@job` when a piece of background work needs **more** database power than the default:\n\n- it must **scan** (fold `events.latest`, walk `membership.list`), and/or\n- it must **publish a View** *while also* doing arbitrary writes (`create`, `patch`, `delete`). A `@derive` can publish a View but cannot `patch` or `delete` a record; a `@job` can do everything.\n\nThe compiler accepts `@job` on a method, including a `@daemon`'s `@scheduled` method. Tag a scheduled task `@job` when it needs that full surface (say, a nightly repair that scans a log, fixes rows, and republishes a summary View). A plain, **untagged** `@scheduled` method runs with the **`@action`** surface instead: point reads plus bounded writes, which is all most rollups and cleanups need.\n\n```ts\n@daemon\nclass Jobs {\n // A plain scheduled method: point reads + bounded writes (the @action surface).\n @scheduled('1h')\n rollup(): void { /* get a counter, patch a summary row */ }\n\n // A scheduled method that also needs SCANS and to PUBLISH a View: tag it @job.\n @scheduled('1d')\n @job\n nightlyRepair(): void {\n // @job unlocks scan-class reads (events.latest / membership.list)\n // and publishing a View, on top of ordinary reads and writes.\n }\n}\n```\n\n### `@job` versus `@derive`\n\nThey overlap (both run off the request path, both may scan, both may publish a View), but they are triggered differently and sized differently:\n\n- **`@derive`** is **change-triggered**: it re-runs automatically whenever its source data changes, and its narrow job is to keep one **View** in sync. It cannot `create` / `patch` / `delete` records (only append, counter-add, and publish). Reach for it when a read is too slow and you want it kept fresh on every write.\n- **`@job`** is **you-drive-it** background work (typically a `@scheduled` daemon method) with the **full** write surface. Reach for it when the work is clock-driven or one-off and needs to both scan and mutate freely.\n\n> **Rule of thumb.** Clock-driven and needs the full database surface: a `@job` (usually inside a `@daemon`). Change-driven and only maintains a View: a `@derive`. Clock-driven with modest reads and writes: a plain `@scheduled` method (no `@job` needed).\n\nFor the complete permission grid see the [function-kind matrix](../database/setup.md#how-access-is-gated-query-action-and-friends), and for the decorator itself see [every decorator](../concepts/decorators.md#database-function-kinds-data-access-policy).\n\n## Related\n\n- [Daemons and scheduled jobs](./daemons.md): `@daemon`, `@scheduled`, interval vs cron, and how a single global worker fails over safely.\n- [Derived views (`@derive`)](./derive.md): keeping a materialized View in sync with its source data.\n- [Compute tiers (L1 to L4)](../concepts/tiers.md): where daemons and request handlers each run on the edge.\n- [Views](../database/views.md) and [Events](../database/events.md): the database families a `@derive` reads from and writes to.\n",
16
- "cli/README.md": "# The toiljs CLI\n\nThe `toiljs` command is how you scaffold, run, build, self-host, and diagnose a toiljs app. This page lists every command and every flag, with copy-pasteable examples.\n\n## What it is\n\nWhen you install the `toiljs` package, it adds one executable to your project: `toiljs`. Everything you do day to day (start the dev server, produce a production build, check your setup) goes through it.\n\nYou almost never type the full path. A freshly scaffolded project already has npm scripts that call it, so you run `npm run dev` and `npm run build`. When you want a command that has no script (like `doctor` or `db`), run it with `npx`:\n\n```bash\nnpx toiljs doctor\n```\n\n`toiljs` needs **Node.js 24 or newer** (older versions will not run it).\n\n## Command overview\n\n| Command | What it does |\n| --- | --- |\n| `toiljs create [name]` | Scaffold a brand new toiljs app in a new folder. |\n| `toiljs dev` | Start the local development server with hot reload. |\n| `toiljs build` | Produce the optimized production build (client bundle + server WebAssembly). |\n| `toiljs start` | Self-host the built app on a fast production HTTP server. |\n| `toiljs configure` | Turn styling features (Sass/Less/Stylus, Tailwind) on or off in an existing project. |\n| `toiljs doctor` | Diagnose your project setup and (with `--fix`) repair common wiring. |\n| `toiljs update` | Check npm for newer dependency versions and apply the ones you pick. |\n| `toiljs db <action>` | Inspect, reset, snapshot, or restore the local dev database. |\n| `toiljs help` | Print the built-in help. Also `--help` or `-h`. |\n| `toiljs --version` | Print the installed toiljs version. Also `-v`. |\n\nRun `toiljs help` any time to see this list in your terminal.\n\n## Global behavior\n\nA few things apply to every command.\n\n- **`--root <dir>`**: run the command against a project in another directory instead of the current one. Every command accepts it.\n- **Automatic update check**: on every run (including `npm run dev`, which calls the CLI under the hood), toiljs quietly asks the npm registry whether a newer `toiljs` exists and prints a one-line notice on stderr if you are behind. It never blocks or slows the command in a meaningful way (the answer is cached for an hour and the network call is capped at two seconds). To turn it off, set the environment variable `TOILJS_NO_UPDATE_CHECK=1`. It also respects the common `NO_UPDATE_NOTIFIER` and `CI` variables.\n\n```bash\n# Run any command against a project in another folder.\ntoiljs build --root ./apps/marketing\n\n# Silence the \"newer version available\" notice.\nTOILJS_NO_UPDATE_CHECK=1 toiljs dev\n```\n\n## `toiljs create`\n\nScaffolds a new project into a new folder. By default it is interactive: it asks a short series of questions (project name, template, styling, and so on), then writes the files and installs dependencies. Pass flags to skip questions, or `-y` to accept every default and run with no questions at all.\n\n```bash\n# Interactive: answer the prompts.\nnpx toiljs create\n\n# Give it a name up front.\nnpx toiljs create my-app\n\n# Fully non-interactive (great for scripts and CI).\nnpx toiljs create my-app --yes --template app --style css\n```\n\n### What it sets up\n\nEvery new project comes wired for you: the enforced TypeScript 7, Oxlint + tsgolint, and Prettier presets, file-based routing, a `toil.config.ts`, a `toilconfig.json` (the server compiler settings), a `.gitignore`, and the native TypeScript 7 editor settings. It also scaffolds a `server/migrations/` folder (where ToilDB schema migrations live) and, unless you opt out, a set of AI assistant helper files.\n\nThe scaffolded `package.json` includes these scripts:\n\n| Script | Runs |\n| --- | --- |\n| `npm run dev` | `toiljs dev` |\n| `npm run build` | `toiljs build` |\n| `npm run build:server` | `toiljs build --server` |\n| `npm run lint` | `oxlint --type-aware client` |\n| `npm run typecheck` | `tsc --noEmit` |\n| `npm run format` | `prettier --write ...` |\n\n### Generated docs and AI-assistant pointers\n\nEvery project carries a full copy of this documentation set at `.toil/docs/`. You do not maintain it: toiljs regenerates it from the installed toiljs version on every `toiljs dev` and `toiljs build`, so it always matches the version you are on. Do not edit those files by hand (your changes are overwritten on the next dev or build).\n\nUnless you opt out, `toiljs create` also writes small **pointer files** at the project root that tell AI coding assistants to read `.toil/docs/` before touching the project: `CLAUDE.md` (Claude Code), `AGENTS.md` (Codex and others), `.cursor/rules/toiljs.mdc` (Cursor), and `.github/copilot-instructions.md` (GitHub Copilot). These are written once, committed, and yours to edit. Control them with `--ai` / `--no-ai` (when you pass neither, `create` asks).\n\n### create options\n\n| Flag | Meaning |\n| --- | --- |\n| `[name]` | The project folder name (a positional argument). If omitted, you are asked. |\n| `-t, --template <app\\|minimal>` | `app` is the full starter (landing page, layout, styles, demo routes). `minimal` is just a layout and a home route. Default `app`. |\n| `--style <css\\|sass\\|less\\|stylus>` | Which CSS flavor to set up. Default `css` (plain CSS). |\n| `--tailwind` / `--no-tailwind` | Add or skip Tailwind CSS (v4). Off by default. |\n| `--ai` / `--no-ai` | Include or skip AI assistant files (like `CLAUDE.md`). When omitted, you are asked. |\n| `--images` / `--no-images` | Enable or skip build-time image optimization. On by default. |\n| `--git` / `--no-git` | Initialize a git repository. When omitted, you are asked (default yes). |\n| `--install` / `--no-install` | Install dependencies after scaffolding. When omitted, you are asked (default yes). |\n| `--pm <npm\\|pnpm\\|yarn\\|bun>` | Which package manager to install with. Default `npm`. |\n| `-y, --yes` | Accept all defaults and skip every prompt. |\n\nFor a walkthrough, see [Create a project](../getting-started/create-project.md).\n\n## `toiljs dev`\n\nStarts the local development server with hot reload, so you edit a file and the browser updates in place. This is the command you leave running while you build.\n\n```bash\nnpm run dev\n# or, to pick a port:\nnpx toiljs dev --port 4000\n```\n\n### What the dev server emulates\n\nThe whole point of the dev server is to run your app locally the same way the real Dacely edge runs it in production, so what you see locally is what you ship. It emulates three things:\n\n1. **The edge.** For a project with a server (any project with a `toilconfig.json`), a small local HTTP server takes the public port and dispatches incoming requests into your compiled `server.wasm` using the exact same request envelope the production edge uses. Anything your server does not claim (page routes, static assets, the hot-reload websocket) is proxied to Vite behind the scenes. So your React frontend and your WebAssembly backend run together, on one URL, exactly like production.\n2. **The database.** ToilDB runs in-process as a local emulator. Every family (documents, views, unique, events, counters, membership, capacity) works, and the data is written to `.toil/devdata.json` so it survives restarts. You manage that file with [`toiljs db`](#toiljs-db).\n3. **The host functions.** The platform services your server code calls (email, environment variables and secrets, time, crypto, rate limiting, auth) are wired up locally so they behave like the edge. Email actually sends if you configure a provider (see [Email](../services/email.md)).\n\nA **client-only** project (no `toilconfig.json`) just gets the plain Vite dev server on your port, unchanged.\n\n### How hot reload works\n\n```mermaid\nflowchart TD\n E[\"You edit a file\"] --> Q{\"Which file?\"}\n Q -->|\"client/ (React, CSS)\"| V[\"Vite hot-swaps it in the browser\"]\n Q -->|\"server/ (a @rest / @data / @service file)\"| S[\"toilscript recompiles the server\"]\n S --> G[\"shared/server.ts is regenerated\"]\n G --> V2[\"Vite hot-swaps the new typed client\"]\n S --> W[\"the dev server hot-swaps the new server.wasm\"]\n V --> B[\"Browser updates, no full reload\"]\n V2 --> B\n W --> B\n```\n\nClient edits go straight through Vite's hot module replacement. Server edits trigger a toilscript rebuild: toiljs recompiles your backend, regenerates `shared/server.ts` (the typed client the browser imports to call your server), and hot-swaps the recompiled WebAssembly, all without you touching the browser. Rebuilds are debounced (grouped over about 150 milliseconds) so a \"save all\" or a formatter pass does not trigger a storm of builds.\n\nIf your project has an `emails/` folder, the dev server also prints an email-preview URL at `/__toil/emails`.\n\n### dev options\n\n| Flag | Meaning |\n| --- | --- |\n| `--port <n>` | Port to listen on. Default `3000` (or `client.port` from your config). |\n| `--root <dir>` | Run against a project in another directory. |\n\nPress `Ctrl+C` to stop. toiljs restores your terminal and force-exits even if a native listener is slow to close, so you never end up with an orphaned dev server rebuilding in the background.\n\n## `toiljs build`\n\nProduces the optimized production build. Run this before you deploy or before `toiljs start`.\n\n```bash\nnpm run build\n# or build only the server:\nnpx toiljs build --server\n```\n\n### What it produces\n\nA full build runs in a careful order so the pieces line up:\n\n```mermaid\nflowchart LR\n A[\"Your project\"] --> S[\"1. Compile the server<br/>(toilscript)\"]\n S --> W[\"build/server/release.wasm\"]\n S --> R[\"shared/server.ts<br/>(typed client, regenerated)\"]\n R --> C[\"2. Bundle the client<br/>(Vite)\"]\n C --> O[\"build/client/<br/>(HTML, JS, CSS, assets)\"]\n C --> X[\"3. Prerender + SEO<br/>(SSG pages, robots.txt,<br/>sitemap.xml, llms.txt)\"]\n```\n\n1. **The server is built first.** toilscript compiles every decorated server file (not just the entry) into `build/server/release.wasm`, and regenerates `shared/server.ts`. Doing this first means the client always bundles against a current, correct typed server client. A project that also declares `@stream` or `@daemon` surfaces compiles those into their own artifacts (`release-stream.wasm`, `release-cold.wasm`).\n2. **The client is bundled** by Vite into `build/client/` (your HTML, JavaScript, CSS, and optimized assets). The dev toolbar and error overlay are stripped out of the production bundle.\n3. **Static pages and SEO files are generated**: any route that opts into static generation is prerendered to HTML, and if you configured `client.seo`, toiljs writes `robots.txt`, `sitemap.xml`, and `llms.txt`. Routes that opt into server-side rendering get their HTML-with-holes templates baked for the edge.\n\n### build options\n\n| Flag | Meaning |\n| --- | --- |\n| `--server` | Build **only** the server (recompile the wasm and regenerate `shared/server.ts`), and skip the client bundle. Fast when you only touched backend code. This is what `npm run build:server` runs. |\n| `--root <dir>` | Run against a project in another directory. |\n\nA client-only project (no `toilconfig.json`) skips step 1 and just bundles the client.\n\n## `toiljs start`\n\nSelf-hosts the app you just built, on a fast production HTTP server (hyper-express, backed by uWebSockets.js). It serves your static client, runs your `server.wasm` for dynamic requests, does server-side rendering, supports daemons, and exposes a `/_toil` websocket channel. Use it to run your app on your own machine or server instead of deploying to the Dacely edge.\n\n```bash\nnpm run build # start needs a build to serve\nnpx toiljs start\nnpx toiljs start --port 8080 --host 0.0.0.0 --threads 4\n```\n\n`start` fails fast if there is no build yet (it looks for `build/client/index.html`), so run `toiljs build` first.\n\n### start options\n\n| Flag | Meaning |\n| --- | --- |\n| `--port <n>` | Port to listen on. Default `3000` (or `client.port`). |\n| `--host <host>` | Address to bind. Default `127.0.0.1` (loopback only). Pass `0.0.0.0` to accept connections from other machines. |\n| `--threads <n>` | Number of HTTP worker processes. Default is automatic (one per available CPU). Pass `1` to disable the worker pool. `--workers` is an accepted alias. This can also be set as `server.threads` in your config. |\n| `--root <dir>` | Run against a project in another directory. |\n\n## `toiljs configure`\n\nToggles a project's client styling features (the CSS preprocessor and Tailwind) after the fact, on an existing app. It detects your current setup, asks what you want, then rewrites the stylesheets and your app entry's imports, edits `package.json`, and syncs `node_modules` so removed packages are actually uninstalled.\n\n```bash\n# Interactive.\nnpx toiljs configure\n\n# Non-interactive: switch to Sass and turn Tailwind on.\nnpx toiljs configure --style sass --tailwind\n```\n\n### configure options\n\n| Flag | Meaning |\n| --- | --- |\n| `--style <css\\|sass\\|less\\|stylus>` | Switch the CSS preprocessor. |\n| `--tailwind` / `--no-tailwind` | Turn Tailwind on or off. |\n| `--images` / `--no-images` | Turn build-time image optimization on or off (sets `client.images` in your config). |\n| `--no-install` | Edit the files but do not run the package manager. You then run install yourself. |\n| `--root <dir>` | Run against a project in another directory. |\n\nPassing any of `--style`, `--tailwind`, or `--images` makes the command non-interactive (it skips the prompts you did not answer with a flag). See [Styling](../frontend/styling.md) for the full picture.\n\n## `toiljs doctor`\n\nRead-only project diagnostics. It gathers facts from disk (your `package.json`, lockfiles, the resolved config, your app entry, `index.html`, your routes, and the server target), runs a set of checks, and prints a grouped report. It never changes anything unless you pass `--fix`, and it never crashes on a partial or non-toiljs project (missing pieces just become warnings or failures). It exits with a non-zero status when any check **fails** (warnings do not fail), so it is safe to run in CI.\n\n```bash\n# Human-readable report.\nnpx toiljs doctor\n\n# Machine-readable, for CI.\nnpx toiljs doctor --json\n\n# Auto-repair the common wiring.\nnpx toiljs doctor --fix\n```\n\n### What it checks\n\nThe report is grouped:\n\n| Group | Example checks |\n| --- | --- |\n| **Environment** | Node.js version, that `toiljs` and its peer dependencies (React and so on) are installed and new enough, that TypeScript 7 is installed and its declared range stays within 7.x, that Oxlint and tsgolint are configured for type-aware linting, that a lockfile exists, and that your scripts do not wrap `toiljs` in a stray `npx`. |\n| **Project + routing** | The `client/` and `routes/` folders exist, `index.html` has a `<div id=\"root\">`, your app entry calls `mount(...)` with the `slots` argument, at least one route exists, no two routes collide on the same URL, and no asset paths are written in a way that 404s on nested routes. |\n| **Config + assets** | Your `toil.config` loads, the base path is well formed, `client.seo` has a `url` if SEO is configured, and your styling packages are actually installed. |\n| **Server / WASM** | The `toilconfig.json` and its entry files exist, `toilscript` is installed, a compiled `.wasm` exists, the typed-RPC wiring is in place, your `@rest` controllers are actually dispatched, the Prettier and editor plugins are wired, and a `migrations/` folder exists. |\n| **Security** | If your server uses auth, whether `AUTH_SESSION_SECRET` is set (an unset secret means sessions fall back to a published dev key, which is forgeable). |\n\n### What `--fix` repairs\n\n`--fix` migrates unsupported or missing TypeScript declarations to `^7.0.2` and enables the native TypeScript editor in any project. The rest only touch a server project (one with a `toilconfig.json`), repairing the wiring that is easy to get wrong or that older projects predate:\n\n- adds `--rpcModule shared/server.ts` to your server build scripts,\n- adds `shared` and the `shared/*` path alias to `tsconfig.json`,\n- adds `shared/server.ts` to `.gitignore`,\n- lifts the `toilscript` version floor if it is too old,\n- adds the `toiljs/prettier-plugin` to your Prettier config (so Prettier does not choke on server decorators),\n- removes the legacy toilscript JavaScript language-service plugin, which cannot run in the TypeScript 7 native language server; `toiljs build --server` provides the WebAssembly dialect diagnostics,\n- refreshes the editor-only server globals declaration file.\n\nIt is idempotent: it only writes files it actually needs to change, and it tells you which ones changed and which need a manual edit (for example a `tsconfig.json` that contains comments). If it changed `package.json`, run your installer afterward.\n\n### doctor options\n\n| Flag | Meaning |\n| --- | --- |\n| `--json` | Emit machine-readable JSON instead of the human report (the banner is suppressed so stdout stays valid JSON). |\n| `--fix` | Repair the server wiring in place, as above. |\n| `--root <dir>` | Run against a project in another directory. |\n\n## `toiljs update`\n\nA friendly wrapper over `npm-check-updates`. It checks the registry for newer versions of your dependencies, groups them by how big the jump is (major, minor, patch), lets you pick which to apply (or `-y` to apply all), bumps `package.json`, and runs your package manager's install. It also makes sure your `server/migrations/` folder exists (older projects predate it). `npm-check-updates` runs via `npx`, so it never becomes a permanent dependency of your project.\n\nTypeScript updates must stay within supported **7.x** releases. Older and future compiler majors,\nunbounded ranges, and prereleases are withheld. An older project is offered the latest 7.x migration\neven with `--target patch`. Select that migration before applying other updates. The command writes\nthe exact reviewed versions before installing, so a second registry query cannot substitute a\ndifferent compiler major.\n\n```bash\n# Interactive picker.\nnpx toiljs update\n\n# Apply everything, non-interactively.\nnpx toiljs update --yes\n\n# Only patch-level updates.\nnpx toiljs update --target patch\n```\n\n### update options\n\n| Flag | Meaning |\n| --- | --- |\n| `-y, --yes` | Apply all available updates without the picker. |\n| `--target <latest\\|minor\\|patch\\|newest\\|greatest>` | How far to bump. Default `latest`. |\n| `--root <dir>` | Run against a project in another directory. |\n\n## `toiljs db`\n\nManages the local dev database: the on-disk ToilDB store your dev server writes to `.toil/devdata.json`. Use it to inspect data, wipe a corrupt state, save a snapshot to share as a fixture, or restore one. The snapshot is exactly the JSON the dev database uses, so an exported file imports cleanly.\n\n```bash\n# See what is stored.\nnpx toiljs db status\n\n# Wipe all dev data.\nnpx toiljs db reset\n\n# Save a snapshot (to a file, or to stdout if you omit the file).\nnpx toiljs db export fixture.json\n\n# Restore a snapshot.\nnpx toiljs db import fixture.json\n\n# Print the on-disk path (scriptable).\nnpx toiljs db path\n```\n\n### db actions\n\n| Action | What it does |\n| --- | --- |\n| `status` (alias `info`) | Show the database path, its size, and per-family row counts. |\n| `reset` (alias `purge`) | Delete all dev data (removes `devdata.json`). |\n| `export [file]` | Write a formatted snapshot to `file`, or to stdout if you omit it (pipe-friendly). |\n| `import <file>` | Replace the dev database with the snapshot in `file`. It refuses a file that is not a valid snapshot. |\n| `path` | Print the `devdata.json` path and nothing else. |\n\nThe dev database is per-project and lives under `.toil/`, which is gitignored. See [Database setup](../database/setup.md) for what actually populates it.\n\n## The dev, build, and deploy flow\n\nHere is how the commands fit together across your workflow.\n\n```mermaid\nflowchart LR\n subgraph Develop\n D[\"toiljs dev<br/>(hot reload, local edge + DB)\"]\n end\n subgraph Ship\n B[\"toiljs build<br/>(client bundle + server.wasm)\"]\n end\n subgraph Run\n ST[\"toiljs start<br/>(self-host locally)\"]\n DP[\"deploy to the Dacely edge\"]\n end\n D --> B\n B --> ST\n B --> DP\n```\n\n- **Develop** with `toiljs dev`. Everything runs locally and reloads as you type.\n- **Ship** with `toiljs build`. This produces the artifacts: the client bundle in `build/client/` and the server WebAssembly in `build/server/`.\n- **Run** the build two ways: `toiljs start` self-hosts it on your own machine, or you deploy the same build to the Dacely edge to serve it worldwide.\n\n## Gotchas\n\n- **`start` needs a build.** `toiljs start` serves what is in `build/`. Run `toiljs build` first, or it exits with an error.\n- **Run `toiljs`, not `npx toiljs`, inside npm scripts.** Under `npm run`, `node_modules/.bin` is already on your PATH, so an extra `npx` layer is redundant and can leave your terminal in a broken input mode after `Ctrl+C`. `doctor` warns about this; the scaffold gets it right.\n- **`--host` and `--threads` are `start`-only.** `toiljs dev` always binds locally and does not take `--host` or `--threads`.\n- **`--server` builds the backend only.** Use it while iterating on server code, but run a full `toiljs build` before you deploy so the client bundle is current.\n- **The update notice is not an error.** The \"newer toiljs available\" line is informational and prints to stderr. Set `TOILJS_NO_UPDATE_CHECK=1` to hide it.\n\n## Related\n\n- [Installation](../getting-started/installation.md) and [Create a project](../getting-started/create-project.md)\n- [Project structure](../getting-started/project-structure.md)\n- [Configuration reference (`toil.config.ts`)](../concepts/config.md)\n- [Styling](../frontend/styling.md)\n- [Database setup](../database/setup.md) and the [database overview](../database/README.md)\n- [Compute tiers (L1 to L4)](../concepts/tiers.md)\n",
16
+ "cli/README.md": "# The toiljs CLI\n\nThe `toiljs` command is how you scaffold, run, build, self-host, and diagnose a toiljs app. This page lists every command and every flag, with copy-pasteable examples.\n\n## What it is\n\nWhen you install the `toiljs` package, it adds one executable to your project: `toiljs`. Everything you do day to day (start the dev server, produce a production build, check your setup) goes through it.\n\nYou almost never type the full path. A freshly scaffolded project already has npm scripts that call it, so you run `npm run dev` and `npm run build`. When you want a command that has no script (like `doctor` or `db`), run it with `npx`:\n\n```bash\nnpx toiljs doctor\n```\n\n`toiljs` needs **Node.js 24 or newer** (older versions will not run it).\n\n## Command overview\n\n| Command | What it does |\n| --- | --- |\n| `toiljs create [name]` | Scaffold a brand new toiljs app in a new folder. |\n| `toiljs dev` | Start the local development server with hot reload. |\n| `toiljs build` | Produce the optimized production build (client bundle + server WebAssembly). |\n| `toiljs start` | Self-host the built app on a fast production HTTP server. |\n| `toiljs configure` | Turn styling features (Sass/Less/Stylus, Tailwind) on or off in an existing project. |\n| `toiljs doctor` | Diagnose your project setup and (with `--fix`) repair common wiring. |\n| `toiljs update` | Check npm for newer dependency versions and apply the ones you pick. |\n| `toiljs db <action>` | Inspect, reset, snapshot, or restore the local dev database. |\n| `toiljs generate auth` | Generate private auth keys and the matching client public key, preserving existing credentials. |\n| `toiljs help` | Print the built-in help. Also `--help` or `-h`. |\n| `toiljs --version` | Print the installed toiljs version. Also `-v`. |\n\nRun `toiljs help` any time to see this list in your terminal.\n\n## Global behavior\n\nA few things apply to every command.\n\n- **`--root <dir>`**: run the command against a project in another directory instead of the current one. Every command accepts it.\n- **Automatic update check**: on every run (including `npm run dev`, which calls the CLI under the hood), toiljs quietly asks the npm registry whether a newer `toiljs` exists and prints a one-line notice on stderr if you are behind. It never blocks or slows the command in a meaningful way (the answer is cached for an hour and the network call is capped at two seconds). To turn it off, set the environment variable `TOILJS_NO_UPDATE_CHECK=1`. It also respects the common `NO_UPDATE_NOTIFIER` and `CI` variables.\n\n```bash\n# Run any command against a project in another folder.\ntoiljs build --root ./apps/marketing\n\n# Silence the \"newer version available\" notice.\nTOILJS_NO_UPDATE_CHECK=1 toiljs dev\n```\n\n## `toiljs generate auth`\n\n```bash\ntoiljs generate auth\ntoiljs generate auth --root ./apps/marketing\n```\n\nSaves private authentication keys in `.env.secrets` (mode `0600`) and\n`VITE_AUTH_KEM_PUBLIC_KEY` in `.env`, and adds both files to `.gitignore`. It preserves\nunrelated settings and existing credentials, and can recover a missing public key from the\nexisting private key. It refuses invalid or mismatched keys rather than rotating them.\nNo secret values are printed. Use this for initial setup or completing an existing key set.\n\nPin the public key in your client's auth options, build with that `.env` on the build machine,\nand install the private settings in your backend's secure env store. See\n[auth configuration](../auth/configuration.md) for the client snippet and toil-backend deployment steps.\n\n## `toiljs create`\n\nScaffolds a new project into a new folder. By default it is interactive: it asks a short series of questions (project name, template, styling, and so on), then writes the files and installs dependencies. Pass flags to skip questions, or `-y` to accept every default and run with no questions at all.\n\n```bash\n# Interactive: answer the prompts.\nnpx toiljs create\n\n# Give it a name up front.\nnpx toiljs create my-app\n\n# Fully non-interactive (great for scripts and CI).\nnpx toiljs create my-app --yes --template app --style css\n```\n\n### What it sets up\n\nEvery new project comes wired for you: the enforced TypeScript 7, Oxlint + tsgolint, and Prettier presets, file-based routing, a `toil.config.ts`, a `toilconfig.json` (the server compiler settings), a `.gitignore`, and the native TypeScript 7 editor settings. It also scaffolds a `server/migrations/` folder (where ToilDB schema migrations live) and, unless you opt out, a set of AI assistant helper files.\n\nThe scaffolded `package.json` includes these scripts:\n\n| Script | Runs |\n| --- | --- |\n| `npm run dev` | `toiljs dev` |\n| `npm run build` | `toiljs build` |\n| `npm run build:server` | `toiljs build --server` |\n| `npm run lint` | `oxlint --type-aware client` |\n| `npm run typecheck` | `tsc --noEmit` |\n| `npm run format` | `prettier --write ...` |\n\n### Generated docs and AI-assistant pointers\n\nEvery project carries a full copy of this documentation set at `.toil/docs/`. You do not maintain it: toiljs regenerates it from the installed toiljs version on every `toiljs dev` and `toiljs build`, so it always matches the version you are on. Do not edit those files by hand (your changes are overwritten on the next dev or build).\n\nUnless you opt out, `toiljs create` also writes small **pointer files** at the project root that tell AI coding assistants to read `.toil/docs/` before touching the project: `CLAUDE.md` (Claude Code), `AGENTS.md` (Codex and others), `.cursor/rules/toiljs.mdc` (Cursor), and `.github/copilot-instructions.md` (GitHub Copilot). These are written once, committed, and yours to edit. Control them with `--ai` / `--no-ai` (when you pass neither, `create` asks).\n\n### create options\n\n| Flag | Meaning |\n| --- | --- |\n| `[name]` | The project folder name (a positional argument). If omitted, you are asked. |\n| `-t, --template <app\\|minimal>` | `app` is the full starter (landing page, layout, styles, demo routes). `minimal` is just a layout and a home route. Default `app`. |\n| `--style <css\\|sass\\|less\\|stylus>` | Which CSS flavor to set up. Default `css` (plain CSS). |\n| `--tailwind` / `--no-tailwind` | Add or skip Tailwind CSS (v4). Off by default. |\n| `--ai` / `--no-ai` | Include or skip AI assistant files (like `CLAUDE.md`). When omitted, you are asked. |\n| `--images` / `--no-images` | Enable or skip build-time image optimization. On by default. |\n| `--git` / `--no-git` | Initialize a git repository. When omitted, you are asked (default yes). |\n| `--install` / `--no-install` | Install dependencies after scaffolding. When omitted, you are asked (default yes). |\n| `--pm <npm\\|pnpm\\|yarn\\|bun>` | Which package manager to install with. Default `npm`. |\n| `-y, --yes` | Accept all defaults and skip every prompt. |\n\nFor a walkthrough, see [Create a project](../getting-started/create-project.md).\n\n## `toiljs dev`\n\nStarts the local development server with hot reload, so you edit a file and the browser updates in place. This is the command you leave running while you build.\n\n```bash\nnpm run dev\n# or, to pick a port:\nnpx toiljs dev --port 4000\n```\n\n### What the dev server emulates\n\nThe whole point of the dev server is to run your app locally the same way the real Dacely edge runs it in production, so what you see locally is what you ship. It emulates three things:\n\n1. **The edge.** For a project with a server (any project with a `toilconfig.json`), a small local HTTP server takes the public port and dispatches incoming requests into your compiled `server.wasm` using the exact same request envelope the production edge uses. Anything your server does not claim (page routes, static assets, the hot-reload websocket) is proxied to Vite behind the scenes. So your React frontend and your WebAssembly backend run together, on one URL, exactly like production.\n2. **The database.** ToilDB runs in-process as a local emulator. Every family (documents, views, unique, events, counters, membership, capacity) works, and the data is written to `.toil/devdata.json` so it survives restarts. You manage that file with [`toiljs db`](#toiljs-db).\n3. **The host functions.** The platform services your server code calls (email, environment variables and secrets, time, crypto, rate limiting, auth) are wired up locally so they behave like the edge. Email actually sends if you configure a provider (see [Email](../services/email.md)).\n\nA **client-only** project (no `toilconfig.json`) just gets the plain Vite dev server on your port, unchanged.\n\n### How hot reload works\n\n```mermaid\nflowchart TD\n E[\"You edit a file\"] --> Q{\"Which file?\"}\n Q -->|\"client/ (React, CSS)\"| V[\"Vite hot-swaps it in the browser\"]\n Q -->|\"server/ (a @rest / @data / @service file)\"| S[\"toilscript recompiles the server\"]\n S --> G[\"shared/server.ts is regenerated\"]\n G --> V2[\"Vite hot-swaps the new typed client\"]\n S --> W[\"the dev server hot-swaps the new server.wasm\"]\n V --> B[\"Browser updates, no full reload\"]\n V2 --> B\n W --> B\n```\n\nClient edits go straight through Vite's hot module replacement. Server edits trigger a toilscript rebuild: toiljs recompiles your backend, regenerates `shared/server.ts` (the typed client the browser imports to call your server), and hot-swaps the recompiled WebAssembly, all without you touching the browser. Rebuilds are debounced (grouped over about 150 milliseconds) so a \"save all\" or a formatter pass does not trigger a storm of builds.\n\nIf your project has an `emails/` folder, the dev server also prints an email-preview URL at `/__toil/emails`.\n\n### dev options\n\n| Flag | Meaning |\n| --- | --- |\n| `--port <n>` | Port to listen on. Default `3000` (or `client.port` from your config). |\n| `--root <dir>` | Run against a project in another directory. |\n\nPress `Ctrl+C` to stop. toiljs restores your terminal and force-exits even if a native listener is slow to close, so you never end up with an orphaned dev server rebuilding in the background.\n\n## `toiljs build`\n\nProduces the optimized production build. Run this before you deploy or before `toiljs start`.\n\n```bash\nnpm run build\n# or build only the server:\nnpx toiljs build --server\n```\n\n### What it produces\n\nA full build runs in a careful order so the pieces line up:\n\n```mermaid\nflowchart LR\n A[\"Your project\"] --> S[\"1. Compile the server<br/>(toilscript)\"]\n S --> W[\"build/server/release.wasm\"]\n S --> R[\"shared/server.ts<br/>(typed client, regenerated)\"]\n R --> C[\"2. Bundle the client<br/>(Vite)\"]\n C --> O[\"build/client/<br/>(HTML, JS, CSS, assets)\"]\n C --> X[\"3. Prerender + SEO<br/>(SSG pages, robots.txt,<br/>sitemap.xml, llms.txt)\"]\n```\n\n1. **The server is built first.** toilscript compiles every decorated server file (not just the entry) into `build/server/release.wasm`, and regenerates `shared/server.ts`. Doing this first means the client always bundles against a current, correct typed server client. A project that also declares `@stream` or `@daemon` surfaces compiles those into their own artifacts (`release-stream.wasm`, `release-cold.wasm`).\n2. **The client is bundled** by Vite into `build/client/` (your HTML, JavaScript, CSS, and optimized assets). The dev toolbar and error overlay are stripped out of the production bundle.\n3. **Static pages and SEO files are generated**: any route that opts into static generation is prerendered to HTML, and if you configured `client.seo`, toiljs writes `robots.txt`, `sitemap.xml`, and `llms.txt`. Routes that opt into server-side rendering get their HTML-with-holes templates baked for the edge.\n\n### build options\n\n| Flag | Meaning |\n| --- | --- |\n| `--server` | Build **only** the server (recompile the wasm and regenerate `shared/server.ts`), and skip the client bundle. Fast when you only touched backend code. This is what `npm run build:server` runs. |\n| `--root <dir>` | Run against a project in another directory. |\n\nA client-only project (no `toilconfig.json`) skips step 1 and just bundles the client.\n\n## `toiljs start`\n\nSelf-hosts the app you just built, on a fast production HTTP server (hyper-express, backed by uWebSockets.js). It serves your static client, runs your `server.wasm` for dynamic requests, does server-side rendering, supports daemons, and exposes a `/_toil` websocket channel. Use it to run your app on your own machine or server instead of deploying to the Dacely edge.\n\n```bash\nnpm run build # start needs a build to serve\nnpx toiljs start\nnpx toiljs start --port 8080 --host 0.0.0.0 --threads 4\n```\n\n`start` fails fast if there is no build yet (it looks for `build/client/index.html`), so run `toiljs build` first.\n\n### start options\n\n| Flag | Meaning |\n| --- | --- |\n| `--port <n>` | Port to listen on. Default `3000` (or `client.port`). |\n| `--host <host>` | Address to bind. Default `127.0.0.1` (loopback only). Pass `0.0.0.0` to accept connections from other machines. |\n| `--threads <n>` | Number of HTTP worker processes. Default is automatic (one per available CPU). Pass `1` to disable the worker pool. `--workers` is an accepted alias. This can also be set as `server.threads` in your config. |\n| `--root <dir>` | Run against a project in another directory. |\n\n## `toiljs configure`\n\nToggles a project's client styling features (the CSS preprocessor and Tailwind) after the fact, on an existing app. It detects your current setup, asks what you want, then rewrites the stylesheets and your app entry's imports, edits `package.json`, and syncs `node_modules` so removed packages are actually uninstalled.\n\n```bash\n# Interactive.\nnpx toiljs configure\n\n# Non-interactive: switch to Sass and turn Tailwind on.\nnpx toiljs configure --style sass --tailwind\n```\n\n### configure options\n\n| Flag | Meaning |\n| --- | --- |\n| `--style <css\\|sass\\|less\\|stylus>` | Switch the CSS preprocessor. |\n| `--tailwind` / `--no-tailwind` | Turn Tailwind on or off. |\n| `--images` / `--no-images` | Turn build-time image optimization on or off (sets `client.images` in your config). |\n| `--no-install` | Edit the files but do not run the package manager. You then run install yourself. |\n| `--root <dir>` | Run against a project in another directory. |\n\nPassing any of `--style`, `--tailwind`, or `--images` makes the command non-interactive (it skips the prompts you did not answer with a flag). See [Styling](../frontend/styling.md) for the full picture.\n\n## `toiljs doctor`\n\nRead-only project diagnostics. It gathers facts from disk (your `package.json`, lockfiles, the resolved config, your app entry, `index.html`, your routes, and the server target), runs a set of checks, and prints a grouped report. It never changes anything unless you pass `--fix`, and it never crashes on a partial or non-toiljs project (missing pieces just become warnings or failures). It exits with a non-zero status when any check **fails** (warnings do not fail), so it is safe to run in CI.\n\n```bash\n# Human-readable report.\nnpx toiljs doctor\n\n# Machine-readable, for CI.\nnpx toiljs doctor --json\n\n# Auto-repair the common wiring.\nnpx toiljs doctor --fix\n```\n\n### What it checks\n\nThe report is grouped:\n\n| Group | Example checks |\n| --- | --- |\n| **Environment** | Node.js version, that `toiljs` and its peer dependencies (React and so on) are installed and new enough, that TypeScript 7 is installed and its declared range stays within 7.x, that Oxlint and tsgolint are configured for type-aware linting, that a lockfile exists, and that your scripts do not wrap `toiljs` in a stray `npx`. |\n| **Project + routing** | The `client/` and `routes/` folders exist, `index.html` has a `<div id=\"root\">`, your app entry calls `mount(...)` with the `slots` argument, at least one route exists, no two routes collide on the same URL, and no asset paths are written in a way that 404s on nested routes. |\n| **Config + assets** | Your `toil.config` loads, the base path is well formed, `client.seo` has a `url` if SEO is configured, and your styling packages are actually installed. |\n| **Server / WASM** | The `toilconfig.json` and its entry files exist, `toilscript` is installed, a compiled `.wasm` exists, the typed-RPC wiring is in place, your `@rest` controllers are actually dispatched, the Prettier and editor plugins are wired, and a `migrations/` folder exists. |\n| **Security** | If your server uses auth, whether `AUTH_SESSION_SECRET` is set (an unset secret means sessions fall back to a published dev key, which is forgeable). |\n\n### What `--fix` repairs\n\n`--fix` migrates unsupported or missing TypeScript declarations to `^7.0.2` and enables the native TypeScript editor in any project. The rest only touch a server project (one with a `toilconfig.json`), repairing the wiring that is easy to get wrong or that older projects predate:\n\n- adds `--rpcModule shared/server.ts` to your server build scripts,\n- adds `shared` and the `shared/*` path alias to `tsconfig.json`,\n- adds `shared/server.ts` to `.gitignore`,\n- lifts the `toilscript` version floor if it is too old,\n- adds the `toiljs/prettier-plugin` to your Prettier config (so Prettier does not choke on server decorators),\n- removes the legacy toilscript JavaScript language-service plugin, which cannot run in the TypeScript 7 native language server; `toiljs build --server` provides the WebAssembly dialect diagnostics,\n- refreshes the editor-only server globals declaration file.\n\nIt is idempotent: it only writes files it actually needs to change, and it tells you which ones changed and which need a manual edit (for example a `tsconfig.json` that contains comments). If it changed `package.json`, run your installer afterward.\n\n### doctor options\n\n| Flag | Meaning |\n| --- | --- |\n| `--json` | Emit machine-readable JSON instead of the human report (the banner is suppressed so stdout stays valid JSON). |\n| `--fix` | Repair the server wiring in place, as above. |\n| `--root <dir>` | Run against a project in another directory. |\n\n## `toiljs update`\n\nA friendly wrapper over `npm-check-updates`. It checks the registry for newer versions of your dependencies, groups them by how big the jump is (major, minor, patch), lets you pick which to apply (or `-y` to apply all), bumps `package.json`, and runs your package manager's install. It also makes sure your `server/migrations/` folder exists (older projects predate it). `npm-check-updates` runs via `npx`, so it never becomes a permanent dependency of your project.\n\nTypeScript updates must stay within supported **7.x** releases. Older and future compiler majors,\nunbounded ranges, and prereleases are withheld. An older project is offered the latest 7.x migration\neven with `--target patch`. Select that migration before applying other updates. The command writes\nthe exact reviewed versions before installing, so a second registry query cannot substitute a\ndifferent compiler major.\n\n```bash\n# Interactive picker.\nnpx toiljs update\n\n# Apply everything, non-interactively.\nnpx toiljs update --yes\n\n# Only patch-level updates.\nnpx toiljs update --target patch\n```\n\n### update options\n\n| Flag | Meaning |\n| --- | --- |\n| `-y, --yes` | Apply all available updates without the picker. |\n| `--target <latest\\|minor\\|patch\\|newest\\|greatest>` | How far to bump. Default `latest`. |\n| `--root <dir>` | Run against a project in another directory. |\n\n## `toiljs db`\n\nManages the local dev database: the on-disk ToilDB store your dev server writes to `.toil/devdata.json`. Use it to inspect data, wipe a corrupt state, save a snapshot to share as a fixture, or restore one. The snapshot is exactly the JSON the dev database uses, so an exported file imports cleanly.\n\n```bash\n# See what is stored.\nnpx toiljs db status\n\n# Wipe all dev data.\nnpx toiljs db reset\n\n# Save a snapshot (to a file, or to stdout if you omit the file).\nnpx toiljs db export fixture.json\n\n# Restore a snapshot.\nnpx toiljs db import fixture.json\n\n# Print the on-disk path (scriptable).\nnpx toiljs db path\n```\n\n### db actions\n\n| Action | What it does |\n| --- | --- |\n| `status` (alias `info`) | Show the database path, its size, and per-family row counts. |\n| `reset` (alias `purge`) | Delete all dev data (removes `devdata.json`). |\n| `export [file]` | Write a formatted snapshot to `file`, or to stdout if you omit it (pipe-friendly). |\n| `import <file>` | Replace the dev database with the snapshot in `file`. It refuses a file that is not a valid snapshot. |\n| `path` | Print the `devdata.json` path and nothing else. |\n\nThe dev database is per-project and lives under `.toil/`, which is gitignored. See [Database setup](../database/setup.md) for what actually populates it.\n\n## The dev, build, and deploy flow\n\nHere is how the commands fit together across your workflow.\n\n```mermaid\nflowchart LR\n subgraph Develop\n D[\"toiljs dev<br/>(hot reload, local edge + DB)\"]\n end\n subgraph Ship\n B[\"toiljs build<br/>(client bundle + server.wasm)\"]\n end\n subgraph Run\n ST[\"toiljs start<br/>(self-host locally)\"]\n DP[\"deploy to the Dacely edge\"]\n end\n D --> B\n B --> ST\n B --> DP\n```\n\n- **Develop** with `toiljs dev`. Everything runs locally and reloads as you type.\n- **Ship** with `toiljs build`. This produces the artifacts: the client bundle in `build/client/` and the server WebAssembly in `build/server/`.\n- **Run** the build two ways: `toiljs start` self-hosts it on your own machine, or you deploy the same build to the Dacely edge to serve it worldwide.\n\n## Gotchas\n\n- **`start` needs a build.** `toiljs start` serves what is in `build/`. Run `toiljs build` first, or it exits with an error.\n- **Run `toiljs`, not `npx toiljs`, inside npm scripts.** Under `npm run`, `node_modules/.bin` is already on your PATH, so an extra `npx` layer is redundant and can leave your terminal in a broken input mode after `Ctrl+C`. `doctor` warns about this; the scaffold gets it right.\n- **`--host` and `--threads` are `start`-only.** `toiljs dev` always binds locally and does not take `--host` or `--threads`.\n- **`--server` builds the backend only.** Use it while iterating on server code, but run a full `toiljs build` before you deploy so the client bundle is current.\n- **The update notice is not an error.** The \"newer toiljs available\" line is informational and prints to stderr. Set `TOILJS_NO_UPDATE_CHECK=1` to hide it.\n\n## Related\n\n- [Installation](../getting-started/installation.md) and [Create a project](../getting-started/create-project.md)\n- [Project structure](../getting-started/project-structure.md)\n- [Configuration reference (`toil.config.ts`)](../concepts/config.md)\n- [Styling](../frontend/styling.md)\n- [Database setup](../database/setup.md) and the [database overview](../database/README.md)\n- [Compute tiers (L1 to L4)](../concepts/tiers.md)\n",
17
17
  "concepts/ai-guide.md": "# Writing toiljs correctly\n\nThis is a fast, high-signal cheat-sheet for writing toiljs and toilscript code that compiles and behaves. It is aimed at AI assistants generating code (and at humans in a hurry): the patterns to follow, the mistakes to avoid, and a link to the deep page whenever you need more. It does not re-teach the framework; each section points you at the authoritative doc.\n\n## Project shape\n\nA toiljs project has three top-level folders, and knowing which one a file belongs to tells you which rules apply:\n\n- **`client/`**: your React app that runs in the browser. Pages live under `client/routes/`, one file per URL, and each route file `export default`s its component. See [Frontend](../frontend/README.md) and [Routing](../frontend/routing.md).\n- **`shared/`**: a generated typed bridge (`shared/server.ts`) that lets the browser call the backend with full types. It is regenerated on every build, so **never hand-edit it**.\n- **`server/`**: your backend, written in TypeScript but compiled by **toilscript** to WebAssembly. It runs on the Dacely edge, not in Node or the browser. See [Backend](../backend/README.md).\n\nMost code runs on the L1 request tier. Long-lived connections (`@stream`) and scheduled work (`@daemon`) are opt-in tiers in their own entry files. See [Compute tiers](./tiers.md).\n\n## Client rules (the browser, React side)\n\n**Call the backend only through the generated `Server` client.** This is the number one rule. Use `Server.REST.<controller>.<route>(args)` for `@rest` HTTP controllers, or `Server.<service>.<method>(args)` / `Server.<remote>(args)` for RPC. Raw `fetch` to your own backend throws away the type safety, the argument and result decoding, and the binary wire codec, and it silently breaks when a route is renamed. `fetch` is only for third-party URLs (an external API, a CDN). See [Fetching data](../frontend/data-fetching.md).\n\n```ts\n// WRONG: raw fetch to your own backend\nawait fetch('/account/session', { method: 'POST' });\n\n// RIGHT: the typed client (renames become compile errors; args + result typed)\nawait Server.REST.account.session();\n```\n\n**Use the ambient `Toil.*` globals with no import.** `Toil` is the `toiljs/client` package exposed as a global and typed via a generated `toil-env.d.ts`, so it autocompletes with no import: `Toil.Link` / `Toil.NavLink` for navigation, `Toil.useParams`, `Toil.useLoaderData`, `Toil.Image`, `Toil.useHead` / `Toil.Head`, `Toil.Form` / `Toil.useAction`, and more. `Server` and `parseError` are also bare globals; `FastMap`, `FastSet`, `DataWriter`, and `DataReader` are bare globals too (write `new DataWriter()`, not `new Toil.DataWriter()`). Full index: [The Toil global](../frontend/toil-global.md).\n\n**Load page data with a `loader`, not a `useEffect` fetch.** A route file exports a `loader`, and the component reads its result with `Toil.useLoaderData`. The loader runs on navigation in parallel with the route chunk, integrates with `loading.tsx`, caching, and SSR hydration. A `useEffect` fetch runs only after mount (slower, invisible to the server).\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n return Server.REST.blog.get({ params: { id: params.id } });\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return <article><h1>{post.title}</h1></article>;\n}\n```\n\n**Mutate with `Toil.useAction` or `<Toil.Form>`, then revalidate.** Both track pending and error state and refetch the affected loader data on success, so the page updates without a manual refetch. Use `<Toil.Form>` for form submits, `useAction` for anything else (a delete button, a toggle). Reach for `Toil.revalidate()` / `router.revalidate()` after a write that these do not cover.\n\n**Per-route SEO with `export const metadata`** (or `Toil.useHead` / `<Toil.Head>` from inside a component). Opt a route into edge SSR with `export const ssr = true`. See [Metadata and SEO](../frontend/metadata.md) and [Rendering and SSR](../frontend/rendering.md).\n\n**Typed hrefs.** `Toil.Link` `href`, `Toil.navigate`, and `router.push` are all type-checked against your real routes, so a typo is a compile error. Use `Toil.href(str)` only when a URL is assembled from data and TypeScript cannot prove it is a real route. See [Navigation](../frontend/navigation.md).\n\n**`getUser()` on the client is display-only.** It reads a readable session cookie with no network call, so it is instant but **forgeable**. Use it to render \"logged in as ...\", never to gate real access. The authoritative check is a server route guarded with `@auth`. See [Auth usage](../auth/usage.md).\n\n## Server rules (toilscript, compiles to WASM)\n\nThe decorators, each doing one job (full list in [Decorators](./decorators.md)):\n\n- **`@data`** on a class: generates the binary and JSON codec so the value can cross the wire and the database. Every field needs a default; field order **is** the binary layout, so add new fields at the end. See [Data types](../backend/data.md).\n- **`@rest('name')`** on a class mounts an HTTP controller at `/name`; **`@get`/`@post`/`@put`/`@del`/`@patch`** on its methods declare routes (`@del`, not `@delete`, which is reserved). See [REST](../backend/rest.md).\n- **`@service`** + **`@remote`** expose typed RPC callable from your own frontend as `Server.<service>.<method>()`. See [RPC](../backend/rpc.md).\n- **`@user`** declares the authenticated-user shape and enables `AuthService.getUser()`. **Exactly one `@user` per project.** **`@auth`** guards a route or a whole `@rest` class. See [Auth](../auth/README.md).\n- **`@ratelimit(strategy, limit, window)`** caps how often a caller may hit a route, rejected at the edge before your code runs. See [Security](./security.md).\n- **`@database`** + **`@collection`** declare a ToilDB schema (below).\n\n```ts\n@rest('players')\nclass Players {\n @get('/:id')\n public get(ctx: RouteContext): Response {\n const id = ctx.param('id');\n return Response.json(`{\"id\":\"${id}\"}`);\n }\n}\n```\n\n**`@auth` rejects with 401 before your handler runs.** So inside an `@auth`-guarded handler, `AuthService.getUser()!` is guaranteed non-null and safe to assert. Without `@auth`, `getUser()` can be null, so null-check it instead. Either way the server re-verifies the signed session, so it (not the client cookie) is the real authorization boundary.\n\n```ts\n@auth\n@get('/settings')\npublic settings(): Response {\n const user = AuthService.getUser()!; // safe: @auth guarantees a session\n return Response.text('hi ' + user.username);\n}\n```\n\n**Server code is a strict, WASM-targeted dialect (AssemblyScript), not full TypeScript.** The real constraints, verified against the toilscript standard library:\n\n- **Use explicit value types**, never `number`: `i32` / `u32` / `i64` / `u64` / `f64` (and `i8`/`u8`/`i16`/`u16`/`f32`, plus `u128`..`u256`), `bool`, `string`, `Uint8Array`. Plain `number` resolves to `f64`, which is wrong for ids and counts. Integer math **wraps** on overflow (it never throws), and integer `/` truncates. See [The type system](./types.md).\n- **No `any`.** Every value has a concrete type; that is what lets it compile to WASM.\n- **No arbitrary npm.** Only the toilscript standard library plus the toiljs host APIs.\n- **Use `null`, not `undefined`** for \"no value\" (`T | null`, narrowed with `if (x != null)` or a `!` assertion).\n- **No usable built-in `RegExp`** (it is a host stub that throws). Parse strings by hand.\n- **Structured values are `@data` classes, not object literals.** Encode and decode raw binary with `DataWriter` / `DataReader` (imported on the server from the `data` module: `import { DataWriter, DataReader } from 'data';`). For dynamic JSON, use the ambient `JSON` value tree.\n- **Read config with `Environment.get(key)` and secrets with `Environment.getSecure(key)`** (each returns `string | null`; the two buckets are disjoint so a secret can never leak through `get`). See [Environment](../services/environment.md).\n- **Return a runtime `Response`** (`Response.json` / `.text` / `.html` / `.bytes` / `.notFound` / ...), or return a `@data` value and let toiljs serialize it.\n\n**ToilDB has seven collection families; pick the one that matches the job** (do not force everything into one). Declare each as a `static` `@collection` field on a `@database` class, typed by its family, and reach it statically (`AppDb.users.get(...)`). See [The database](../database/README.md) and [Setup](../database/setup.md).\n\n- **`Documents<K,V>`**: the default record store, looked up by id (users, posts, orders).\n- **`Unique<K,V>`**: a globally one-of-a-kind claim (usernames, emails, slugs).\n- **`Counter<K>`**: a running total many callers bump at once (likes, views); `add` a delta only.\n- **`Events<K,V>`**: an append-only log kept in order (feeds, audit trails).\n- **`Capacity<K>`**: a limited quantity handed out without overselling (tickets, seats).\n- **`Membership<K,M>`**: sets of who belongs to what (followers, tags, room members).\n- **`View<K,V>`**: a precomputed read-optimized snapshot (leaderboards, home pages).\n\n**Data access is gated by function kind.** A `@get` route is a read-only **Query**; a `@post` route is an **Action** (may write); a plain `@remote` defaults to read-only Query, so add **`@action`** to let it write. Scans (`events.latest`, `membership.list`) are barred from request handlers: do them in a `@derive` and have the request read the resulting `View`. The compiler enforces this and the edge re-checks it.\n\n## Common mistakes\n\nA short do-not list. Every one of these is a real, avoidable error:\n\n- **Raw `fetch` to your own backend.** Go through `Server.*` instead.\n- **Importing `Toil.*`.** They are ambient globals; importing them is wrong.\n- **Fetching page data in a `useEffect`.** Use a route `loader` + `Toil.useLoaderData`.\n- **A plain `<a>` for in-app links.** Use `Toil.Link` (a bare `<a>` triggers a full reload).\n- **More than one `@user`.** Exactly one per project.\n- **Trusting client `getUser()` for authorization.** It is display-only and forgeable; enforce on the server with `@auth`.\n- **`any`, `number`, or `RegExp` in server code.** Use explicit value types; there is no usable `RegExp`.\n- **Reordering `@data` fields** (or changing a field type) on a stored type. Field order is the layout; add at the end and use `@migrate` to evolve. See [Data types](../backend/data.md).\n- **Writing from a plain `@remote` or `@get`.** Reads are the default; a write needs `@action`.\n- **Calling a scan (`events.latest`, `membership.list`) from a request handler.** Do it in a `@derive`.\n- **Hand-editing `shared/server.ts` or `.toil/`.** They are generated; rebuild instead.\n\n## Where to go deeper\n\n- Concepts: [Decorators](./decorators.md), [Type system](./types.md), [Security](./security.md), [Compute tiers](./tiers.md).\n- Backend: [Overview](../backend/README.md), [REST](../backend/rest.md), [RPC](../backend/rpc.md), [Data types](../backend/data.md).\n- Frontend: [Overview](../frontend/README.md), [Fetching data](../frontend/data-fetching.md), [Routing](../frontend/routing.md), [Navigation](../frontend/navigation.md), [The Toil global](../frontend/toil-global.md).\n- Database: [ToilDB overview](../database/README.md), [Setup](../database/setup.md).\n- Auth: [Overview](../auth/README.md), [Usage](../auth/usage.md).\n- Services: [Environment](../services/environment.md), [Rate limiting](../services/ratelimit.md).\n</content>\n</invoke>\n",
18
- "concepts/config.md": "# Configuration (`toil.config.ts`)\n\n`toil.config.ts` is the one file that configures your whole toiljs app: the client (React/Vite) side and the server (WebAssembly) side. Every field is optional and has a sensible default, so most projects keep it tiny.\n\n## The shortest possible config\n\nA scaffolded project ships something like this:\n\n```ts\n// toil.config.ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n // Optimize images at build time (resize and compress imported images).\n images: true,\n },\n});\n```\n\n`defineConfig` does not do anything at runtime: it is an identity helper that gives you full editor autocomplete and type-checking for the config shape. Always wrap your config in it.\n\nAn empty config is valid too. `export default defineConfig({})` gives you a working app with all defaults.\n\n## Where the file lives\n\ntoiljs looks in your project root for the first file that matches, in this order:\n\n```\ntoil.config.ts toil.config.mts toil.config.js toil.config.mjs\ntoiljs.config.ts toiljs.config.mts toiljs.config.js toiljs.config.mjs\n```\n\nUse whichever you like. `toil.config.ts` is the convention.\n\n## Config vs. environment variables\n\nThere are two very different places settings live, and it matters which one you use.\n\n| | `toil.config.ts` (this page) | Environment variables |\n| --- | --- | --- |\n| **When it applies** | Build time and dev time | Runtime, per request |\n| **What it holds** | Framework and build options (routing, styling, SEO, which features to compile) | Values and secrets your running server reads (API keys, feature flags, connection info) |\n| **Committed to git?** | Yes, it is source | No. Local values go in `.env` / `.env.secrets` (both gitignored); production values live on your deploy target |\n| **Read in code with** | Not read from your app code | `Environment.get(...)` / `Environment.getSecure(...)` on the server |\n\nRule of thumb: if it is a **secret** (a password, an API key, a session key), it does **not** go in `toil.config.ts`. It goes in the environment. See [Environment and secrets](../services/environment.md) for the full story.\n\nThere is also a **third** file, `toilconfig.json`, which is a different thing entirely. See [`toil.config.ts` is not `toilconfig.json`](#toilconfigts-is-not-toilconfigjson) at the bottom.\n\n## The top-level shape\n\n```ts\ninterface ToilConfig {\n root?: string; // project root (defaults to the current working directory)\n client?: ClientConfig; // the React / Vite frontend\n server?: ServerConfig; // the toilscript / WebAssembly backend\n}\n```\n\nEverything else lives under `client` or `server`.\n\n## `client` reference\n\nConfigures the frontend: source folders, the dev server, and build-time optimizations.\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `srcDir` | `string` | `\"client\"` | Your frontend source directory, relative to the project root. |\n| `routesDir` | `string` | `\"routes\"` | Your file-based routes directory, relative to `srcDir`. |\n| `publicDir` | `string` | `\"<srcDir>/public\"` | Static assets directory. Holds `index.html` (which you own) plus files served as-is (favicons, images). |\n| `outDir` | `string` | `\"build/client\"` | Where the production client bundle is written. |\n| `base` | `string` | `\"/\"` | The public base path. Use this if you serve the app under a sub-path (a non-root base should start and end with `/`, like `\"/app/\"`). |\n| `port` | `number` | `3000` | The dev server port. `--port` on the CLI overrides it. |\n| `images` | `boolean` | `true` | Optimize imported images at build time (resize and convert). See [Images](../frontend/images.md). |\n| `fonts` | `boolean` | `true` | Preload bundled fonts (inject `<link rel=\"preload\">` for each `@font-face`) so text paints faster. |\n| `viewTransitions` | `boolean` | `false` | Animate page navigations with the browser View Transitions API (a crossfade by default). Respects `prefers-reduced-motion`. |\n| `transitions` | `boolean` | `false` | Wrap navigations in a React transition, keeping the current page visible while the next route's loader runs (instead of showing its loading state immediately). |\n| `devtools` | `boolean` or object | `true` | The floating dev toolbar (route/build info, errors, live controls). It is dev-only and never ships in production. Set `false` to disable, or pass an object to configure its AI integration. |\n| `seo` | object | (off) | Build-time SEO: bakes site-level metadata into the HTML and generates `robots.txt`, `sitemap.xml`, `llms.txt`. See [`client.seo`](#clientseo) below. |\n| `vite` | Vite `InlineConfig` | `{}` | An escape hatch: raw Vite options, deep-merged over the framework's own Vite setup. toiljs owns the Vite config; use this only to override specific options. |\n\n### Example\n\n```ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n images: true,\n fonts: true,\n viewTransitions: true,\n // Serve the app under https://example.com/app/\n base: '/app/',\n },\n});\n```\n\n### `client.devtools`\n\nThe dev toolbar is on by default. To turn it off, or to give it an AI provider (so its \"explain this error\" helpers can call a model), pass an object:\n\n```ts\nimport { defineConfig, AiProvider } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n devtools: {\n ai: {\n provider: AiProvider.Anthropic, // 'anthropic' or 'openai'\n model: 'claude-sonnet-4-6',\n // The name of the env var holding the API key. It is read\n // server-side by the dev process and never sent to the browser.\n apiKeyEnv: 'ANTHROPIC_API_KEY',\n // Optional: a custom POST endpoint ({ prompt } in, { text } out)\n // that overrides `provider` entirely.\n // endpoint: 'http://localhost:5000/ai',\n },\n },\n },\n});\n```\n\n| `devtools.ai` field | Type | What it does |\n| --- | --- | --- |\n| `provider` | `AiProvider` | Built-in provider: `AiProvider.Anthropic` (`'anthropic'`) or `AiProvider.OpenAI` (`'openai'`). |\n| `model` | `string` | The model id, for example `'claude-sonnet-4-6'` or `'gpt-4o'`. |\n| `apiKeyEnv` | `string` | The **name** of the environment variable that holds the API key. The key stays server-side and never reaches the browser. |\n| `endpoint` | `string` | A custom POST endpoint. When set, it takes precedence over `provider`. |\n\nThe toolbar always offers hand-off links to Claude and ChatGPT even without any AI config; this only enables the in-toolbar helpers.\n\n### `client.seo`\n\n`client.seo` turns on build-time search-engine and social metadata. It bakes tags into your HTML `<head>` (so crawlers and link-preview bots that do not run JavaScript still see real metadata) and generates `robots.txt`, `sitemap.xml`, and `llms.txt`. This is a large section with its own guide; see [Metadata and SEO](../frontend/metadata.md) for the complete field list and examples. The one thing worth knowing here: set `seo.url` to your absolute site URL (like `\"https://example.com\"`), because the sitemap and canonical links need it.\n\n```ts\nexport default defineConfig({\n client: {\n seo: {\n url: 'https://example.com',\n title: 'My App',\n description: 'A toiljs app.',\n },\n },\n});\n```\n\n## `server` reference\n\nConfigures the backend: which platform features to compile in, and how the dev server and self-host behave.\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `auth` | `boolean` | `false` | Opt into the framework's built-in post-quantum login. See [`server.auth`](#serverauth) below. |\n| `email` | object | (off) | The non-secret email backend config for the dev server and self-host. See [`server.email`](#serveremail) below. |\n| `daemon` | object | (defaults) | Background-job (L4 daemon) settings for dev and self-host. See [`server.daemon`](#serverdaemon) below. |\n| `nodeMode` | string | `\"all\"` | Which compute layer the single dev/self-host process emulates. See [`server.nodeMode`](#servernodemode) below. |\n| `threads` | `number` or `\"auto\"` | `\"auto\"` | HTTP worker count for `toiljs start`. `\"auto\"` uses one per CPU; `1` disables the worker pool. `--threads` on the CLI overrides it. |\n| `srcDir` | `string` | `\"server\"` | Declarative: your server source directory. See the note below. |\n| `outDir` | `string` | `\"build/server\"` | Declarative: the server build output directory. See the note below. |\n\n> **Note on `server.srcDir` / `server.outDir`.** These fields exist in the config type, but the actual location of your server source and its compiled output is driven by `toilconfig.json` (the toilscript compiler config), not by these two fields. Change your server paths in `toilconfig.json`. These fields are currently declarative and left for forward compatibility. See [not `toilconfig.json`](#toilconfigts-is-not-toilconfigjson) below.\n\n### `server.auth`\n\nSet `auth: true` to get a complete post-quantum login system with no boilerplate. The build appends a shipped `@rest('auth')` controller and its `@user` shape to your server, giving you `/auth/register`, `/auth/login`, `/auth/me`, and `/auth/logout` plus sessions.\n\n```ts\nexport default defineConfig({\n server: {\n auth: true,\n },\n});\n```\n\nTwo things to know:\n\n- If you opt in, your app must **not** declare its own `@user` type. The built-in auth owns the single per-program one.\n- There is an escape hatch: adding `import 'toiljs/server/auth'` in `server/main.ts` turns on the same built-in auth surface without this flag.\n\nAuth has its own configuration (session secrets, the OPRF and KEM keys) that lives in the **environment**, not here. See [Auth configuration](../auth/configuration.md).\n\n### `server.email`\n\nThe **non-secret** part of your email setup: which provider, the \"from\" address, and send caps. The dev server and self-host read it. The API key or SMTP password is a **secret** and never goes here; it comes from `.env.secrets` (`TOIL_EMAIL_API_KEY`). Any `TOIL_EMAIL_*` environment variable overrides the matching field here.\n\n```ts\nexport default defineConfig({\n server: {\n email: {\n provider: 'resend',\n from: 'hello@example.com',\n maxPerMin: 60,\n },\n },\n});\n```\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `provider` | `'resend'` \\| `'gmail'` \\| `'smtp'` | `'resend'` | Which email backend to use. |\n| `from` | `string` | (none) | The \"from\" address. Validated (single address, no line breaks). |\n| `maxPerMin` | `number` | `60` | Per-process send ceiling per minute (rolling). `0` means unlimited. |\n| `maxPerDay` | `number` | `0` | Per-process send ceiling per day (rolling). `0` means unlimited. |\n| `maxPerRecipientPerHour` | `number` | `5` | Per-recipient hourly cap (anti-abuse). |\n| `smtp` | object | (none) | Connection details for the `gmail` / `smtp` providers: `host`, `port` (defaults to 587 STARTTLS; 465 is implicit TLS), and `user` (defaults to `from`). |\n\nSee the full guide at [Email and 2FA](../services/email.md).\n\n### `server.daemon`\n\nSettings for the daemon (L4) background layer, used by the dev process and self-host. In dev, the local process is always the leader, so region fields are informational.\n\n```ts\nexport default defineConfig({\n server: {\n daemon: {\n defaultIntervalMs: 60000,\n maxTasks: 64,\n },\n },\n});\n```\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `region` | `string` | (none) | Region the daemon is pinned to (informational in dev). |\n| `standbyRegion` | `string` | (none) | Warm standby region (informational in dev). |\n| `defaultIntervalMs` | `number` | `60000` | Default interval for a `@scheduled` task that declares none. Values below `1000` are clamped up to `1000` (a sub-second loop would flood the console). |\n| `tickBudgetMs` | `number` | `30000` | Per-tick wall-clock budget before the dev scheduler logs an overrun. |\n| `gasTick` | `number` | `0` | Per-tick gas cap (a dev stub: charged then ignored). |\n| `maxTasks` | `number` | `64` | Maximum number of `@scheduled` tasks. Clamped to the range 1 to 1024. |\n\nSee [Daemons and scheduled jobs](../background/daemons.md).\n\n### `server.nodeMode`\n\nWhich compute layer the single local process emulates. This is a dev and self-host knob only; in production the Dacely edge decides each server's role. Valid values are `hot`, `regional`, `continental`, `daemon`, and `all`. The default, `all`, runs every surface (requests, streams, and daemons) in one process, which is what you want for a full local run. An invalid value falls back to `all` with a warning rather than failing. See [Compute tiers (L1 to L4)](../concepts/tiers.md) for what each layer means.\n\n```ts\nexport default defineConfig({\n server: {\n nodeMode: 'all', // run everything locally (the default)\n },\n});\n```\n\n## Defaults at a glance\n\nIf you write nothing, this is what you get.\n\n| Setting | Default |\n| --- | --- |\n| `client.srcDir` | `\"client\"` |\n| `client.routesDir` | `\"routes\"` |\n| `client.publicDir` | `\"client/public\"` |\n| `client.outDir` | `\"build/client\"` |\n| `client.base` | `\"/\"` |\n| `client.port` | `3000` |\n| `client.images` | `true` |\n| `client.fonts` | `true` |\n| `client.viewTransitions` | `false` |\n| `client.transitions` | `false` |\n| `client.devtools` | on |\n| `client.seo` | off |\n| `server.auth` | `false` |\n| `server.email` | off |\n| `server.nodeMode` | `\"all\"` |\n| `server.threads` | `\"auto\"` |\n| `server.daemon.defaultIntervalMs` | `60000` |\n| `server.daemon.tickBudgetMs` | `30000` |\n| `server.daemon.maxTasks` | `64` |\n\n## A fuller example\n\n```ts\nimport { defineConfig, AiProvider } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n images: true,\n fonts: true,\n viewTransitions: true,\n seo: {\n url: 'https://example.com',\n title: 'Example',\n description: 'Built with toiljs.',\n },\n devtools: {\n ai: { provider: AiProvider.Anthropic, model: 'claude-sonnet-4-6', apiKeyEnv: 'ANTHROPIC_API_KEY' },\n },\n },\n server: {\n auth: true,\n email: { provider: 'resend', from: 'hello@example.com' },\n threads: 'auto',\n },\n});\n```\n\n## `toil.config.ts` is not `toilconfig.json`\n\nThese two look almost the same and are easy to confuse. They are not the same file.\n\n| File | What it is |\n| --- | --- |\n| `toil.config.ts` | **This page.** The framework config: client and server options, styling, SEO, which features to build. You edit it often. |\n| `toilconfig.json` | The **toilscript compiler** config: which server files are entry points, where the `.wasm` is written, and low-level WebAssembly options (optimization level, memory layout, enabled wasm features). It is scaffolded for you and you rarely touch it. Its presence is also what tells toiljs \"this project has a server.\" |\n\nIf you ever need to change where your server source lives or what the compiled artifact is named, that is `toilconfig.json`, not `toil.config.ts`.\n\n## The `toilconfig.json` reference\n\n`toilconfig.json` is the **toilscript compiler** config. toilscript is the compiler that turns your `server/` TypeScript into a `.wasm` file (WebAssembly, the sandboxed binary your backend ships as). This file tells toilscript which server files to compile, where to write the output, and which low-level WebAssembly options to use.\n\n`toiljs create` scaffolds it for you and most projects never touch it. You only edit it if you want to rename the compiled artifact, move your server entry, or hand-tune the WebAssembly codegen. Its presence at the project root is also the signal toiljs uses to decide \"this project has a server\" (a project with no `toilconfig.json` is a client-only app).\n\nA scaffolded file looks like this:\n\n```json\n{\n \"entries\": [\"server/main.ts\"],\n \"targets\": {\n \"release\": {\n \"outFile\": \"build/server/release.wasm\",\n \"textFile\": \"build/server/release.wat\"\n }\n },\n \"options\": {\n \"sourceMap\": false,\n \"optimizeLevel\": 3,\n \"shrinkLevel\": 1,\n \"converge\": true,\n \"noAssert\": false,\n \"enable\": [\n \"sign-extension\",\n \"mutable-globals\",\n \"nontrapping-f2i\",\n \"bulk-memory\",\n \"simd\",\n \"reference-types\",\n \"multi-value\"\n ],\n \"runtime\": \"stub\",\n \"lib\": [\"node_modules/toiljs/server/globals\"],\n \"memoryBase\": 65536,\n \"initialMemory\": 4,\n \"debug\": false,\n \"trapMode\": \"allow\"\n }\n}\n```\n\n### `entries`\n\nAn array of your server entry files: the toilscript starting points for the compile. The scaffold lists just `server/main.ts`, and `main.ts` imports your other surface modules so they all get pulled in. (Under `toiljs build`, toiljs compiles every decorated server file it finds, not only the entries, so a `@rest` or `@data` file you drop in is picked up even if `main.ts` does not import it.)\n\nA project that also has a streams tier or a daemon tier lists their entry files here too, so each tier can compile into its own artifact:\n\n```json\n\"entries\": [\"server/main.ts\", \"server/main.stream.ts\", \"server/main.daemon.ts\"]\n```\n\n### `targets`\n\nA map of named build targets. Each target names its output files. The scaffold has one target, `release`.\n\n| Field | Type | What it does |\n| --- | --- | --- |\n| `outFile` | `string` | Where the compiled `.wasm` is written. The scaffold uses `build/server/release.wasm`. |\n| `textFile` | `string` | Where the `.wat` (WebAssembly **text** format, the human-readable text form of the same module) is written. Handy for inspecting the output; not needed at runtime. |\n\n### `options`\n\nLow-level WebAssembly codegen options passed straight to toilscript. The defaults are already tuned for production, so change these only if you know you need to.\n\n| Field | Type | Scaffold value | What it controls |\n| --- | --- | --- | --- |\n| `sourceMap` | `boolean` | `false` | Emit a source map alongside the `.wasm` so a debugger can map machine code back to your TypeScript. Off by default (it makes the build bigger). |\n| `optimizeLevel` | `number` | `3` | How hard the optimizer works on **speed**, from `0` (none) to `3` (most). `3` is a production release setting. |\n| `shrinkLevel` | `number` | `1` | How hard the optimizer works on **size**, from `0` to `2`. `1` trades a little speed for a smaller binary. |\n| `converge` | `boolean` | `true` | Re-run the optimizer until the output stops getting better. Squeezes out a bit more at the cost of a slower build. |\n| `noAssert` | `boolean` | `false` | Strip `assert(...)` checks from the output (replace them with just their value, no trap). `false` keeps the safety checks in. |\n| `enable` | `string[]` | (see below) | Which modern WebAssembly features the compiled module is allowed to use. |\n| `runtime` | `string` | `\"stub\"` | The memory-management runtime baked into the module. `\"stub\"` is a minimal runtime that never frees memory, which fits toiljs's model exactly: the edge runs one fresh instance per request and throws its whole memory away when the request ends, so there is nothing to garbage-collect. |\n| `lib` | `string[]` | `[\"node_modules/toiljs/server/globals\"]` | Extra library paths whose top-level exports become **ambient globals** (usable with no `import`). This is what makes toiljs's server globals (like `crypto` and the auth primitives) available everywhere in `server/`. |\n| `memoryBase` | `number` | `65536` | The byte offset where your server's static data starts. toiljs reserves the first 64 KiB (`[0, 65536)`) for the **request envelope** the edge writes at offset 0, so a large request body can never overwrite your program's state. Raise it to accept larger request bodies (it costs a little more initial memory). |\n| `initialMemory` | `number` | `4` | How much linear memory the module starts with, in **pages**. One WebAssembly page is 64 KiB, so `4` is 256 KiB. It grows on demand past this. |\n| `debug` | `boolean` | `false` | Include debug information (names and the like) in the binary. Off for production. |\n| `trapMode` | `string` | `\"allow\"` | What happens on a trapping operation (like a bad float-to-int conversion). `\"allow\"` lets it trap (the default and correct choice); `\"clamp\"` replaces traps with clamping instead. |\n\nThe `enable` array turns on WebAssembly features that are off by default in the compiler. The scaffold enables the modern set the compiled server relies on: `sign-extension`, `mutable-globals`, `nontrapping-f2i` (non-trapping float-to-int conversions), `bulk-memory` (fast `memory.copy` / `memory.fill`), `simd` (vector operations), `reference-types`, and `multi-value` (functions that return more than one value). Leave this list as scaffolded unless you have a specific reason to change it; removing an entry can make the module fail to compile or run.\n\n### The `--rpcModule` build flag\n\nOne toilscript flag is worth knowing even though it lives in your npm scripts, not in `toilconfig.json`: `--rpcModule shared/server.ts`. It tells the compiler to also emit `shared/server.ts`, the fully typed client the browser imports to call your server (the `@data` codec plus the typed `Server` surface). toiljs adds this flag for you on the request build. If your `build:server` script is missing it (older projects predate it), `toiljs doctor --fix` injects it. See [the CLI reference](../cli/README.md#what---fix-repairs).\n\n## Gotchas\n\n- **Wrap the config in `defineConfig`.** Without it you lose autocomplete and type errors on typos.\n- **Secrets never go here.** API keys, session secrets, and passwords belong in the environment, not in `toil.config.ts` (which is committed to git). See [Environment and secrets](../services/environment.md).\n- **`server.srcDir` / `server.outDir` do not move your server.** The server source location is governed by `toilconfig.json` entries. Editing these two config fields has no effect today.\n- **`nodeMode` and `daemon` are dev/self-host only.** In production the edge assigns each server its role; these settings never override that.\n- **An invalid `nodeMode` does not crash the build.** It warns and falls back to `all`.\n\n## Related\n\n- [The CLI](../cli/README.md): the commands that read this config.\n- [Environment and secrets](../services/environment.md): runtime values and how they differ from build-time config.\n- [Auth configuration](../auth/configuration.md): the auth-related environment settings behind `server.auth`.\n- [Email and 2FA](../services/email.md): the full email setup behind `server.email`.\n- [Metadata and SEO](../frontend/metadata.md): the full `client.seo` field list.\n- [Daemons and scheduled jobs](../background/daemons.md): the background layer behind `server.daemon`.\n- [Compute tiers (L1 to L4)](../concepts/tiers.md): what `server.nodeMode` selects.\n- [Images](../frontend/images.md) and [Styling](../frontend/styling.md): the features `client.images` and `toiljs configure` control.\n",
18
+ "concepts/config.md": "# Configuration (`toil.config.ts`)\n\n`toil.config.ts` is the one file that configures your whole toiljs app: the client (React/Vite) side and the server (WebAssembly) side. Every field is optional and has a sensible default, so most projects keep it tiny.\n\n## The shortest possible config\n\nA scaffolded project ships something like this:\n\n```ts\n// toil.config.ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n // Optimize images at build time (resize and compress imported images).\n images: true,\n },\n});\n```\n\n`defineConfig` does not do anything at runtime: it is an identity helper that gives you full editor autocomplete and type-checking for the config shape. Always wrap your config in it.\n\nAn empty config is valid too. `export default defineConfig({})` gives you a working app with all defaults.\n\n## Where the file lives\n\ntoiljs looks in your project root for the first file that matches, in this order:\n\n```\ntoil.config.ts toil.config.mts toil.config.js toil.config.mjs\ntoiljs.config.ts toiljs.config.mts toiljs.config.js toiljs.config.mjs\n```\n\nUse whichever you like. `toil.config.ts` is the convention.\n\n## Config vs. environment variables\n\nThere are two very different places settings live, and it matters which one you use.\n\n| | `toil.config.ts` (this page) | Environment variables |\n| --- | --- | --- |\n| **When it applies** | Build time and dev time | Runtime, per request |\n| **What it holds** | Framework and build options (routing, styling, SEO, which features to compile) | Values and secrets your running server reads (API keys, feature flags, connection info) |\n| **Committed to git?** | Yes, it is source | No. Local values go in `.env` / `.env.secrets` (both gitignored); production values live on your deploy target |\n| **Read in code with** | Not read from your app code | `Environment.get(...)` / `Environment.getSecure(...)` on the server |\n\nRule of thumb: if it is a **secret** (a password, an API key, a session key), it does **not** go in `toil.config.ts`. It goes in the environment. See [Environment and secrets](../services/environment.md) for the full story.\n\nThere is also a **third** file, `toilconfig.json`, which is a different thing entirely. See [`toil.config.ts` is not `toilconfig.json`](#toilconfigts-is-not-toilconfigjson) at the bottom.\n\n## The top-level shape\n\n```ts\ninterface ToilConfig {\n root?: string; // project root (defaults to the current working directory)\n client?: ClientConfig; // the React / Vite frontend\n server?: ServerConfig; // the toilscript / WebAssembly backend\n}\n```\n\nEverything else lives under `client` or `server`.\n\n## `client` reference\n\nConfigures the frontend: source folders, the dev server, and build-time optimizations.\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `srcDir` | `string` | `\"client\"` | Your frontend source directory, relative to the project root. |\n| `routesDir` | `string` | `\"routes\"` | Your file-based routes directory, relative to `srcDir`. |\n| `publicDir` | `string` | `\"<srcDir>/public\"` | Static assets directory. Holds `index.html` (which you own) plus files served as-is (favicons, images). |\n| `outDir` | `string` | `\"build/client\"` | Where the production client bundle is written. |\n| `base` | `string` | `\"/\"` | The public base path. Use this if you serve the app under a sub-path (a non-root base should start and end with `/`, like `\"/app/\"`). |\n| `port` | `number` | `3000` | The dev server port. `--port` on the CLI overrides it. |\n| `images` | `boolean` | `true` | Optimize imported images at build time (resize and convert). See [Images](../frontend/images.md). |\n| `fonts` | `boolean` | `true` | Preload bundled fonts (inject `<link rel=\"preload\">` for each `@font-face`) so text paints faster. |\n| `viewTransitions` | `boolean` | `false` | Animate page navigations with the browser View Transitions API (a crossfade by default). Respects `prefers-reduced-motion`. |\n| `transitions` | `boolean` | `false` | Wrap navigations in a React transition, keeping the current page visible while the next route's loader runs (instead of showing its loading state immediately). |\n| `devtools` | `boolean` or object | `true` | The floating dev toolbar (route/build info, errors, live controls). It is dev-only and never ships in production. Set `false` to disable, or pass an object to configure its AI integration. |\n| `seo` | object | (off) | Build-time SEO: bakes site-level metadata into the HTML and generates `robots.txt`, `sitemap.xml`, `llms.txt`. See [`client.seo`](#clientseo) below. |\n| `vite` | Vite `InlineConfig` | `{}` | An escape hatch: raw Vite options, deep-merged over the framework's own Vite setup. toiljs owns the Vite config; use this only to override specific options. |\n\n### Example\n\n```ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n images: true,\n fonts: true,\n viewTransitions: true,\n // Serve the app under https://example.com/app/\n base: '/app/',\n },\n});\n```\n\n### `client.devtools`\n\nThe dev toolbar is on by default. To turn it off, or to give it an AI provider (so its \"explain this error\" helpers can call a model), pass an object:\n\n```ts\nimport { defineConfig, AiProvider } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n devtools: {\n ai: {\n provider: AiProvider.Anthropic, // 'anthropic' or 'openai'\n model: 'claude-sonnet-4-6',\n // The name of the env var holding the API key. It is read\n // server-side by the dev process and never sent to the browser.\n apiKeyEnv: 'ANTHROPIC_API_KEY',\n // Optional: a custom POST endpoint ({ prompt } in, { text } out)\n // that overrides `provider` entirely.\n // endpoint: 'http://localhost:5000/ai',\n },\n },\n },\n});\n```\n\n| `devtools.ai` field | Type | What it does |\n| --- | --- | --- |\n| `provider` | `AiProvider` | Built-in provider: `AiProvider.Anthropic` (`'anthropic'`) or `AiProvider.OpenAI` (`'openai'`). |\n| `model` | `string` | The model id, for example `'claude-sonnet-4-6'` or `'gpt-4o'`. |\n| `apiKeyEnv` | `string` | The **name** of the environment variable that holds the API key. The key stays server-side and never reaches the browser. |\n| `endpoint` | `string` | A custom POST endpoint. When set, it takes precedence over `provider`. |\n\nThe toolbar always offers hand-off links to Claude and ChatGPT even without any AI config; this only enables the in-toolbar helpers.\n\n### `client.seo`\n\n`client.seo` turns on build-time search-engine and social metadata. It bakes tags into your HTML `<head>` (so crawlers and link-preview bots that do not run JavaScript still see real metadata) and generates `robots.txt`, `sitemap.xml`, and `llms.txt`. This is a large section with its own guide; see [Metadata and SEO](../frontend/metadata.md) for the complete field list and examples. The one thing worth knowing here: set `seo.url` to your absolute site URL (like `\"https://example.com\"`), because the sitemap and canonical links need it.\n\n```ts\nexport default defineConfig({\n client: {\n seo: {\n url: 'https://example.com',\n title: 'My App',\n description: 'A toiljs app.',\n },\n },\n});\n```\n\n## `server` reference\n\nConfigures the backend: which platform features to compile in, and how the dev server and self-host behave.\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `auth` | `boolean` | `false` | Opt into the framework's built-in post-quantum login. See [`server.auth`](#serverauth) below. |\n| `email` | object | (off) | The non-secret email backend config for the dev server and self-host. See [`server.email`](#serveremail) below. |\n| `daemon` | object | (defaults) | Background-job (L4 daemon) settings for dev and self-host. See [`server.daemon`](#serverdaemon) below. |\n| `nodeMode` | string | `\"all\"` | Which compute layer the single dev/self-host process emulates. See [`server.nodeMode`](#servernodemode) below. |\n| `threads` | `number` or `\"auto\"` | `\"auto\"` | HTTP worker count for `toiljs start`. `\"auto\"` uses one per CPU; `1` disables the worker pool. `--threads` on the CLI overrides it. |\n| `srcDir` | `string` | `\"server\"` | Declarative: your server source directory. See the note below. |\n| `outDir` | `string` | `\"build/server\"` | Declarative: the server build output directory. See the note below. |\n\n> **Note on `server.srcDir` / `server.outDir`.** These fields exist in the config type, but the actual location of your server source and its compiled output is driven by `toilconfig.json` (the toilscript compiler config), not by these two fields. Change your server paths in `toilconfig.json`. These fields are currently declarative and left for forward compatibility. See [not `toilconfig.json`](#toilconfigts-is-not-toilconfigjson) below.\n\n### `server.auth`\n\nSet `auth: true` to get a complete post-quantum login system with no boilerplate. The build appends a shipped `@rest('auth')` controller and its `@user` shape to your server, giving you `/auth/register`, `/auth/login`, `/auth/me`, and `/auth/logout` plus sessions.\n\n```ts\nexport default defineConfig({\n server: {\n auth: true,\n },\n});\n```\n\nTwo things to know:\n\n- If you opt in, your app must **not** declare its own `@user` type. The built-in auth owns the single per-program one.\n- There is an escape hatch: adding `import 'toiljs/server/auth'` in `server/main.ts` turns on the same built-in auth surface without this flag.\n\nAuth has its own configuration (session secrets, the OPRF and KEM keys) that lives in the **environment**, not here. See [Auth configuration](../auth/configuration.md).\n\n### `server.email`\n\nThe **non-secret** part of your email setup: which provider, the \"from\" address, and send caps. The dev server and self-host read it. The API key or SMTP password is a **secret** and never goes here; it comes from `.env.secrets` (`TOIL_EMAIL_API_KEY`). Any `TOIL_EMAIL_*` environment variable overrides the matching field here.\n\n```ts\nexport default defineConfig({\n server: {\n email: {\n provider: 'resend',\n from: 'hello@example.com',\n maxPerMin: 60,\n },\n },\n});\n```\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `provider` | `'resend'` \\| `'gmail'` \\| `'smtp'` | `'resend'` | Which email backend to use. |\n| `from` | `string` | (none) | The \"from\" address. Validated (single address, no line breaks). |\n| `maxPerMin` | `number` | `60` | Per-process send ceiling per minute (rolling). `0` means unlimited. |\n| `maxPerDay` | `number` | `0` | Per-process send ceiling per day (rolling). `0` means unlimited. |\n| `maxPerRecipientPerHour` | `number` | `5` | Per-recipient hourly cap (anti-abuse). |\n| `smtp` | object | (none) | Connection details for the `gmail` / `smtp` providers: `host`, `port` (defaults to 587 STARTTLS; 465 is implicit TLS), and `user` (defaults to `from`). |\n\nSee the full guide at [Email and 2FA](../services/email.md).\n\n### `server.daemon`\n\nSettings for the daemon (L4) background layer, used by the dev process and self-host. In dev, the local process is always the leader, so region fields are informational.\n\n```ts\nexport default defineConfig({\n server: {\n daemon: {\n defaultIntervalMs: 60000,\n maxTasks: 64,\n },\n },\n});\n```\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `region` | `string` | (none) | Region the daemon is pinned to (informational in dev). |\n| `standbyRegion` | `string` | (none) | Warm standby region (informational in dev). |\n| `defaultIntervalMs` | `number` | `60000` | Default interval for a `@scheduled` task that declares none. Values below `1000` are clamped up to `1000` (a sub-second loop would flood the console). |\n| `tickBudgetMs` | `number` | `30000` | Per-tick wall-clock budget before the dev scheduler logs an overrun. |\n| `gasTick` | `number` | `0` | Per-tick gas cap (a dev stub: charged then ignored). |\n| `maxTasks` | `number` | `64` | Maximum number of `@scheduled` tasks. Clamped to the range 1 to 1024. |\n\nSee [Daemons and scheduled jobs](../background/daemons.md).\n\n### `server.nodeMode`\n\nWhich compute layer the single local process emulates. This is a dev and self-host knob only; in production the Dacely edge decides each server's role. Valid values are `hot`, `regional`, `continental`, `daemon`, and `all`. The default, `all`, runs every surface (requests, streams, and daemons) in one process, which is what you want for a full local run. An invalid value falls back to `all` with a warning rather than failing. See [Compute tiers (L1 to L4)](../concepts/tiers.md) for what each layer means.\n\n```ts\nexport default defineConfig({\n server: {\n nodeMode: 'all', // run everything locally (the default)\n },\n});\n```\n\n## Route execution and transport limits\n\n`server.daemon.tickBudgetMs` applies only to scheduled daemon ticks. It is an overrun warning emitted after a tick returns; it does not cancel the tick or set a REST/RPC route deadline. There is no route execution timeout setting in `toil.config.ts`.\n\nFor `toiljs dev` and the Node self-host (`toiljs start`):\n\n| Limit | Default | What it bounds |\n| --- | --- | --- |\n| HTTP request body | 8 MiB | The body accepted by the front server, before handler dispatch. |\n| HTTP idle timeout | 60 seconds | An idle connection. |\n| HTTP response timeout | 120 seconds | A response stalled while sending under backpressure. |\n| WASM handler execution | No elapsed-time cutoff or gas metering | Synchronous `handle`/`render` execution on its worker. |\n\nThe HTTP timeouts do not interrupt a WASM handler that is still running. A busy loop can occupy its worker indefinitely, so a browser abort or reverse-proxy timeout must not be treated as proof that a write was cancelled. The low-level `startDevServer` / `startBuiltServer` APIs accept `maxBodyLength`; the CLI uses the default above.\n\nThe Dacely production edge meters execution with a per-request **gas budget** (a charge for WASM instructions and host operations), determined by the host's plan and operator configuration. Gas exhaustion traps the request. Gas is a work budget, not a guaranteed number of milliseconds, and the local Node hosts do not emulate it.\n\nProduction asynchronous database waits have their own deadlines: the current defaults are **50 ms for a Query**, **250 ms for an Action**, and **5 seconds for a Job or Derive**. The edge operator configures these through `db_quota`, separately from `toil.config.ts`. They bound DB waits rather than the whole route's execution time, and the local emulator does not enforce them. Deployed transport and database budgets are controlled by the host; there is no universal route time limit in seconds to infer from the daemon setting.\n\nKeep each upload request bounded by bytes, chunk count, and database operations. Continue large jobs across requests or background tasks rather than looping until a wall-clock deadline. See [Documents size limits and file uploads](../database/documents.md#size-limits-and-file-uploads) and [Time](../services/time.md#upload-expiry-versus-elapsed-deadlines).\n\n## Defaults at a glance\n\nIf you write nothing, this is what you get.\n\n| Setting | Default |\n| --- | --- |\n| `client.srcDir` | `\"client\"` |\n| `client.routesDir` | `\"routes\"` |\n| `client.publicDir` | `\"client/public\"` |\n| `client.outDir` | `\"build/client\"` |\n| `client.base` | `\"/\"` |\n| `client.port` | `3000` |\n| `client.images` | `true` |\n| `client.fonts` | `true` |\n| `client.viewTransitions` | `false` |\n| `client.transitions` | `false` |\n| `client.devtools` | on |\n| `client.seo` | off |\n| `server.auth` | `false` |\n| `server.email` | off |\n| `server.nodeMode` | `\"all\"` |\n| `server.threads` | `\"auto\"` |\n| `server.daemon.defaultIntervalMs` | `60000` |\n| `server.daemon.tickBudgetMs` | `30000` |\n| `server.daemon.maxTasks` | `64` |\n\n## A fuller example\n\n```ts\nimport { defineConfig, AiProvider } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n images: true,\n fonts: true,\n viewTransitions: true,\n seo: {\n url: 'https://example.com',\n title: 'Example',\n description: 'Built with toiljs.',\n },\n devtools: {\n ai: { provider: AiProvider.Anthropic, model: 'claude-sonnet-4-6', apiKeyEnv: 'ANTHROPIC_API_KEY' },\n },\n },\n server: {\n auth: true,\n email: { provider: 'resend', from: 'hello@example.com' },\n threads: 'auto',\n },\n});\n```\n\n## `toil.config.ts` is not `toilconfig.json`\n\nThese two look almost the same and are easy to confuse. They are not the same file.\n\n| File | What it is |\n| --- | --- |\n| `toil.config.ts` | **This page.** The framework config: client and server options, styling, SEO, which features to build. You edit it often. |\n| `toilconfig.json` | The **toilscript compiler** config: which server files are entry points, where the `.wasm` is written, and low-level WebAssembly options (optimization level, memory layout, enabled wasm features). It is scaffolded for you and you rarely touch it. Its presence is also what tells toiljs \"this project has a server.\" |\n\nIf you ever need to change where your server source lives or what the compiled artifact is named, that is `toilconfig.json`, not `toil.config.ts`.\n\n## The `toilconfig.json` reference\n\n`toilconfig.json` is the **toilscript compiler** config. toilscript is the compiler that turns your `server/` TypeScript into a `.wasm` file (WebAssembly, the sandboxed binary your backend ships as). This file tells toilscript which server files to compile, where to write the output, and which low-level WebAssembly options to use.\n\n`toiljs create` scaffolds it for you and most projects never touch it. You only edit it if you want to rename the compiled artifact, move your server entry, or hand-tune the WebAssembly codegen. Its presence at the project root is also the signal toiljs uses to decide \"this project has a server\" (a project with no `toilconfig.json` is a client-only app).\n\nA scaffolded file looks like this:\n\n```json\n{\n \"entries\": [\"server/main.ts\"],\n \"targets\": {\n \"release\": {\n \"outFile\": \"build/server/release.wasm\",\n \"textFile\": \"build/server/release.wat\"\n }\n },\n \"options\": {\n \"sourceMap\": false,\n \"optimizeLevel\": 3,\n \"shrinkLevel\": 1,\n \"converge\": true,\n \"noAssert\": false,\n \"enable\": [\n \"sign-extension\",\n \"mutable-globals\",\n \"nontrapping-f2i\",\n \"bulk-memory\",\n \"simd\",\n \"reference-types\",\n \"multi-value\"\n ],\n \"runtime\": \"stub\",\n \"lib\": [\"node_modules/toiljs/server/globals\"],\n \"memoryBase\": 65536,\n \"initialMemory\": 4,\n \"debug\": false,\n \"trapMode\": \"allow\"\n }\n}\n```\n\n### `entries`\n\nAn array of your server entry files: the toilscript starting points for the compile. The scaffold lists just `server/main.ts`, and `main.ts` imports your other surface modules so they all get pulled in. (Under `toiljs build`, toiljs compiles every decorated server file it finds, not only the entries, so a `@rest` or `@data` file you drop in is picked up even if `main.ts` does not import it.)\n\nA project that also has a streams tier or a daemon tier lists their entry files here too, so each tier can compile into its own artifact:\n\n```json\n\"entries\": [\"server/main.ts\", \"server/main.stream.ts\", \"server/main.daemon.ts\"]\n```\n\n### `targets`\n\nA map of named build targets. Each target names its output files. The scaffold has one target, `release`.\n\n| Field | Type | What it does |\n| --- | --- | --- |\n| `outFile` | `string` | Where the compiled `.wasm` is written. The scaffold uses `build/server/release.wasm`. |\n| `textFile` | `string` | Where the `.wat` (WebAssembly **text** format, the human-readable text form of the same module) is written. Handy for inspecting the output; not needed at runtime. |\n\n### `options`\n\nLow-level WebAssembly codegen options passed straight to toilscript. The defaults are already tuned for production, so change these only if you know you need to.\n\n| Field | Type | Scaffold value | What it controls |\n| --- | --- | --- | --- |\n| `sourceMap` | `boolean` | `false` | Emit a source map alongside the `.wasm` so a debugger can map machine code back to your TypeScript. Off by default (it makes the build bigger). |\n| `optimizeLevel` | `number` | `3` | How hard the optimizer works on **speed**, from `0` (none) to `3` (most). `3` is a production release setting. |\n| `shrinkLevel` | `number` | `1` | How hard the optimizer works on **size**, from `0` to `2`. `1` trades a little speed for a smaller binary. |\n| `converge` | `boolean` | `true` | Re-run the optimizer until the output stops getting better. Squeezes out a bit more at the cost of a slower build. |\n| `noAssert` | `boolean` | `false` | Strip `assert(...)` checks from the output (replace them with just their value, no trap). `false` keeps the safety checks in. |\n| `enable` | `string[]` | (see below) | Which modern WebAssembly features the compiled module is allowed to use. |\n| `runtime` | `string` | `\"stub\"` | The memory-management runtime baked into the module. `\"stub\"` is a minimal runtime that never frees memory, which fits toiljs's model exactly: the edge runs one fresh instance per request and throws its whole memory away when the request ends, so there is nothing to garbage-collect. |\n| `lib` | `string[]` | `[\"node_modules/toiljs/server/globals\"]` | Extra library paths whose top-level exports become **ambient globals** (usable with no `import`). This is what makes toiljs's server globals (like `crypto` and the auth primitives) available everywhere in `server/`. |\n| `memoryBase` | `number` | `65536` | The byte offset where your server's static data starts. toiljs reserves the first 64 KiB (`[0, 65536)`) for the **request envelope** the edge writes at offset 0, so a large request body can never overwrite your program's state. Raise it to accept larger request bodies (it costs a little more initial memory). |\n| `initialMemory` | `number` | `4` | How much linear memory the module starts with, in **pages**. One WebAssembly page is 64 KiB, so `4` is 256 KiB. It grows on demand past this. |\n| `debug` | `boolean` | `false` | Include debug information (names and the like) in the binary. Off for production. |\n| `trapMode` | `string` | `\"allow\"` | What happens on a trapping operation (like a bad float-to-int conversion). `\"allow\"` lets it trap (the default and correct choice); `\"clamp\"` replaces traps with clamping instead. |\n\nThe `enable` array turns on WebAssembly features that are off by default in the compiler. The scaffold enables the modern set the compiled server relies on: `sign-extension`, `mutable-globals`, `nontrapping-f2i` (non-trapping float-to-int conversions), `bulk-memory` (fast `memory.copy` / `memory.fill`), `simd` (vector operations), `reference-types`, and `multi-value` (functions that return more than one value). Leave this list as scaffolded unless you have a specific reason to change it; removing an entry can make the module fail to compile or run.\n\n### The `--rpcModule` build flag\n\nOne toilscript flag is worth knowing even though it lives in your npm scripts, not in `toilconfig.json`: `--rpcModule shared/server.ts`. It tells the compiler to also emit `shared/server.ts`, the fully typed client the browser imports to call your server (the `@data` codec plus the typed `Server` surface). toiljs adds this flag for you on the request build. If your `build:server` script is missing it (older projects predate it), `toiljs doctor --fix` injects it. See [the CLI reference](../cli/README.md#what---fix-repairs).\n\n## Gotchas\n\n- **Wrap the config in `defineConfig`.** Without it you lose autocomplete and type errors on typos.\n- **Secrets never go here.** API keys, session secrets, and passwords belong in the environment, not in `toil.config.ts` (which is committed to git). See [Environment and secrets](../services/environment.md).\n- **`server.srcDir` / `server.outDir` do not move your server.** The server source location is governed by `toilconfig.json` entries. Editing these two config fields has no effect today.\n- **`nodeMode` and `daemon` are dev/self-host only.** In production the edge assigns each server its role; these settings never override that.\n- **An invalid `nodeMode` does not crash the build.** It warns and falls back to `all`.\n\n## Related\n\n- [The CLI](../cli/README.md): the commands that read this config.\n- [Environment and secrets](../services/environment.md): runtime values and how they differ from build-time config.\n- [Auth configuration](../auth/configuration.md): the auth-related environment settings behind `server.auth`.\n- [Email and 2FA](../services/email.md): the full email setup behind `server.email`.\n- [Metadata and SEO](../frontend/metadata.md): the full `client.seo` field list.\n- [Daemons and scheduled jobs](../background/daemons.md): the background layer behind `server.daemon`.\n- [Compute tiers (L1 to L4)](../concepts/tiers.md): what `server.nodeMode` selects.\n- [Images](../frontend/images.md) and [Styling](../frontend/styling.md): the features `client.images` and `toiljs configure` control.\n",
19
19
  "concepts/decorators.md": "# Decorators reference\n\nEvery feature of a toiljs backend, an HTTP route, an RPC method, a database collection, a scheduled job, is switched on by a **decorator**. This page lists them all, grouped by what they do, so you can find the right one at a glance and jump to the page that covers it in depth.\n\n## What a decorator is\n\nA **decorator** is the `@name` you write on the line just above a class, a method, or a field. It attaches meaning to that code without changing what the code itself does. You are labelling it so the compiler knows how to wire it up.\n\n```ts\n@rest('users') // <- a class decorator: \"this class is an HTTP controller\"\nclass Users {\n @get('/:id') // <- a method decorator: \"this method answers GET /users/:id\"\n public byId(): Response { /* ... */ }\n}\n```\n\nThree things to notice, because they decide where each decorator can go:\n\n- Some decorators apply to a **class** (`@rest`, `@service`, `@stream`, `@daemon`, `@database`, `@data`, `@user`).\n- Some apply to a **method** or a free **function** (`@get`, `@remote`, `@scheduled`, `@query`).\n- One applies to a **field** (`@collection`).\n\nSome take arguments, like `@get('/:id')` or `@cache(60)`; the bare ones, like `@rest` or `@daemon`, do not. You never register anything by hand: tagging the code is enough, and the build discovers it.\n\nEach decorator also belongs to a **tier** (where its code runs) or is **shared** (compiled into every tier). If tiers are new to you, read [Compute tiers](./tiers.md) first; the short version is L1 = per-request at the edge, L2 / L3 = long-lived stream connections, L4 = one global daemon.\n\n## Routing and HTTP (L1)\n\nTurn a class into an HTTP controller and its methods into routes. All run on the L1 request tier. Covered in [REST](../backend/rest.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@rest` | class | Marks the class an HTTP controller, mounted at a prefix (`@rest('users')` -> `/users`). |\n| `@route` | method | Declares a route with an explicit method + path: `@route({ method: Methods.GET, path: '/' })`. |\n| `@get` | method | Shorthand for a `GET` route: `@get('/:id')`. |\n| `@post` | method | Shorthand for a `POST` route. |\n| `@put` | method | Shorthand for a `PUT` route. |\n| `@del` | method | Shorthand for a `DELETE` route (named `del` because `delete` is a reserved word). |\n| `@patch` | method | Shorthand for a `PATCH` route. |\n| `@head` | method | Shorthand for a `HEAD` route. |\n| `@options` | method | Shorthand for an `OPTIONS` route. |\n\n## RPC (L1)\n\nExpose server functions that your own frontend calls like typed async functions. Run on L1. Covered in [RPC](../backend/rpc.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@service` | class | Marks an RPC service; its `@remote` methods are namespaced under the generated client as `Server.<service>.<method>()`. |\n| `@remote` | method / function | Marks a method (of a `@service`) or a top-level function as a client-callable RPC endpoint. |\n\n## Guards and policy (L1)\n\nAttach a rule to a route (or a whole controller). These stack above a route method. Run on L1.\n\n| Decorator | Applies to | What it does | Covered in |\n| --- | --- | --- | --- |\n| `@auth` | class / method | Requires a valid session; returns `401` otherwise. On a class it guards every route. | [Auth usage](../auth/usage.md) |\n| `@cache` | method | Caches the response at the edge and browser: `@cache(edgeMinutes, browserSeconds?, privateScope?, allowAuth?)`. | [Caching](../services/caching.md) |\n| `@ratelimit` | method | Rate-limits a route: `@ratelimit(strategy, limit, window)`. | [Rate limiting](../services/ratelimit.md) |\n\n## Realtime streams (L2 / L3)\n\nHandle a long-lived connection, keeping state per connected client. The `@stream` class runs on the L2 / L3 stream tier; its lifecycle-hook methods fire as events arrive. Covered in [Realtime streams](../realtime/streams.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@stream` | class | Marks a stream protocol handler. `@stream('name')` sets the mount name; `@stream({ scope })` picks Regional (L2) or Continental (L3). |\n| `@connect` | method | Lifecycle hook: fires when a client connects (returns a `StreamOutbound` accept/reject). |\n| `@message` | method | Lifecycle hook: fires on each inbound packet. |\n| `@close` | method | Lifecycle hook: fires on a graceful close. |\n| `@disconnect` | method | Lifecycle hook: fires on an abrupt transport loss. |\n\nA server-side broadcast hook, `@channel`, is planned but not live in the current runtime. See [Channels](../realtime/channels.md) for the status and the working client-side `useChannel` hook you can use today.\n\n## Background and daemon (L4)\n\nRecurring, run-once-globally background work. `@daemon` runs on the single L4 leader. Covered in [Daemons](../background/daemons.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@daemon` | class | Marks the single daemon class (at most one per project). May declare a zero-arg `onStart()` run once at boot. |\n| `@scheduled` | method | Runs the method on a cadence: an interval (`'30s'`, `'5m'`, `'1h'`, `'1d'`) or a 5-field cron string (`'15 9 * * 1-5'`). |\n\n## Database structure (shared)\n\nDeclare your database schema. These carry no tier of their own; they are compiled into every artifact so any tier can use them. Covered in the [database section](../database/README.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@database` | class | Marks a class as a ToilDB database; each `@collection` field becomes a typed handle (`App.users.get(...)`). |\n| `@collection` | field | Declares a field as a collection handle (`Documents` / `View` / `Unique` / `Counter` / `Events` / `Membership` / `Capacity`). |\n| `@data` | class | Marks a class serializable: the compiler generates a binary codec so it can cross the wire and the database. See [Data types](../backend/data.md). |\n| `@migrate` | function | Schema migration for a `@data` type: a free function that upgrades a record written under an old layout to the current shape (lives in `migrations/<Type>.migration.ts`). See [Data types](../backend/data.md). |\n\n## Database function kinds (data-access policy)\n\nA **function kind** labels a function or method with *which* database operations it is allowed to issue, and the compiler enforces it (a read-only `@query` that tries to write is a compile error). Covered across the [database section](../database/README.md).\n\n| Decorator | Applies to | What it does | Covered in |\n| --- | --- | --- | --- |\n| `@query` | function / method | Read-only data access. | [Database](../database/README.md) |\n| `@action` | function / method | Read plus bounded writes and claims. | [Database](../database/README.md) |\n| `@derive` | function / method | Publishes materialized views and rollups (a background materializer). | [@derive](../background/derive.md) |\n| `@job` | function / method | Background work. | [Background work](../background/README.md) |\n| `@admin` | function / method | Control-plane only operations. | [Database](../database/README.md) |\n\n## Auth and structure\n\n| Decorator | Applies to | What it does | Covered in |\n| --- | --- | --- | --- |\n| `@user` | class | Declares the authenticated-user shape; enables the typed `AuthService.getUser()`. At most one per project. | [Auth usage](../auth/usage.md) |\n| `@main` | function | Marks a single top-level function as the module entry point (exported as the WASM `main`). Rarely written by hand: toiljs supplies the entry glue. | [Project structure](../getting-started/project-structure.md) |\n\n## A note on low-level decorators\n\nAssemblyScript (the language toilscript is built on) has its own low-level decorators such as `@inline`, `@unsafe`, and `@operator`. They tune how a symbol compiles and are for advanced library authors, not everyday app code. You will not need them to build a normal toiljs app, and they are out of scope here.\n\n## Related\n\n- [Compute tiers](./tiers.md): where each decorator's code runs (L1 to L4).\n- [The type system](./types.md): the types (`u64`, `string`, `@data`) these decorators work with.\n- [REST](../backend/rest.md) and [RPC](../backend/rpc.md): the L1 surfaces.\n- [Realtime streams](../realtime/streams.md): the `@stream` surface.\n- [Daemons](../background/daemons.md) and [@derive](../background/derive.md): L4 and background work.\n- [The database (ToilDB)](../database/README.md): `@database`, `@collection`, and the function kinds.\n- [Auth](../auth/README.md): `@auth` and `@user`.\n",
20
20
  "concepts/security.md": "# Security\n\nSecurity in toil is not a single feature you switch on. It is a set of defaults that are already on, spread across the whole stack, so that the safe path is also the default path.\n\n## The model\n\nToil is built for **defense in depth**: many independent layers, so that one weak spot is not one breach. The goal is the top of our internal security bar, the Resilience and Scale Grade (see [../../RSG.md](../../RSG.md)), where each of these holds at the same time:\n\n- All traffic is encrypted in transit with modern TLS (Transport Layer Security, the encryption every `https://` connection uses).\n- The login design never lets the server see a password it could reuse.\n- Secrets never live in your code.\n- The browser verifies every asset it loads before running it.\n- There is no implicit trust between the parts of the system (zero trust): each request is checked on its own merits.\n\nThe rest of this page walks through each layer and points to where it is documented in full.\n\n## The backend is a sandbox\n\nEvery site (every tenant) on toil runs inside a **sandbox**: a walled-off compartment that can only touch what the host explicitly hands it, and can never reach the host itself or another tenant. Your server code is compiled to **WebAssembly** (wasm, a portable binary instruction format that runs in a tightly controlled virtual machine), and the edge loads each tenant as its own isolated wasm module.\n\nThat isolation is backed by **hard per-host limits**, enforced by the host and not by your code:\n\n- **A committed-memory cap.** Each tenant may commit only a bounded amount of real memory (on the order of tens of MiB). A tenant cannot grow without bound and starve its neighbors.\n- **A compute cap.** Each request gets a bounded compute budget, and when a request exceeds it, the host stops it. A tenant cannot spin forever on the request path.\n- **Rate limits.** How often one client may call a route is capped (see [Transport and rate limiting](#transport-and-rate-limiting) below).\n- **Hostile-wasm containment.** The runtime is hardened to run untrusted, even deliberately malicious, wasm without letting it escape.\n\nThe point of all of this: a buggy or malicious tenant can crash or hang only itself. It cannot escape its sandbox, exhaust the host, or read or corrupt another tenant's data.\n\n## Passwords: post-quantum login\n\nToil ships a complete login system you turn on with one line, and it is built so the server never handles a usable password. See [../auth/README.md](../auth/README.md) for the overview and [../auth/how-it-works.md](../auth/how-it-works.md) for the mechanism.\n\nHere is why that matters. The pattern almost the whole internet uses is \"send the password over TLS, then hash it on the server.\" That is the accepted baseline, but it caps security below the top grade, because for a moment the server holds a live, replayable credential. Anything that reads server memory or a stray log line at the wrong instant walks away with real passwords, and that is one of the most common breach paths in practice.\n\nToil designs that surface away. The proof that you know your password happens without the password ever crossing the wire in a form anyone can replay, so a fully breached server still yields no usable credentials.\n\n## Subresource Integrity (SRI) on the client bundle\n\nThis is the layer that protects the code running in your users' browsers.\n\n### The risk\n\nYour app's JavaScript and CSS are static files served over the network, often through a CDN (Content Delivery Network, the fleet of edge caches that sits between your server and the browser). If any point in that path is compromised (the host, a cache, or the network itself), an attacker can serve **tampered** JavaScript or CSS: a modified bundle that steals sessions, skims form input, or rewrites the page. The browser, on its own, has no way to know the bytes it received are not the bytes you shipped.\n\n### What toil does at build time\n\n**Subresource Integrity (SRI)** is a browser feature that closes this gap. You attach a cryptographic fingerprint of a file to the tag that loads it, and the browser refuses to use the file unless its bytes match that fingerprint.\n\nAt build time (verified in [../../src/compiler/sri.ts](../../src/compiler/sri.ts)), toil adds that fingerprint to every **local** asset tag in the built HTML:\n\n- every `<script src=...>`,\n- every `<link rel=\"modulepreload\" href=...>`,\n- every `<link rel=\"stylesheet\" href=...>`.\n\nTo each it adds two attributes: `integrity=\"sha384-<base64>\"` and `crossorigin=\"anonymous\"`.\n\n- **`sha384`** is the hash algorithm: the file's bytes are run through SHA-384 (a member of the SHA-2 family), producing a fixed fingerprint that changes if even one byte changes. The `integrity` value is that fingerprint, base64-encoded, with a `sha384-` prefix.\n- **`crossorigin=\"anonymous\"`** is required for the browser to actually verify the fingerprint. It is a no-op when the asset is same-origin, so adding it is always safe.\n\nTags that are not toil's to hash are left completely untouched: external URLs (anything with a scheme like `https:`, a protocol-relative `//cdn/...`, or a `data:` URI), inline scripts (a `<script>` with no `src`), and any tag that already carries its own `integrity`.\n\n### Why an import map is also emitted\n\nA per-tag `integrity` only protects the **entry** script: the one file named directly in the tag. But a modern app is an ES module graph (ECMAScript modules, the browser's native `import` system). The entry script statically imports other chunks (for example the shared React vendor chunk), and it lazily pulls in each route with a dynamic `import()` as the user navigates. Those follow-on files are fetched by the browser's **module loader**, which does **not** inherit the entry tag's `integrity`. Without more, everything past the entry point would be unverified.\n\nSo toil also emits an **import map**: a `<script type=\"importmap\">` tag (a standard browser mechanism for describing module resolution). Its `integrity` section maps every emitted JavaScript chunk to its own `sha384` fingerprint. This is the platform mechanism that extends verification to the module loader's fetches, so the **full** dynamic `import()` graph is covered, not just the entry tag. Browsers that do not yet support the `integrity` key simply ignore it (progressive enhancement: nothing breaks on older browsers, they just get one fewer check).\n\n### Why it stays stable\n\nA fingerprint is only useful if it keeps matching the bytes actually served. Toil guarantees that by **content-hashing** every asset it puts SRI on: the build tool (Vite) names each JavaScript chunk and each CSS file with a hash of its own contents baked into the URL (verified in [../../src/compiler/vite.ts](../../src/compiler/vite.ts)). Because the URL is pinned to the exact bytes, the fingerprint baked into the HTML can never falsely mismatch.\n\nThis prevents a real failure mode. Imagine a stylesheet kept a stable URL but its bytes changed on a new deploy. A user still holding an older HTML page (with the old baked fingerprint) would fetch the new bytes, the hashes would disagree, and the browser would **block** the stylesheet, breaking the page for that user. Content-hashing sidesteps this: new bytes get a new URL, so old HTML keeps pointing at the old bytes, and every fingerprint always matches. (Images and fonts keep stable names on purpose: they carry no SRI and are often referenced by fixed path.)\n\n### What the browser does\n\nFor each asset that carries an `integrity` value (from a tag or from the import map), the browser hashes the bytes it received and compares them to the expected fingerprint. If they match, it runs the script or applies the stylesheet. If they do not match, it refuses: the asset is blocked and tampered code never runs.\n\n```mermaid\nflowchart TD\n A[\"Browser loads the built page HTML\"] --> B[\"Reads integrity tags plus the importmap integrity section\"]\n B --> C[\"Fetches an asset: entry script, vendor chunk, route chunk, or stylesheet\"]\n C --> D[\"Computes the sha384 hash of the received bytes\"]\n D --> E{\"Hash matches the baked fingerprint?\"}\n E -->|yes| F[\"Runs the script or applies the stylesheet\"]\n E -->|no| G[\"Refuses: blocks the asset, tampered code never runs\"]\n```\n\n### When it applies\n\nSRI is part of the **production build output**. The dev server hands the browser un-hashed modules straight from Vite, where SRI would be meaningless, so it is not applied there. You get the protection automatically in what you ship, with nothing to configure.\n\n## Secrets\n\nSecrets (API keys, signing keys, and the like) are **never compiled into the wasm**. They are provided per-host at runtime through the secure environment store and read with `Environment.getSecure(key)`, which is kept strictly separate from the plain-variable `Environment.get(key)`: a secret can never leak out through a code path that only reads plain config. Because secrets live outside your code, they are never in your repository or its history. See [../services/environment.md](../services/environment.md) for the full model.\n\n## Transport and rate limiting\n\nAll traffic runs over modern **TLS**, so nothing sensitive travels in the clear. The edge speaks **HTTP/3** (built on QUIC, a faster, head-of-line-blocking-free transport) with graceful fallback to HTTP/2 and HTTP/1.1, so newer clients get the modern path and older clients still connect and work.\n\nOn top of that, you can cap how often a client may hit any given route. Adding the `@ratelimit` decorator to a route makes the edge reject a client that goes over the limit **before your code runs**, which blunts brute-force and abuse and helps absorb bursts and denial-of-service (DDoS) pressure at the edge rather than inside your handler. Only routes that opt in pay any cost; everything else stays on the untouched fast path. See [../services/ratelimit.md](../services/ratelimit.md) for the strategies and per-user (keyed) limits.\n\n## Related\n\n- [Authentication](../auth/README.md): the post-quantum login system and `@auth`-guarded routes.\n- [Environment](../services/environment.md): plain variables versus secrets, and where they live.\n- [Rate limiting](../services/ratelimit.md): the `@ratelimit` decorator, strategies, and keyed limits.\n- [Design principles](../introduction/design-principles.md): the ideas the framework is built on.\n",
21
21
  "concepts/tiers.md": "# Compute tiers\n\nYour toiljs backend runs across four **compute tiers**, from a per-request handler at the very edge (L1) up to a single worldwide coordinator (L4). You write one project; the build decides which code belongs to which tier.\n\n## What a tier is\n\nA **tier** is a place your server code runs, defined by two things: how *close* it sits to the user, and how *long* one copy of it lives. The Dacely **edge** is a fleet of servers in many cities. Some work wants to run as close to the user as possible for speed. Other work wants exactly one copy in the whole world so it stays coordinated. A tier is the framework's answer to \"where and for how long should this piece of code live?\"\n\nThere are four tiers, named L1 through L4. The number is the **scope**: L1 is the smallest and nearest (one edge node), and each higher number widens the scope until L4 covers the entire globe.\n\n```mermaid\nflowchart TB\n subgraph L4[\"L4 - Global (one leader worldwide)\"]\n direction TB\n D[\"@daemon / @scheduled<br/>rollups, cleanup, polling\"]\n subgraph L3[\"L3 - Continental (per continent)\"]\n direction TB\n C[\"@stream (Continental scope)<br/>cross-region coordination\"]\n subgraph L2[\"L2 - Regional (per region)\"]\n direction TB\n R[\"@stream (Regional scope)<br/>chat, presence, live cursors\"]\n subgraph L1[\"L1 - Hot / edge (per request)\"]\n H[\"@rest, @service, @remote<br/>HTTP + RPC, the default\"]\n end\n end\n end\n end\n```\n\nRead the diagram as concentric rings: L1 is the inner ring nearest the user, and each outer ring is a wider, longer-lived scope. Most of your code lives in the innermost ring.\n\n## The four tiers at a glance\n\n| Tier | Name | How close | How many copies | Lives for | Runs |\n| --- | --- | --- | --- | --- | --- |\n| **L1** | Hot / edge | The exact node nearest the user | A fresh one per request | One request | `@rest`, `@service` / `@remote` |\n| **L2** | Regional | A region (a group of nearby cities) | One box per connection | The connection | `@stream` (Regional scope) |\n| **L3** | Continental | A continent | One box per connection | The connection | `@stream` (Continental scope) |\n| **L4** | Global / daemon | One place worldwide | Exactly one leader (plus a warm standby) | Until redeployed | `@daemon`, `@scheduled` |\n\nThe tier names come straight from the edge runtime (its node roles are `Hot`, `Regional`, `Continental`, and `Daemon`).\n\n## L1: Hot (request/response)\n\n**What:** a plain request-in, response-out handler. This is the default tier and where most code lives.\n\n**How it behaves:** a fresh copy of your handler serves each request, on whatever node is nearest the caller. Nothing you store on a field survives to the next request, because the next request may be a brand-new copy on the other side of the planet. This is called being **stateless**. It is what lets your app scale to the whole world with no coordination. When you need something to persist, you write it to [ToilDB](../database/README.md), the shared database.\n\nL1 is the lowest latency tier: the code runs on the node the user already reached, so there is no extra hop.\n\n```ts\n// L1: a fresh box per request. `hits` does NOT persist across requests.\n@rest('hello')\nclass Hello {\n private hits: i32 = 0; // reset every request\n\n @get('/')\n public greet(): Response {\n this.hits = this.hits + 1; // always 1\n return Response.text('hi\\n');\n }\n}\n```\n\nSee [REST](../backend/rest.md) and [RPC](../backend/rpc.md).\n\n## L2 and L3: streams (long-lived connections)\n\n**What:** a **stream** is a long-lived connection (think a chat socket) where the server keeps state *per connected client* for as long as they stay connected.\n\n**How it behaves:** when a client connects, the edge creates one resident **box** (one running copy of your `@stream` class) and pins it to a single worker. That same box handles every message from that client and is torn down when they leave. Because the box stays alive, its fields *do* persist across events, unlike L1. This is what makes a running message count, a game session, or a live cursor possible.\n\nA `@stream` picks its scope with `@stream({ scope })`:\n\n- **`StreamScope.Regional`** puts the box in the user's region (L2). Closest, lowest latency, smallest blast radius. Use it for per-connection state that only that one client cares about.\n- **`StreamScope.Continental`** puts the box at a continent-wide node (L3). A little farther, but many clients across a continent can be steered to the *same* box, which is what you want when connected users must share state (a shared room, a leaderboard everyone updates).\n\nThe trade-off is the theme of this whole page: nearer (L2) is faster; wider (L3) lets more clients coordinate on one box.\n\n```ts\n// L2/L3: the box is resident, so `count` survives every message.\n@stream({ scope: StreamScope.Regional })\nclass Echo {\n private count: i32 = 0;\n\n @connect onConnect(): StreamOutbound {\n this.count = 0;\n return StreamOutbound.accept();\n }\n @message onMessage(pkt: StreamPacket): StreamOutbound {\n this.count = this.count + 1; // persists across events\n return StreamOutbound.empty();\n }\n @close onClose(): void { /* box torn down after this */ }\n}\n```\n\nSee [Realtime streams](../realtime/streams.md).\n\n## L4: the daemon (global coordination)\n\n**What:** the daemon is a single, elected leader for your whole app, running recurring background work on a schedule.\n\n**How it behaves:** unlike every other tier, there is exactly **one** live daemon box in the entire world for your domain (with a warm standby ready to take over if the leader fails). It is not tied to any request or connection. It wakes on a cadence (an interval like `\"1h\"`, or a cron schedule) and does coordination work: rolling up analytics, cleaning up expired rows, polling an upstream API. Because there is only ever one leader, a `@scheduled` task runs **at most once** per due tick across the planet, which is exactly what you want for work that must not double-fire.\n\nL4 is the highest-latency, most-consistent tier: it is far from any single user, but it is the one place where \"do this exactly once, globally\" is true.\n\n```ts\n// L4: one leader worldwide runs this hourly.\n@daemon\nclass Jobs {\n @scheduled('1h')\n hourly(): void {\n // rollups, cleanup, polling an upstream, ...\n }\n}\n```\n\nSee [Daemons](../background/daemons.md).\n\n## How code is routed to a tier\n\nYou do not configure tiers. You opt into one by adding an **entry file** with the right name and using the matching **surface decorator**. A surface decorator is the top-level decorator that says what kind of endpoint a class is (`@rest`, `@stream`, `@daemon`).\n\nAt build time, `toiljs build` runs the compiler once per tier and hands each pass only the code that belongs to it. Each tier compiles into its own WebAssembly file (its own `.wasm` **artifact**), so the request build never carries stream or daemon code, and vice versa.\n\n| Entry file (`server/`) | Surface decorator | Artifact | Tier |\n| --- | --- | --- | --- |\n| `main.ts` | `@rest`, `@service` / `@remote` | `release.wasm` | L1 request |\n| `main.stream.ts` | `@stream` | `release-stream.wasm` | L2 / L3 stream |\n| `main.daemon.ts` | `@daemon`, `@scheduled` | `release-cold.wasm` | L4 daemon |\n\nShared building blocks carry no tier of their own. Your [`@data` classes](../backend/data.md) and your [`@database` schema](../database/README.md) are compiled into *every* artifact, because any tier may need to read or write the database. The stream and daemon tiers are opt-in: a project with no `@stream` and no `@daemon` just builds the single `release.wasm` and behaves exactly like a request-only app.\n\nFor the entry-file conventions and the full build flow, see [Project structure](../getting-started/project-structure.md).\n\n## Why tiers matter: latency vs consistency\n\nPicking a tier is a trade-off between two things a junior developer should keep in mind:\n\n- **Latency** (how fast a single user gets an answer): lower tiers win. L1 runs on the node the user already reached.\n- **Consistency and shared state** (many actors agreeing on one thing): higher tiers win. L4 has exactly one copy in the world, so it is trivially consistent, but it is far from everyone.\n\nA rule of thumb:\n\n- Answering one request as fast as possible: **L1**.\n- Keeping per-connection state for one live client: **L2**.\n- Letting many connected clients across a continent share one box's state: **L3**.\n- Doing a job exactly once, globally, on a schedule: **L4**.\n\n## Gotchas\n\n- **L1 fields reset every request.** If you set a field in one request and read it in the next, it will be gone. Persist to [ToilDB](../database/README.md) instead.\n- **A project using `@stream` may not also declare `@service` or `@remote`** anywhere (the compiler enforces this). Streams and RPC live in different artifacts. Keep them in separate entry files (`main.stream.ts` vs `main.ts`).\n- **There is at most one `@daemon` class per project.** It compiles only into the cold artifact; a `@daemon` in the request build is a compile error.\n- **Higher tier is not \"better\".** Do not reach for L4 to hold shared state you read on every request; that adds a long hop. Use the database for shared reads, and the daemon only for scheduled, run-once work.\n\n## Related\n\n- [Backend overview](../backend/README.md): the L1 request model in depth.\n- [REST](../backend/rest.md) and [RPC](../backend/rpc.md): the L1 surfaces.\n- [Realtime streams](../realtime/streams.md): the L2 / L3 surface and `StreamScope`.\n- [Daemons](../background/daemons.md) and [@derive](../background/derive.md): scheduled and background work.\n- [The database (ToilDB)](../database/README.md): where shared, persistent state lives.\n- [Decorators](./decorators.md): the full list of surface decorators and their tiers.\n- [Project structure](../getting-started/project-structure.md): entry files and the build.\n",
22
22
  "concepts/types.md": "# The server type system\n\nYour backend is TypeScript, but a stricter, faster dialect of it. Numbers have an exact bit width (`u32`, `i64`, `f64`, and friends) instead of one loose `number`, and a few TypeScript features do not exist. This page explains the types you write in `server/` code.\n\n## Why the server types are different\n\nYour frontend TypeScript runs in a browser, where a JavaScript engine figures out types as it goes. Your **server** code is different: the **toilscript** compiler turns it into [WebAssembly](../backend/README.md) (WASM), a compact machine-like binary. To emit that binary, the compiler must know the *exact* size of every value ahead of time. \"A number\" is not enough; it needs to know \"a 32-bit unsigned integer\" so it can pick the right machine instruction and lay out memory.\n\ntoilscript is built on **AssemblyScript**, a strict subset of TypeScript designed to compile to WASM. So the server language looks and reads like TypeScript, but every number is a fixed-width type, and there is no room for the fuzzy parts of JavaScript. The payoff is speed and safety: your code compiles to something close to hand-written native code, and whole classes of bugs cannot occur.\n\nYou already know the syntax. You only need to learn the number types and a handful of rules.\n\n## The number types\n\nThere is no plain `number` in server code. Instead you pick the exact type. The name tells you three things: signed or unsigned, how many bits, integer or float.\n\n- The letter: `u` = unsigned integer (zero and up), `i` = signed integer (can be negative), `f` = floating-point (has a fractional part).\n- The number: how many bits wide it is.\n\n| Type | Kind | Bits | Range | Use it for |\n| --- | --- | --- | --- | --- |\n| `bool` | boolean | 1 | `true` / `false` | flags, yes/no |\n| `u8` | unsigned int | 8 | 0 to 255 | a byte, a small enum |\n| `u16` | unsigned int | 16 | 0 to 65,535 | ports, small counts |\n| `u32` | unsigned int | 32 | 0 to ~4.29 billion | sizes, ids, most counts |\n| `u64` | unsigned int | 64 | 0 to ~1.8e19 | timestamps (ms), big counters, large ids |\n| `i8` | signed int | 8 | -128 to 127 | tiny signed values |\n| `i16` | signed int | 16 | -32,768 to 32,767 | small signed values |\n| `i32` | signed int | 32 | ~-2.1 to 2.1 billion | the default integer; loop counters, deltas |\n| `i64` | signed int | 64 | ~-9.2e18 to 9.2e18 | large signed counts, database counters |\n| `f32` | float | 32 | ~7 decimal digits | low-precision decimals (rare) |\n| `f64` | float | 64 | ~15-16 decimal digits | any value with a fraction: prices, ratios, math |\n\nThere are also 128-bit and 256-bit integers (`u128`, `i128`, `u256`, `i256`) for cryptography and very large ids. See [Big integers](#big-integers-u128-to-u256) below.\n\n### Which one should I pick?\n\nWhen in doubt, use these defaults and you will rarely be wrong:\n\n- A counter, a size, a loop index, an id you do not do math on: **`u32`** (or `i32` if it can go negative).\n- A millisecond timestamp, a like count that might grow huge, or a value that must never wrap: **`u64`** / **`i64`**.\n- Anything with a decimal point (money, averages, percentages): **`f64`**.\n- A true/false flag: **`bool`**.\n- One raw byte, or an element of a byte buffer: **`u8`**.\n\nSmaller types do not make your program meaningfully faster; they exist to match binary layouts and save memory in big arrays. Default to 32-bit unless a value is genuinely large or genuinely a byte.\n\n## Integer overflow: it wraps, it does not throw\n\nThis is the single most important difference from everyday JavaScript. Integer math is **modular**: if a value goes past the top of its range, it silently wraps around to the bottom (two's complement). It never throws an error and never turns into a bigger type on its own.\n\n```ts\nlet x: u8 = 255;\nx = x + 1; // x is now 0, not 256 (wrapped around)\n\nlet y: u8 = 0;\ny = y - 1; // y is now 255 (wrapped the other way)\n\nlet z: i32 = 2147483647; // the largest i32\nz = z + 1; // z is now -2147483648 (wrapped to the minimum)\n```\n\nThe lesson: choose a type wide enough that your values cannot reach its edge. A counter that could exceed ~4.29 billion needs `u64`, not `u32`. Timestamps in milliseconds always use `u64`.\n\nFloating-point (`f32` / `f64`) does not wrap; it follows the usual IEEE rules (very large values become `Infinity`, `0/0` is `NaN`).\n\n## Casting between number types\n\nBecause types are explicit, the compiler will not silently mix them. To convert, cast. There are two equivalent spellings; use whichever reads better:\n\n```ts\nconst big: u64 = 300;\nconst small: u8 = big as u8; // \"as\" form\nconst small2 = u8(big); // call the type like a function\n```\n\nCasting a wider integer into a narrower one **truncates**: it keeps only the low bits, so `u8(300)` is `44` (300 wraps modulo 256). Casting a float to an integer drops the fraction toward zero (`i32(3.9)` is `3`). None of these throw, so cast deliberately.\n\n```ts\nconst f: f64 = 3.9;\nconst i = i32(f); // 3 (fraction dropped)\nconst n = u8(300); // 44 (truncated to 8 bits)\n```\n\nInteger division also truncates (it is not float division):\n\n```ts\nconst half = 5 / 2; // both operands are i32 -> result is 2, not 2.5\nconst real = 5.0 / 2.0; // f64 division -> 2.5\n```\n\n## `bool`\n\n`bool` is a real 1-bit type, not \"any truthy value\". Comparisons (`==`, `<`, `>=`) produce a `bool`, and `if`/`while` expect one. It stores as 0 or 1.\n\n## Big integers: `u128` to `u256`\n\nFor values too large for 64 bits, toilscript provides `u128`, `i128`, `u256`, and `i256` as native global types (no import needed). They support the normal operators (`+`, `-`, `*`, `/`, `==`, `<`, `<<`, and so on) through operator overloading, plus static helpers like `u256.fromString(...)` and instance methods like `.toString()`, `.toBytes()`, and `.toUint8Array()`.\n\n```ts\nconst a = u256.fromString('123456789012345678901234567890');\nconst b = a + u256.One;\nconst hex = b.toString(16);\n```\n\nThese are mainly for cryptography and very large ids. Everyday counts and sizes should stay in `u32` / `u64`.\n\n### `ToilUserId`: a 256-bit identity\n\nThe built-in auth system represents a logged-in user as a **`ToilUserId`**, a stable 256-bit value (four `u64` words) derived from their key, identifier, and your domain. It is a global type, like `crypto`. It is the right key to store per-user data on. Comparison is overloaded and O(1), and `.toHex()` gives you a 64-character string key.\n\n```ts\nconst id: ToilUserId = AuthService.userId()!; // the current user's stable id\nconst key = id.toHex(); // a convenient string key\n```\n\nFull reference (including the `==` null-check gotcha) is in [Extending auth](../auth/extending.md).\n\n## Strings\n\nStrings work as you expect: `'hello'`, template literals, `.length`, `.substring(...)`, `+` to concatenate. Under the hood a server string is **UTF-16** (the same 16-bit code units as JavaScript), so `.length` counts code units, not visible characters or bytes.\n\nWhen you need the **bytes** of a string (to hash it, write it to a binary body, or send it over a stream), encode it explicitly. UTF-8 is almost always what you want on the wire:\n\n```ts\nconst buf: ArrayBuffer = String.UTF8.encode('hello'); // UTF-8 bytes\nconst text: string = String.UTF8.decode(buf); // back to a string\n```\n\n`String.UTF8.byteLength(s)` gives the encoded byte length. There is a matching `String.UTF16` namespace when you need raw UTF-16 bytes.\n\n## Binary data: `Uint8Array`\n\nRaw bytes are a `Uint8Array` (a fixed-length array of `u8`), exactly like the browser type. It is the standard currency for request bodies, hashes, crypto keys, and stream packets.\n\n```ts\nconst bytes = new Uint8Array(32);\nbytes[0] = 0xff; // each element is a u8\nconst n: i32 = bytes.length;\n```\n\nYou will also see `StaticArray<u8>` (a fixed-size, lower-overhead byte array) and `ArrayBuffer` (a raw buffer that `Uint8Array` and the encoders view). For most app code, `Uint8Array` is all you need.\n\n## Arrays and collections\n\nTyped arrays and the familiar collection types are available and must be typed:\n\n```ts\nconst nums: i32[] = [1, 2, 3]; // Array<i32>\nconst names = new Array<string>();\nconst seen = new Map<string, u32>();\nconst tags = new Set<string>();\n```\n\n`Array`, `Map`, and `Set` behave like their JavaScript counterparts (`.push`, `.get`, `.has`, `.forEach`), but every element type is fixed at compile time. There is no mixed-type array.\n\n## Objects: `@data` classes, not object literals\n\nServer code has no free-form object type. A structured value is a **class**, and to send it between the browser and the server (or in and out of the database) you tag it `@data`. That makes the compiler generate a binary codec plus a matching client type. See [Data types](../backend/data.md).\n\n```ts\n@data\nclass Player {\n username: string = '';\n score: u64 = 0;\n constructor(username: string = '', score: u64 = 0) {\n this.username = username;\n this.score = score;\n }\n}\n```\n\n## Key differences from normal TypeScript\n\n| Thing | Normal TypeScript | Server (toilscript) |\n| --- | --- | --- |\n| Numbers | one `number` | explicit `u8` / `i32` / `u64` / `f64` / ... |\n| `number` type | everywhere | avoid it; pick a fixed-width type |\n| `any` | allowed | not allowed: everything is typed |\n| Integer overflow | numbers just get bigger | wraps around silently |\n| `/` on integers | `5 / 2` is `2.5` | `5 / 2` is `2` (truncates) |\n| Mixing number types | implicit | explicit cast (`x as u8` / `u8(x)`) |\n| npm packages | `import` anything | only toilscript's standard library and toiljs APIs |\n| `undefined` | common | use `null` (a `T | null`), not `undefined` |\n| Objects | `{ a: 1 }` literals | a `class` (usually `@data`) |\n| Strings | UTF-16 | UTF-16 (same), encode to bytes for the wire |\n\nA few rules worth stating plainly:\n\n- **No `any`.** Every value has a concrete type. This is what makes the code compile to WASM at all.\n- **No arbitrary npm.** Server code cannot pull in npm packages; it uses the toilscript standard library (the types on this page) plus the toiljs host APIs (database, crypto, email, and so on). This is the sandbox that keeps the edge safe.\n- **No plain `number`.** If you write `number`, it resolves to `f64` (a float), which is almost never what you want for an id or a count. Always pick an explicit type.\n- **Use `null`, not `undefined`,** for \"no value\": a nullable is written `T | null` and you narrow it with an `if (x != null)` check or a `!` assertion when you are sure.\n\n## How server types cross to the browser\n\nWhen a `@data` value or an RPC result travels to your frontend, the compiler maps each server type to a matching TypeScript type in the generated client. You do not write this mapping; it is generated. It matters because it tells you what your React code receives.\n\n| Server type | Browser (generated TS) |\n| --- | --- |\n| `u8`, `u16`, `u32`, `i8`, `i16`, `i32`, `f32`, `f64` | `number` |\n| `u64`, `i64`, `u128`, `i128`, `u256`, `i256` | `bigint` |\n| `bool` | `boolean` |\n| `string` | `string` |\n| a `@data` class `T` | the generated class `T` |\n| `T[]` | `T[]` |\n\nThe important row is the 64-bit-and-larger integers: they become `bigint` on the client and travel as decimal strings on the JSON wire, so they stay **exact at any size** (they never lose precision the way a giant JavaScript `number` would). See [RPC](../backend/rpc.md) for the generated client.\n\n## Related\n\n- [Data types (`@data`)](../backend/data.md): defining structs that cross the wire and the database.\n- [RPC and the generated client](../backend/rpc.md): how server types map to browser types.\n- [Extending auth](../auth/extending.md): the `ToilUserId` 256-bit identity in full.\n- [The database (ToilDB)](../database/README.md): the collections your typed keys and values live in.\n- [Decorators](./decorators.md): the decorators referenced throughout this page.\n",
23
23
  "database/capacity.md": "# Capacity (limited stock without overselling)\n\nA `Capacity` collection models a **finite resource that must never be oversold**:\nconcert tickets, seats, limited-drop stock, rate grants. It uses a two-phase\n`reserve` then `confirm`/`cancel` pattern so two buyers can never grab the same\nlast unit.\n\n## What and why\n\nSome resources are strictly limited. There are 100 tickets, and selling 101 is a\nreal, expensive mistake. The classic bug is a race: two requests both read \"1\nleft\", both decide they can sell it, and both sell it. Now you have oversold.\n\nA `Capacity` collection prevents that. Instead of \"read, then decrement\" (two\nsteps a race can slip between), it does an atomic **hold**: the check that enough\nis available and the decrement happen as one indivisible step, at one place. And\nit adds a second phase so a shopper who starts a checkout but never pays does not\nlock up stock forever.\n\nThe lifecycle has three moves:\n\n- **`reserve`**: hold some units for a short time. The units come out of\n \"available\" immediately, and you get back a **reservation id**. If there is not\n enough available, the reserve fails (and nothing is oversold).\n- **`confirm`**: the shopper paid. Turn the hold into a permanent sale.\n- **`cancel`**: the shopper backed out. Release the hold back to available.\n\nIf a hold is neither confirmed nor cancelled within its time limit (its **TTL**,\ntime to live), it **auto-releases**. So an abandoned checkout heals itself.\n\n```mermaid\nsequenceDiagram\n participant Buyer\n participant App as Your route\n participant Cap as Capacity ledger\n Buyer->>App: start checkout\n App->>Cap: reserve(key, 2, ttl)\n Cap-->>App: reservationId (available drops by 2)\n Note over Cap: 2 units are held, not yet sold\n alt payment succeeds\n Buyer->>App: pay\n App->>Cap: confirm(key, reservationId)\n Cap-->>App: true (hold becomes a permanent sale)\n else buyer cancels\n Buyer->>App: cancel\n App->>Cap: cancel(key, reservationId)\n Cap-->>App: true (2 units returned to available)\n else buyer disappears\n Note over Cap: TTL passes, hold auto-releases (2 units returned)\n end\n```\n\nUse `Capacity` whenever selling one too many is unacceptable: tickets, seats,\ninventory for a limited drop, a pool of licences, or any \"N and no more\" resource.\n\n## Why not just a counter?\n\nA [Counter](./counters.md) can only `add` and `get`. To sell a ticket with a\ncounter you would read the remaining count, check it is above zero, and then\nsubtract one. Those are separate steps. Two requests can both read \"1 left\"\nbefore either subtracts, and both proceed: **oversold**. A counter has no way to\natomically check-and-decrement, and no notion of a temporary hold that expires.\n\n`Capacity` fixes both problems:\n\n- The availability check and the hold are **one atomic step**, so contenders are\n serialized: the first reserve takes the unit, the second sees zero available\n and fails.\n- The two-phase **reserve then confirm/cancel** means an unpaid, abandoned\n checkout does not permanently consume stock: its hold expires on the TTL.\n\nIf a small over- or under-count is harmless (likes, page views, an\napproximate inventory display), a counter is simpler and cheaper. If overselling\nby one is a real problem, use `Capacity`.\n\n## The type\n\n```ts\nCapacity<K>\n```\n\n`Capacity` has a single type parameter, the **key** (`K`): it picks *which*\nresource. There is no value type, because the value is a private ledger the\nplatform manages for you (it tracks the total, the confirmed sales, and the live\nholds). For one ticketed event, the key is the event id.\n\nDeclare it as a `@collection` field on a `@database` class:\n\n```ts\n@data\nclass ShowKey {\n show: string = '';\n constructor(show: string = '') { this.show = show; }\n}\n\n@database\nclass TicketsDb {\n @collection static seats: Capacity<ShowKey>;\n}\n```\n\n## Operations\n\nFive operations, matching the toilscript API exactly. Exact signatures:\n\n| Operation | Signature | What it does |\n| --- | --- | --- |\n| `available` | `available(key: K): i64` | How many units can be reserved right now. |\n| `reserve` | `reserve(key: K, amount: i64, ttlMs: i64): u64` | Hold `amount` units for `ttlMs` milliseconds. Returns a reservation id (> 0), or `0` if there was not enough. |\n| `confirm` | `confirm(key: K, reservationId: u64): bool` | Turn a hold into a permanent sale. Returns whether the id was valid. |\n| `cancel` | `cancel(key: K, reservationId: u64): bool` | Release a hold back to available. Returns whether it was released. |\n| `setTotal` | `setTotal(key: K, total: i64): void` | Set the ceiling (seed or restock). Background task only. |\n\nThe core accounting the platform maintains for you:\n\n```txt\navailable = total - confirmed - held\n```\n\n`total` is the ceiling you set. `confirmed` is units permanently sold. `held` is\nunits currently reserved but not yet confirmed. `available` never goes below zero.\n\n### `available`\n\nReads how many units could be reserved right now. It is a keyed read, legal from\nany handler including a `@get`.\n\n```ts\nconst left = TicketsDb.seats.available(new ShowKey('jazz-night'));\n```\n\n### `reserve`\n\nHolds `amount` units for `ttlMs` milliseconds and returns a reservation id. This\nis the operation that prevents overselling.\n\n```ts\nconst id = TicketsDb.seats.reserve(new ShowKey('jazz-night'), 2, 120_000); // hold 2 for 2 minutes\nif (id == 0) {\n // Not enough available. This is a normal outcome, NOT an error: no oversell.\n return Response.conflict('sold out');\n}\n// id > 0: you now hold 2 units for up to 2 minutes. Keep the id.\n```\n\nTwo things to know:\n\n- A return of `0` means \"not enough available\". It is the safe, expected answer\n when the resource is (nearly) sold out. Always check for it.\n- The hold auto-releases after `ttlMs` if you do not confirm or cancel it. Choose\n a TTL that covers a realistic checkout (a couple of minutes), not hours.\n\n`reserve` is a **write**, so call it from an action handler (`@post`, `@put`,\n`@patch`, `@del`), not from a `@get`.\n\n### `confirm`\n\nTurns a hold into a permanent sale. Call it once payment (or whatever finalizes\nthe deal) succeeds.\n\n```ts\nconst ok = TicketsDb.seats.confirm(new ShowKey('jazz-night'), id);\n// ok === true -> the id was valid; those units are now permanently sold\n// ok === false -> the id was unknown (never reserved, or already expired)\n```\n\nA confirmed sale is **permanent**: it can never be cancelled and its TTL never\nreclaims it. Confirm is safe to call more than once for the same id (it stays\nconfirmed).\n\n### `cancel`\n\nReleases a hold back to available. Call it when the shopper backs out before\npaying.\n\n```ts\nconst released = TicketsDb.seats.cancel(new ShowKey('jazz-night'), id);\n// released === true -> the hold was released, units are available again\n// released === false -> the id was unknown, already expired, OR already confirmed\n```\n\nYou **cannot** cancel a confirmed sale (a sale is final). `cancel` returns `false`\nin that case.\n\n### `setTotal`\n\nSets the ceiling: the total number of units. You call it to **seed** a new\nresource (\"this show has 100 seats\") and to **restock** (\"we opened the balcony,\nnow 150\"). Existing holds and confirmed sales are untouched; only the ceiling\nmoves, and `available` reflects the new total.\n\n`setTotal` is a **privileged** operation: it may only run from a background task\n(a `@job`), never from a request handler. Seeding and restocking are\nadministrative actions, so they live off the request path. See\n[background tasks](../background/README.md).\n\n```ts\n@database\nclass TicketsDb {\n @collection static seats: Capacity<ShowKey>;\n\n // A background task seeds/restocks the ceiling. This runs off the request path.\n @job\n openSeats(): void {\n TicketsDb.seats.setTotal(new ShowKey('jazz-night'), 100);\n }\n}\n```\n\nNote that lowering the total below what is already sold plus held does not\ncancel anything (confirmed sales are permanent); `available` simply floors at\nzero until holds clear.\n\n## The lifecycle in words\n\n1. **Seed.** A background `@job` calls `setTotal(key, 100)`. Now `available` is\n 100.\n2. **Reserve.** A buyer starts checkout. Your `@post` calls `reserve(key, 1,\n ttl)` and gets an id. `available` drops to 99. The unit is *held*, not sold.\n3. **Finalize, one of:**\n - **Confirm.** Payment succeeds; your `@post` calls `confirm(key, id)`. The\n unit is now permanently sold. `available` stays 99.\n - **Cancel.** The buyer backs out; your `@post` calls `cancel(key, id)`. The\n unit returns to available; `available` goes back to 100.\n - **Expire.** The buyer vanishes. After `ttl` passes, the hold auto-releases;\n `available` goes back to 100 on its own.\n\nAt no point can the confirmed count exceed the total. That is the guarantee.\n\n## Worked example: selling limited tickets\n\n```ts\nimport { ShowKey } from '../models/ShowKey';\nimport { ReserveRequest } from '../models/ReserveRequest';\nimport { FinalizeRequest } from '../models/FinalizeRequest';\nimport { ReserveResult } from '../models/ReserveResult';\n\n@database\nclass TicketsDb {\n @collection static seats: Capacity<ShowKey>;\n\n // Background task: seed the show's capacity (and restock if you reopen it).\n @job\n openSeats(): void {\n TicketsDb.seats.setTotal(new ShowKey('jazz-night'), 100);\n }\n}\n\n@rest('tickets')\nclass Tickets {\n // How many seats are left (a keyed read, legal in a GET).\n @get('/jazz-night/available')\n public left(): i64 {\n return TicketsDb.seats.available(new ShowKey('jazz-night'));\n }\n\n // Start checkout: hold the seats. Returns a reservation id, or 0 if sold out.\n @post('/jazz-night/reserve')\n public reserve(input: ReserveRequest): ReserveResult {\n const id = TicketsDb.seats.reserve(new ShowKey('jazz-night'), input.count, 120_000);\n // id === 0 means not enough available. No oversell; tell the buyer honestly.\n return new ReserveResult(id, id != 0);\n }\n\n // Payment succeeded: make the hold a permanent sale.\n @post('/jazz-night/confirm')\n public confirm(input: FinalizeRequest): bool {\n return TicketsDb.seats.confirm(new ShowKey('jazz-night'), input.reservationId);\n }\n\n // Buyer backed out before paying: release the hold.\n @post('/jazz-night/cancel')\n public cancel(input: FinalizeRequest): bool {\n return TicketsDb.seats.cancel(new ShowKey('jazz-night'), input.reservationId);\n }\n}\n```\n\nThe models:\n\n```ts\n@data\nexport class ReserveRequest {\n count: i64 = 1;\n}\n\n@data\nexport class FinalizeRequest {\n reservationId: u64 = 0;\n}\n\n@data\nexport class ReserveResult {\n reservationId: u64 = 0;\n ok: bool = false;\n constructor(reservationId: u64 = 0, ok: bool = false) {\n this.reservationId = reservationId;\n this.ok = ok;\n }\n}\n```\n\nEven if a thousand people hit `reserve` at the same instant for the last few\nseats, the ledger serializes them: the first buyers get ids, the rest get `0`.\nThe total is never exceeded.\n\n## Consistency\n\nUnlike most ToilDB families, `Capacity` is **strongly consistent**, and that is\nthe whole point. Every reserve/confirm/cancel for a key is routed to that key's\none home location and applied there in order, at a single serialization point. So\neven though ToilDB spans many regions, there is exactly one place that decides\nwhether a reserve succeeds. That is what makes \"never oversell\" a hard guarantee\nrather than a best effort. `available` reflects that home ledger.\n\n## Gotchas\n\n- **Check for `0` from `reserve`.** A `0` is \"not enough available\", the normal\n sold-out answer, not an error. Do not treat it as a failure to retry blindly.\n- **Always finalize a reservation.** After a successful `reserve`, you should\n `confirm` it (paid) or `cancel` it (abandoned). If you do neither, the hold\n sits until its TTL expires, temporarily reducing availability.\n- **Pick a sensible TTL.** Too short and a slow-but-real checkout loses its hold\n mid-payment. Too long and abandoned carts starve real buyers. A couple of\n minutes is typical for a checkout.\n- **Confirmed sales are permanent.** You cannot `cancel` a confirmed sale, and\n its TTL never reclaims it. If you support refunds, model that as your own\n restock (raise `setTotal`), not as a cancel.\n- **`reserve` is not automatically deduplicated.** Each call to `reserve` creates\n a new, distinct hold. If a client might retry the same reserve, avoid holding\n twice: reserve once, keep the returned id, and drive `confirm`/`cancel` off\n that id.\n- **`setTotal` is background-only.** You cannot set the total from a route. Seed\n and restock from a `@job`.\n- **There is a per-key cap on live holds.** A key can carry only so many\n simultaneous unconfirmed holds; a flood of holds beyond that is rejected as\n \"not enough available\" (and abandoned holds clear on their TTL). Normal traffic\n never hits this; it is a guardrail against a hold flood.\n\n## Related\n\n- [Counters](./counters.md): a running total when overselling is not a concern\n (and why it cannot guarantee a limit).\n- [Documents](./documents.md): store the order/booking record that a confirmed\n sale produces.\n- [background tasks](../background/README.md): where `setTotal` (seeding/restock)\n runs.\n- [Data types (`@data`)](../concepts/types.md): how the capacity key is stored.\n- [Decorators](../concepts/decorators.md): which handler kinds may reserve,\n confirm, and cancel.\n",
24
24
  "database/counters.md": "# Counters\n\nThe **Counter** family is a distributed running total that many callers can increase at the same time, from anywhere in the world, without ever losing a count. It is how you build likes, view counts, and inventory tallies correctly.\n\n## What and why\n\nA **Counter collection** maps a `@data` key to a single whole number. You do just two things with it: read the current total (`get`) and add to it (`add`). There is deliberately **no \"set to this value\"** operation, and that missing operation is the entire point of the family, as the next section explains.\n\nReach for a Counter whenever the thing you are storing is a number that many requests bump concurrently:\n\n- likes or upvotes on a post\n- page views or play counts\n- a stock or quantity tally\n- any \"increase this by N\" metric\n\nDeclare one by typing a `@collection` as `Counter<Key>`. Notice there is no value type: the value is always a number the database manages for you.\n\n```ts\n@data\nclass PostId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@database\nclass AppDb {\n @collection static likes: Counter<PostId>;\n @collection static views: Counter<PostId>;\n}\n```\n\n## The operations\n\n`K` is the key type. The value is always a 64-bit signed integer (`i64`).\n\n| Operation | Signature | Returns | Use it to |\n| --- | --- | --- | --- |\n| `get` | `get(key: K): i64` | the current total (`0` if nothing has been added) | read the count |\n| `add` | `add(key: K, delta: i64): void` | nothing | change the count by `delta` (may be negative) |\n\n`get` is a read (works in any function). `add` is a write, so it needs an **Action** (a `@post` route or an `@action`); see [Setup](./setup.md#how-access-is-gated-query-action-and-friends).\n\n### `get`\n\n`get` returns the current total. A counter that has never been touched reads as `0`, not `null`: there is no \"absent\" state to handle.\n\n```ts\nconst likeCount: i64 = AppDb.likes.get(new PostId('p_42'));\n```\n\n### `add`\n\n`add` changes the total by a delta. The delta can be positive to increase or negative to decrease:\n\n```ts\nAppDb.likes.add(new PostId('p_42'), 1); // one more like\nAppDb.likes.add(new PostId('p_42'), -1); // undo a like\nAppDb.stock.add(new SkuId('sku_9'), -3); // sold three\n```\n\n`add` saturates at the `i64` limits: it will not wrap around from a huge positive to a negative if you somehow overflow. Note that `add` returns nothing, not the new total. If you need the total right after adding, call `get` (bearing in mind the consistency note below).\n\n## Why a Counter, and not a number in a Documents record?\n\nThis is the question the family exists to answer, so it is worth walking through.\n\nSuppose you stored the like count as a field in a Documents record and incremented it the obvious way: read the record, add one, write it back. Now two servers on opposite sides of the world each get a like at the same moment:\n\n```mermaid\nsequenceDiagram\n participant Tokyo as Server (Tokyo)\n participant Paris as Server (Paris)\n participant DB as Documents record\n\n Note over DB: likes = 10\n Tokyo->>DB: read likes -> 10\n Paris->>DB: read likes -> 10\n Tokyo->>DB: write likes = 11\n Paris->>DB: write likes = 11\n Note over DB: likes = 11 (one like LOST)\n```\n\nBoth read `10`, both wrote `11`, and one like vanished. This is a **lost update**, and it happens whenever two callers do read-modify-write on the same value at once. It is not a rare edge case on a busy post; it is the normal case.\n\nA Counter avoids this entirely because you never send a final value, you send a **delta**. Each server just says \"add 1.\" The database merges the two deltas into \"add 1, add 1 = add 2,\" so the total goes from 10 to 12 and nothing is lost. Deltas are **commutative** (the order they arrive in does not matter) and **additive**, so concurrent increments from anywhere in the world always combine correctly. That property is what \"conflict-free\" means: there is no conflict to resolve, because two adds are never at odds.\n\nThis is also why there is no `set` operation. A `set` would reintroduce the exact race above (two callers overwriting each other). By offering only `add`, the Counter family makes the lost-update bug impossible to write.\n\n**Use a Counter when** the number is bumped concurrently by many requests. **Use a Documents field instead when** the number is only ever changed by a single logical owner in a controlled way (for example, a value you always overwrite with a freshly computed absolute figure, not an increment), where you would use `patch` to store the whole new value.\n\n## Worked example: page views and likes\n\nHere is a small feature that counts views on every read and likes on demand, then reports both.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@data\nclass PostId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass Stats {\n views: i64 = 0;\n likes: i64 = 0;\n}\n\n@database\nclass AppDb {\n @collection static views: Counter<PostId>;\n @collection static likes: Counter<PostId>;\n}\n\n@rest('posts')\nclass Posts {\n // POST /posts/:id/view -> count a view, return the running totals (Action)\n @post('/:id/view')\n public view(ctx: RouteContext): Stats {\n const key = new PostId(ctx.param('id'));\n AppDb.views.add(key, 1);\n const s = new Stats();\n s.views = AppDb.views.get(key);\n s.likes = AppDb.likes.get(key);\n return s;\n }\n\n // POST /posts/:id/like -> add a like (Action)\n @post('/:id/like')\n public like(ctx: RouteContext): Stats {\n const key = new PostId(ctx.param('id'));\n AppDb.likes.add(key, 1);\n const s = new Stats();\n s.views = AppDb.views.get(key);\n s.likes = AppDb.likes.get(key);\n return s;\n }\n\n // GET /posts/:id/stats -> just read the totals (Query: read-only)\n @get('/:id/stats')\n public stats(ctx: RouteContext): Stats {\n const key = new PostId(ctx.param('id'));\n const s = new Stats();\n s.views = AppDb.views.get(key);\n s.likes = AppDb.likes.get(key);\n return s;\n }\n}\n```\n\nCounting a view lives in a `@post` because it is a write (a counter `add`), even though it feels like part of a read. The `@get` stats route only reads, so it runs as a Query.\n\n## Consistency notes\n\n- **Counters are eventually consistent.** `get` returns the total known to the copy nearest you, which may briefly trail increments applied moments ago in other regions. The copies converge quickly, so the count catches up on its own.\n- **No increment is ever lost.** That is the strong guarantee: every `add` from anywhere is eventually included in the total, because deltas merge. The total may be a little behind, but it is never wrong in the \"lost a like\" sense.\n- **Monotonic when you only add positives.** If every `add` is positive, the total only ever goes up as it converges. Mixing in negative `add`s (undoing a like, selling stock) is fine and correct; it simply means the total is not monotonic.\n- Because of the lag, do not treat the value `get` returns immediately after an `add` as a globally final figure. It is a fast, close-enough total, which is exactly right for likes and views.\n\n## Gotchas\n\n- **There is no `set`.** To move a counter to a specific number, add the difference (`add(key, target - get(key))`), but be aware that read-then-add reintroduces a race if two callers do it at once. Prefer to think in deltas.\n- **`add` does not return the new total.** Call `get` after if you need it, remembering it may lag concurrent remote adds.\n- **A never-touched counter reads `0`, not `null`.** There is no \"does this counter exist\" check; every key answers `0` until something is added.\n- **Counters hold a number, not a record.** If you need per-key fields alongside the count, keep the record in [Documents](./documents.md) and the tally in a Counter, keyed the same way (the example above effectively does this with two counters).\n\n## Related\n\n- [ToilDB overview](./README.md): the seven families and how to choose.\n- [Setup](./setup.md): declaring the collection and which function kinds may write.\n- [Documents](./documents.md): when the number is part of a record you overwrite wholesale.\n- [Events](./events.md): when you need the individual events, not just a total.\n- [Data types (`@data`)](../backend/data.md): the counter's key type.\n",
25
- "database/documents.md": "# Documents\n\nThe **Documents** family is ToilDB's general-purpose record store: you keep a value under a key and look it up, update it, or delete it by that key. It is the family you will use most, and the one to reach for whenever the other six do not obviously fit.\n\n## What and why\n\nA **Documents collection** maps a `@data` key to a `@data` value: one value per key, which you can read, replace, and remove. Think users by user id, posts by post id, orders by order number. If you are storing \"a thing with fields that I look up and update by its id,\" this is the family.\n\nDeclare one by typing a `@collection` as `Documents<Key, Value>`:\n\n```ts\n@data\nclass UserId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass User {\n id: string = '';\n name: string = '';\n email: string = '';\n score: u64 = 0;\n}\n\n@database\nclass AppDb {\n @collection static users: Documents<UserId, User>;\n}\n```\n\n## The operations\n\nHere is every operation, its shape, and what it gives back. `K` is your key type, `V` your value type.\n\n| Operation | Signature | Returns | Use it to |\n| --- | --- | --- | --- |\n| `get` | `get(key: K): V \\| null` | the value, or `null` if absent | read one record |\n| `require` | `require(key: K): V` | the value; **traps** if absent | read a record you are sure exists |\n| `getMany` | `getMany(keys: K[]): Array<V \\| null>` | one entry per key, in order, each value or `null` | read several records in one call |\n| `exists` | `exists(key: K): bool` | `true` if the record is present | check presence without reading the value |\n| `create` | `create(key: K, value: V): bool` | `true` if inserted, `false` if the key was already taken | add a **new** record without overwriting |\n| `patch` | `patch(key: K, value: V): V` | the newly stored value; **traps** if the record is absent | replace an **existing** record's value |\n| `upsert` | `upsert(key: K, value: V): UpsertResult` | `UpsertResult.Created` or `.Updated`; `.Conflict` on a `@unique` collision (nothing written) | create **or** overwrite in one op (last-writer-wins) |\n| `enqueue` | `enqueue(key: K, value: V): bool` | `true` if applied, `false` if a concurrent write won first or the record is absent | a version-checked (compare-and-swap) overwrite of an existing record |\n| `delete` | `delete(key: K): void` | nothing (idempotent) | remove a record |\n| `getDelete` | `getDelete(key: K): V \\| null` | the value that was there, or `null`; removes it atomically | consume a record exactly once |\n\nWhich kind of function may call which operation is covered in [Setup](./setup.md#how-access-is-gated-query-action-and-friends). In short: reads (`get`, `getMany`, `exists`) work anywhere; writes (`create`, `patch`, `upsert`, `enqueue`, `delete`, `getDelete`) need an **Action** (a `@post` route or an `@action`).\n\n### Reading: `get`, `require`, `exists`, `getMany`\n\n`get` is the everyday read. It returns the value or `null`, so you handle \"not found\" explicitly:\n\n```ts\nconst user = AppDb.users.get(new UserId('u_123'));\nif (user == null) {\n return Response.notFound();\n}\n// user is a fully typed User here\n```\n\n`require` is `get` for the case where absence is a bug, not a normal outcome: it returns the value directly and traps (aborts the request) if the record is missing. Use it only when you have already guaranteed the record exists.\n\n`exists` answers \"is there a record here?\" without paying to decode the value. It is handy as a cheap precondition:\n\n```ts\nif (AppDb.users.exists(new UserId(name))) {\n // username already registered, do not overwrite\n}\n```\n\n`getMany` reads several keys in a single operation. You hand it an array of keys; you get back an array the same length and in the same order, each entry either the value or `null` for a key that was absent. Reach for it instead of a loop of `get` calls when you already know the handful of keys you need.\n\n```ts\nconst ids = [new UserId('a'), new UserId('b'), new UserId('c')];\nconst found: Array<User | null> = AppDb.users.getMany(ids);\n// found[0] lines up with ids[0], and so on; each is a User or null\n```\n\n`getMany` is a **bounded batch of point reads**, not a scan: the number of keys you may pass is capped by the request budget, and it never walks the whole collection. There is no \"get all records\" operation on a request path, by design (an unbounded scan could fan out across a huge collection). If you need \"the latest N of something,\" model it as [Events](./events.md) or precompute a [View](./views.md).\n\n### Writing: `create` vs `upsert` vs `patch` vs `enqueue`\n\nThese four all put a value under a key, but they differ in one important way each. Choosing correctly is the heart of using this family.\n\n```mermaid\nflowchart TD\n START[\"I want to write a record\"] --> Q1{\"Must this write fail if another<br/>request changed the record<br/>since I read it?\"}\n Q1 -->|\"Yes, guard against a lost update\"| ENQUEUE[\"enqueue<br/>(version-checked CAS;<br/>false = retry)\"]\n Q1 -->|\"No, last write wins\"| Q2{\"Does the record<br/>already exist?\"}\n Q2 -->|\"Must be new (never clobber)\"| CREATE[\"create<br/>(false if the key is taken)\"]\n Q2 -->|\"May or may not exist\"| UPSERT[\"upsert<br/>(create-or-overwrite in one op)\"]\n Q2 -->|\"Definitely exists\"| PATCH[\"patch<br/>(overwrite; returns the value;<br/>traps if absent)\"]\n```\n\n**`create` inserts a new record.** It only writes if the key is free. If the key already has a record, `create` does nothing and returns `false`. This is your tool for \"sign up a new user\" or \"claim this order id,\" where accidentally overwriting an existing record would be a bug. Because every key is serialized at its home (see [eventual consistency](./README.md#eventual-consistency-in-plain-words)), `create` is race-safe: if two requests create the same key at the same instant, exactly one gets `true` and the other gets `false`.\n\n```ts\nconst ok = AppDb.users.create(new UserId(input.id), input);\nif (!ok) {\n return Response.text('that id is taken', 409);\n}\n```\n\n**`patch` overwrites an existing record** and returns the value now stored. The record **must already exist**: `patch` on a missing key traps (aborts the request), so create it first. Despite the name, `patch` replaces the whole value; there is no field-level partial update, so read the current value, change the fields you want, and patch the whole thing back:\n\n```ts\nconst current = AppDb.users.get(new UserId('u_123'));\nif (current == null) return Response.notFound();\ncurrent.score = current.score + 10;\nconst saved: User = AppDb.users.patch(new UserId('u_123'), current);\n// saved is what is now stored\n```\n\n**`enqueue` is a version-checked overwrite (a compare-and-swap).** Like `patch`, it replaces the whole value of an **existing** record, but it does so *only if the record has not changed since you read it*. A **compare-and-swap** (CAS) is exactly that: \"write my new value, but only if the current value is still the one I saw.\" It returns a `bool`: `true` means your write was applied; `false` means either a concurrent write changed the record first (someone else beat you to it) or the record is absent. A `false` is **not an error**; it is the signal to **re-read and try again**. This approach is called **optimistic concurrency**: rather than locking the record, you assume nobody else will touch it, and you simply re-run the update on the rare occasion someone did.\n\nReach for `enqueue` when several requests may update the *same* record at once and you must not silently lose any of their changes. A plain `patch` cannot promise that: two overlapping patches clobber each other (the last writer wins and the earlier update just vanishes). The intended pattern for `enqueue` is always a read-modify-CAS **retry loop**:\n\n```ts\nconst key = new UserId('u_123');\nfor (let attempt = 0; attempt < 5; attempt++) {\n const current = AppDb.users.get(key); // 1. read the current value\n if (current == null) return Response.notFound();\n current.score = current.score + 10; // 2. modify your copy\n if (AppDb.users.enqueue(key, current)) { // 3. try to commit it\n return Response.text('ok'); // true: applied, we are done\n }\n // false: someone wrote between our get and our enqueue.\n // Loop: re-read the now-newer value and reapply the change on top of it.\n}\nreturn Response.text('too much contention, try again later', 409);\n```\n\nBecause every retry re-reads the latest value, the two updates **compose** (both `+10`s land) instead of one silently overwriting the other. If you do not need this guard (only one writer touches the key, or last-write-wins is genuinely fine), a plain `patch` is simpler and also hands you the stored value back directly.\n\n**`upsert` creates the record or overwrites it, in one operation.** It is the \"just store this value, I do not care whether a row was already there\" call, and it is the right tool for a save that runs repeatedly (a profile edit, a settings blob) where the first save inserts and every later save replaces. It returns an `UpsertResult` telling you which happened:\n\n```ts\nconst key = new UserId('u_123');\nconst result = AppDb.users.upsert(key, user);\n// result is UpsertResult.Created (row was absent) or UpsertResult.Updated (overwrote)\n```\n\nBefore `upsert`, that meant two operations, `create` then `patch`:\n\n```ts\n// the old two-op idiom - upsert replaces it\nif (!AppDb.users.create(key, user)) {\n AppDb.users.patch(key, user);\n}\n```\n\n`upsert` collapses both into a single write, so the common \"already exists\" path costs one round trip instead of two.\n\nLike `patch`, it is a **last-writer-wins** overwrite, not a compare-and-swap: concurrent upserts never fail or retry, the later one simply wins. That makes it correct for writing a **whole value**, and wrong for a read-modify-write of accumulating state (a running total, a like count) where a concurrent write would be silently lost, use `enqueue`'s retry loop or a [Counter](./counters.md) there. On a collection with a `@unique` field, `upsert` returns `UpsertResult.Conflict` (and writes nothing) when the value's unique field is already held by a **different** record, the one case it cannot resolve by overwriting:\n\n```ts\nconst outcome = AppDb.handles.upsert(key, profile);\nif (outcome == UpsertResult.Conflict) {\n return Response.text('that handle is taken', 409);\n}\n```\n\n### Removing: `delete` and `getDelete`\n\n`delete` removes a record. It is **idempotent**: deleting a key that is already gone is not an error, it just does nothing. So you can call it without first checking that the record exists.\n\n```ts\nAppDb.users.delete(new UserId('u_123'));\n```\n\n`getDelete` is the atomic **fetch-and-remove**: in one indivisible step it reads the current value and deletes it, returning what it removed (or `null` if there was nothing). \"Atomic\" here means no other request can slip in between the read and the delete, so exactly one caller can ever receive a given value. That makes it the right tool for **consume-once** data: one-time login challenges, single-use invite codes, password-reset tokens. The PQ-auth demo uses it to consume a login challenge exactly once, so a challenge cannot be replayed:\n\n```ts\nconst challenge = AppDb.challenges.getDelete(new ChallengeId(cid));\nif (challenge == null) return fail(); // unknown, already used, or expired\n// ...verify against challenge...\n```\n\nIf two requests race to `getDelete` the same key, only one gets the value; the other gets `null`. That is the guarantee a plain `get` then `delete` cannot give you, because two racers could both `get` the value before either `delete`s it.\n\n## A full worked example: a small CRUD entity\n\nPutting it together, here is a complete `notes` resource: create, read, update, and delete, backed by a Documents collection.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n// ---- key + value ----\n@data\nclass NoteId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass Note {\n id: string = '';\n title: string = '';\n body: string = '';\n updatedAt: u64 = 0;\n}\n\n// ---- database ----\n@database\nclass NotesDb {\n @collection static notes: Documents<NoteId, Note>;\n}\n\n// ---- routes ----\n@rest('notes')\nclass Notes {\n // GET /notes/:id -> read one (Query: read-only)\n @get('/:id')\n public read(ctx: RouteContext): Response {\n const note = NotesDb.notes.get(new NoteId(ctx.param('id')));\n if (note == null) return Response.notFound();\n return Response.json(note.toJSON().toString());\n }\n\n // POST /notes -> create a new one, refusing a duplicate id (Action: may write)\n @post('/')\n public create(input: Note): Response {\n input.updatedAt = <u64>(Date.now() / 1000);\n if (!NotesDb.notes.create(new NoteId(input.id), input)) {\n return Response.text('id already exists', 409);\n }\n return Response.json(input.toJSON().toString());\n }\n\n // POST /notes/:id -> overwrite an existing note (Action)\n @post('/:id')\n public update(input: Note, ctx: RouteContext): Response {\n const key = new NoteId(ctx.param('id'));\n if (!NotesDb.notes.exists(key)) return Response.notFound();\n input.id = ctx.param('id');\n input.updatedAt = <u64>(Date.now() / 1000);\n const saved = NotesDb.notes.patch(key, input);\n return Response.json(saved.toJSON().toString());\n }\n\n // POST /notes/:id/delete -> remove one (Action)\n @post('/:id/delete')\n public remove(ctx: RouteContext): Response {\n NotesDb.notes.delete(new NoteId(ctx.param('id')));\n return Response.text('deleted');\n }\n}\n```\n\nThat is a complete persistent CRUD entity. Run it under `toiljs dev` and the notes survive across requests; deploy it and the same code stores them worldwide on the edge.\n\n## Consistency notes\n\nDocuments follows ToilDB's general model (see [the overview](./README.md#eventual-consistency-in-plain-words)):\n\n- **Writes to one key are serialized at that key's home**, so `create` is race-safe and `patch`/`upsert`/`enqueue`/`getDelete` never corrupt a record under concurrency. `upsert` is last-writer-wins like `patch`: serialization means the writes do not tear, but the later one still overwrites the earlier, so it does not by itself prevent a lost update (use `enqueue` for that).\n- **Reads are eventually consistent across regions.** Right after a write, a read from a far-away region may briefly still see the old value (or, for a just-created record, not see it yet). The copies converge within moments.\n- Because `patch` replaces the whole value, two updates to *different* fields of the same record can clobber each other if they overlap (read-modify-write races). `enqueue`'s version check is exactly the guard against that: use the read-modify-CAS retry loop shown above so a lost update turns into a retry instead of silent data loss. If you find yourself contending on one hot record a lot, a counter or a set is often a better fit than a Documents value. See [Counters](./counters.md) and [Membership](./membership.md).\n\n## Gotchas\n\n- **`patch` requires an existing record.** Calling it on a missing key traps the request. Use `create` for new records, or `upsert` when the record may or may not exist yet.\n- **`patch` replaces the whole value.** There is no field-level merge; read, modify, and write back the full value.\n- **`enqueue` returning `false` is not a failure.** It means a concurrent write beat you to the record (or the record is absent), so re-read and retry in a loop; never ignore the return value or treat `false` as a hard error. `enqueue` also does not hand back the stored value; use `patch` when you want the value returned and last-write-wins is acceptable.\n- **`upsert` is last-writer-wins, not a merge.** It writes the whole value you pass, so it is for storing a complete value, not a read-modify-write of one field of a shared record. Two overlapping upserts do not error, but the earlier one is overwritten (lost). When several writers contend on one record and must not lose each other's changes, use `enqueue`'s retry loop or a [Counter](./counters.md).\n- **No \"get all.\"** There is no scan on the request path. Use `getMany` for known keys, and [Events](./events.md) or a [View](./views.md) for \"the latest N.\"\n- **`getDelete`, not `get` + `delete`, for consume-once.** Only `getDelete` guarantees exactly one caller receives the value.\n\n## Related\n\n- [ToilDB overview](./README.md): the seven families and how to choose.\n- [Setup](./setup.md): declaring the collection and which function kinds may write.\n- [Data types (`@data`)](../backend/data.md): keys and values.\n- [Counters](./counters.md): when you are really just counting.\n- [Events](./events.md) and [Views](./views.md): for \"the latest N\" and precomputed reads.\n",
26
- "database/events.md": "# Events (append-only logs)\n\nAn `Events` collection is an **append-only log**: a growing list of immutable\nfacts under a key. You add events to the end and read the newest ones back. You\nnever edit or delete an event once it is written.\n\n## What and why\n\nAn \"append-only log\" means exactly two things happen to it:\n\n1. You **append** a new event (it goes on the end).\n2. You **read** events back (newest first).\n\nThat is it. There is no \"update event 5\" and no \"delete event 3\". Each event is a\npermanent record of something that happened: \"Ada signed the guestbook\", \"order\n1234 was refunded\", \"user logged in from a new device\".\n\nUse an `Events` log when you care about the **history**, not just the latest\nstate:\n\n- **Activity feeds:** \"what happened in this room, newest first\".\n- **Audit trails:** a tamper-resistant record of every important action.\n- **Event sourcing:** treat the log as the source of truth and compute other\n values (totals, summaries, leaderboards) from it.\n\nIf you only ever need the *current* value of something (a user's profile, a\nshopping cart), use [Documents](./documents.md) instead. If you only need a\nrunning total (likes, page views), use a [Counter](./counters.md). Events are\nfor keeping the full list of what happened.\n\n```mermaid\nflowchart LR\n A[\"append(user, event)\"] --> L[(\"Event log for user<br/>(oldest ... newest)\")]\n B[\"append(user, event)\"] --> L\n L --> R[\"latest(user, 20)<br/>returns newest 20\"]\n```\n\n## The type\n\nAn `Events` collection has two type parameters: the **key** (which log) and the\n**event** (what each entry stores).\n\n```ts\nEvents<K, V>\n```\n\n- `K` is the key type: it picks *which* log you are appending to. One key, one\n log. For a per-user activity feed, the key is the user id.\n- `V` is the event type: the shape of a single entry. Both `K` and `V` are\n [`@data`](../concepts/types.md) classes so they can be stored as bytes.\n\nYou declare it as a `@collection` field inside a `@database` class:\n\n```ts\n@data\nclass UserKey {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass Activity {\n action: string = ''; // 'login', 'post.create', 'comment', ...\n detail: string = '';\n at: u64 = 0; // unix seconds\n}\n\n@database\nclass FeedDb {\n @collection static activity: Events<UserKey, Activity>;\n}\n```\n\nThe field name (`activity`) names the collection; the class name (`FeedDb`) names\nthe database. You reach the log through `FeedDb.activity`.\n\n## Operations\n\nThere are three operations, exactly matching the [toilscript](../concepts/decorators.md)\nAPI. Their exact signatures:\n\n| Operation | Signature | What it does |\n| --- | --- | --- |\n| `append` | `append(key: K, event: V): void` | Add one event to the end of the log. |\n| `appendOnce` | `appendOnce(key: K, eventId: string, event: V): bool` | Add one event, but only if that `eventId` was never seen before. |\n| `latest` | `latest(key: K, limit: i32): V[]` | Read up to `limit` events, newest first. |\n| `since` | `since(key: K, limit: i32): V[]` | Read the NEXT batch of events past a `@derive`'s checkpoint, oldest first. For folding a growing log incrementally. |\n\n### `append`\n\nThe everyday operation. It adds one event and returns nothing.\n\n```ts\nFeedDb.activity.append(\n new UserKey('ada'),\n new Activity('login', 'from Paris', now),\n);\n```\n\n`append` is a **write**, so you may call it from an action handler (`@post`,\n`@put`, `@patch`, `@del`) but not from a read-only `@get`. See\n[decorators](../concepts/decorators.md) for what each handler kind may do.\n\n### `appendOnce` (idempotent append)\n\n`appendOnce` adds an event **at most once** for a given id. \"Idempotent\" means\nrunning it more than once has the same effect as running it once: the extra calls\nchange nothing.\n\nWhy you want this: networks retry. A client (or another service) may send the\nsame request twice because the first reply got lost. With plain `append`, that\ndouble-delivery writes the event twice. With `appendOnce`, you pass a stable\n`eventId` (a string you choose that is unique for that logical event), and the\nlog records it only once.\n\n```ts\n// `orderId` uniquely identifies this order event. If the request is retried,\n// the second call is a no-op and the log still has exactly one entry.\nconst first = FeedDb.activity.appendOnce(\n new UserKey('ada'),\n `order-refunded:${orderId}`, // the idempotency id\n new Activity('order.refund', orderId, now),\n);\n// first === true -> this call appended the event\n// first === false -> a previous call already appended it; nothing happened now\n```\n\nThe return value tells you which happened:\n\n- `true`: this call appended the event (it was the first with that id).\n- `false`: a call with the same id already appended it, so this was a no-op.\n\nPick an `eventId` that is truly unique per event: an order id, a payment id, a\nmessage uuid. Do not reuse one id for two different events, or the second will be\nsilently dropped as a \"duplicate\".\n\n### `latest`\n\nReads the newest events back, up to a limit, newest first.\n\n```ts\nconst recent = FeedDb.activity.latest(new UserKey('ada'), 20);\n// recent[0] is the most recent event, recent[1] the next, ...\n```\n\n`latest` is a **scan**: it can walk many rows. Scans are barred on the request\npath, so you **cannot** call `latest` from a `@get` or a `@post`. This is a\ndeliberate guardrail: a log can grow without bound, and a per-request scan would\nget slower and more expensive as the log grows.\n\nInstead you scan the log **off** the request path, in a\n[`@derive`](../background/derive.md), and publish a small precomputed\n[View](./views.md) that your routes read cheaply. The next section shows the full\npattern.\n\n### `since` (incremental read for a `@derive`)\n\n`latest` rescans the tail every time. For a log that grows without bound, that gets slower and slower. `since`\nreads only the events you have **not folded yet**: the next batch past a saved cursor, oldest first, up to a\nlimit. It is how a [`@derive`](../background/derive.md) folds a huge log incrementally instead of rescanning\nit from the start on every change.\n\nYou do not pass or manage the cursor. The host owns it: it seeds `since` from a durable checkpoint, advances\nit as it hands you events, and saves it after your fold's view publish lands. So you just loop until the\nbatch is empty:\n\n```ts\n@derive\nrollup(): void {\n const key = new StatsKey('global');\n const view = StatsDb.summary.get(key) ?? new Summary(); // the running view\n let batch = StatsDb.events.since(key, 500);\n while (batch.length > 0) { // drain in bounded batches\n for (let i = 0; i < batch.length; i++) view.apply(batch[i]);\n batch = StatsDb.events.since(key, 500);\n }\n StatsDb.summary.publish(key, view);\n}\n```\n\nTwo rules:\n\n- **`@derive` or `@job` only.** Like `latest`, `since` is a scan, so it is barred on the request path (`@get`\n / `@post`). Calling it there is rejected.\n- **Your fold must be idempotent per event.** A rare crash-recovery case (a \"healed\" event that arrives at an\n older position) makes the host re-read the log from the start that one run, so applying an event twice must\n not change the result. Use set-style updates (`view.byId[e.id] = e.value`), not blind accumulation\n (`view.count += 1`). If your fold cannot be idempotent, use `latest` and recompute instead.\n\n`since` is the efficient path; `latest` (recompute) is the simple path. See [`@derive`](../background/derive.md)\nfor the full picture.\n\n## Worked example: an activity feed\n\nHere is the whole loop: an action appends an event, a `@derive` folds the log\ninto a view, and a `@get` serves the view.\n\n```ts\nimport { Activity } from '../models/Activity';\nimport { UserKey } from '../models/UserKey';\nimport { FeedView } from '../models/FeedView';\nimport { NewActivity } from '../models/NewActivity';\n\n@database\nclass FeedDb {\n // The source of truth: every activity, appended forever.\n @collection static activity: Events<UserKey, Activity>;\n // The precomputed snapshot the GET serves: the newest 20 activities.\n @collection static feed: View<UserKey, FeedView>;\n\n // A @derive runs OFF the request path, so it is allowed to scan the log\n // (`latest`) and publish the view. The runtime runs it right after an append,\n // so the view reflects the new event on the next read.\n @derive\n rebuild(): void {\n const key = new UserKey('ada'); // (real code would loop per active user)\n const view = new FeedView();\n view.items = FeedDb.activity.latest(key, 20);\n FeedDb.feed.publish(key, view);\n }\n}\n\n@rest('feed')\nclass Feed {\n // GET reads the precomputed view: a single keyed read, NOT a scan.\n @get('/')\n public list(): FeedView {\n const view = FeedDb.feed.get(new UserKey('ada'));\n return view == null ? new FeedView() : view;\n }\n\n // POST appends one activity. The @derive republishes `feed` right after.\n @post('/')\n public record(input: NewActivity): FeedView {\n const at = <u64>(Date.now() / 1000);\n FeedDb.activity.append(new UserKey('ada'), new Activity(input.action, input.detail, at));\n // We do NOT scan here (that would be barred). We just acknowledge; the GET\n // serves the updated list from the view the derive rebuilds.\n return new FeedView();\n }\n}\n```\n\nThe two `@data` models:\n\n```ts\n@data\nexport class Activity {\n action: string = '';\n detail: string = '';\n at: u64 = 0;\n}\n\n@data\nexport class FeedView {\n items: Activity[] = [];\n}\n```\n\nThe key idea: the **log** is the durable history, and the **view** is a small,\nread-cheap snapshot derived from it. Reads hit the view; the log grows quietly in\nthe background.\n\n## Ordering and consistency\n\nToilDB is a **worldwide** database: your data lives in many regions at once. A\nfew honest details about what that means for events:\n\n- **Ordering within a log is stable.** Each key (each log) has one \"home\" location\n where appends are serialized and stamped with an increasing sequence number. So\n `latest` always returns a consistent newest-first order for that key.\n- **`latest` is newest first.** Index `0` is the most recent event.\n- **Reads can lag slightly across regions.** An append is applied at the log's\n home first, then copied out to other regions in the background (this is called\n *asynchronous replication*, and the result is *eventual consistency*: given a\n little time, every region converges to the same log). A read served from a far\n region may briefly miss the very newest event. A `@derive` runs right after the\n append that triggered it, so the view it publishes reflects that write.\n- **`appendOnce` dedups by your id.** The at-most-once guarantee is anchored to\n the `eventId` you pass, so retries are safe even across a flaky network.\n- **A log grows without bound.** Appending never shrinks it. Keep events small,\n and read them through a bounded `latest(key, N)` in a derive rather than trying\n to hold the whole log in memory.\n\n## Gotchas\n\n- **You cannot read the log from a route.** `latest` is a scan and is only legal\n in a `@derive` or a `@job`. If you try to call it from a `@get`/`@post`, the\n compiler rejects it. Read a [View](./views.md) from your route instead.\n- **Events are immutable.** There is no \"edit\" or \"delete an event\". If a fact\n changes, append a new event that records the change (for example, an\n `order.refund` event, not an edit to the original `order.create`).\n- **Choose `eventId` carefully for `appendOnce`.** It must be unique per logical\n event. Reusing an id drops the new event as a duplicate.\n- **Do not treat a counter as a log.** A [Counter](./counters.md) gives you a\n running total but forgets the individual events. If you need the list, use\n `Events`.\n\n## Related\n\n- [Documents](./documents.md): mutable, one value per key (edit in place).\n- [Counters](./counters.md): a single running total per key.\n- [Views](./views.md): the precomputed snapshot your routes read.\n- [`@derive`](../background/derive.md): folds an event log into a View, off the\n request path.\n- [Data types (`@data`)](../concepts/types.md): how keys and events are stored.\n- [Decorators](../concepts/decorators.md): which handler kinds may append vs scan.\n",
25
+ "database/documents.md": "# Documents\n\nThe **Documents** family is ToilDB's general-purpose record store: you keep a value under a key and look it up, update it, or delete it by that key. It is the family you will use most, and the one to reach for whenever the other six do not obviously fit.\n\n## What and why\n\nA **Documents collection** maps a `@data` key to a `@data` value: one value per key, which you can read, replace, and remove. Think users by user id, posts by post id, orders by order number. If you are storing \"a thing with fields that I look up and update by its id,\" this is the family.\n\nDeclare one by typing a `@collection` as `Documents<Key, Value>`:\n\n```ts\n@data\nclass UserId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass User {\n id: string = '';\n name: string = '';\n email: string = '';\n score: u64 = 0;\n}\n\n@database\nclass AppDb {\n @collection static users: Documents<UserId, User>;\n}\n```\n\n## The operations\n\nHere is every operation, its shape, and what it gives back. `K` is your key type, `V` your value type.\n\n| Operation | Signature | Returns | Use it to |\n| --- | --- | --- | --- |\n| `get` | `get(key: K): V \\| null` | the value, or `null` if absent | read one record |\n| `require` | `require(key: K): V` | the value; **traps** if absent | read a record you are sure exists |\n| `getMany` | `getMany(keys: K[]): Array<V \\| null>` | one entry per key, in order, each value or `null` | read several records in one call |\n| `exists` | `exists(key: K): bool` | `true` if the record is present | check presence without reading the value |\n| `create` | `create(key: K, value: V): bool` | `true` if inserted, `false` if the key was already taken | add a **new** record without overwriting |\n| `patch` | `patch(key: K, value: V): V` | the newly stored value; **traps** if the record is absent | replace an **existing** record's value |\n| `upsert` | `upsert(key: K, value: V): UpsertResult` | `UpsertResult.Created` or `.Updated`; `.Conflict` on a `@unique` collision (nothing written) | create **or** overwrite in one op (last-writer-wins) |\n| `enqueue` | `enqueue(key: K, value: V): bool` | acceptance status | submit a replacement of an existing record; not caller-observed CAS |\n| `delete` | `delete(key: K): void` | nothing (idempotent) | remove a record |\n| `getDelete` | `getDelete(key: K): V \\| null` | the value that was there, or `null`; removes it atomically | consume a record exactly once |\n\nWhich kind of function may call which operation is covered in [Setup](./setup.md#how-access-is-gated-query-action-and-friends). In short: reads (`get`, `getMany`, `exists`) work anywhere; writes (`create`, `patch`, `upsert`, `enqueue`, `delete`, `getDelete`) need an **Action** (a `@post` route or an `@action`).\n\n### Reading: `get`, `require`, `exists`, `getMany`\n\n`get` is the everyday read. It returns the value or `null`, so you handle \"not found\" explicitly:\n\n```ts\nconst user = AppDb.users.get(new UserId('u_123'));\nif (user == null) {\n return Response.notFound();\n}\n// user is a fully typed User here\n```\n\n`require` is `get` for the case where absence is a bug, not a normal outcome: it returns the value directly and traps (aborts the request) if the record is missing. Use it only when you have already guaranteed the record exists.\n\n`exists` answers \"is there a record here?\" without paying to decode the value. It is handy as a cheap precondition:\n\n```ts\nif (AppDb.users.exists(new UserId(name))) {\n // username already registered, do not overwrite\n}\n```\n\n`getMany` reads several keys in a single operation. You hand it an array of keys; you get back an array the same length and in the same order, each entry either the value or `null` for a key that was absent. Reach for it instead of a loop of `get` calls when you already know the handful of keys you need.\n\n```ts\nconst ids = [new UserId('a'), new UserId('b'), new UserId('c')];\nconst found: Array<User | null> = AppDb.users.getMany(ids);\n// found[0] lines up with ids[0], and so on; each is a User or null\n```\n\n`getMany` is a **bounded batch of point reads**, not a scan: the number of keys you may pass is capped by the request budget, and it never walks the whole collection. There is no \"get all records\" operation on a request path, by design (an unbounded scan could fan out across a huge collection). If you need \"the latest N of something,\" model it as [Events](./events.md) or precompute a [View](./views.md).\n\nIn `toiljs dev` and the current production host, one `getMany` call accepts up to **32 keys in a Query**, **64 in an Action**, or **1024 in background function kinds**. Each encoded key may be at most **4096 bytes**, and the complete encoded keys frame (the count, length prefixes, and key bytes) may be at most **2097152 bytes (2 MiB)**. Split a larger list into smaller calls; production also applies cumulative request budgets, so separate requests may be needed for a large job.\n\n### Size limits and file uploads\n\nA stored value may be at most **2097152 encoded bytes (2 MiB)**, including the `@data` message id, field lengths, and other codec overhead. The limit applies in both `toiljs dev` and production. A photo can be stored whole when its encoded record fits this limit; a photo with 2 MiB of raw content will exceed it once wrapped in a record. `create`, `patch`, `upsert`, and `enqueue` all enforce the same limit; changing the write method does not bypass it. Events, View values, Unique owners, and Membership members have the same encoded value cap.\n\nProduction also applies host-configured limits to individual database results and the total result bytes in a request. A `getMany` result includes all returned rows and their framing, so its key-count limit does not guarantee that the result fits. The local emulator enforces value, key, and batch-size caps, but does not emulate these result-byte budgets or production DB wait deadlines. Size reads for the deployed host's request budgets as well as the 2 MiB per-value cap.\n\nFor files larger than the encoded value or request budgets allow, store the content in external file storage and keep its reference and metadata in Documents, or store chunks under separate keys:\n\n1. Give each upload an id and store each chunk under `(uploadId, chunkIndex)`. Reserve room below 2 MiB for the record's codec overhead, and choose smaller chunks if the deployed request result budget requires them. Check your complete encoded record and result sizes before writing it.\n2. Store upload metadata separately: content type, total byte count, expected chunk count, and an expiry. Bound those counts and the number of chunks processed by any one request.\n3. Validate that every expected chunk arrived before publishing a completed upload. Readers use the completion marker so an interrupted upload is not mistaken for a complete file. Retry chunk writes with `upsert` when replacing that chunk is acceptable.\n4. Read known chunk keys with bounded `getMany` calls or individual `get` calls, keeping every result and the request's total under the deployed budgets. Remove abandoned chunks using background work. Allow for replication delay when verifying newly written chunks from a different cell.\n\nThe local HTTP request body limit is a separate **8 MiB** default: accepting an upload body does not make that body a valid database value. See [route execution and transport limits](../concepts/config.md#route-execution-and-transport-limits) and [upload expiry versus elapsed deadlines](../services/time.md#upload-expiry-versus-elapsed-deadlines).\n\n### Writing: `create` vs `upsert` vs `patch` vs `enqueue`\n\nThese four all put a value under a key, but they differ in one important way each. Choosing correctly is the heart of using this family.\n\n```mermaid\nflowchart TD\n START[\"I want to write a record\"] --> Q1{\"Must this write fail if another<br/>request changed the record<br/>since I read it?\"}\n Q1 -->|\"Yes, guard against a lost update\"| ENQUEUE[\"Counter or explicit<br/>conditional-write protocol\"]\n Q1 -->|\"No, last write wins\"| Q2{\"Does the record<br/>already exist?\"}\n Q2 -->|\"Must be new (never clobber)\"| CREATE[\"create<br/>(false if the key is taken)\"]\n Q2 -->|\"May or may not exist\"| UPSERT[\"upsert<br/>(create-or-overwrite in one op)\"]\n Q2 -->|\"Definitely exists\"| PATCH[\"patch<br/>(overwrite; returns the value;<br/>traps if absent)\"]\n```\n\n**`create` inserts a new record.** It only writes if the key is free. If the key already has a record, `create` does nothing and returns `false`. This is your tool for \"sign up a new user\" or \"claim this order id,\" where accidentally overwriting an existing record would be a bug. Because every key is serialized at its home (see [eventual consistency](./README.md#eventual-consistency-in-plain-words)), `create` is race-safe: if two requests create the same key at the same instant, exactly one gets `true` and the other gets `false`.\n\n```ts\nconst ok = AppDb.users.create(new UserId(input.id), input);\nif (!ok) {\n return Response.text('that id is taken', 409);\n}\n```\n\n**`patch` overwrites an existing record** and returns the value now stored. The record **must already exist**: `patch` on a missing key traps (aborts the request), so create it first. Despite the name, `patch` replaces the whole value; there is no field-level partial update, so read the current value, change the fields you want, and patch the whole thing back:\n\n```ts\nconst current = AppDb.users.get(new UserId('u_123'));\nif (current == null) return Response.notFound();\ncurrent.score = current.score + 10;\nconst saved: User = AppDb.users.patch(new UserId('u_123'), current);\n// saved is what is now stored\n```\n\n**`enqueue` submits a replacement of an existing record.** Its current ABI carries\nonly the new value. It does not carry the version or bytes observed by your earlier\n`get`, so it cannot protect an application read-modify-write from lost updates.\nThe production host may compare a version read *inside* the write operation; that\nis not a comparison against the application's earlier read. A retry loop around\n`get` and `enqueue` does not fix this gap. Acceptance also must not be treated as a\nportable durable-commit acknowledgement.\n\nUse a Counter for independent additive totals. Financial state spanning several\nrecords requires a tested conditional-write/recovery protocol. The native host\nnow exposes `data.compare_exchange` (explicit expected bytes) and\n`data.get_current` (owner read), mirrored by the development emulator. These are\nlow-level host imports; the typed Documents API does not yet wrap them. They\ncurrently require local key ownership and reject remote-owner execution.\n\n**`upsert` creates the record or overwrites it, in one operation.** It is the \"just store this value, I do not care whether a row was already there\" call, and it is the right tool for a save that runs repeatedly (a profile edit, a settings blob) where the first save inserts and every later save replaces. It returns an `UpsertResult` telling you which happened:\n\n```ts\nconst key = new UserId('u_123');\nconst result = AppDb.users.upsert(key, user);\n// result is UpsertResult.Created (row was absent) or UpsertResult.Updated (overwrote)\n```\n\nBefore `upsert`, that meant two operations, `create` then `patch`:\n\n```ts\n// the old two-op idiom - upsert replaces it\nif (!AppDb.users.create(key, user)) {\n AppDb.users.patch(key, user);\n}\n```\n\n`upsert` collapses both into a single write, so the common \"already exists\" path costs one round trip instead of two.\n\nLike `patch`, it is a **last-writer-wins** overwrite, not a compare-and-swap: concurrent upserts never fail or retry, the later one simply wins. That makes it correct for writing a **whole value**, and wrong for a read-modify-write of accumulating state (a running total, a like count) where a concurrent write would be silently lost, use a [Counter](./counters.md) for additive state or an explicit conditional-write protocol there. On a collection with a `@unique` field, `upsert` returns `UpsertResult.Conflict` (and writes nothing) when the value's unique field is already held by a **different** record, the one case it cannot resolve by overwriting:\n\n```ts\nconst outcome = AppDb.handles.upsert(key, profile);\nif (outcome == UpsertResult.Conflict) {\n return Response.text('that handle is taken', 409);\n}\n```\n\n### Removing: `delete` and `getDelete`\n\n`delete` removes a record. It is **idempotent**: deleting a key that is already gone is not an error, it just does nothing. So you can call it without first checking that the record exists.\n\n```ts\nAppDb.users.delete(new UserId('u_123'));\n```\n\n`getDelete` is the atomic **fetch-and-remove**: in one indivisible step it reads the current value and deletes it, returning what it removed (or `null` if there was nothing). \"Atomic\" here means no other request can slip in between the read and the delete, so exactly one caller can ever receive a given value. That makes it the right tool for **consume-once** data: one-time login challenges, single-use invite codes, password-reset tokens. The PQ-auth demo uses it to consume a login challenge exactly once, so a challenge cannot be replayed:\n\n```ts\nconst challenge = AppDb.challenges.getDelete(new ChallengeId(cid));\nif (challenge == null) return fail(); // unknown, already used, or expired\n// ...verify against challenge...\n```\n\nIf two requests race to `getDelete` the same key, only one gets the value; the other gets `null`. That is the guarantee a plain `get` then `delete` cannot give you, because two racers could both `get` the value before either `delete`s it.\n\n## A full worked example: a small CRUD entity\n\nPutting it together, here is a complete `notes` resource: create, read, update, and delete, backed by a Documents collection.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n// ---- key + value ----\n@data\nclass NoteId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass Note {\n id: string = '';\n title: string = '';\n body: string = '';\n updatedAt: u64 = 0;\n}\n\n// ---- database ----\n@database\nclass NotesDb {\n @collection static notes: Documents<NoteId, Note>;\n}\n\n// ---- routes ----\n@rest('notes')\nclass Notes {\n // GET /notes/:id -> read one (Query: read-only)\n @get('/:id')\n public read(ctx: RouteContext): Response {\n const note = NotesDb.notes.get(new NoteId(ctx.param('id')));\n if (note == null) return Response.notFound();\n return Response.json(note.toJSON().toString());\n }\n\n // POST /notes -> create a new one, refusing a duplicate id (Action: may write)\n @post('/')\n public create(input: Note): Response {\n input.updatedAt = <u64>(Date.now() / 1000);\n if (!NotesDb.notes.create(new NoteId(input.id), input)) {\n return Response.text('id already exists', 409);\n }\n return Response.json(input.toJSON().toString());\n }\n\n // POST /notes/:id -> overwrite an existing note (Action)\n @post('/:id')\n public update(input: Note, ctx: RouteContext): Response {\n const key = new NoteId(ctx.param('id'));\n if (!NotesDb.notes.exists(key)) return Response.notFound();\n input.id = ctx.param('id');\n input.updatedAt = <u64>(Date.now() / 1000);\n const saved = NotesDb.notes.patch(key, input);\n return Response.json(saved.toJSON().toString());\n }\n\n // POST /notes/:id/delete -> remove one (Action)\n @post('/:id/delete')\n public remove(ctx: RouteContext): Response {\n NotesDb.notes.delete(new NoteId(ctx.param('id')));\n return Response.text('deleted');\n }\n}\n```\n\nThat is a complete persistent CRUD entity. Run it under `toiljs dev` and the notes survive across requests; deploy it and the same code stores them worldwide on the edge.\n\n## Consistency notes\n\nDocuments follows ToilDB's general model (see [the overview](./README.md#eventual-consistency-in-plain-words)):\n\n- **Writes to one key are serialized at that key's home**, so `create` is race-safe and `patch`/`upsert`/`enqueue`/`getDelete` never corrupt a record under concurrency. `upsert` is last-writer-wins like `patch`: serialization means the writes do not tear, but the later one still overwrites the earlier, so it does not by itself prevent a lost update (use an explicit caller-observed conditional write for that).\n- **Reads are eventually consistent across regions.** Right after a write, a read from a far-away region may briefly still see the old value (or, for a just-created record, not see it yet). The copies converge within moments.\n- Because `patch` replaces the whole value, two updates to *different* fields of the same record can clobber each other if they overlap (read-modify-write races). Neither `patch` nor `enqueue` accepts an expectation from the earlier application read. If you find yourself contending on one hot record a lot, a counter or a set is often a better fit than a Documents value. See [Counters](./counters.md) and [Membership](./membership.md).\n\n## Gotchas\n\n- **`patch` requires an existing record.** Calling it on a missing key traps the request. Use `create` for new records, or `upsert` when the record may or may not exist yet.\n- **`patch` replaces the whole value.** There is no field-level merge; read, modify, and write back the full value.\n- **`enqueue` is not caller-observed CAS.** Do not use its boolean result as proof that a prior application read was still current.\n- **`upsert` is last-writer-wins, not a merge.** It writes the whole value you pass, so it is for storing a complete value, not a read-modify-write of one field of a shared record. Two overlapping upserts do not error, but the earlier one is overwritten (lost). When several writers contend on one record and must not lose each other's changes, use an explicit conditional-write protocol or a [Counter](./counters.md).\n- **No \"get all.\"** There is no scan on the request path. Use `getMany` for known keys, and [Events](./events.md) or a [View](./views.md) for \"the latest N.\"\n- **`getDelete`, not `get` + `delete`, for consume-once.** Only `getDelete` guarantees exactly one caller receives the value.\n\n## Related\n\n- [ToilDB overview](./README.md): the seven families and how to choose.\n- [Setup](./setup.md): declaring the collection and which function kinds may write.\n- [Data types (`@data`)](../backend/data.md): keys and values.\n- [Counters](./counters.md): when you are really just counting.\n- [Events](./events.md) and [Views](./views.md): for \"the latest N\" and precomputed reads.\n",
26
+ "database/events.md": "# Events (append-only logs)\n\nAn `Events` collection is an **append-only log**: a growing list of immutable\nfacts under a key. You add events to the end and read the newest ones back. You\nnever edit or delete an event once it is written.\n\n## What and why\n\nAn \"append-only log\" means exactly two things happen to it:\n\n1. You **append** a new event (it goes on the end).\n2. You **read** events back (newest first).\n\nThat is it. There is no \"update event 5\" and no \"delete event 3\". Each event is a\npermanent record of something that happened: \"Ada signed the guestbook\", \"order\n1234 was refunded\", \"user logged in from a new device\".\n\nUse an `Events` log when you care about the **history**, not just the latest\nstate:\n\n- **Activity feeds:** \"what happened in this room, newest first\".\n- **Audit trails:** a tamper-resistant record of every important action.\n- **Event sourcing:** treat the log as the source of truth and compute other\n values (totals, summaries, leaderboards) from it.\n\nIf you only ever need the *current* value of something (a user's profile, a\nshopping cart), use [Documents](./documents.md) instead. If you only need a\nrunning total (likes, page views), use a [Counter](./counters.md). Events are\nfor keeping the full list of what happened.\n\n```mermaid\nflowchart LR\n A[\"append(user, event)\"] --> L[(\"Event log for user<br/>(oldest ... newest)\")]\n B[\"append(user, event)\"] --> L\n L --> R[\"latest(user, 20)<br/>returns newest 20\"]\n```\n\n## The type\n\nAn `Events` collection has two type parameters: the **key** (which log) and the\n**event** (what each entry stores).\n\n```ts\nEvents<K, V>\n```\n\n- `K` is the key type: it picks *which* log you are appending to. One key, one\n log. For a per-user activity feed, the key is the user id.\n- `V` is the event type: the shape of a single entry. Both `K` and `V` are\n [`@data`](../concepts/types.md) classes so they can be stored as bytes.\n\nYou declare it as a `@collection` field inside a `@database` class:\n\n```ts\n@data\nclass UserKey {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass Activity {\n action: string = ''; // 'login', 'post.create', 'comment', ...\n detail: string = '';\n at: u64 = 0; // unix seconds\n}\n\n@database\nclass FeedDb {\n @collection static activity: Events<UserKey, Activity>;\n}\n```\n\nThe field name (`activity`) names the collection; the class name (`FeedDb`) names\nthe database. You reach the log through `FeedDb.activity`.\n\n## Operations\n\nThe [toilscript](../concepts/decorators.md) API exposes these operations:\n\n| Operation | Signature | What it does |\n| --- | --- | --- |\n| `append` | `append(key: K, event: V): void` | Add one event to the end of the log. |\n| `appendOnce` | `appendOnce(key: K, eventId: string, event: V): bool` | Add one event, but only if that `eventId` was never seen before. |\n| `get` | `get(key: K, eventId: string): V \\| null` | Read the event stored by `appendOnce` under this ID. Bounded request-path read. |\n| `last` | `last(key: K): V \\| null` | Read the newest appended event. Bounded request-path read. |\n| `latest` | `latest(key: K, limit: i32): V[]` | Read up to `limit` events, newest first. |\n| `since` | `since(key: K, limit: i32): V[]` | Read the NEXT batch of events past a `@derive`'s checkpoint, oldest first. For folding a growing log incrementally. |\n\n### `append`\n\nThe everyday operation. It adds one event and returns nothing.\n\n```ts\nFeedDb.activity.append(\n new UserKey('ada'),\n new Activity('login', 'from Paris', now),\n);\n```\n\n`append` is a **write**, so you may call it from an action handler (`@post`,\n`@put`, `@patch`, `@del`) but not from a read-only `@get`. See\n[decorators](../concepts/decorators.md) for what each handler kind may do.\n\n### `appendOnce` (idempotent append)\n\n`appendOnce` adds an event **at most once** for a given id. \"Idempotent\" means\nrunning it more than once has the same effect as running it once: the extra calls\nchange nothing.\n\nWhy you want this: networks retry. A client (or another service) may send the\nsame request twice because the first reply got lost. With plain `append`, that\ndouble-delivery writes the event twice. With `appendOnce`, you pass a stable\n`eventId` (a string you choose that is unique for that logical event), and the\nlog records it only once.\n\n```ts\n// `orderId` uniquely identifies this order event. If the request is retried,\n// the second call is a no-op and the log still has exactly one entry.\nconst first = FeedDb.activity.appendOnce(\n new UserKey('ada'),\n `order-refunded:${orderId}`, // the idempotency id\n new Activity('order.refund', orderId, now),\n);\n// first === true -> this call appended the event\n// first === false -> a previous call already appended it; nothing happened now\n```\n\nThe return value tells you which happened:\n\n- `true`: this call appended the event (it was the first with that id).\n- `false`: a call with the same id already appended it, so this was a no-op.\n\nPick an `eventId` that is truly unique per event: an order id, a payment id, a\nmessage uuid. Do not reuse one id for two different events, or the second will be\nsilently dropped as a \"duplicate\".\n\n### `get` and `last`\n\nRead one event directly from a route, query, action, derive, or job:\n\n```ts\nconst event = FeedDb.activity.get(new UserKey('ada'), 'order-refunded:123');\nconst newest = FeedDb.activity.last(new UserKey('ada'));\n```\n\n`get` uses the exact string ID supplied to `appendOnce`, scoped to that collection\nand stream key. Ordinary `append` does not expose an ID for `get`. `last` reads the\nnewest appended event from either write method; it does not interpret application\nrevision numbers or sort event IDs. A duplicate `appendOnce` neither replaces the\noriginal event nor moves it to the end of the stream.\n\nBoth operations return `null` when absent. As with Documents reads, inspect\n`Db.lastError()` immediately after a null result to distinguish absence from a\nretryable storage failure. Values decode using their stored schema version, so\n`@migrate` applies without rewriting the immutable event.\n\nThese are bounded point reads, permitted on the request path. `latest` and `since`\nremain background-only scans. Point reads do not add compare-and-swap semantics,\nor validate revision sequences. Production async reads fetch from the stream’s home\ncell; synchronous execution reports unavailable when the home is remote. Event IDs\nand stream keys are limited to 4096 encoded bytes. Deploy the updated backend and\nmesh peers before deploying modules that call these methods.\n\nDev snapshots created before event ID positions were recorded cannot reliably\nassociate old deduplication IDs with events. For such IDs, `get` reports\n`DbError.Unavailable`; `last`, `latest`, and `since` can still read the stored events.\nNew `appendOnce` entries retain their ID positions across dev-server restarts.\n\n### `latest`\n\nReads the newest events back, up to a limit, newest first.\n\n```ts\nconst recent = FeedDb.activity.latest(new UserKey('ada'), 20);\n// recent[0] is the most recent event, recent[1] the next, ...\n```\n\n`latest` is a **scan**: it can walk many rows. Scans are barred on the request\npath, so you **cannot** call `latest` from a `@get` or a `@post`. This is a\ndeliberate guardrail: a log can grow without bound, and a per-request scan would\nget slower and more expensive as the log grows.\n\nInstead you scan the log **off** the request path, in a\n[`@derive`](../background/derive.md), and publish a small precomputed\n[View](./views.md) that your routes read cheaply. The next section shows the full\npattern.\n\n### `since` (incremental read for a `@derive`)\n\n`latest` rescans the tail every time. For a log that grows without bound, that gets slower and slower. `since`\nreads only the events you have **not folded yet**: the next batch past a saved cursor, oldest first, up to a\nlimit. It is how a [`@derive`](../background/derive.md) folds a huge log incrementally instead of rescanning\nit from the start on every change.\n\nYou do not pass or manage the cursor. The host owns it: it seeds `since` from a durable checkpoint, advances\nit as it hands you events, and saves it after your fold's view publish lands. So you just loop until the\nbatch is empty:\n\n```ts\n@derive\nrollup(): void {\n const key = new StatsKey('global');\n const view = StatsDb.summary.get(key) ?? new Summary(); // the running view\n let batch = StatsDb.events.since(key, 500);\n while (batch.length > 0) { // drain in bounded batches\n for (let i = 0; i < batch.length; i++) view.apply(batch[i]);\n batch = StatsDb.events.since(key, 500);\n }\n StatsDb.summary.publish(key, view);\n}\n```\n\nTwo rules:\n\n- **`@derive` or `@job` only.** Like `latest`, `since` is a scan, so it is barred on the request path (`@get`\n / `@post`). Calling it there is rejected.\n- **Your fold must be idempotent per event.** A rare crash-recovery case (a \"healed\" event that arrives at an\n older position) makes the host re-read the log from the start that one run, so applying an event twice must\n not change the result. Use set-style updates (`view.byId[e.id] = e.value`), not blind accumulation\n (`view.count += 1`). If your fold cannot be idempotent, use `latest` and recompute instead.\n\n`since` is the efficient path; `latest` (recompute) is the simple path. See [`@derive`](../background/derive.md)\nfor the full picture.\n\n## Worked example: an activity feed\n\nHere is the whole loop: an action appends an event, a `@derive` folds the log\ninto a view, and a `@get` serves the view.\n\n```ts\nimport { Activity } from '../models/Activity';\nimport { UserKey } from '../models/UserKey';\nimport { FeedView } from '../models/FeedView';\nimport { NewActivity } from '../models/NewActivity';\n\n@database\nclass FeedDb {\n // The source of truth: every activity, appended forever.\n @collection static activity: Events<UserKey, Activity>;\n // The precomputed snapshot the GET serves: the newest 20 activities.\n @collection static feed: View<UserKey, FeedView>;\n\n // A @derive runs OFF the request path, so it is allowed to scan the log\n // (`latest`) and publish the view. The runtime runs it right after an append,\n // so the view reflects the new event on the next read.\n @derive\n rebuild(): void {\n const key = new UserKey('ada'); // (real code would loop per active user)\n const view = new FeedView();\n view.items = FeedDb.activity.latest(key, 20);\n FeedDb.feed.publish(key, view);\n }\n}\n\n@rest('feed')\nclass Feed {\n // GET reads the precomputed view: a single keyed read, NOT a scan.\n @get('/')\n public list(): FeedView {\n const view = FeedDb.feed.get(new UserKey('ada'));\n return view == null ? new FeedView() : view;\n }\n\n // POST appends one activity. The @derive republishes `feed` right after.\n @post('/')\n public record(input: NewActivity): FeedView {\n const at = <u64>(Date.now() / 1000);\n FeedDb.activity.append(new UserKey('ada'), new Activity(input.action, input.detail, at));\n // We do NOT scan here (that would be barred). We just acknowledge; the GET\n // serves the updated list from the view the derive rebuilds.\n return new FeedView();\n }\n}\n```\n\nThe two `@data` models:\n\n```ts\n@data\nexport class Activity {\n action: string = '';\n detail: string = '';\n at: u64 = 0;\n}\n\n@data\nexport class FeedView {\n items: Activity[] = [];\n}\n```\n\nThe key idea: the **log** is the durable history, and the **view** is a small,\nread-cheap snapshot derived from it. Reads hit the view; the log grows quietly in\nthe background.\n\n## Ordering and consistency\n\nToilDB is a **worldwide** database: your data lives in many regions at once. A\nfew honest details about what that means for events:\n\n- **Ordering within a log is stable.** Each key (each log) has one \"home\" location\n where appends are serialized and stamped with an increasing sequence number. So\n `latest` always returns a consistent newest-first order for that key.\n- **`latest` is newest first.** Index `0` is the most recent event.\n- **Reads can lag slightly across regions.** An append is applied at the log's\n home first, then copied out to other regions in the background (this is called\n *asynchronous replication*, and the result is *eventual consistency*: given a\n little time, every region converges to the same log). A read served from a far\n region may briefly miss the very newest event. A `@derive` runs right after the\n append that triggered it, so the view it publishes reflects that write.\n- **`appendOnce` dedups by your id.** The at-most-once guarantee is anchored to\n the `eventId` you pass, so retries are safe even across a flaky network.\n- **A log grows without bound.** Appending never shrinks it. Keep events small,\n and read them through a bounded `latest(key, N)` in a derive rather than trying\n to hold the whole log in memory.\n\n## Gotchas\n\n- **Routes can read individual events, but cannot scan the log.** Use `get` or\n `last` for a single event. `latest` and `since` require a `@derive` or `@job`;\n use a [View](./views.md) when a route needs a precomputed list.\n- **Events are immutable.** There is no \"edit\" or \"delete an event\". If a fact\n changes, append a new event that records the change (for example, an\n `order.refund` event, not an edit to the original `order.create`).\n- **Choose `eventId` carefully for `appendOnce`.** It must be unique per logical\n event. Reusing an id drops the new event as a duplicate.\n- **Do not treat a counter as a log.** A [Counter](./counters.md) gives you a\n running total but forgets the individual events. If you need the list, use\n `Events`.\n\n## Related\n\n- [Documents](./documents.md): mutable, one value per key (edit in place).\n- [Counters](./counters.md): a single running total per key.\n- [Views](./views.md): the precomputed snapshot your routes read.\n- [`@derive`](../background/derive.md): folds an event log into a View, off the\n request path.\n- [Data types (`@data`)](../concepts/types.md): how keys and events are stored.\n- [Decorators](../concepts/decorators.md): which handler kinds may append vs scan.\n",
27
27
  "database/membership.md": "# Membership (sets and relationships)\n\nA `Membership` collection stores **sets**: unordered groups of members under a\nkey. You add and remove members, ask whether a member is in the set, and list a\nset's members. It is the natural fit for many-to-many relationships like group\nmembers, followers, and tags.\n\n## What and why\n\nA \"set\" is a collection with no duplicates and no order: a member is either in it\nor not. `Membership` gives you one set per key. The key names the set; each member\nis one item in it.\n\nThis maps directly onto **many-to-many relationships**, where each thing on one\nside can relate to many things on the other:\n\n- **Group members:** the key is a group id, the members are user ids.\n- **Followers:** the key is a user id, the members are the ids of their followers.\n- **Tags:** the key is an article id, the members are tag names.\n- **Access control:** the key is a resource id, the members are the users allowed\n in.\n\nYou could try to store these as an array field inside a [Document](./documents.md)\n(for example `group.memberIds: string[]`). That works for **small, rarely-changed**\nsets, but it has real limits:\n\n| | Array in a Document | `Membership` set |\n| --- | --- | --- |\n| Add/remove one member | Read the whole array, edit it, write it all back | One direct `add`/`remove`, no read-modify-write |\n| Concurrent edits | Two writers can clobber each other's array | Each `add`/`remove` is its own operation |\n| Big sets | The whole array is loaded on every read/write | Members are stored separately; you read a bounded page |\n| \"Is X a member?\" | Load the array and search it | One direct `contains` check |\n\nRule of thumb: a **handful** of stable items (a user's two or three roles) is fine\nas an array in a Document. A set that **grows** or is **edited concurrently**\n(followers, group members, tags across many articles) should be a `Membership`.\n\n```mermaid\nflowchart LR\n K[\"Group key: 'eng'\"] --> S[(\"Membership set\")]\n S --> M1[\"ada\"]\n S --> M2[\"grace\"]\n S --> M3[\"linus\"]\n```\n\n## The type\n\n```ts\nMembership<K, M>\n```\n\n- `K` is the key type: it picks *which* set. For group membership, the key is the\n group id.\n- `M` is the member type: the shape of one member. For user membership, that is a\n user id. Both are [`@data`](../concepts/types.md) classes.\n\nDeclare it as a `@collection` field on a `@database` class:\n\n```ts\n@data\nclass GroupKey {\n group: string = '';\n constructor(group: string = '') { this.group = group; }\n}\n\n@data\nclass Member {\n userId: string = '';\n constructor(userId: string = '') { this.userId = userId; }\n}\n\n@database\nclass GroupsDb {\n @collection static members: Membership<GroupKey, Member>;\n}\n```\n\n## Operations\n\nFour operations, matching the toilscript API exactly. Exact signatures:\n\n| Operation | Signature | What it does |\n| --- | --- | --- |\n| `contains` | `contains(key: K, member: M): bool` | Is `member` in the set? |\n| `add` | `add(key: K, member: M): void` | Put `member` in the set (idempotent). |\n| `remove` | `remove(key: K, member: M): void` | Take `member` out of the set (idempotent). |\n| `list` | `list(key: K, limit: i32): M[]` | Up to `limit` members of the set. |\n\n### `contains`\n\nA direct membership check. It is a keyed read, so it is allowed from any handler,\nincluding a read-only `@get`.\n\n```ts\nif (GroupsDb.members.contains(new GroupKey('eng'), new Member('ada'))) {\n // ada is in the 'eng' group\n}\n```\n\n### `add` and `remove`\n\nBoth are **writes**, so you call them from an action handler (`@post`, `@put`,\n`@patch`, `@del`), not from a `@get`. Both are **idempotent**: \"idempotent\"\nmeans doing it again has no extra effect.\n\n- `add` on a member already in the set: no change, no error.\n- `remove` on a member not in the set: no change, no error.\n\n```ts\nGroupsDb.members.add(new GroupKey('eng'), new Member('ada')); // ada joins\nGroupsDb.members.add(new GroupKey('eng'), new Member('ada')); // no-op, still one 'ada'\nGroupsDb.members.remove(new GroupKey('eng'), new Member('linus')); // linus leaves (or no-op)\n```\n\nBecause they are idempotent, you do not need to check `contains` before calling\nthem. Just `add` to join and `remove` to leave.\n\n### `list`\n\nReturns up to `limit` members of a set.\n\n```ts\nconst roster = GroupsDb.members.list(new GroupKey('eng'), 100);\n```\n\n`list` is a **scan** (it can walk many rows), so it is barred on the request\npath: you **cannot** call it from a `@get` or a `@post`. Like reading a whole\nevent log, listing a whole set belongs off the request path, in a\n[`@derive`](../background/derive.md) or a `@job`, which publishes a\n[View](./views.md) your routes read. The [worked example](#worked-example-group-membership)\nbelow shows this.\n\n`list` returns up to `limit` members. A set can be larger than one page, so treat\nthe result as \"a bounded page of members\", not necessarily \"every member\".\n\n## Worked example: group membership\n\nA user joins or leaves a group from a route; a derive lists the roster into a\nview; a route reads the view. Membership checks happen directly in the route.\n\n```ts\nimport { GroupKey } from '../models/GroupKey';\nimport { Member } from '../models/Member';\nimport { Roster } from '../models/Roster';\nimport { JoinRequest } from '../models/JoinRequest';\n\n@database\nclass GroupsDb {\n // The set: who is in each group.\n @collection static members: Membership<GroupKey, Member>;\n // A precomputed roster for the GET (listing is a scan, barred on routes).\n @collection static roster: View<GroupKey, Roster>;\n\n @derive\n rebuild(): void {\n const key = new GroupKey('eng');\n const r = new Roster();\n r.users = GroupsDb.members.list(key, 500); // scan, allowed in a derive\n GroupsDb.roster.publish(key, r);\n }\n}\n\n@rest('groups')\nclass Groups {\n // GET the roster from the view (a keyed read, not a scan).\n @get('/eng')\n public list(): Roster {\n const r = GroupsDb.roster.get(new GroupKey('eng'));\n return r == null ? new Roster() : r;\n }\n\n // Direct membership check: `contains` is a keyed read, legal in a GET.\n @get('/eng/is-member')\n public isMember(userId: string): bool {\n return GroupsDb.members.contains(new GroupKey('eng'), new Member(userId));\n }\n\n // Join: `add` is idempotent, so joining twice is harmless.\n @post('/eng/join')\n public join(input: JoinRequest): bool {\n GroupsDb.members.add(new GroupKey('eng'), new Member(input.userId));\n return true; // the @derive rebuilds the roster view right after\n }\n\n // Leave: `remove` is idempotent, so leaving when not a member is harmless.\n @post('/eng/leave')\n public leave(input: JoinRequest): bool {\n GroupsDb.members.remove(new GroupKey('eng'), new Member(input.userId));\n return true;\n }\n}\n```\n\nThe models:\n\n```ts\n@data\nexport class Member {\n userId: string = '';\n}\n\n@data\nexport class Roster {\n users: Member[] = [];\n}\n```\n\nThe split is the point: `contains`/`add`/`remove` act on a single member and are\nlegal on the request path; `list` scans the whole set and lives in a derive that\nfeeds a view.\n\n## Consistency\n\n- **Add and remove serialize at the set's home.** Each set key has one home\n location where its writes are applied in order, so an `add` followed by a\n `remove` of the same member lands in that order and the set ends up correct.\n- **Reads can lag slightly across regions.** ToilDB is worldwide. A change made\n at a set's home is copied to other regions in the background (asynchronous\n replication, giving eventual consistency: every region converges after a short\n delay). A `contains` or a `list` served from a far region may briefly miss a\n just-made change.\n- **`add`/`remove` are idempotent**, so retries are safe: re-adding an existing\n member or re-removing an absent one does nothing.\n- **A set can grow without bound.** Adding never shrinks it. Read it through a\n bounded `list(key, N)` in a derive, not all at once.\n\n## Gotchas\n\n- **You cannot `list` from a route.** `list` is a scan, legal only in a `@derive`\n or `@job`. To show a roster on a page, publish it to a [View](./views.md) from a\n derive and `get` that view in the route.\n- **`list` returns a bounded page.** A large set may have more members than your\n `limit`. Do not assume the returned array is the entire set.\n- **Sets are unordered.** Do not rely on the order `list` returns members in.\n- **Use `Membership` for growth or concurrency.** A small, stable set is fine as\n an array in a Document. Reach for `Membership` when the set grows, is edited\n concurrently, or you frequently ask \"is X a member?\".\n\n## Related\n\n- [Documents](./documents.md): when a small array field is enough (and when it is\n not).\n- [Views](./views.md): where a roster/list belongs so routes can read it cheaply.\n- [`@derive`](../background/derive.md): runs `list` off the request path.\n- [Data types (`@data`)](../concepts/types.md): how set keys and members are stored.\n- [Decorators](../concepts/decorators.md): which handler kinds may add/remove vs scan.\n",
28
28
  "database/README.md": "# ToilDB\n\nToilDB is the database built into toiljs. It is **global**, it needs **no setup or connection string**, and your backend talks to it with plain typed method calls.\n\nIf you have used Postgres, MySQL, or MongoDB before, ToilDB will feel a little different, and this page explains why. If you have never used a database, that is fine too: read on and every term is defined as it appears.\n\n## What \"a globally distributed edge database\" means\n\nLet us take that phrase one word at a time.\n\n- A **database** is a place your program stores data so it is still there on the next request. Your handler runs, finishes, and forgets everything in memory (a fresh WebAssembly instance runs each request), so anything you want to keep, a user account, a like count, a guestbook entry, has to go into a database.\n- The **edge** is a network of servers spread around the world, each one physically close to some of your users. Your compiled backend runs on the edge server nearest whoever is calling it. (For the full picture of where code runs, see [Tiers](../concepts/tiers.md).)\n- **Globally distributed** means the database is not one machine in one city. Its data lives on many machines in many regions at once. A user in Tokyo and a user in Paris each read from a copy near them, so reads are fast everywhere instead of fast in one place and slow everywhere else.\n\nPut together: ToilDB is storage that lives out on that same worldwide edge, next to your code, so a database read does not have to fly across an ocean and back.\n\n```mermaid\nflowchart LR\n subgraph edge[\"Dacely edge (worldwide)\"]\n direction LR\n H1[\"Your backend<br/>(Tokyo)\"] --> D1[(\"ToilDB<br/>copy near Tokyo\")]\n H2[\"Your backend<br/>(Paris)\"] --> D2[(\"ToilDB<br/>copy near Paris\")]\n end\n D1 <-->|\"replication<br/>(keeps copies in sync)\"| D2\n UA[\"User in Tokyo\"] --> H1\n UB[\"User in Paris\"] --> H2\n```\n\nYou never pick a region, open a connection, or run a migration script. You declare what you want to store (see [Setup](./setup.md)) and ToilDB is there.\n\n## Why seven families instead of one big \"table\"\n\nMost databases give you one general-purpose tool: a table (or a collection) that you read and write however you like. That is flexible, but on a system spread across the whole planet, the flexible tool is also the slow and error-prone tool. The classic example is a counter. If two servers on opposite sides of the world both try to do \"read the number, add one, write it back\" at the same moment, one of the two increments is silently lost, because each read the same starting value.\n\nToilDB solves this by giving you **seven specialized collection types**, called **families**. Each family is tuned for one job and exposes only the operations that are safe and fast for that job. A counter family, for instance, has no \"set to this value\" operation at all: you can only `add` a delta, and the database merges concurrent deltas from around the world without losing any. You cannot misuse it, because the unsafe operation does not exist.\n\nSo instead of one generic table you reach for, you pick the family that matches what you are doing. Picking the right one is the main skill, and the guide below walks you through it.\n\nThe seven families are:\n\n| Family | It stores | Read a page |\n| --- | --- | --- |\n| **Documents** | Records you look up by id (users, posts, orders). | [Documents](./documents.md) |\n| **Unique** | A claim on a value that must be one-of-a-kind (usernames, emails, slugs). | [Unique](./unique.md) |\n| **Counter** | A running total that many callers increment at once (likes, views). | [Counters](./counters.md) |\n| **Events** | An append-only log of things that happened (feeds, audit trails). | [Events](./events.md) |\n| **Capacity** | A limited quantity you hand out without overselling (tickets, seats). | [Capacity](./capacity.md) |\n| **Membership** | Sets of \"who belongs to what\" (followers, tags, room members). | [Membership](./membership.md) |\n| **View** | A precomputed, read-optimized snapshot (home pages, leaderboards). | [Views](./views.md) |\n\n## The one idea shared by all seven: key then value\n\nEvery family, underneath, works the same way: it maps a **key** to a **value**.\n\n- A **key** is how you find your data. Think of it as the label on a drawer: a user id, a username, a room name.\n- A **value** is what is in the drawer: the user record, the owner of a username, the members of a room.\n\nIn ToilDB, both the key and the value are ordinary TypeScript classes that you tag with `@data`. The `@data` tag tells toilscript (the compiler that turns your backend into WebAssembly) how to pack that class into bytes for storage and unpack it again. You write a normal class with normal fields, give each field a default, and the compiler does the rest.\n\n```ts\n// A key: how you address one record.\n@data\nclass UserId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n// A value: what you store under that key.\n@data\nclass User {\n id: string = '';\n name: string = '';\n score: u64 = 0;\n}\n```\n\nYou then declare a collection that maps that key type to that value type, for example `Documents<UserId, User>`. Every family is generic over its key and value types in exactly this way, so once you understand `@data` keys and values you understand all seven. The full `@data` reference is on the [data types page](../backend/data.md); how to declare collections is on [Setup](./setup.md).\n\n## Eventual consistency, in plain words\n\nBecause ToilDB keeps copies of your data in many regions, there is one honest trade-off you need to understand: **eventual consistency**.\n\nEvery key has one **home**: a single region that officially owns that key's data. All **writes** to a key travel to its home, where they are applied one at a time, in order. That is what makes writes safe: even if a thousand servers write the same key at once, the home lines them up so nothing is lost or corrupted.\n\n**Reads**, on the other hand, are served from the copy nearest the reader, which is fast but may be a beat behind. After a write lands at the home, it takes a brief moment (usually milliseconds) to fan out to the other regions' copies. During that moment, a reader in another region might still see the previous value. The copies always catch up; they are *eventually* consistent, not *instantly* consistent everywhere.\n\nWhat this means in practice:\n\n- **It is usually invisible.** The lag is tiny, and most apps never notice.\n- **Do not assume a write is visible everywhere the instant it returns.** For example, right after you create a record, a read from a far-away region might briefly not see it yet.\n- **Some families are stronger.** Because writes to a single key are serialized at its home, operations that must never race, claiming a unique username, reserving the last ticket, are decided at the home and are safe. The Unique and Capacity families lean on this so two callers can never both win. See [Unique](./unique.md) and [Capacity](./capacity.md).\n\nEach family page spells out its own consistency behavior, so you always know what to expect.\n\n## Choosing a family: the decision guide\n\nStart from what you are trying to do, not from a data structure. This table maps a real need to the family that was built for it.\n\n| I need to... | Use | Why |\n| --- | --- | --- |\n| Store and update a thing I look up by id (a user, a post, an order). | **Documents** | The general-purpose record store: create, read, update, delete by key. |\n| Guarantee a value is used by only one owner across the whole world (username, email, slug). | **Unique** | Claims are decided at the key's home, so two people cannot claim the same name. |\n| Count something that many people bump at the same time (likes, page views, stock tally). | **Counter** | Concurrent `add`s from anywhere merge without ever losing an increment. |\n| Keep a growing list of things that happened, in order (activity feed, audit log). | **Events** | Append-only: you add events and read the newest, but never edit history. |\n| Hand out a limited quantity and never sell more than exist (tickets, seats, inventory). | **Capacity** | Reserve/confirm/cancel holds at the home prevent overselling. |\n| Track which members belong to a set (followers, tags, room members, permissions). | **Membership** | Add/remove/check membership without loading a whole record to edit a list. |\n| Serve a precomputed, ready-to-render result quickly (leaderboard, home page snapshot). | **View** | A background job computes it once; requests read it with a single fast lookup. |\n\nIf two families seem to fit, this flowchart resolves it. Read it top to bottom.\n\n```mermaid\nflowchart TD\n START[\"What are you storing?\"] --> Q1{\"Is it a running<br/>total (a number many<br/>callers increase)?\"}\n Q1 -->|Yes| COUNTER[\"Counter\"]\n Q1 -->|No| Q2{\"Must a value be<br/>one-of-a-kind across<br/>everyone (username,<br/>email, slug)?\"}\n Q2 -->|Yes| UNIQUE[\"Unique\"]\n Q2 -->|No| Q3{\"Is it a limited stock<br/>you must not oversell<br/>(tickets, seats)?\"}\n Q3 -->|Yes| CAPACITY[\"Capacity\"]\n Q3 -->|No| Q4{\"Is it an append-only<br/>log of events that<br/>happened, kept in order?\"}\n Q4 -->|Yes| EVENTS[\"Events\"]\n Q4 -->|No| Q5{\"Is it a set of members<br/>you add/remove/check<br/>(followers, tags)?\"}\n Q5 -->|Yes| MEMBERSHIP[\"Membership\"]\n Q5 -->|No| Q6{\"Is it a precomputed<br/>read-only snapshot a<br/>background job builds?\"}\n Q6 -->|Yes| VIEW[\"View\"]\n Q6 -->|No| DOCUMENTS[\"Documents<br/>(the default record store)\"]\n```\n\nA quick sanity check when you land somewhere: **Documents is the default.** If you are storing \"a thing with fields that I update by id,\" it is almost always Documents. The other six exist for the specific jobs above where Documents would be slow, unsafe under concurrency, or awkward.\n\nReal apps mix families freely. The demo guestbook, for example, uses **Events** for the log of signatures, a **Counter** for the running total, and a **View** for the ready-to-serve snapshot, all in one small feature.\n\n## Where to go next\n\n- [Setup](./setup.md): declare a database, its collections, and reach them from a handler.\n- [Documents](./documents.md): the general-purpose record store (start here).\n- [Unique](./unique.md), [Counters](./counters.md), [Events](./events.md), [Views](./views.md), [Membership](./membership.md), [Capacity](./capacity.md): the specialized families.\n\n## Related\n\n- [Setup](./setup.md): how to declare `@database`, `@collection`, and `@data` types.\n- [Data types (`@data`)](../backend/data.md): the typed keys and values every family uses.\n- [Tiers](../concepts/tiers.md): where your backend and its data run.\n- [Decorators](../concepts/decorators.md): `@query`, `@action`, `@derive`, and friends.\n",
29
- "database/setup.md": "# Declaring a database\n\nYou set up ToilDB by **declaring** it in code: a `@database` class listing your collections, plus the `@data` key and value types those collections use. There is no schema file, no migration command, and no connection string.\n\n## What and why\n\nA **database declaration** tells toiljs three things at compile time: which collections exist, which family each one is, and what key and value types it stores. The compiler bakes that list into your server's WebAssembly, and both the dev server and the production edge read it back to set up your collections automatically. You declare; the platform provisions.\n\nReach for this on day one: before you can read or write any data, you need a `@database` with at least one `@collection`.\n\n## How: the three pieces\n\nA working database is always these three pieces together.\n\n1. **`@data` key and value types.** Plain classes, tagged `@data`, that describe what you store.\n2. **A `@database` class** whose static fields are `@collection`s, each typed by a family.\n3. **A handler** (a `@rest` route or an RPC method) that reads and writes those collections.\n\nHere is a complete, minimal example: a store of users you look up by id.\n\n```ts\n// A @data key: how you address one user.\n@data\nclass UserId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n// A @data value: what you store for each user.\n@data\nclass User {\n id: string = '';\n name: string = '';\n score: u64 = 0;\n}\n\n// The database declaration. Each @collection is one collection,\n// typed by its family (here, Documents) with <Key, Value>.\n@database\nclass AppDb {\n @collection static users: Documents<UserId, User>;\n}\n\n// A route that reads and writes the collection.\n@rest('users')\nclass Users {\n @get('/:id')\n public getUser(ctx: RouteContext): User {\n const user = AppDb.users.get(new UserId(ctx.param('id')));\n return user == null ? new User() : user;\n }\n\n @post('/')\n public createUser(input: User): User {\n AppDb.users.create(new UserId(input.id), input);\n return input;\n }\n}\n```\n\nThat is the whole setup. Run `toiljs dev` and `AppDb.users` works immediately.\n\n### The `@data` types\n\nBoth the key and the value are `@data` classes. `@data` is what makes a class storable: the compiler synthesizes a binary codec (pack to bytes, unpack from bytes) so ToilDB can persist it. The rules that matter here:\n\n- **Give every field a default** (`= ''`, `= 0`, and so on). The decoder builds an empty instance and fills it, so it needs defaults.\n- **A value type must be default-constructible** (creatable with `new User()` and no arguments). Keys often add a convenience constructor, as `UserId` does above, so you can write `new UserId('abc')`.\n- Fields may be numbers (`u8`..`u256`, `i8`..`i256`, `f32`, `f64`), `bool`, `string`, another `@data` class, or an array of any of these.\n\nThe full reference, including how the same type becomes a typed client type, is on the [data types page](../backend/data.md).\n\n### The `@database` class and `@collection` fields\n\n```ts\n@database\nclass AppDb {\n @collection static users: Documents<UserId, User>;\n @collection static likes: Counter<UserId>;\n}\n```\n\n- `@database` marks the class as a database. You can have more than one `@database` class; each is a separate namespace of collections.\n- Each `@collection` is one collection. Declare it as a **`static`** field with **no initializer**: you write the type, and the compiler wires up the actual handle for you.\n- The field's **type is its family**: `Documents<K, V>`, `Counter<K>`, `Events<K, V>`, `Unique<K, V>`, `Membership<K, M>`, `Capacity<K>`, or `View<K, V>`. The compiler reads the family straight from this type, so getting it right here is how you pick a family (see [Choosing a family](./README.md#choosing-a-family-the-decision-guide)).\n\nYou reach a collection through the class, statically: `AppDb.users.get(...)`, `AppDb.likes.add(...)`. There is nothing to instantiate.\n\n### Reaching collections from a handler\n\nAny backend function can read and write collections by referencing them on the database class. What that function is allowed to do depends on its **kind**, covered next.\n\n## How access is gated: `@query`, `@action`, and friends\n\nToilDB will not let every function do everything. Each backend function runs as one **function kind**, and each kind is allowed a different slice of database operations. This is a safety rail: a read-only endpoint physically cannot write, and an expensive scan cannot run on the hot request path.\n\nYou rarely write these decorators by hand, because routes get a sensible kind automatically:\n\n- A **`@get`** route (a safe, read-only HTTP method) runs as a **Query**.\n- A **`@post`** route (a mutating method) runs as an **Action**.\n- A plain RPC **`@remote`** method defaults to a **Query** (read-only) because it has no HTTP method to infer from. Tag it `@action` if it writes.\n\nYou can override the default with `@query` or `@action` on the method when the automatic choice is wrong (for example, a `@get` that genuinely needs to write, though that is unusual).\n\nWhat each kind may do:\n\n| Kind | Set by | May do | May **not** do |\n| --- | --- | --- | --- |\n| **Query** | `@get`, plain `@remote`, or `@query` | Point reads: `get`, `getMany`, `exists`, `lookup`, `contains`, counter `get`, view `get`, capacity `available`. | Any write. Any scan. |\n| **Action** | `@post` or `@action` | Everything a Query can, plus bounded writes: `create`, `patch`, `delete`, `getDelete`, `enqueue`, `append`, `appendOnce`, counter `add`, membership `add`/`remove`, unique `claim`/`release`, capacity `reserve`/`confirm`/`cancel`. | Scans. Publishing a View. |\n| **Derive / Job** | `@derive` / `@job` (background work) | Reads including **scans** (`latest`, membership `list`), plus `publish` a View. | (Run off the request path; see below.) |\n\nTwo rules trip people up, so they are worth stating plainly:\n\n- **Scans are barred from request handlers.** Reading \"the newest N events\" (`events.latest`) or \"the members of this set\" (`membership.list`) can fan out across many rows, so a `@get` or `@post` cannot call them. Do the scan in a `@derive` (a small function that recomputes a snapshot off the request path) and have the request read the snapshot. See [Views](./views.md) and [@derive](../background/derive.md).\n- **Only a `@derive` or `@job` may `publish` a View.** Requests read views; background work writes them.\n\nBoth gates are enforced twice: the compiler rejects an illegal call at build time, and the edge rejects it again at runtime, so a hand-edited module cannot sneak past. For the decorator catalog, see [Decorators](../concepts/decorators.md).\n\n## No manual provisioning: how it actually gets set up\n\nYou never create a table or run a migration. Here is the machinery, so the \"it just works\" is not a mystery.\n\nWhen toilscript compiles your backend, it scans every `@database` class and writes a small catalog into the `.wasm` file: for each collection, its name, its family, its key and value type names, and the value's schema version. This catalog rides *inside* the compiled module.\n\nThen, wherever your backend runs, the host reads that catalog once at startup and builds your collections to match:\n\n- Under **`toiljs dev`**, the host is an in-process, in-memory emulator. It reads the catalog and stands up all seven families in memory. This is a development store: single process, single tenant, and cleared when you restart. It exists so you can build and test against real ToilDB behavior with no services to run.\n- On the **Dacely edge**, the host is the real ToilDB, backed by a globally distributed ScyllaDB cluster. It reads the *same* catalog and serves the *same* operations, now durable and worldwide.\n\nBecause both sides read the same catalog, **the same code runs unchanged in dev and in production.** There is no connection string to swap and no provisioning step to run.\n\n```mermaid\nflowchart TD\n SRC[\"@database AppDb<br/>@collection static users: Documents...<br/>@data UserId / User\"]\n SRC -->|toilscript compile| WASM[\"server.wasm<br/>(with an embedded<br/>toildb catalog section)\"]\n WASM -->|toiljs dev reads the catalog| DEV[\"In-memory dev store<br/>(one process, cleared on restart)\"]\n WASM -->|edge reads the same catalog| EDGE[(\"ToilDB on the Dacely edge<br/>(ScyllaDB, worldwide, durable)\")]\n```\n\n## Gotchas\n\n- **Declare collections as `static` fields with no initializer.** Do not try to `new` a collection or assign a handle yourself; the compiler owns that.\n- **Every `@data` field needs a default, and value types must be default-constructible.** A missing default fails to compile.\n- **Dev data is not durable.** The `toiljs dev` store lives in memory and resets on restart. Do not rely on it to persist across dev runs; that is what the edge is for.\n- **Changing a `@data` type is a format change.** Reordering fields or changing a field's type changes the stored layout. Add new fields at the end, and use a migration when you evolve a stored type. See [data types](../backend/data.md).\n- **Pick the family at the type.** The family is read from the collection's declared type, so `Counter<K>` versus `Documents<K, V>` is a real, load-bearing choice, not a hint. Revisit [Choosing a family](./README.md#choosing-a-family-the-decision-guide) if unsure.\n\n## Related\n\n- [ToilDB overview](./README.md): the seven families and how to choose.\n- [Documents](./documents.md): the general-purpose record family (a good first collection).\n- [Data types (`@data`)](../backend/data.md): keys, values, and the codec.\n- [Decorators](../concepts/decorators.md): `@database`, `@collection`, `@query`, `@action`, `@derive`.\n- [@derive](../background/derive.md): recompute snapshots and run scans off the request path.\n",
30
- "database/unique.md": "# Unique\n\nThe **Unique** family enforces that a value is claimed by only one owner across the entire world. It is how you make usernames, email addresses, and URL slugs one-of-a-kind, safely, even when two people try to grab the same one at the same instant.\n\n## What and why\n\nA **Unique collection** maps a `@data` **claim key** (the thing that must be unique, like a username) to a `@data` **owner value** (who or what claimed it, like a user id). At any moment a claim key is either **unclaimed** or **owned by exactly one owner**. The family gives you three operations: look up who owns a key, claim a key, and release it.\n\nReach for Unique whenever a value must be globally singular:\n\n- usernames or handles\n- email addresses\n- URL slugs or workspace names\n- any \"reserve this name for me\" scenario\n\nWhy not just check a Documents record first? Because a check-then-write has a race: two requests can both check \"is `alice` free?\", both see yes, and both create it. Unique closes that gap. The claim is decided at the key's single **home** (see [consistency](./README.md#eventual-consistency-in-plain-words)), where claims are processed one at a time, so exactly one of the two racers wins.\n\nDeclare one by typing a `@collection` as `Unique<ClaimKey, OwnerValue>`:\n\n```ts\n@data\nclass Username {\n name: string = '';\n constructor(name: string = '') { this.name = name; }\n}\n\n@data\nclass OwnerId {\n userId: string = '';\n constructor(userId: string = '') { this.userId = userId; }\n}\n\n@database\nclass AppDb {\n @collection static usernames: Unique<Username, OwnerId>;\n}\n```\n\n## The operations\n\n`K` is the claim-key type, `V` the owner value type.\n\n| Operation | Signature | Returns | Use it to |\n| --- | --- | --- | --- |\n| `lookup` | `lookup(key: K): V \\| null` | the current owner, or `null` if unclaimed | find out who owns a name |\n| `claim` | `claim(key: K, value: V): ClaimResult<V>` | a `ClaimResult` (see below) | try to take a name for an owner |\n| `release` | `release(key: K, value: V): void` | nothing; **traps** if you are not the owner | give a name back |\n\n`lookup` is a read, so it works in any function. `claim` and `release` are writes, so they need an **Action** (a `@post` route or an `@action`); see [Setup](./setup.md#how-access-is-gated-query-action-and-friends).\n\n### `ClaimResult`\n\n`claim` returns a small object that tells you what happened:\n\n```ts\nclass ClaimResult<V> {\n claimed: bool; // true if YOU own the key now\n owner: V | null; // when claimed is false, who owns it instead\n}\n```\n\n- **`claimed == true`**: you own the key. This covers both a fresh claim and an **idempotent re-claim**: if you claim a key you already own (same owner value), you still get `true`, so retrying a claim is safe. `owner` is `null` in this case.\n- **`claimed == false`**: someone else got there first. `owner` is their value, so you can tell the user \"that name is taken.\"\n\n### `lookup`\n\n`lookup` just reads the current owner without changing anything:\n\n```ts\nconst owner = AppDb.usernames.lookup(new Username('alice'));\nif (owner == null) {\n // 'alice' is free\n} else {\n // owner.userId currently holds 'alice'\n}\n```\n\nNote that `lookup` is subject to [eventual consistency](./README.md#eventual-consistency-in-plain-words): a claim made moments ago in another region may not show up in a far-away `lookup` yet. Do not use `lookup` as your uniqueness guard. `lookup` is for display (\"this name is taken by ...\"); the real guarantee comes from `claim`, which is decided at the home and cannot race.\n\n### `claim`\n\n`claim` is the operation that actually enforces uniqueness. You pass the key and the owner value:\n\n```ts\nconst result = AppDb.usernames.claim(new Username('alice'), new OwnerId('u_123'));\nif (result.claimed) {\n // 'alice' is now yours\n} else {\n // taken; result.owner is the current owner\n}\n```\n\nBecause claims are serialized at the key's home, this is race-safe. If two requests anywhere in the world call `claim('alice', ...)` at the same moment, the home applies them in order: the first gets `claimed: true`, the second gets `claimed: false` with the first as `owner`. There is no window where both win.\n\n### `release`\n\n`release` gives a claim back so the key becomes available again. Only the **current owner** may release: you pass both the key and the owner value, and if that value is not the current owner, `release` **traps** (aborts the request). This prevents one user from releasing another user's name.\n\n```ts\nAppDb.usernames.release(new Username('alice'), new OwnerId('u_123'));\n```\n\nRelease when a name is being changed or an account is deleted, so the name returns to the pool.\n\n## The claim / release lifecycle\n\nA claim is a small state machine: unclaimed, owned, and back to unclaimed.\n\n```mermaid\nstateDiagram-v2\n [*] --> Unclaimed\n Unclaimed --> Owned: claim(key, me) -> claimed: true\n Owned --> Owned: claim(key, me) again -> claimed: true (idempotent)\n Owned --> Owned: claim(key, someoneElse) -> claimed: false, owner: me\n Owned --> Unclaimed: release(key, me)\n```\n\nThe key insight: **claiming is the guard, releasing is cleanup.** You claim to reserve, you release to free. A claim that is never released stays owned forever (there is no automatic expiry; if you want time-limited holds, that is what the [Capacity](./capacity.md) family's TTL reservations are for).\n\n## Worked example: reserving a username on signup\n\nThe safe signup pattern is: claim the username first, then create the account, and if creating the account fails, release the claim so the name is not stranded.\n\n```ts\nimport { Response } from 'toiljs/server/runtime';\n\n@data\nclass Username {\n name: string = '';\n constructor(name: string = '') { this.name = name; }\n}\n\n@data\nclass OwnerId {\n userId: string = '';\n constructor(userId: string = '') { this.userId = userId; }\n}\n\n@data\nclass UserId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass User {\n id: string = '';\n username: string = '';\n}\n\n@data\nclass SignupInput {\n username: string = '';\n userId: string = '';\n}\n\n@database\nclass AppDb {\n @collection static usernames: Unique<Username, OwnerId>;\n @collection static users: Documents<UserId, User>;\n}\n\n@rest('signup')\nclass Signup {\n // POST /signup (Action: may claim and write)\n @post('/')\n public signup(input: SignupInput): Response {\n const owner = new OwnerId(input.userId);\n\n // 1. Claim the username. This is the uniqueness guard.\n const claim = AppDb.usernames.claim(new Username(input.username), owner);\n if (!claim.claimed) {\n return Response.text('username taken', 409);\n }\n\n // 2. Create the account. If this fails, undo the claim so the name is free.\n const user = new User();\n user.id = input.userId;\n user.username = input.username;\n if (!AppDb.users.create(new UserId(input.userId), user)) {\n AppDb.usernames.release(new Username(input.username), owner);\n return Response.text('could not create account', 409);\n }\n\n return Response.json(user.toJSON().toString());\n }\n}\n```\n\nTwo things make this correct:\n\n- **The claim happens before the account write**, so uniqueness is decided up front.\n- **The claim is released if the follow-up write fails**, so a failed signup does not permanently burn a username.\n\nBecause `claim` is idempotent for the same owner, a client that retries the whole request after a network hiccup does not get a false \"taken\" error: the second `claim('alice', u_123)` returns `claimed: true` again.\n\n## Consistency notes\n\n- **`claim` and `release` are strongly consistent at the key's home.** Two callers can never both own the same key; the home serializes claims. This is the whole point of the family.\n- **`lookup` is eventually consistent.** It reads a possibly-nearby copy, so a very recent claim from elsewhere may not appear yet. Never gate uniqueness on `lookup`; gate it on the result of `claim`.\n- **Claims do not expire.** A claim stays until someone releases it. Use [Capacity](./capacity.md) if you need holds that auto-release after a timeout.\n\n## Gotchas\n\n- **Do not use `lookup` as the uniqueness check.** Its answer can be stale. Call `claim` and trust its `claimed` flag.\n- **Release on failure and on account deletion.** A claim you forget to release stays owned forever, quietly blocking that name.\n- **`release` traps for a non-owner.** Pass the correct owner value, or you abort the request. If you are not sure you own it, `lookup` first (for display) but expect `release` to enforce ownership regardless.\n- **The owner value is data you choose.** Store enough in it (a user id, say) to know who holds the claim, so you can display and release it later.\n\n## Related\n\n- [ToilDB overview](./README.md): the seven families and how to choose.\n- [Setup](./setup.md): declaring the collection and which function kinds may claim.\n- [Documents](./documents.md): the account record you create alongside a claim.\n- [Capacity](./capacity.md): time-limited holds that auto-release (a different kind of \"reserve\").\n- [Data types (`@data`)](../backend/data.md): the claim key and owner value.\n",
29
+ "database/setup.md": "# Declaring a database\n\nYou set up ToilDB by **declaring** it in code: a `@database` class listing your collections, plus the `@data` key and value types those collections use. There is no schema file, no migration command, and no connection string.\n\n## What and why\n\nA **database declaration** tells toiljs three things at compile time: which collections exist, which family each one is, and what key and value types it stores. The compiler bakes that list into your server's WebAssembly, and both the dev server and the production edge read it back to set up your collections automatically. You declare; the platform provisions.\n\nReach for this on day one: before you can read or write any data, you need a `@database` with at least one `@collection`.\n\nGroup related data into collections within a small number of databases. A new\nfeature usually needs a new collection, not another `@database` class. Database\nnamespaces do not replace application authorization: enforce account and\norganization access in server routes and scope collection keys appropriately.\n\n## How: the three pieces\n\nA working database is always these three pieces together.\n\n1. **`@data` key and value types.** Plain classes, tagged `@data`, that describe what you store.\n2. **A `@database` class** whose static fields are `@collection`s, each typed by a family.\n3. **A handler** (a `@rest` route or an RPC method) that reads and writes those collections.\n\nHere is a complete, minimal example: a store of users you look up by id.\n\n```ts\n// A @data key: how you address one user.\n@data\nclass UserId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n// A @data value: what you store for each user.\n@data\nclass User {\n id: string = '';\n name: string = '';\n score: u64 = 0;\n}\n\n// The database declaration. Each @collection is one collection,\n// typed by its family (here, Documents) with <Key, Value>.\n@database\nclass AppDb {\n @collection static users: Documents<UserId, User>;\n}\n\n// A route that reads and writes the collection.\n@rest('users')\nclass Users {\n @get('/:id')\n public getUser(ctx: RouteContext): User {\n const user = AppDb.users.get(new UserId(ctx.param('id')));\n return user == null ? new User() : user;\n }\n\n @post('/')\n public createUser(input: User): User {\n AppDb.users.create(new UserId(input.id), input);\n return input;\n }\n}\n```\n\nThat is the whole setup. Run `toiljs dev` and `AppDb.users` works immediately.\n\n### The `@data` types\n\nBoth the key and the value are `@data` classes. `@data` is what makes a class storable: the compiler synthesizes a binary codec (pack to bytes, unpack from bytes) so ToilDB can persist it. The rules that matter here:\n\n- **Give every field a default** (`= ''`, `= 0`, and so on). The decoder builds an empty instance and fills it, so it needs defaults.\n- **A value type must be default-constructible** (creatable with `new User()` and no arguments). Keys often add a convenience constructor, as `UserId` does above, so you can write `new UserId('abc')`.\n- Fields may be numbers (`u8`..`u256`, `i8`..`i256`, `f32`, `f64`), `bool`, `string`, another `@data` class, or an array of any of these.\n\nThe full reference, including how the same type becomes a typed client type, is on the [data types page](../backend/data.md).\n\n### The `@database` class and `@collection` fields\n\n```ts\n@database\nclass AppDb {\n @collection static users: Documents<UserId, User>;\n @collection static likes: Counter<UserId>;\n}\n```\n\n- `@database` marks the class as a database. You can have more than one `@database` class; each is a separate namespace of collections.\n- Each `@collection` is one collection. Declare it as a **`static`** field with **no initializer**: you write the type, and the compiler wires up the actual handle for you.\n- The field's **type is its family**: `Documents<K, V>`, `Counter<K>`, `Events<K, V>`, `Unique<K, V>`, `Membership<K, M>`, `Capacity<K>`, or `View<K, V>`. The compiler reads the family straight from this type, so getting it right here is how you pick a family (see [Choosing a family](./README.md#choosing-a-family-the-decision-guide)).\n\nYou reach a collection through the class, statically: `AppDb.users.get(...)`, `AppDb.likes.add(...)`. There is nothing to instantiate.\n\n### Reaching collections from a handler\n\nAny backend function can read and write collections by referencing them on the database class. What that function is allowed to do depends on its **kind**, covered next.\n\n## How access is gated: `@query`, `@action`, and friends\n\nToilDB will not let every function do everything. Each backend function runs as one **function kind**, and each kind is allowed a different slice of database operations. This is a safety rail: a read-only endpoint physically cannot write, and an expensive scan cannot run on the hot request path.\n\nYou rarely write these decorators by hand, because routes get a sensible kind automatically:\n\n- A **`@get`** route (a safe, read-only HTTP method) runs as a **Query**.\n- A **`@post`** route (a mutating method) runs as an **Action**.\n- A plain RPC **`@remote`** method defaults to a **Query** (read-only) because it has no HTTP method to infer from. Tag it `@action` if it writes.\n\nYou can override the default with `@query` or `@action` on the method when the automatic choice is wrong (for example, a `@get` that genuinely needs to write, though that is unusual).\n\nWhat each kind may do:\n\n| Kind | Set by | May do | May **not** do |\n| --- | --- | --- | --- |\n| **Query** | `@get`, plain `@remote`, or `@query` | Point reads: `get`, `getMany`, `exists`, `lookup`, `contains`, counter `get`, view `get`, Events `get`/`last`, capacity `available`. | Any write. Any scan. |\n| **Action** | `@post` or `@action` | Everything a Query can, plus bounded writes: `create`, `patch`, `delete`, `getDelete`, `enqueue`, `append`, `appendOnce`, counter `add`, membership `add`/`remove`, unique `claim`/`release`, capacity `reserve`/`confirm`/`cancel`. | Scans. Publishing a View. |\n| **Derive / Job** | `@derive` / `@job` (background work) | Reads including **scans** (`latest`, membership `list`), plus `publish` a View. | (Run off the request path; see below.) |\n\nTwo rules trip people up, so they are worth stating plainly:\n\n- **Scans are barred from request handlers.** Reading \"the newest N events\" (`events.latest`) or \"the members of this set\" (`membership.list`) can fan out across many rows, so a `@get` or `@post` cannot call them. Do the scan in a `@derive` (a small function that recomputes a snapshot off the request path) and have the request read the snapshot. See [Views](./views.md) and [@derive](../background/derive.md).\n- **Only a `@derive` or `@job` may `publish` a View.** Requests read views; background work writes them.\n\nBoth gates are enforced twice: the compiler rejects an illegal call at build time, and the edge rejects it again at runtime, so a hand-edited module cannot sneak past. For the decorator catalog, see [Decorators](../concepts/decorators.md).\n\n## No manual provisioning: how it actually gets set up\n\nYou never create a table or run a migration. Here is the machinery, so the \"it just works\" is not a mystery.\n\nWhen toilscript compiles your backend, it scans every `@database` class and writes a small catalog into the `.wasm` file: for each collection, its name, its family, its key and value type names, and the value's schema version. This catalog rides *inside* the compiled module.\n\nThen, wherever your backend runs, the host reads that catalog once at startup and builds your collections to match:\n\n- Under **`toiljs dev`**, the host is an in-process, in-memory emulator. It reads the catalog and stands up all seven families in memory. This is a development store: single process, single tenant, and cleared when you restart. It exists so you can build and test against real ToilDB behavior with no services to run.\n- On the **Dacely edge**, the host is the real ToilDB, backed by a globally distributed ScyllaDB cluster. It reads the *same* catalog and serves the *same* operations, now durable and worldwide.\n\nBecause both sides read the same catalog, **the same code runs unchanged in dev and in production.** There is no connection string to swap and no provisioning step to run.\n\n```mermaid\nflowchart TD\n SRC[\"@database AppDb<br/>@collection static users: Documents...<br/>@data UserId / User\"]\n SRC -->|toilscript compile| WASM[\"server.wasm<br/>(with an embedded<br/>toildb catalog section)\"]\n WASM -->|toiljs dev reads the catalog| DEV[\"In-memory dev store<br/>(one process, cleared on restart)\"]\n WASM -->|edge reads the same catalog| EDGE[(\"ToilDB on the Dacely edge<br/>(ScyllaDB, worldwide, durable)\")]\n```\n\n## Gotchas\n\n- **Declare collections as `static` fields with no initializer.** Do not try to `new` a collection or assign a handle yourself; the compiler owns that.\n- **Every `@data` field needs a default, and value types must be default-constructible.** A missing default fails to compile.\n- **Dev data is not durable.** The `toiljs dev` store lives in memory and resets on restart. Do not rely on it to persist across dev runs; that is what the edge is for.\n- **Changing a `@data` type is a format change.** Reordering fields or changing a field's type changes the stored layout. Add new fields at the end, and use a migration when you evolve a stored type. See [data types](../backend/data.md).\n- **Pick the family at the type.** The family is read from the collection's declared type, so `Counter<K>` versus `Documents<K, V>` is a real, load-bearing choice, not a hint. Revisit [Choosing a family](./README.md#choosing-a-family-the-decision-guide) if unsure.\n\n## Related\n\n- [ToilDB overview](./README.md): the seven families and how to choose.\n- [Documents](./documents.md): the general-purpose record family (a good first collection).\n- [Data types (`@data`)](../backend/data.md): keys, values, and the codec.\n- [Decorators](../concepts/decorators.md): `@database`, `@collection`, `@query`, `@action`, `@derive`.\n- [@derive](../background/derive.md): recompute snapshots and run scans off the request path.\n",
30
+ "database/unique.md": "# Unique\n\nThe **Unique** family enforces that a value is claimed by only one owner across the entire world. It is how you make usernames, email addresses, and URL slugs one-of-a-kind, safely, even when two people try to grab the same one at the same instant.\n\n## What and why\n\nA **Unique collection** maps a `@data` **claim key** (the thing that must be unique, like a username) to a `@data` **owner value** (who or what claimed it, like a user id). At any moment a claim key is either **unclaimed** or **owned by exactly one owner**. The family lets you look up one or several owners, claim a key, and release it.\n\nReach for Unique whenever a value must be globally singular:\n\n- usernames or handles\n- email addresses\n- URL slugs or workspace names\n- any \"reserve this name for me\" scenario\n\nWhy not just check a Documents record first? Because a check-then-write has a race: two requests can both check \"is `alice` free?\", both see yes, and both create it. Unique closes that gap. The claim is decided at the key's single **home** (see [consistency](./README.md#eventual-consistency-in-plain-words)), where claims are processed one at a time, so exactly one of the two racers wins.\n\nDeclare one by typing a `@collection` as `Unique<ClaimKey, OwnerValue>`:\n\n```ts\n@data\nclass Username {\n name: string = '';\n constructor(name: string = '') { this.name = name; }\n}\n\n@data\nclass OwnerId {\n userId: string = '';\n constructor(userId: string = '') { this.userId = userId; }\n}\n\n@database\nclass AppDb {\n @collection static usernames: Unique<Username, OwnerId>;\n}\n```\n\n## The operations\n\n`K` is the claim-key type, `V` the owner value type.\n\n| Operation | Signature | Returns | Use it to |\n| --- | --- | --- | --- |\n| `lookup` | `lookup(key: K): V \\| null` | the current owner, or `null` if unclaimed | find out who owns a name |\n| `claim` | `claim(key: K, value: V): ClaimResult<V>` | a `ClaimResult` (see below) | try to take a name for an owner |\n| `release` | `release(key: K, value: V): void` | nothing; **traps** if you are not the owner | give a name back |\n\n`lookup` is a read, so it works in any function. `claim` and `release` are writes, so they need an **Action** (a `@post` route or an `@action`); see [Setup](./setup.md#how-access-is-gated-query-action-and-friends).\n\n### `ClaimResult`\n\n`claim` returns a small object that tells you what happened:\n\n```ts\nclass ClaimResult<V> {\n claimed: bool; // true if YOU own the key now\n owner: V | null; // when claimed is false, who owns it instead\n}\n```\n\n- **`claimed == true`**: you own the key. This covers both a fresh claim and an **idempotent re-claim**: if you claim a key you already own (same owner value), you still get `true`, so retrying a claim is safe. `owner` is `null` in this case.\n- **`claimed == false`**: someone else got there first. `owner` is their value, so you can tell the user \"that name is taken.\"\n\n### `lookup`\n\n`lookup` just reads the current owner without changing anything:\n\n```ts\nconst owner = AppDb.usernames.lookup(new Username('alice'));\nif (owner == null) {\n // 'alice' is free\n} else {\n // owner.userId currently holds 'alice'\n}\n```\n\nNote that `lookup` is subject to [eventual consistency](./README.md#eventual-consistency-in-plain-words): a claim made moments ago in another region may not show up in a far-away `lookup` yet. Do not use `lookup` as your uniqueness guard. `lookup` is for display (\"this name is taken by ...\"); the real guarantee comes from `claim`, which is decided at the home and cannot race.\n\n### `lookupMany`\n\n`lookupMany(keys)` returns owners in input order, with `null` for unclaimed keys.\nIt uses one database operation for the batch and the same tenant isolation and\nconsistency rules as `lookup`. The host enforces its batch-size limit (32 keys for queries, 64 for actions); split larger lists into bounded pages. The batch is not a transaction\nor a consistent snapshot across keys. It never replaces `claim` as a uniqueness\ncheck.\n\n```ts\nconst owners = AppDb.usernames.lookupMany([new Username('alice'), new Username('bob')]);\n```\n\n### `claim`\n\n`claim` is the operation that actually enforces uniqueness. You pass the key and the owner value:\n\n```ts\nconst result = AppDb.usernames.claim(new Username('alice'), new OwnerId('u_123'));\nif (result.claimed) {\n // 'alice' is now yours\n} else {\n // taken; result.owner is the current owner\n}\n```\n\nBecause claims are serialized at the key's home, this is race-safe. If two requests anywhere in the world call `claim('alice', ...)` at the same moment, the home applies them in order: the first gets `claimed: true`, the second gets `claimed: false` with the first as `owner`. There is no window where both win.\n\n### `release`\n\n`release` gives a claim back so the key becomes available again. Only the **current owner** may release: you pass both the key and the owner value, and if that value is not the current owner, `release` **traps** (aborts the request). This prevents one user from releasing another user's name.\n\n```ts\nAppDb.usernames.release(new Username('alice'), new OwnerId('u_123'));\n```\n\nRelease when a name is being changed or an account is deleted, so the name returns to the pool.\n\n## The claim / release lifecycle\n\nA claim is a small state machine: unclaimed, owned, and back to unclaimed.\n\n```mermaid\nstateDiagram-v2\n [*] --> Unclaimed\n Unclaimed --> Owned: claim(key, me) -> claimed: true\n Owned --> Owned: claim(key, me) again -> claimed: true (idempotent)\n Owned --> Owned: claim(key, someoneElse) -> claimed: false, owner: me\n Owned --> Unclaimed: release(key, me)\n```\n\nThe key insight: **claiming is the guard, releasing is cleanup.** You claim to reserve, you release to free. A claim that is never released stays owned forever (there is no automatic expiry; if you want time-limited holds, that is what the [Capacity](./capacity.md) family's TTL reservations are for).\n\n## Worked example: reserving a username on signup\n\nThe safe signup pattern is: claim the username first, then create the account, and if creating the account fails, release the claim so the name is not stranded.\n\n```ts\nimport { Response } from 'toiljs/server/runtime';\n\n@data\nclass Username {\n name: string = '';\n constructor(name: string = '') { this.name = name; }\n}\n\n@data\nclass OwnerId {\n userId: string = '';\n constructor(userId: string = '') { this.userId = userId; }\n}\n\n@data\nclass UserId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass User {\n id: string = '';\n username: string = '';\n}\n\n@data\nclass SignupInput {\n username: string = '';\n userId: string = '';\n}\n\n@database\nclass AppDb {\n @collection static usernames: Unique<Username, OwnerId>;\n @collection static users: Documents<UserId, User>;\n}\n\n@rest('signup')\nclass Signup {\n // POST /signup (Action: may claim and write)\n @post('/')\n public signup(input: SignupInput): Response {\n const owner = new OwnerId(input.userId);\n\n // 1. Claim the username. This is the uniqueness guard.\n const claim = AppDb.usernames.claim(new Username(input.username), owner);\n if (!claim.claimed) {\n return Response.text('username taken', 409);\n }\n\n // 2. Create the account. If this fails, undo the claim so the name is free.\n const user = new User();\n user.id = input.userId;\n user.username = input.username;\n if (!AppDb.users.create(new UserId(input.userId), user)) {\n AppDb.usernames.release(new Username(input.username), owner);\n return Response.text('could not create account', 409);\n }\n\n return Response.json(user.toJSON().toString());\n }\n}\n```\n\nTwo things make this correct:\n\n- **The claim happens before the account write**, so uniqueness is decided up front.\n- **The claim is released if the follow-up write fails**, so a failed signup does not permanently burn a username.\n\nBecause `claim` is idempotent for the same owner, a client that retries the whole request after a network hiccup does not get a false \"taken\" error: the second `claim('alice', u_123)` returns `claimed: true` again.\n\n## Consistency notes\n\n- **`claim` and `release` are strongly consistent at the key's home.** Two callers can never both own the same key; the home serializes claims. This is the whole point of the family.\n- **`lookup` is eventually consistent.** It reads a possibly-nearby copy, so a very recent claim from elsewhere may not appear yet. Never gate uniqueness on `lookup`; gate it on the result of `claim`.\n- **Claims do not expire.** A claim stays until someone releases it. Use [Capacity](./capacity.md) if you need holds that auto-release after a timeout.\n\n## Gotchas\n\n- **Do not use `lookup` as the uniqueness check.** Its answer can be stale. Call `claim` and trust its `claimed` flag.\n- **Release on failure and on account deletion.** A claim you forget to release stays owned forever, quietly blocking that name.\n- **`release` traps for a non-owner.** Pass the correct owner value, or you abort the request. If you are not sure you own it, `lookup` first (for display) but expect `release` to enforce ownership regardless.\n- **The owner value is data you choose.** Store enough in it (a user id, say) to know who holds the claim, so you can display and release it later.\n\n## Related\n\n- [ToilDB overview](./README.md): the seven families and how to choose.\n- [Setup](./setup.md): declaring the collection and which function kinds may claim.\n- [Documents](./documents.md): the account record you create alongside a claim.\n- [Capacity](./capacity.md): time-limited holds that auto-release (a different kind of \"reserve\").\n- [Data types (`@data`)](../backend/data.md): the claim key and owner value.\n",
31
31
  "database/views.md": "# Views (materialized views)\n\nA `View` collection is a **precomputed, read-optimized result** that you publish\nonce and read many times. Reads are a single cheap keyed lookup, no scanning, no\nrecomputing.\n\n## What and why\n\nA \"materialized view\" is a fancy name for a simple idea: instead of computing an\nanswer every time someone asks, you compute it **once**, store the finished\nanswer, and just hand it out on each request. \"Materialized\" means the result is\nmade real and saved (as opposed to computed on the fly). \"View\" means it is a\nread-friendly projection of your underlying data.\n\nThink of a leaderboard. Computing \"the top 10 players by score\" means scanning\nevery player and sorting. You do not want to do that on every page load. So you\ncompute it in the background, save the finished top-10 list as a view, and every\npage load just reads that saved list. Fast, cheap, and it does not get slower as\nyou add players.\n\nUse a `View` when:\n\n- A page needs a result that is **expensive to compute** (a scan, a sort, a fold\n over many rows).\n- That result is **read far more often than it changes** (a home page, a feed, a\n leaderboard, a \"latest N\" list, a rendered summary).\n\nThe rule of thumb: if a route wants \"the newest N of something\" or \"the top N of\nsomething\", that is a scan, and scans are barred on the request path. A `View`\nis how you serve that result without scanning per request.\n\n```mermaid\nflowchart LR\n subgraph off[\"Off the request path (@derive)\"]\n S[(\"Source data<br/>events / counters / records\")] --> D[\"@derive: scan + compute\"]\n D --> P[\"publish(key, result)\"]\n end\n P --> V[(\"View\")]\n subgraph on[\"On the request path (@get)\"]\n G[\"view.get(key)\"] --> V\n end\n```\n\n## The type\n\nA `View` has two type parameters: the **key** (which view) and the **value**\n(the precomputed result).\n\n```ts\nView<K, V>\n```\n\n- `K` is the key type: it picks *which* view to read or publish. For a single\n global leaderboard, the key can be a fixed constant like `'main'`.\n- `V` is the value type: the finished result you serve. Both are\n [`@data`](../concepts/types.md) classes.\n\nDeclare it as a `@collection` field inside a `@database` class, alongside the\nsources it is built from:\n\n```ts\n@database\nclass BoardDb {\n @collection static scores: Events<GameKey, ScoreEvent>; // a source\n @collection static board: View<GameKey, Leaderboard>; // the view\n}\n```\n\n## Operations\n\nA `View` has exactly two operations you use directly (plus `require`, a\nconvenience wrapper). Exact signatures:\n\n| Operation | Signature | What it does |\n| --- | --- | --- |\n| `get` | `get(key: K): V \\| null` | Read the published view, or `null` if nothing has been published yet. |\n| `require` | `require(key: K): V` | Like `get`, but traps (aborts the request) if nothing is published. |\n| `publish` | `publish(key: K, value: V): void` | Overwrite the view for `key` with a fresh result. |\n\n### `get`\n\nA plain keyed read. It is cheap and is allowed from **any** handler, including a\nread-only `@get` route.\n\n```ts\nconst board = BoardDb.board.get(new GameKey('main'));\nif (board == null) {\n // Nothing published yet (e.g. brand-new game). Serve an empty default.\n return new Leaderboard();\n}\nreturn board;\n```\n\nAlways handle the `null` case: until a `@derive` (or `@job`) has published at\nleast once, `get` returns `null`.\n\n### `publish`\n\nOverwrites the stored view with a new value. This is how the view gets its\ncontent.\n\n```ts\nBoardDb.board.publish(new GameKey('main'), freshBoard);\n```\n\n`publish` is **restricted**: you may only call it from a\n[`@derive`](../background/derive.md) or a `@job` (a background task), never from a\nrequest handler (`@get`/`@post`/...). The compiler enforces this, and so does the\nedge at runtime. The reason: a view is meant to be maintained off the request\npath from the source of truth, not written ad hoc by whichever request happens to\nrun.\n\n### How `publish` differs from a Documents write\n\nA `View`'s `publish` looks like writing a value, but it is not the same as a\n[Documents](./documents.md) `patch`/`create`. The differences matter:\n\n| | `Documents` write (`create` / `patch`) | `View` `publish` |\n| --- | --- | --- |\n| Who may call it | Any action handler (`@post`, ...) | Only a `@derive` or `@job` |\n| Meaning | The record IS the source of truth | The view is a **copy** derived from a source |\n| Version control | You may version-check (optimistic concurrency) | The host assigns the version; a later publish always wins (last writer wins) |\n| Read path | Standard keyed read | A read-optimized fast path (views are heavily read) |\n| Who owns correctness | You (each writer edits the true value) | The derive (it recomputes from the source, so the view always converges) |\n\nIn short: a Document is the truth; a View is a saved snapshot of a computation\nover the truth. If the view is ever wrong or stale, the derive just recomputes and\nrepublishes it. You never hand-edit a view from a route.\n\n## Automatic maintenance with `@derive`\n\nYou rarely call `publish` by hand. The normal pattern is a\n[`@derive`](../background/derive.md): a method on your `@database` class that\nreads the sources, builds the value, and publishes it. The runtime runs it for\nyou:\n\n- **Right after a write to a source.** When a request writes one of the\n database's source collections (an `append`, a `counter.add`, a record write),\n the database's derives run right after the response is produced. So the view\n reflects the new data on the next read.\n- **On box load.** When the server starts or reloads, the views are rebuilt from\n their sources before the first read is served.\n\nYou never call a derive yourself, and a derive's own `publish` never re-triggers\nit. See [`@derive`](../background/derive.md) for the full rules.\n\n## Worked example: a leaderboard\n\nA game appends score events; a derive folds them into a top-10 list; a route\nserves that list with a single `get`.\n\n```ts\nimport { ScoreEvent } from '../models/ScoreEvent';\nimport { Leaderboard } from '../models/Leaderboard';\nimport { GameKey } from '../models/GameKey';\nimport { NewScore } from '../models/NewScore';\n\n@database\nclass BoardDb {\n // The source of truth: every score, appended as it happens.\n @collection static scores: Events<GameKey, ScoreEvent>;\n // The materialized view: the current top entries, ready to serve.\n @collection static board: View<GameKey, Leaderboard>;\n\n // Off the request path: scan the recent scores, sort, publish the top 10.\n @derive\n rebuild(): void {\n const key = new GameKey('main');\n const recent = BoardDb.scores.latest(key, 500); // a scan, allowed in a derive\n const board = new Leaderboard();\n board.entries = topTen(recent); // your own sort/aggregate\n BoardDb.board.publish(key, board);\n }\n}\n\n@rest('leaderboard')\nclass LeaderboardRoutes {\n // GET reads the precomputed view: one keyed read, no scan.\n @get('/')\n public top(): Leaderboard {\n const board = BoardDb.board.get(new GameKey('main'));\n return board == null ? new Leaderboard() : board;\n }\n\n // POST records a score. The @derive rebuilds `board` right after.\n @post('/')\n public submit(input: NewScore): Leaderboard {\n BoardDb.scores.append(new GameKey('main'), new ScoreEvent(input.player, input.score));\n return new Leaderboard(); // ack; the GET serves the updated board from the view\n }\n}\n```\n\nThe models:\n\n```ts\n@data\nexport class ScoreEvent {\n player: string = '';\n score: u64 = 0;\n}\n\n@data\nexport class Leaderboard {\n entries: ScoreEvent[] = [];\n}\n```\n\nA \"latest N\" view is the same shape: the derive calls `latest(key, N)` and\npublishes the list. See the [Events](./events.md) page for that variant end to\nend.\n\n## Consistency\n\n- **Last writer wins.** The host assigns each `publish` a version, and a later\n publish always supersedes an earlier one. Because a derive recomputes the whole\n view from the source of truth, the view converges to a correct snapshot; you do\n not need to coordinate concurrent publishes.\n- **A view can be briefly stale.** ToilDB is worldwide, and a view is published\n at its home then copied to other regions in the background (asynchronous\n replication). A read from a far region can lag the newest publish by a moment.\n This is usually fine for the things views hold (feeds, leaderboards, summaries):\n a leaderboard that is a second behind is still a good leaderboard.\n- **`get` returns `null` until the first publish.** Always default the empty case.\n\n## Gotchas\n\n- **`publish` is derive/job only.** You cannot publish from a route. If you need\n to update a view in response to a request, write the *source* (append an event,\n add to a counter) from the action and let the `@derive` republish.\n- **Handle `null` from `get`.** A never-published view reads as `null`, not as an\n empty value. Return a sensible default.\n- **A view is a copy, not the truth.** Never store data *only* in a view. Keep\n the real data in a source family (Documents, Events, Counters) and treat the\n view as a disposable, recomputable projection.\n- **Keep views bounded.** A view value is read whole on every request, so build\n it from a bounded read (top N, latest N, a total), not the entire history.\n\n## Related\n\n- [`@derive`](../background/derive.md): the normal way a view is maintained.\n- [Events](./events.md): the append-only log a view is often folded from.\n- [Counters](./counters.md): another common source for a view (totals).\n- [Documents](./documents.md): mutable source-of-truth records (and how a write\n differs from a publish).\n- [Data types (`@data`)](../concepts/types.md): how view keys and values are stored.\n",
32
32
  "frontend/components.md": "# Components\n\nA toiljs frontend is a normal React app, so the components you write are just React components: functions that return JSX, hold state with hooks, and compose however you like. There is no toiljs-specific base class, no decorator, and no registration step. On top of your own components, toiljs ships a small set of ready-made ones on the `Toil` global (an image, a script loader, a form, and a few more) for the jobs a plain React app makes you wire up by hand. This page covers both: how your components fit in, and a reference for every `Toil.*` component.\n\n## Writing your own components\n\nAnything you already know about writing React components applies unchanged. You write ordinary functions, use `useState` / `useEffect` / `useMemo` and any hooks you like, and return JSX:\n\n```tsx\n// client/components/Counter.tsx\nimport { useState } from 'react';\n\nexport default function Counter({ start = 0 }: { start?: number }) {\n const [n, setN] = useState(start);\n return <button onClick={() => setN(n + 1)}>Clicked {n} times</button>;\n}\n```\n\nYour reusable components live in `client/components/`, and you import them the normal way from anywhere in `client/`:\n\n```tsx\n// client/routes/index.tsx\nimport Counter from '../components/Counter';\n\nexport default function Home() {\n return (\n <main>\n <h1>Welcome</h1>\n <Counter start={10} />\n </main>\n );\n}\n```\n\nA few things are worth spelling out, because they are the parts that are handled for you rather than by you:\n\n- **Route files must `export default` a component.** A file under `client/routes/` becomes a page only if it default-exports a component (see [Routing](./routing.md)). Files under `client/components/` have no such rule; export them however you please (default or named).\n- **The `Toil.*` globals need no import.** `Toil.Link`, `Toil.useParams()`, `Toil.Image`, and everything else on `Toil` are ambient. The compiler generates a `toil-env.d.ts` that types the global, so your editor autocompletes `Toil.` and type-checks it with no `import` line. The same goes for `Server` (the typed backend client) and the `FastMap` / `DataWriter` data utilities. See the [Frontend overview](./README.md) for the whole ambient surface.\n- **The JSX runtime is configured for you.** You do not write `import React from 'react'` at the top of every file. The build sets up the automatic JSX runtime, so JSX just works. Import named hooks and types from `react` when you need them (`import { useState, type ReactNode } from 'react'`), but the bare React import for JSX is unnecessary.\n\nIn other words, a component in a toiljs app is indistinguishable from a component in any React app. Two things are genuinely special, and both are opt-in.\n\n### Special case 1: components inside an SSR route\n\nIf a route opts into server rendering with `export const ssr = true`, its component tree is rendered once at build time into a template, then filled per request on the edge (see [Rendering and SSR](./rendering.md)). For that to work, any part of the JSX that changes per request (a value from the URL, a list from a loader, a block of user HTML) has to be wrapped in one of the [SSR marker primitives](#ssr-marker-primitives) below, or isolated in a `Toil.Island`. Anything you leave unwrapped gets frozen into the template at its build-time value.\n\nYou do not have to get this perfect: if an SSR route (or a layout above it) cannot render on the server, toiljs skips SSR for that route at build, prints a warning telling you what to move, and falls back to plain client rendering. So the route still works, it just loses its server first paint until you address the warning.\n\n### Special case 2: importing images and other assets\n\nImporting an image with the `?toil` suffix gives you an object carrying the resolved URL, the intrinsic width and height, and an auto-generated blur placeholder (a `blurDataURL`), ready to hand straight to `Toil.Image`:\n\n```tsx\nimport hero from './hero.webp?toil';\n// hero is { src, width, height, blurDataURL }\n\n<Toil.Image src={hero} alt=\"Our office\" />;\n```\n\nPassing the whole object lets `Toil.Image` reserve the correct aspect ratio and paint the blur while the real image loads, with no extra props. See [Images](./images.md) for the full treatment.\n\nPlain asset imports are typed for you as well. A bare `import logo from './logo.svg'` (or `.png`, `.webp`, and so on) resolves to the hashed URL string, and the vite-imagetools query forms (`?url`, `?as=srcset`, `?as=metadata`) are typed too, so you get autocomplete and type-checking on all of them without declaring any modules yourself.\n\n## The toiljs component primitives\n\nThese are the components toiljs provides on the `Toil` global. They cover the framework-level jobs a plain React app leaves to you. All are ambient (no import), and all are fully typed.\n\n| Component | What it is | Renders |\n| --- | --- | --- |\n| `Toil.Image` | Layout-shift-free `<img>` replacement with lazy-load and blur placeholder. | An `<img>` (optionally wrapped in a sizing box). |\n| `Toil.Script` | One-time external or inline `<script>` loader with a load strategy. | Nothing (`null`). |\n| `Toil.Form` | A `<form>` that runs an action on submit and revalidates loader data. | A `<form>`. |\n| `Toil.Slot` | Renders a named parallel-route slot for the current URL. | The slot's route tree, or a fallback. |\n| `Toil.Head` | Declarative `<head>` contribution (title, meta, link). | Nothing (`null`). |\n| `Toil.Metadata` | Declarative route-style metadata applied from any component. | Nothing (`null`). |\n\nThe [SSR marker primitives](#ssr-marker-primitives) (`Toil.Hole`, `Toil.Repeat`, `Toil.RawHtml`, `Toil.attr`, `Toil.Island`) are a separate group, covered in their own section below.\n\n### `Toil.Image`\n\nA drop-in `<img>` replacement that prevents layout shift and lazy-loads by default. You give it `width` and `height` (or `fill`) so it reserves space before the image arrives, it decodes asynchronously, it lazy-loads unless you mark it `priority`, and it can fade in from a blur placeholder. It accepts either a string URL or a `?toil` import object (which auto-fills the size and blur):\n\n```tsx\nimport hero from './hero.webp?toil';\n\nexport default function Home() {\n return (\n <>\n <Toil.Image src={hero} alt=\"Our office\" priority placeholder=\"blur\" />\n <Toil.Image src=\"/team.jpg\" alt=\"The team\" width={800} height={600} />\n </>\n );\n}\n```\n\nKey props are `src`, `alt` (required, pass `alt=\"\"` for a decorative image), `width` / `height` or `fill`, `priority` (for an above-the-fold hero), and `placeholder` (`'empty'` or `'blur'`). It also accepts any standard `<img>` attribute. This is kept brief on purpose: [Images](./images.md) covers sizing, `fill`, blur placeholders, and how it stops layout shift in full.\n\n### `Toil.Script`\n\nLoads an external or inline `<script>` exactly once for the whole life of the app (even across client-side navigations), and lets you choose when it runs with a `strategy` prop. Use it instead of a hand-written `<script>` tag for third-party snippets like analytics or chat widgets, which a plain `<script>` in a single-page app runs unreliably or twice:\n\n```tsx\n// client/layout.tsx\nexport default function Layout({ children }: { children?: React.ReactNode }) {\n return (\n <div className=\"app\">\n <Toil.Script src=\"https://cdn.example-analytics.com/analytics.js\" />\n {children}\n </div>\n );\n}\n```\n\nThe `strategy` is `afterInteractive` (default), `lazyOnload`, or `beforeInteractive`, and there are `onLoad` / `onReady` / `onError` callbacks. `Toil.Script` renders nothing. See [Scripts](./scripts.md) for the strategies, inline scripts, dedup rules, and the full prop table.\n\n### `Toil.Form`\n\nA `<form>` that submits without reloading the page. On submit it runs your `action` (which receives the form's `FormData`), tracks pending and error state, and on success revalidates the current route's loader data so the page reflects the write. It is the convenient front end of the loader/action data loop:\n\n```tsx\n// A guestbook that refreshes its list after a successful sign.\nexport default function Guestbook() {\n const entries = Toil.useLoaderData<typeof loader>();\n\n const sign = async (data: FormData) => {\n const author = String(data.get('author'));\n const message = String(data.get('message'));\n await Server.REST.guestbook.sign({ body: new NewMessage(author, message) });\n };\n\n return (\n <Toil.Form action={sign} resetOnSuccess>\n {({ pending, error }) => (\n <>\n <input name=\"author\" placeholder=\"Your name\" />\n <textarea name=\"message\" />\n <button disabled={pending}>{pending ? 'Signing...' : 'Sign'}</button>\n {error ? <p className=\"err\">Could not sign, try again.</p> : null}\n </>\n )}\n </Toil.Form>\n );\n}\n```\n\nPass a render function as the child to read the live submit state: it receives `{ pending, error, data }`, which is how you disable the button while pending or show an error. The props:\n\n| Prop | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `action` | `(data: FormData) => void \\| Promise<void>` | (required) | Runs on submit, receiving the form's `FormData`. May be async. |\n| `revalidate` | `RevalidateTarget` | `true` | Which loader data to refetch after a successful submit. `true` is the current route. |\n| `onSuccess` | `() => void` | (none) | Called after a successful submit. |\n| `onError` | `(error: unknown) => void` | (none) | Called when the action throws. |\n| `resetOnSuccess` | `boolean` | `false` | Reset the form fields after a successful submit. |\n| `className` | `string` | (none) | Class on the `<form>` element. |\n| `children` | `ReactNode` or `(state) => ReactNode` | (none) | Form contents. A function child receives `{ pending, error, data }`. |\n\nFor writes that are not form submits (a delete button, a like toggle), reach for the underlying `Toil.useAction` hook instead. Both are covered in [Fetching data](./data-fetching.md).\n\n### `Toil.Slot`\n\nRenders the named parallel-route slot for the current URL. A folder starting with `@` (like `@modal`) under `client/routes/` is a whole second route tree that matches the URL independently of the main page; placing `<Toil.Slot name=\"modal\" />` is where that tree renders. If no slot route matches the current URL, it renders the `fallback` (nothing by default):\n\n```tsx\n// client/routes/gallery/layout.tsx\nexport default function GalleryLayout({ children }: { children?: React.ReactNode }) {\n return (\n <div>\n {children}\n <Toil.Slot name=\"modal\" fallback={null} />\n </div>\n );\n}\n```\n\n| Prop | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `name` | `string` | (required) | The slot name: the `@name` directory under `routes/`, without the `@`. |\n| `fallback` | `ReactNode` | `null` | Rendered when no slot route matches the current URL. |\n\nParallel slots and the intercepting routes that fill them (the \"click a photo, open it in a modal\" pattern) are explained in [Routing](./routing.md).\n\n### `Toil.Head`\n\nA declarative way for any component (a page, a layout, a deep child) to contribute to the document `<head>`: a title, `<meta>` tags, and `<link>` tags. It renders nothing; it just applies its head entries for the lifetime of the component and reverts them on unmount. Entries compose across the tree, with later or deeper ones winning per key:\n\n```tsx\nexport default function ArticlePage() {\n const post = Toil.useLoaderData<typeof loader>();\n return (\n <>\n <Toil.Head\n title={post.title}\n meta={[\n { name: 'description', content: post.summary },\n { property: 'og:title', content: post.title },\n ]}\n link={[{ rel: 'canonical', href: `https://example.com/blog/${post.id}` }]}\n />\n <article>{/* ... */}</article>\n </>\n );\n}\n```\n\n`<Toil.Head>` takes `title`, `meta` (an array of `{ name | property, content }` tags), and `link` (an array of `{ rel, href }` tags). The hook form `Toil.useHead(spec)` and the shorthand `Toil.useTitle(title)` do the same job imperatively. See [Metadata and SEO](./metadata.md).\n\n### `Toil.Metadata`\n\nThe declarative, convenience-shaped cousin of `Toil.Head`. Instead of raw meta and link arrays, you pass a route-style metadata object (the same shape a route file's `export const metadata` uses), and toiljs expands the convenience fields (`description`, `keywords`, `canonical`, `openGraph`, and so on) into the right tags. It renders nothing, and applies for the component's lifetime. Use it to set metadata from a component that is not itself a route file (a rendered article, a widget):\n\n```tsx\n<Toil.Metadata\n title=\"Our pricing\"\n description=\"Simple, flat pricing for teams of any size.\"\n openGraph={{ title: 'Pricing', image: 'https://example.com/og/pricing.png' }}\n/>\n```\n\nBecause a route's own `metadata` export is applied last (highest priority), `Toil.Metadata` fills in for routes that declare none and yields to a route that sets the same key. The full field list lives in [Metadata and SEO](./metadata.md).\n\n## SSR marker primitives\n\nThese five primitives matter only for routes with `export const ssr = true`. They are how you tell the build which parts of a server-rendered page are dynamic (filled per request) versus static (baked into the template once). On a client-only route they do nothing special, so you never need them unless you opt a route into SSR.\n\n**The mental model.** In the browser these markers are transparent: `<Toil.Hole>` renders its children, `<Toil.Repeat>` maps its rows, `<Toil.RawHtml>` renders a raw-HTML wrapper, and `Toil.attr(id, value)` returns the value unchanged. Your client-side app runs exactly as written. But under the build-time SSR extractor, each marker instead emits a sentinel token that marks an insertion point. The extractor strips those tokens and records their positions, producing a static template with numbered holes. Per request, your compiled backend fills only the hole values, and the edge splices them into the template. Because the static scaffold around each hole is React's own rendering output and the hole values are escaped exactly as React escapes them, the browser hydrates the result byte-for-byte, with no mismatch.\n\n| Marker | Use it for | Shape |\n| --- | --- | --- |\n| `Toil.Hole` | A single dynamic **text** value. | `<Toil.Hole id=\"...\">{value}</Toil.Hole>` |\n| `Toil.Repeat` | A **list**: one row template repeated over an `each` array. | `<Toil.Repeat id=\"...\" each={rows}>{(item, i) => ...}</Toil.Repeat>` |\n| `Toil.RawHtml` | A block of **trusted, pre-rendered HTML**. | `<Toil.RawHtml id=\"...\" html={s} as=\"section\" />` |\n| `Toil.attr` | A dynamic value in **attribute position** (an `href`, a `class`). | `href={Toil.attr('id', value)}` (a function call) |\n| `Toil.Island` | Content that must render **only in the browser** (escape hatch). | `<Toil.Island>{children}</Toil.Island>` |\n\nA few rules that keep the template valid:\n\n- **Every marker needs a stable `id`**, a short name unique within the page. The build maps each id to a numbered slot, so keep the ids constant across builds.\n- **`Toil.Repeat` needs at least one row at build time.** It captures that first row as the sub-template for every row, so the build render must see a sample with one or more items. An empty `each` gives it nothing to capture.\n- **`Toil.RawHtml` renders inside a wrapper element** (a `<div>` by default, `as=\"section\"` to change the tag), and you own sanitising that HTML, exactly like React's `dangerouslySetInnerHTML`.\n- **`Toil.attr` is a function, not an element.** An attribute is not a child node, so it cannot be a JSX element. You call `Toil.attr(id, value)` right where the attribute value goes, and it composes with literal text around it (`` className={`btn ${Toil.attr('kind', d.kind)}`} ``).\n- **`Toil.Island` renders nothing on the server and on the first (hydration) render**, then reveals its children after mount. So an island gets no server first paint and no SEO, by design. It is the place for anything that genuinely cannot run on the server (reads `window`, calls `Date.now()`, depends on the live URL).\n\nHere is a compact SSR route wiring several of them together. The title, tag list, and author link all come from the route's `loader`:\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const ssr = true;\n\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n // Illustrative shape: { title, tags, authorUrl }.\n return Server.REST.blog.get({ params: { id: params.id } });\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return (\n <article>\n <h1>\n <Toil.Hole id=\"title\">{post.title}</Toil.Hole>\n </h1>\n\n {/* A dynamic attribute: call attr() in attribute position. */}\n <a href={Toil.attr('authorUrl', post.authorUrl)}>By the author</a>\n\n {/* A list: one row template, stamped once per item on the server. */}\n <ul>\n <Toil.Repeat id=\"tags\" each={post.tags}>\n {(tag) => <li key={tag}>{tag}</li>}\n </Toil.Repeat>\n </ul>\n </article>\n );\n}\n```\n\nFor the whole SSR story (the template extraction flow, keeping a route SSR-safe, and the current SSR limitations), see [Rendering and SSR](./rendering.md).\n\n## Related\n\n- [Rendering and SSR](./rendering.md): how SSR routes render, hydrate, and use the marker primitives.\n- [Images](./images.md): the full `Toil.Image` reference, blur placeholders, and layout shift.\n- [Scripts](./scripts.md): `Toil.Script` strategies, inline scripts, and dedup.\n- [Fetching data](./data-fetching.md): `Toil.Form`, `useAction`, loaders, and the typed backend clients.\n- [Metadata and SEO](./metadata.md): `Toil.Head`, `Toil.Metadata`, and per-route metadata.\n- [Routing](./routing.md): pages, layouts, and the parallel slots `Toil.Slot` renders.\n",
33
33
  "frontend/data-fetching.md": "# Fetching data\n\nYour React frontend and your toiljs backend live in one project, so toiljs generates a **typed client** that lets the browser call your server with full type safety and no hand-written `fetch` boilerplate. This page covers loading data for a page, calling the backend directly, submitting forms, and reading who is logged in.\n\n## The generated `Server` client\n\nWhen you build the server, toiljs writes a file at `shared/server.ts` that contains your `@data` classes and a typed description of every backend endpoint. Importing anything from `shared/server` attaches the runtime clients to a global called `Server`. From then on you call your backend through `Server`, fully typed, with editor autocomplete.\n\nThere are two surfaces under `Server`:\n\n- **`Server.REST.<controller>.<route>(args)`**: a real, typed `fetch` client for your `@rest` HTTP controllers. This is the working, recommended way to call the backend today.\n- **`Server.<service>.<method>(args)` and `Server.<remote>(args)`**: the typed RPC surface for `@service` / `@remote` functions.\n\nBoth are generated from your server code, so if you rename a route or change an argument type, the call site is a compile error until you fix it.\n\n> **Rule: never call your own backend with raw `fetch`. Always go through `Server`.**\n> Use `Server.REST.<controller>.<route>(args)` for `@rest` HTTP controllers, or\n> `Server.<service>.<method>(args)` / `Server.<remote>(args)` for `@service` / `@remote`\n> RPC. Raw `fetch` throws away the type safety, the argument/return decoding, and the\n> binary wire format that the generated client handles for you, and it silently breaks\n> when a route is renamed. This applies to AI assistants generating code too.\n>\n> ```ts\n> // WRONG: raw fetch to your backend\n> await fetch('/account/session', { method: 'POST', credentials: 'same-origin' });\n>\n> // RIGHT: the typed client (renames become compile errors, args + result are typed)\n> await Server.REST.account.session();\n> ```\n>\n> `fetch` is only for URLs that are NOT your toiljs backend (a third-party API, a CDN).\n\n## Calling a REST endpoint\n\n`Server.REST` mirrors your `@rest` controllers. If your backend has a `players` controller with routes on it, you call them like this:\n\n```tsx\nimport { NewPlayer, ScoreDelta } from 'shared/server';\n\n// POST /players with a typed @data body -> typed Promise<Player>\nconst player = await Server.REST.players.create({ body: new NewPlayer('Ada') });\n\n// POST /players/:id/score with a path param AND a body\nconst updated = await Server.REST.players.addScore({\n params: { id: 1 },\n body: new ScoreDelta(5n),\n});\n\n// GET /leaderboard -> typed Promise<Standings>\nconst board = await Server.REST.leaderboard.top();\n```\n\nThe single argument is an object with up to four optional parts:\n\n| Key | What it is |\n| --- | --- |\n| `params` | Path parameters, e.g. `{ id: 1 }` for a `/players/:id` route. |\n| `body` | The request body, usually a `@data` class instance. |\n| `query` | Query-string values. |\n| `headers` | Extra request headers. |\n\nReturn values are typed and decoded for you. A route that returns a `@data` type hands you the parsed class instance. A route that returns a raw `Response` hands you the raw fetch `Response`, so you can inspect the status and headers yourself:\n\n```tsx\n// This route returns a Response, so you get the raw fetch Response.\nconst res = await Server.REST.players.get({ params: { id: 1 } });\nif (!res.ok) {\n console.log('status', res.status);\n} else {\n const p = await res.json();\n}\n```\n\n`@data` classes are the typed values that cross the wire. You import them from `shared/server` and construct them normally (`new NewPlayer('Ada')`). See [Data types](../backend/data.md) for how they are defined on the server.\n\n### Handling errors\n\nA failed call throws. The global `parseError` helper turns any caught value into a readable message, which is handy in a `catch`:\n\n```tsx\ntry {\n const board = await Server.REST.leaderboard.top();\n} catch (err) {\n console.error(parseError(err));\n}\n```\n\n## Loading data for a page (loaders)\n\nCalling the backend from a button handler (as above) is fine for actions. For the data a page needs to *render*, use a route **loader** instead of a `useEffect`. A loader runs on navigation, in parallel with loading the page's code, and the page suspends (showing its `loading.tsx`) until the data is ready:\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n return Server.REST.blog.get({ params: { id: params.id } });\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return <article><h1>{post.title}</h1></article>;\n}\n```\n\n`useLoaderData<typeof loader>()` is fully typed from the loader's return. This keeps data fetching declarative and out of effects, and it is what lets server-rendered pages seed the browser with the server's data so hydration stays clean.\n\n### Caching and revalidating loader data\n\nLoader results are cached by URL. You control how long with `export const revalidate`:\n\n| `revalidate` value | Behavior |\n| --- | --- |\n| `0` (default) | Re-run the loader on every navigation to the route. |\n| a number `n` | Reuse cached data for `n` seconds, then refetch. |\n| `false` | Cache until you invalidate it manually. |\n\nAfter a mutation (say you just POSTed a new comment), refresh the current page's data with `revalidate()` or the router:\n\n```tsx\nconst router = Toil.useRouter();\nawait Server.REST.comments.add({ body: new NewComment(text) });\nrouter.revalidate(); // refetch the active route's loader\n// or target another route: router.revalidate('/posts');\n```\n\n`router.refresh()` re-runs the current loader and clears the cache; `router.revalidate(href)` invalidates a specific route.\n\n### Invalidating loader data by hand\n\n`revalidate()` and the `<Toil.Form>` `revalidate` prop are both built on one lower-level primitive: `Toil.invalidateLoaderData(href?)`. It drops cached loader data so the next render refetches it, but it does **not** trigger the re-render itself (that is the difference from `revalidate()`, which does both):\n\n- `invalidateLoaderData()` with no argument clears every route's cached data.\n- `invalidateLoaderData('/posts')` clears just that one route's entry.\n\nYou rarely need it directly; `revalidate()` (which pairs it with a re-render) is what you usually want. Reach for the primitive when you want to mark data stale *now* but let a later navigation do the actual refetch:\n\n```tsx\n// After a background sync finished, mark the dashboard stale so its\n// next visit refetches, without disturbing the page the user is on.\nToil.invalidateLoaderData('/dashboard');\n```\n\nSignature: `invalidateLoaderData(href?: string): void`.\n\n## Forms\n\nFor the common \"submit a form, then refresh the page's data\" loop, use `Toil.Form`. It runs an async action on submit (no page reload), tracks pending and error state, and revalidates the route's loader data on success:\n\n```tsx\nimport { NewMessage } from 'shared/server';\n\nexport default function Guestbook() {\n const entries = Toil.useLoaderData<typeof loader>();\n\n const sign = async (data: FormData) => {\n const author = String(data.get('author'));\n const message = String(data.get('message'));\n await Server.REST.guestbook.sign({ body: new NewMessage(author, message) });\n };\n\n return (\n <Toil.Form action={sign} resetOnSuccess>\n {({ pending }) => (\n <>\n <input name=\"author\" />\n <input name=\"message\" />\n <button disabled={pending}>Sign</button>\n </>\n )}\n </Toil.Form>\n );\n}\n```\n\nKey `Toil.Form` props:\n\n| Prop | Default | What it does |\n| --- | --- | --- |\n| `action` | (required) | Runs on submit, receiving the form's `FormData`. May be async. |\n| `revalidate` | `true` (current route) | Which loader data to refetch after a successful submit. |\n| `resetOnSuccess` | `false` | Clear the form fields after success. |\n| `onSuccess` / `onError` | | Callbacks for the two outcomes. |\n\nPassing a **render function** as `children` gives you live submit state (`pending`, `error`), so you can disable the button while the request is in flight. On success, `Form` revalidates the loader, so the page's data updates automatically without a manual refetch.\n\n## Mutations without a form: `useAction`\n\n`Toil.Form` is convenient sugar over a more general primitive, `Toil.useAction`. Use `useAction` for any write that is not a form submit: a delete button, a \"like\" toggle, a \"mark as read\" action. It runs your async function on demand and tracks the whole lifecycle (`pending`, `error`, `data`), then revalidates the affected loader data on success so the page reflects the change, exactly like `Form` does.\n\nYou call it with the function to run and an optional options object, and it hands back a handle:\n\n```tsx\nexport default function PostRow({ id }: { id: number }) {\n const del = Toil.useAction(\n (postId: number) => Server.REST.posts.remove({ params: { id: postId } }),\n { revalidate: true, onError: (e) => alert(parseError(e)) },\n );\n\n return (\n <button disabled={del.pending} onClick={() => void del.run(id)}>\n {del.pending ? 'Deleting...' : 'Delete'}\n </button>\n );\n}\n```\n\nThe handle it returns:\n\n| Field | What it is |\n| --- | --- |\n| `run(input)` | Runs the action. Resolves to the result, or `undefined` if it threw. The error is captured in `error` rather than rejected, so a fire-and-forget `onClick` cannot leak an unhandled rejection. |\n| `pending` | `true` while a run is in flight. Drive a spinner or a disabled button off this. |\n| `error` | The error from the last failed run, or `undefined`. |\n| `data` | The value the last successful run returned, or `undefined`. |\n| `reset()` | Clear `pending` / `error` / `data` back to idle. |\n\nThe options mirror `Toil.Form`:\n\n| Option | Default | What it does |\n| --- | --- | --- |\n| `revalidate` | `true` (current route) | Which loader data to refetch after a successful run: `true` (the active route), an `href` or array of hrefs (those routes), or `false` (none). |\n| `onSuccess` | | Called with the action's return value after a successful run. |\n| `onError` | | Called with the thrown value when the action fails. |\n\n`Toil.Form` is just this hook wired to a `<form>`'s submit event, with the form's `FormData` passed as the input. Anything a form does, you can do by hand with `useAction`.\n\n## Reading who is logged in\n\nTo render \"logged in as ...\" you need the current user. toiljs generates a `getUser()` helper (from the backend's `@user` surface) that reads the current user from a readable session cookie with no network round-trip. It is instant, which makes it perfect for display, but it is **untrusted**: a value read on the client can be forged, so never gate anything security-sensitive on it. For a trusted check, call a guarded backend route that re-verifies the signed session.\n\nThe full auth flow (post-quantum login, sessions, `getUser()`, and guarding routes) is its own guide:\n\n- [Auth usage](../auth/usage.md): reading the session, `getUser()`, and guarding pages.\n\n## RPC (`@service` / `@remote`)\n\nAlongside REST, toiljs generates a typed RPC surface for `@service` methods and free `@remote` functions. These read like plain function calls with no URL:\n\n```tsx\nconst n = await Server.ping(10); // a free @remote\nconst count = await Server.stats.playerCount(); // a @service method\n```\n\nThe types come straight from your server, so `Server.ping` knows it takes a number and returns a number. Under the hood each call encodes its arguments and POSTs them to a single reserved endpoint (`/__toil_rpc`) with a compact method id, then decodes the typed result. Both the local dev server and the production edge dispatch this endpoint, so RPC works end to end.\n\nIf a call throws an \"unavailable\" error, it means the generated client has not attached yet: build the server (`npm run build:server`) to regenerate `shared/server.ts`, and import from `shared/server` so the client loads (see the gotchas below). See [Typed RPC](../backend/rpc.md) for the backend side and when to choose RPC over REST.\n\n## Gotchas\n\n- **Import from `shared/server` to attach the clients.** `Server.REST` (and the RPC client) attach when you import from `shared/server`. Importing your `@data` classes from there does it naturally; if `Server.REST.foo()` throws an \"unavailable\" error, make sure the server has been built (`npm run build:server`) and that `shared/server` is imported.\n- **The server runs a fresh instance per request.** In the examples, in-memory writes are previews that do not persist. To keep data, write it to the database (see [Database](../database/README.md)). This is a backend property, but it explains why a create returns a value that is not there on the next request.\n- **Fetch page data in a `loader`, not `useEffect`.** A loader runs in parallel with the route chunk and integrates with `loading.tsx`, suspense, caching, and SSR hydration. A `useEffect` fetch runs only after the page mounts (slower, and invisible to the server).\n- **`getUser()` is display-only.** It is fast because it does not verify anything. Do real authorization on the server.\n\n## Related\n\n- [Routing](./routing.md): loaders, `loading.tsx`, and navigation.\n- [Backend HTTP routes](../backend/rest.md): the `@rest` controllers behind `Server.REST`.\n- [Typed RPC](../backend/rpc.md): the `@service` / `@remote` surface behind `Server`.\n- [Data types](../backend/data.md): the `@data` classes that cross the wire.\n- [Auth usage](../auth/usage.md): sessions and `getUser()`.\n",
@@ -68,5 +68,5 @@ export const TOIL_DOCS = {
68
68
  "services/environment.md": "# Environment and secrets\n\n**`Environment` gives your app configuration values and secrets that are set outside your code**, so your compiled backend (`.wasm`) never carries a password or an API key. You read them at runtime; you never write them from code.\n\n## Why this exists\n\nYour backend often needs two kinds of outside values:\n\n- **Config**: non-sensitive settings that can change per deployment, like a public API base URL, a region name, or a feature flag.\n- **Secrets**: sensitive values you must never leak, like a payment provider's API key.\n\nThe wrong way to handle these is to type them into your source code. Then your secret is in your git history, in your build output, and in the `.wasm` shipped to every edge node. The right way, and the way toiljs uses, is the **GitHub Actions model**: you set these values out of band (today a dashboard, backed by the edge database), and your code reads them by name at runtime. Your compiled program carries **no credentials**.\n\n`Environment` is an ambient global: you use it with no import, like `Time` or `crypto`.\n\n## The two buckets: `get` vs `getSecure`\n\nThere are two **disjoint** buckets, exactly like GitHub Actions' `vars` versus `secrets`:\n\n- **`Environment.get(key)`** reads a **plain variable** (config). Returns the string, or `null` if it is not set.\n- **`Environment.getSecure(key)`** reads a **secret**. Returns the string, or `null` if it is not set.\n\n\"Disjoint\" means the buckets never overlap: a secret is **never** returned by `get()`, and a plain var is never returned by `getSecure()`. This is a deliberate safety property. It means a secret cannot leak through a code path that logs the result of a `get()`, because `get()` can never see a secret in the first place. Keys are case-sensitive and matched exactly.\n\n```mermaid\nflowchart LR\n subgraph Store[\"Per-app environment (set out of band)\"]\n V[\"vars<br/>(config)\"]\n S[\"secrets\"]\n end\n V -->|Environment.get key| G[\"your handler\"]\n S -->|Environment.getSecure key| G\n V -. never .-x GS[\"getSecure\"]\n S -. never .-x GT[\"get\"]\n```\n\n## How: a worked example reading a secret\n\nHere a route reads a public config value and a secret, then uses the secret to authorize a call to a third party. Note what it does **not** do: it never logs the secret and never returns it to the client.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('billing')\nclass Billing {\n @get('/status')\n show(ctx: RouteContext): Response {\n const base = Environment.get('PUBLIC_API_BASE'); // plain var, or null\n const key = Environment.getSecure('STRIPE_KEY'); // secret, or null\n\n if (key == null) {\n return Response.text('billing not configured\\n', 503);\n }\n\n // Use `key` to authorize an outbound call. NEVER log it,\n // NEVER put it in the response, NEVER copy it into the client bundle.\n // (Outbound calls are made from a daemon or service; see the links below.)\n\n return Response.text(base != null ? base : 'unset');\n }\n}\n```\n\nBoth getters return `string | null`. Always handle the `null` case: a value that is not configured comes back as `null`, not an empty string or a crash.\n\n> **Secrets are plaintext in your module at runtime.** That is the point: you need the actual bytes to call out to a third party. So the rule is simple. Do not log a secret, do not put it in a response, and do not copy it into anything the browser receives.\n\n## Where the values live\n\nYou set vars and secrets in **two separate places**, so the split is structural (the secrets store can be locked down on its own).\n\n### On the edge\n\nEach host has its own environment, backed by the edge database and, as a fallback, by two dotenv files kept **outside** your deployed project (so the config watcher never even sees a credential):\n\n```bash\n# $TOIL_ENV_DIR/<host>.env (default dir: /run/toil/env) -> Environment.get(...)\nPUBLIC_API_BASE=https://api.example.com\nREGION=eu\n\n# $TOIL_ENV_DIR/<host>.env.secrets (mode 0600) -> Environment.getSecure(...)\nSTRIPE_KEY=sk_live_xxx\n```\n\nA **dotenv file** is just `KEY=value`, one per line, with `#` comments, optional `export`, and optional quotes.\n\nFramework-reserved keys use the `TOIL_` prefix (today, the `TOIL_EMAIL_*` email provider config). These are **host-only**: resolved and used in Rust where the framework needs them, and **stripped from both guest buckets**, so your code can never read them with `get` or `getSecure`. You do not read email config yourself; see [Email](./email.md).\n\n### In local development\n\n`toiljs dev` reads two files at your project root:\n\n```bash\n# .env (plain vars)\nPUBLIC_API_BASE=http://localhost:4000\n\n# .env.secrets (secrets; mode 0600; gitignored by the scaffold)\nSTRIPE_KEY=sk_test_xxx\n```\n\nIt also overlays `process.env` as plain vars, for convenience. The behavior is identical to the edge, so code that reads env in dev reads it the same way in production. Both files are gitignored by the project scaffold so you do not commit them by accident.\n\n## Secrets never enter the `.wasm`\n\nThis is the whole point, so it is worth stating plainly. Your source code contains only the **names** you look up (`'STRIPE_KEY'`), never the values. The compiler turns your names into calls to a host function; the actual value is resolved on the trusted server side, from that host's environment, at the moment you ask. The value is never baked into the compiled module and never shipped to the browser.\n\nOn the edge, a secret is **zeroized** (wiped from memory) when the host goes cold, so it does not linger.\n\n## How caching works (and why it is safe)\n\nReading env goes through a host lookup, so the edge caches values so a busy app is not paying for a fresh lookup on every request. The cache is designed to be abuse-proof:\n\n- **Lazy**: env is loaded the first time your code reads it, not eagerly. A host that never reads env costs nothing.\n- **Shared and read-only**: the data lives in one place and is never copied per request.\n- **Bounded with idle eviction (a TTL cache)**: entries are dropped after they sit unused, and the total size is capped. So a flood of requests to many different hosts can never grow memory without bound, and secrets are wiped when a host goes cold.\n\nYou do not manage any of this. You just call `get` / `getSecure`.\n\n## Build-time config vs runtime environment\n\nDo not confuse two different things that both feel like \"configuration\":\n\n- **Build-time config** lives in your project's config file (for example `toil.config.ts`). It shapes how your app is *built and wired*: routes, tiers, which features are on. It is baked into the build. See [Configuration](../concepts/config.md).\n- **Runtime environment** (this page) is resolved *while your code runs*, per host, and is never baked in. It is where secrets and per-deployment values belong.\n\nRule of thumb: if a value is a secret, or it differs between your dev machine and production, it belongs in the runtime environment, not in build-time config.\n\n## Gotchas\n\n- **`Environment` is read-only.** There is no `set`. You configure values out of band (dashboard / dotenv files), never from the module.\n- **Handle `null`.** An unset key returns `null`. Do not assume a value is present.\n- **Never log or return a secret.** `getSecure` hands you real credentials; treat them like one.\n- **Do not put a secret in build-time config.** That would bake it into the `.wasm`, defeating the whole design.\n- **`TOIL_`-prefixed keys are invisible to your code.** They are host-only framework config; you cannot read them with `get`/`getSecure`.\n\n## Related\n\n- [Email](./email.md): configured through the reserved, host-only `TOIL_EMAIL_*` keys.\n- [Configuration](../concepts/config.md): build-time config, and how it differs from runtime env.\n- [Crypto](./crypto.md): use a secret from `getSecure` as a signing or encryption key.\n- [Cookies](./cookies.md): `SecureCookies` takes a key you can source from a secret.\n",
69
69
  "services/ratelimit.md": "# Rate limiting\n\n**Rate limiting caps how often one client can call a route.** You add the `@ratelimit` decorator to a route, and the edge rejects a client that goes over the limit **before your code runs**.\n\n## Why you want it\n\nSome routes are attractive to abuse. A login route invites password guessing (a \"brute-force\" attack: trying thousands of passwords). A \"send me a code\" route invites spam. A public write endpoint invites floods. Rate limiting puts a hard ceiling on how fast any single client can hammer a route, so an attacker cannot try millions of guesses and a script cannot drown your app.\n\nBecause it runs at the edge before your handler (and before the auth check), an over-the-limit request costs you almost nothing: your WebAssembly code never even wakes up.\n\n## How: the `@ratelimit` decorator\n\nPut it above a route, alongside the verb decorator (`@get`, `@post`, and so on):\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('auth')\nclass Auth {\n // At most 5 login attempts per 60 seconds, per client.\n @ratelimit(RateLimit.SlidingWindow, 5, 60)\n @post('/login')\n login(ctx: RouteContext): Response {\n // ...only runs if the caller is under the limit...\n return Response.text('ok\\n');\n }\n}\n```\n\nThe shape is `@ratelimit(strategy, limit, window)`:\n\n- **`strategy`** is a `RateLimit` value: `RateLimit.FixedWindow`, `RateLimit.SlidingWindow`, or `RateLimit.TokenBucket`. It is an ambient global, so you do not import it.\n- **`limit`** and **`window`** are two whole numbers. Their exact meaning depends on the strategy (see the table below). For the window strategies, it reads as \"at most `limit` requests per `window` seconds\".\n\nBoth numbers must be plain integer literals. A malformed decorator emits no guard at all (it fails safe) rather than compiling something wrong, the same rule as [`@cache`](./caching.md).\n\n## What a rejected request looks like\n\nWhen a client is over the limit, the edge returns **`429 Too Many Requests`** with a **`Retry-After`** header telling the client how many whole seconds to wait before trying again. Your handler does not run.\n\n```mermaid\nflowchart TD\n R[\"Request to a<br/>rate-limited route\"] --> C{\"This client<br/>under the limit?\"}\n C -->|Yes| A[\"Run @auth, then your handler\"]\n C -->|No| L[\"429 Too Many Requests<br/>Retry-After: 30\"]\n```\n\nA `429` is the standard \"slow down\" status. A well-behaved client library will read `Retry-After` and back off automatically.\n\n## The three strategies\n\nA **window** is a slice of time the limiter counts within. The strategies differ in how they draw those slices.\n\n| Strategy | `limit`, `window` mean | Behavior |\n| --- | --- | --- |\n| `FixedWindow` | `limit` events per `window` seconds | Cheapest. Counts in fixed wall-clock buckets (for example, each minute on the clock). A caller who times a burst around a bucket boundary can briefly get up to ~2x `limit` across two adjacent buckets. |\n| `SlidingWindow` | `limit` events per `window` seconds | Smooths that boundary spike by weighting the previous window. **The best general choice** for \"N per period\". |\n| `TokenBucket` | `limit` = burst size, `window` = refill rate **per second** | Allows an initial burst of up to `limit`, then refills at a steady `window` tokens per second. Good for APIs that are bursty but must stay bounded on average. |\n\nExamples:\n\n```ts\n// 100 requests per minute, smoothed. A good default for a public API.\n@ratelimit(RateLimit.SlidingWindow, 100, 60)\n\n// Burst of 20, then 5 per second sustained.\n@ratelimit(RateLimit.TokenBucket, 20, 5)\n\n// Exactly 3 per hour, cheapest counting.\n@ratelimit(RateLimit.FixedWindow, 3, 3600)\n```\n\nIf you are unsure, use `SlidingWindow`. It behaves the way people intuitively expect \"N per period\" to behave.\n\n## How a client is identified\n\nBy default the limiter keys on the **client IP address**: specifically the network peer address the edge actually observed, not a header like `X-Forwarded-For` (which a client can forge). That is what makes this a real abuse control: a caller cannot reset their own bucket by lying in a header.\n\nThe count is **exact across all of the edge's workers**: a given IP always maps to one authoritative place that keeps the count, so the limit is global for the route, not per worker. And each rate-limited route has its **own independent limiter**: a limit on `/login` does not eat into `/signup`'s budget.\n\nOnly routes that opt in with `@ratelimit` pay any cost; everything else runs on the untouched fast path.\n\n## Per-user limits with a custom key\n\nThe `@ratelimit` decorator always keys on the client IP. Sometimes you want to limit by **account** instead: \"at most 5 password changes per user per hour,\" no matter which IP the user comes from. The runtime supports this through a callable you invoke yourself, `RateLimitService.guardKeyed`. The decorator does not yet expose keyed limiting, so this callable is how you get it today.\n\nLike `crypto` and `AuthService`, `RateLimitService` and the `RateLimit` enum are **ambient globals**: use them with no import.\n\n```ts\n// Signature (ambient global; no import):\nRateLimitService.guardKeyed(\n routeId: i32, // a stable integer naming this limiter (see below)\n strategy: i32, // a RateLimit value: FixedWindow / SlidingWindow / TokenBucket\n limit: i32,\n window: i32,\n key: string, // the identity to limit by; an empty string falls back to the peer IP\n): Response | null\n```\n\nIt returns `null` when the caller is under the limit (proceed), or a ready-made **`429`** `Response` (with a `Retry-After` header) when they are over it. You call it near the top of your handler and **early-return** the response when it is non-null:\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('account')\nclass Account {\n @auth\n @post('/change-password')\n changePassword(ctx: RouteContext): Response {\n // Key the limit on the logged-in user's stable id (hex form), so the\n // budget follows the account across devices and IP addresses.\n const key = AuthService.hasSession() ? AuthService.userId()!.toHex() : '';\n\n // At most 5 attempts per user per hour. routeId 1001 names THIS limiter.\n const limited = RateLimitService.guardKeyed(1001, RateLimit.SlidingWindow, 5, 3600, key);\n if (limited != null) return limited; // over the limit: 429, the handler stops here\n\n // ...under the limit and authenticated: do the work...\n return Response.text('password changed\\n');\n }\n}\n```\n\nA few things to know:\n\n- **`routeId` names the counter.** Each distinct `routeId` is an **independent** limiter, scoped to your app (it never collides with another tenant's). Pick a stable integer per logical limiter and keep it constant, use a different value from any other keyed limiter you add, and avoid reusing a value the `@ratelimit` decorator already assigned to a route.\n- **An empty `key` falls back to the peer IP,** so `guardKeyed(..., '')` behaves like the decorator's default. Handle the not-logged-in case explicitly (as above) rather than passing a blank string by accident.\n- **You control the ordering.** Unlike the decorator (which the compiler always places before `@auth`), a manual `guardKeyed` runs wherever you put it. If you key on the user id, call it **after** `@auth` has established the session, as in the example.\n- **Same strategies and meanings** as the decorator: see [the three strategies](#the-three-strategies) for what `strategy`, `limit`, and `window` do.\n\n## Worked example: protecting login and a write route\n\nLogin is the classic case. This mirrors what the built-in auth system does for you (`register` and `login` already carry `@ratelimit(SlidingWindow, 5, 60)`), so you rarely have to add it there yourself. See [Using auth](../auth/usage.md).\n\nFor your own sensitive routes, the pattern is the same. Here a public \"post a comment\" endpoint is both rate-limited and auth-guarded:\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('comments')\nclass Comments {\n // 10 comments per minute per client, and you must be logged in.\n @ratelimit(RateLimit.SlidingWindow, 10, 60)\n @auth\n @post('/')\n create(ctx: RouteContext): Response {\n // Runs only if the caller is under the limit AND authenticated.\n return Response.text('posted\\n');\n }\n}\n```\n\nThe order is fixed and safe: rate limiting runs **first** (so even unauthenticated floods are capped), then the `@auth` guard, then your handler. You do not manage that ordering.\n\n## Choosing a limit\n\n- **Login / signup / \"send a code\":** keep it tight. 5 to 10 per minute per client is plenty for a human and painful for a guesser.\n- **Public read APIs:** a `SlidingWindow` of 60 to 300 per minute is a reasonable starting point; raise it if legitimate clients hit it.\n- **Bursty clients (dashboards that fire several calls on load):** consider `TokenBucket` so a short burst is allowed but the sustained rate stays bounded.\n- Start conservative and loosen if you see legitimate users getting `429`s. It is easier to relax a limit than to recover from an abuse incident.\n\n## Gotchas and current limits\n\n- **Route-level only.** Put `@ratelimit` on each route you want limited. There is no controller-wide form yet (unlike `@auth`, which you can put on a whole class).\n- **Keyed on IP today.** The decorator keys on the client IP. Users behind a shared IP (a corporate network, a mobile carrier gateway) share a bucket, so do not set login limits so low that a busy office trips them. A per-user key (limiting by account instead of IP) exists in the runtime but is not yet exposed through the decorator. You can still get per-user limiting today by calling [`RateLimitService.guardKeyed`](#per-user-limits-with-a-custom-key) directly.\n- **`TokenBucket`'s second number is a rate, not a duration.** For the bucket, `window` means \"tokens refilled per second\", not \"seconds\". Re-read the table if it surprises you.\n- **Same behavior in dev.** `toiljs dev` runs a single-process mirror of all three strategies, so a limited route behaves the same locally as on the edge.\n\n## Related\n\n- [Auth, sessions, and `@user`](../auth/usage.md): `@ratelimit` runs before the `@auth` guard, so it protects the login itself.\n- [Email](./email.md): pair `@ratelimit` with email triggers (verification codes, password resets) to blunt abuse.\n- [Caching](./caching.md): the other pre-handler guard; both fail safe on a malformed decorator.\n- [Every decorator](../concepts/decorators.md).\n",
70
70
  "services/README.md": "# Platform services\n\nYour toiljs backend runs on the **Dacely edge**: a fleet of servers spread around the world that sit close to your users and run your compiled backend (a small, sandboxed WebAssembly program). \"Platform services\" are the batteries the edge includes so you do not have to run your own: response caching, rate limiting, secrets, email, analytics, cryptography, cookies, and a clock.\n\nEach service is either a **route decorator** (an annotation you put above a route, like `@cache`) or an **ambient global** (an object you can use without importing it, like `Environment` or `Time`). Nothing here needs a separate server, a database you manage, or a third-party account for the basics.\n\n## What each service is (and when you reach for it)\n\n- **[Caching](./caching.md)** stores a copy of a response so the edge can hand it back instantly, without re-running your code. Reach for it when a route returns the same public data to many users (a leaderboard, a product listing, a marketing page). Skip it for anything personalized.\n- **[Rate limiting](./ratelimit.md)** caps how often one client can hit a route. Reach for it on anything abusable: login, signup, \"send me a code\", a public write API. It stops brute-force and spam before your code even runs.\n- **[Environment and secrets](./environment.md)** gives your app configuration (like a public API base URL) and secrets (like a payment provider key) that are set outside your code, so your compiled program never carries a credential. Reach for it any time you call a third party or need a value that differs between dev and production.\n- **[Email](./email.md)** sends transactional email (verification codes, password resets, receipts) through a provider you configure once. Reach for it when your app needs to email a user.\n- **[Analytics](./analytics.md)** counts requests, statuses, bytes, and cache hits per site, with no code changes. Reach for it to see traffic and cache effectiveness.\n- **[Crypto](./crypto.md)** is a small, safe cryptography toolkit (hashing, HMAC, AES, random bytes) available with no import. Reach for it to sign or encrypt your own data. It is the engine under signed cookies.\n- **[Cookies](./cookies.md)** is a complete cookie layer: read incoming cookies, set them on a response, and optionally sign or encrypt their values. Reach for it for preferences, feature flags, and any small piece of state you keep in the browser.\n- **[Time](./time.md)** is the edge's clock: the current wall-clock time, read through a single blessed API. Reach for it whenever you need a timestamp or need to compute an expiry.\n\n## Pick a service by what you need\n\n| I want to... | Use | Shape |\n| --- | --- | --- |\n| Serve the same public response fast, without re-running code | [Caching](./caching.md) | `@cache(...)` / `Response.cache(...)` |\n| Stop brute-force logins or spammy writes | [Rate limiting](./ratelimit.md) | `@ratelimit(...)` |\n| Read a config value or a secret set outside my code | [Environment](./environment.md) | `Environment.get` / `getSecure` |\n| Email a user a code or a receipt | [Email](./email.md) | `EmailService` |\n| See traffic and cache-hit numbers per site | [Analytics](./analytics.md) | `Analytics` |\n| Hash, sign, encrypt, or make random bytes | [Crypto](./crypto.md) | `crypto` |\n| Remember a preference or flag in the browser | [Cookies](./cookies.md) | `Cookie` / `Cookies` / `SecureCookies` |\n| Stamp or compare an instant in time | [Time](./time.md) | `Time.nowMillis()` / `Time.nowSeconds()` |\n| Know who is logged in and protect routes | [Auth](../auth/README.md) | `@auth` / `@user` |\n\n## How they fit together\n\nDecorators and globals compose. A single route can be rate-limited, auth-guarded, and cached at once, and it can read a secret and stamp the time inside the handler:\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('api')\nclass Api {\n @ratelimit(RateLimit.SlidingWindow, 100, 60) // at most 100/min per client IP\n @cache(1) // edge-cache for 1 minute\n @get('/status')\n status(ctx: RouteContext): Response {\n const region = Environment.get('REGION'); // config value, or null\n const now = Time.nowSeconds(); // seconds since 1970\n return Response.json(`{\"region\":${JSON.stringify(region)},\"at\":${now}}`);\n }\n}\n```\n\nThe order of guards is fixed by the framework: rate limiting runs first (so floods are rejected cheaply), then auth, then your handler, and caching is applied to whatever your handler returns. You do not have to think about that order; it is safe by construction.\n\n## Related\n\n- [Every decorator, in one place](../concepts/decorators.md)\n- [Configuration: build-time config vs runtime environment](../concepts/config.md)\n- [Compute tiers (where your code runs)](../concepts/tiers.md)\n- [Backend overview](../backend/README.md) and [HTTP routes](../backend/rest.md)\n- [Auth, sessions, and `@user`](../auth/README.md)\n",
71
- "services/time.md": "# Time\n\n**`Time` is the edge's clock: the blessed way to read the current time in your backend.** It gives you the number of milliseconds or seconds since a fixed reference point, so you can stamp events and compute expiries.\n\n## How to use it\n\n`Time` is an ambient global (no import needed), and is also exported from `toiljs/server/runtime`:\n\n```ts\nconst ms = Time.nowMillis(); // milliseconds since the Unix epoch (a u64)\nconst s = Time.nowSeconds(); // whole seconds since the Unix epoch (a u64)\n```\n\n| Member | Returns | Description |\n| --- | --- | --- |\n| `Time.nowMillis()` | `u64` | Milliseconds since the **Unix epoch** (midnight UTC on 1 January 1970). This is the same value as JavaScript's `Date.now()`. |\n| `Time.nowSeconds()` | `u64` | Whole seconds since the epoch (`nowMillis() / 1000`). This is the unit sessions and login challenges use. |\n\nThe **Unix epoch** is just the agreed \"time zero\" that computers count from. \"Milliseconds since the epoch\" is a single number that unambiguously names an instant, with no time zone to worry about.\n\n## Why time is a \"host import\" (and what that means)\n\nYour backend runs as a **WebAssembly** module: a small, sandboxed program. A sandbox is deliberately sealed off from the outside world. It has no clock of its own, no network, no files, nothing except the memory it was given and a short list of functions the host chooses to hand it. Each of those handed-in functions is called a **host import**.\n\nThis is on purpose. WebAssembly aims to be **deterministic**: given the same inputs, the same code produces the same outputs, every time. That makes it safe to run untrusted code, easy to cache, and easy to reason about. But the current time is not deterministic: it is different every time you ask. So a WebAssembly module cannot just \"know\" the time; it has to **ask the host** for it. Reading the clock is a call across the sandbox boundary.\n\n`Time` is the thin, friendly wrapper over that call. Under the hood it uses the host's `Date.now()` binding (`env.Date.now`).\n\n```mermaid\nflowchart LR\n subgraph Sandbox[\"Your backend (WebAssembly sandbox)\"]\n C[\"Time.nowMillis()\"]\n end\n subgraph Host[\"Dacely edge host\"]\n D[\"env.Date.now\"]\n K[\"real system clock\"]\n end\n C -->|host import| D\n D --> K\n K -->|milliseconds| C\n```\n\nBoth the edge and the dev server provide this binding, so time behaves **identically** in `toiljs dev` and in production.\n\n## `Time` vs calling `Date.now()`\n\nToilScript's `Date.now()` lowers to the exact same host import, so you *can* call it directly. Prefer `Time`:\n\n- It makes the host boundary (and the single millisecond unit) explicit and easy to find in a codebase.\n- It gives you `nowSeconds()` without writing an open-coded `/ 1000` at every call site.\n\nBoth are the same clock, so timing you do in a handler lines up with timing the framework does. For example, the auth system uses `Time.nowSeconds()` for session issue and expiry times.\n\n## Correct usage: timestamps and TTLs\n\n`Time` is **wall-clock time**, exactly like a browser's `Date.now()`. It follows the system clock, which means it can occasionally **step backward** (for instance when the machine corrects its clock against a time server).\n\nDo:\n\n- **Stamp an instant.** Record when something happened, so you can compare it later: an event time, a \"last seen\" marker.\n- **Compute an expiry (a TTL).** A **TTL** (\"time to live\") is a deadline. You set it by adding a duration to *now*, and you check it by comparing *now* against the stored deadline:\n\n```ts\nconst ttlSeconds: u64 = 3600; // valid for one hour\nconst expiresAt = Time.nowSeconds() + ttlSeconds;\n\n// ...later, on another request...\nconst expired = Time.nowSeconds() >= expiresAt;\n```\n\nDo not:\n\n- **Measure elapsed time with it.** Do not use it as a stopwatch to time how long an operation took. Because the wall clock can step backward, `end - start` could come out zero or negative. `Time` answers \"what instant is it now?\", not \"how much time has passed?\".\n\n## Gotchas\n\n- **It is wall-clock, not monotonic.** It can jump backward across a clock correction. Fine for timestamps and expiries; wrong for measuring durations.\n- **Milliseconds vs seconds.** `nowMillis()` and `nowSeconds()` differ by a factor of 1000. Mixing them (comparing a millisecond stamp to a second stamp) is a common bug. Sessions and login challenges use **seconds**.\n- **UTC only.** These are epoch counts, with no time zone. Format for display on the client if you need a local time.\n\n## Related\n\n- [Auth, sessions, and `@user`](../auth/usage.md): sessions use `Time.nowSeconds()` for their issue and expiry stamps.\n- [Caching](./caching.md): TTLs are the same idea, a deadline after which a saved value is stale.\n- [Compute tiers](../concepts/tiers.md): more on the WebAssembly sandbox your backend runs in.\n",
71
+ "services/time.md": "# Time\n\n**`Time` is the edge's clock: the blessed way to read the current time in your backend.** It gives you the number of milliseconds or seconds since a fixed reference point, so you can stamp events and compute expiries.\n\n## How to use it\n\n`Time` is an ambient global (no import needed), and is also exported from `toiljs/server/runtime`:\n\n```ts\nconst ms = Time.nowMillis(); // milliseconds since the Unix epoch (a u64)\nconst s = Time.nowSeconds(); // whole seconds since the Unix epoch (a u64)\n```\n\n| Member | Returns | Description |\n| --- | --- | --- |\n| `Time.nowMillis()` | `u64` | Milliseconds since the **Unix epoch** (midnight UTC on 1 January 1970). This is the same value as JavaScript's `Date.now()`. |\n| `Time.nowSeconds()` | `u64` | Whole seconds since the epoch (`nowMillis() / 1000`). This is the unit sessions and login challenges use. |\n\nThe **Unix epoch** is just the agreed \"time zero\" that computers count from. \"Milliseconds since the epoch\" is a single number that unambiguously names an instant, with no time zone to worry about.\n\n## Why time is a \"host import\" (and what that means)\n\nYour backend runs as a **WebAssembly** module: a small, sandboxed program. A sandbox is deliberately sealed off from the outside world. It has no clock of its own, no network, no files, nothing except the memory it was given and a short list of functions the host chooses to hand it. Each of those handed-in functions is called a **host import**.\n\nThis is on purpose. WebAssembly aims to be **deterministic**: given the same inputs, the same code produces the same outputs, every time. That makes it safe to run untrusted code, easy to cache, and easy to reason about. But the current time is not deterministic: it is different every time you ask. So a WebAssembly module cannot just \"know\" the time; it has to **ask the host** for it. Reading the clock is a call across the sandbox boundary.\n\n`Time` is the thin, friendly wrapper over that call. Under the hood it uses the host's `Date.now()` binding (`env.Date.now`).\n\n```mermaid\nflowchart LR\n subgraph Sandbox[\"Your backend (WebAssembly sandbox)\"]\n C[\"Time.nowMillis()\"]\n end\n subgraph Host[\"Dacely edge host\"]\n D[\"env.Date.now\"]\n K[\"real system clock\"]\n end\n C -->|host import| D\n D --> K\n K -->|milliseconds| C\n```\n\nBoth the edge and the dev server provide this binding, so time behaves **identically** in `toiljs dev` and in production.\n\n## `Time` vs calling `Date.now()`\n\nToilScript's `Date.now()` lowers to the exact same host import, so you *can* call it directly. Prefer `Time`:\n\n- It makes the host boundary (and the single millisecond unit) explicit and easy to find in a codebase.\n- It gives you `nowSeconds()` without writing an open-coded `/ 1000` at every call site.\n\nBoth return the same wall-clock timestamps. For example, the auth system uses `Time.nowSeconds()` for session issue and expiry times.\n\n## Correct usage: timestamps and TTLs\n\n`Time` is **wall-clock time**, exactly like a browser's `Date.now()`. It follows the system clock, which means it can occasionally **step backward** (for instance when the machine corrects its clock against a time server).\n\nDo:\n\n- **Stamp an instant.** Record when something happened, so you can compare it later: an event time, a \"last seen\" marker.\n- **Compute an expiry (a TTL).** A **TTL** (\"time to live\") is a deadline. You set it by adding a duration to *now*, and you check it by comparing *now* against the stored deadline:\n\n```ts\nconst ttlSeconds: u64 = 3600; // valid for one hour\nconst expiresAt = Time.nowSeconds() + ttlSeconds;\n\n// ...later, on another request...\nconst expired = Time.nowSeconds() >= expiresAt;\n```\n\nDo not:\n\n- **Measure elapsed time with it.** Do not use it as a stopwatch to time how long an operation took. Because the wall clock can step backward, `end - start` could come out zero or negative. `Time` answers \"what instant is it now?\", not \"how much time has passed?\".\n\n## Upload expiry versus elapsed deadlines\n\nAn upload that spans several requests can have a stored **absolute expiry**:\n\n```ts\n// When the upload is created, allow ten minutes to finish it.\nconst expiresAtMillis: u64 = Time.nowMillis() + 600000;\n// Persist expiresAtMillis with the upload metadata.\n\n// On a later request, compare instants directly.\nconst expired = Time.nowMillis() >= expiresAtMillis;\n```\n\nThis is a wall-clock TTL, so a clock correction can make it expire sooner or later. It is suitable for expiring abandoned uploads when that behavior is acceptable. Compare the timestamps directly: subtracting two `u64` timestamps after the clock moves backward can underflow into a very large duration.\n\nAn **elapsed execution deadline** answers a different question: how long has this request or operation been running? `Time` cannot reliably enforce it. `Time.nowMillis() + budget` and a loop that checks the wall clock can both be defeated by a backward clock correction, and cannot interrupt work between checks. The current guest host surface does not provide a monotonic clock; the standard-library `process.hrtime()` requires `env.performance.now`, which the toiljs/Dacely hosts do not currently supply.\n\nMeasure client upload duration with the browser's monotonic `performance.now()` and apply a client timeout if needed. That timeout controls the client's wait; it does not roll back server writes. Bound server work by bytes, chunk count, and database operations, give chunk requests stable identifiers, and check upload completion after a timed-out request. See [Documents file uploads](../database/documents.md#size-limits-and-file-uploads) and [route execution and transport limits](../concepts/config.md#route-execution-and-transport-limits).\n\n## Gotchas\n\n- **It is wall-clock, not monotonic.** It can jump backward across a clock correction. Fine for timestamps and expiries; wrong for measuring durations.\n- **Milliseconds vs seconds.** `nowMillis()` and `nowSeconds()` differ by a factor of 1000. Mixing them (comparing a millisecond stamp to a second stamp) is a common bug. Sessions and login challenges use **seconds**.\n- **UTC only.** These are epoch counts, with no time zone. Format for display on the client if you need a local time.\n\n## Related\n\n- [Auth, sessions, and `@user`](../auth/usage.md): sessions use `Time.nowSeconds()` for their issue and expiry stamps.\n- [Caching](./caching.md): TTLs are the same idea, a deadline after which a saved value is stale.\n- [Compute tiers](../concepts/tiers.md): more on the WebAssembly sandbox your backend runs in.\n",
72
72
  };