toiljs 0.0.99 → 0.0.101
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +10 -0
- package/build/client/.tsbuildinfo +1 -1
- package/build/client/auth.d.ts +35 -1
- package/build/client/auth.js +75 -0
- package/build/client/index.d.ts +2 -2
- package/build/client/index.js +1 -1
- package/build/compiler/.tsbuildinfo +1 -1
- package/build/compiler/docs.js +26 -0
- package/build/compiler/toil-docs.generated.js +5 -4
- package/docs/README.md +2 -1
- package/docs/concepts/ai-guide.md +132 -0
- package/docs/frontend/data-fetching.md +17 -0
- package/docs/frontend/rendering.md +5 -5
- package/docs/llms.txt +1 -0
- package/docs/services/cookies.md +2 -2
- package/package.json +1 -1
- package/server/auth/AuthController.ts +29 -1
- package/src/client/auth.ts +125 -0
- package/src/client/index.ts +9 -1
- package/src/compiler/docs.ts +26 -0
- package/src/compiler/toil-docs.generated.ts +5 -4
- package/test/pqauth-e2e.test.ts +5 -5
- package/test/pqauth-email.test.ts +83 -44
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
|
|
6
6
|
/** The framework guides written into `.toil/docs/`, keyed by filename, generated from `docs/`. */
|
|
7
7
|
export const TOIL_DOCS: Record<string, string> = {
|
|
8
|
-
"README.md": "# toiljs\n\ntoiljs is a full-stack web framework. You write your **frontend in React** and your **backend in\nTypeScript**, and toiljs turns the backend into a tiny, fast **WebAssembly** program that runs at the edge\n(on servers close to your users, all over the world). One language, one project, one deploy.\n\nIf you have used Next.js, this will feel familiar: file-based routes, server code next to client code, a dev\nserver with hot reload. The difference is what happens underneath. Your server code is compiled by\n**toilscript** (a TypeScript-to-WebAssembly compiler) into a sandboxed `.wasm` module, and it runs on the\n**Dacely edge** with a built-in worldwide database (**ToilDB**), streaming, background jobs, and auth all\nincluded.\n\n## The mental model\n\n```mermaid\nflowchart LR\n A[\"Your project<br/>(TypeScript + React)\"] -->|toiljs build| B[\"client bundle<br/>(React, runs in the browser)\"]\n A -->|toilscript compile| C[\"server.wasm<br/>(your backend, sandboxed)\"]\n B --> U[\"User's browser\"]\n C --> E[\"Dacely edge<br/>(worldwide)\"]\n E --> D[(\"ToilDB<br/>global database\")]\n U <-->|HTTP / WebTransport| E\n```\n\n- **client/** is your React app (pages, components, styles). It runs in the browser.\n- **server/** is your backend (routes, database, auth). It compiles to WebAssembly and runs on the edge.\n- **shared/** is the typed bridge: toiljs generates a client here so the browser calls your server with full\n type safety.\n\n## Hello, toiljs\n\n```ts\n// server/routes/Hello.ts (a backend HTTP route)\nimport { Response } from 'toiljs/server/runtime';\n\n@rest('hello')\nclass Hello {\n @get('/')\n public hi(): Response {\n return Response.text('Hello from the edge!\\n');\n }\n}\n```\n\n```tsx\n// client/routes/index.tsx (a frontend page)\nexport default function Home() {\n return <main><h1>Welcome</h1></main>;\n}\n```\n\nRun `toiljs dev`, open the browser, and both are live with hot reload. That is the whole loop.\n\n## Learn toiljs\n\n**Understand toil first**\n- [Understanding toil](./introduction/README.md): what toil is and the one big idea, then\n [why toil and who it is for](./introduction/why-toil.md),\n [the modern stack you get](./introduction/modern-stack.md),\n [how it works](./introduction/how-it-works.md),\n [what makes it hyper-scalable](./introduction/hyperscale.md),\n [how it is distributed](./introduction/distributed.md),\n [toil versus other frameworks](./introduction/vs-other-frameworks.md), and\n [why it is built this way](./introduction/design-principles.md).\n\n**Start here**\n- [Getting started](./getting-started/README.md): install, create a project, project structure, your first\n app, and migrating an existing React app.\n- [The CLI](./cli/README.md): `dev`, `build`, `create`, `doctor`, and every flag.\n- [Deploy](./getting-started/deploy.md): build for production, self-host it, and how the managed edge fits in.\n\n**Build the frontend**\n- [Frontend overview](./frontend/README.md), [Routing](./frontend/routing.md),\n [Navigation](./frontend/navigation.md), [Components](./frontend/components.md),\n [Rendering and SSR](./frontend/rendering.md), [Styling](./frontend/styling.md),\n [Images](./frontend/images.md), [Metadata and SEO](./frontend/metadata.md),\n [Fetching data](./frontend/data-fetching.md), [Scripts](./frontend/scripts.md),\n [Search](./frontend/search.md), [The Toil global (reference)](./frontend/toil-global.md).\n\n**Build the backend**\n- [Backend overview](./backend/README.md), [HTTP routes (`@rest`)](./backend/rest.md),\n [Typed RPC (`@service`/`@remote`)](./backend/rpc.md), [Data types (`@data`)](./backend/data.md).\n\n**The database (ToilDB)**\n- [Database overview and choosing a family](./database/README.md), [Setup (`@database`)](./database/setup.md),\n [Documents](./database/documents.md), [Unique](./database/unique.md), [Counters](./database/counters.md),\n [Events](./database/events.md), [Views and `@derive`](./database/views.md),\n [Membership](./database/membership.md), [Capacity](./database/capacity.md).\n\n**Auth**: [the full auth guide](./auth/README.md) covers post-quantum login, sessions, and `ToilUserId`.\n\n**Realtime and background**\n- [Streams](./realtime/README.md) and [channels](./realtime/channels.md), [Daemons and scheduled\n jobs](./background/daemons.md), [Derived views (`@derive`)](./background/derive.md).\n\n**Platform services**\n- [Caching](./services/caching.md), [Rate limiting](./services/ratelimit.md),\n [Environment and secrets](./services/environment.md), [Email and 2FA](./services/email.md),\n [Analytics](./services/analytics.md), [Crypto](./services/crypto.md), [Cookies](./services/cookies.md),\n [Time](./services/time.md).\n\n**Concepts and reference**\n- [Compute tiers (L1 to L4)](./concepts/tiers.md), [Types (u64, u256, and friends)](./concepts/types.md),\n [Every decorator](./concepts/decorators.md), [Configuration](./concepts/config.md),\n [Security and SRI](./concepts/security.md).\n",
|
|
8
|
+
"README.md": "# toiljs\n\ntoiljs is a full-stack web framework. You write your **frontend in React** and your **backend in\nTypeScript**, and toiljs turns the backend into a tiny, fast **WebAssembly** program that runs at the edge\n(on servers close to your users, all over the world). One language, one project, one deploy.\n\nIf you have used Next.js, this will feel familiar: file-based routes, server code next to client code, a dev\nserver with hot reload. The difference is what happens underneath. Your server code is compiled by\n**toilscript** (a TypeScript-to-WebAssembly compiler) into a sandboxed `.wasm` module, and it runs on the\n**Dacely edge** with a built-in worldwide database (**ToilDB**), streaming, background jobs, and auth all\nincluded.\n\n## The mental model\n\n```mermaid\nflowchart LR\n A[\"Your project<br/>(TypeScript + React)\"] -->|toiljs build| B[\"client bundle<br/>(React, runs in the browser)\"]\n A -->|toilscript compile| C[\"server.wasm<br/>(your backend, sandboxed)\"]\n B --> U[\"User's browser\"]\n C --> E[\"Dacely edge<br/>(worldwide)\"]\n E --> D[(\"ToilDB<br/>global database\")]\n U <-->|HTTP / WebTransport| E\n```\n\n- **client/** is your React app (pages, components, styles). It runs in the browser.\n- **server/** is your backend (routes, database, auth). It compiles to WebAssembly and runs on the edge.\n- **shared/** is the typed bridge: toiljs generates a client here so the browser calls your server with full\n type safety.\n\n## Hello, toiljs\n\n```ts\n// server/routes/Hello.ts (a backend HTTP route)\nimport { Response } from 'toiljs/server/runtime';\n\n@rest('hello')\nclass Hello {\n @get('/')\n public hi(): Response {\n return Response.text('Hello from the edge!\\n');\n }\n}\n```\n\n```tsx\n// client/routes/index.tsx (a frontend page)\nexport default function Home() {\n return <main><h1>Welcome</h1></main>;\n}\n```\n\nRun `toiljs dev`, open the browser, and both are live with hot reload. That is the whole loop.\n\n## Learn toiljs\n\n**Understand toil first**\n- [Understanding toil](./introduction/README.md): what toil is and the one big idea, then\n [why toil and who it is for](./introduction/why-toil.md),\n [the modern stack you get](./introduction/modern-stack.md),\n [how it works](./introduction/how-it-works.md),\n [what makes it hyper-scalable](./introduction/hyperscale.md),\n [how it is distributed](./introduction/distributed.md),\n [toil versus other frameworks](./introduction/vs-other-frameworks.md), and\n [why it is built this way](./introduction/design-principles.md).\n\n**Start here**\n- [Getting started](./getting-started/README.md): install, create a project, project structure, your first\n app, and migrating an existing React app.\n- [The CLI](./cli/README.md): `dev`, `build`, `create`, `doctor`, and every flag.\n- [Deploy](./getting-started/deploy.md): build for production, self-host it, and how the managed edge fits in.\n\n**Build the frontend**\n- [Frontend overview](./frontend/README.md), [Routing](./frontend/routing.md),\n [Navigation](./frontend/navigation.md), [Components](./frontend/components.md),\n [Rendering and SSR](./frontend/rendering.md), [Styling](./frontend/styling.md),\n [Images](./frontend/images.md), [Metadata and SEO](./frontend/metadata.md),\n [Fetching data](./frontend/data-fetching.md), [Scripts](./frontend/scripts.md),\n [Search](./frontend/search.md), [The Toil global (reference)](./frontend/toil-global.md).\n\n**Build the backend**\n- [Backend overview](./backend/README.md), [HTTP routes (`@rest`)](./backend/rest.md),\n [Typed RPC (`@service`/`@remote`)](./backend/rpc.md), [Data types (`@data`)](./backend/data.md).\n\n**The database (ToilDB)**\n- [Database overview and choosing a family](./database/README.md), [Setup (`@database`)](./database/setup.md),\n [Documents](./database/documents.md), [Unique](./database/unique.md), [Counters](./database/counters.md),\n [Events](./database/events.md), [Views and `@derive`](./database/views.md),\n [Membership](./database/membership.md), [Capacity](./database/capacity.md).\n\n**Auth**: [the full auth guide](./auth/README.md) covers post-quantum login, sessions, and `ToilUserId`.\n\n**Realtime and background**\n- [Streams](./realtime/README.md) and [channels](./realtime/channels.md), [Daemons and scheduled\n jobs](./background/daemons.md), [Derived views (`@derive`)](./background/derive.md).\n\n**Platform services**\n- [Caching](./services/caching.md), [Rate limiting](./services/ratelimit.md),\n [Environment and secrets](./services/environment.md), [Email and 2FA](./services/email.md),\n [Analytics](./services/analytics.md), [Crypto](./services/crypto.md), [Cookies](./services/cookies.md),\n [Time](./services/time.md).\n\n**Concepts and reference**\n- [Writing toiljs correctly (AI + human quick rules)](./concepts/ai-guide.md),\n [Compute tiers (L1 to L4)](./concepts/tiers.md), [Types (u64, u256, and friends)](./concepts/types.md),\n [Every decorator](./concepts/decorators.md), [Configuration](./concepts/config.md),\n [Security and SRI](./concepts/security.md).\n",
|
|
9
9
|
"auth/configuration.md": "# Configuring auth for production\n\nBuilt-in auth runs locally with **zero configuration**, it falls back to published, insecure DEV secrets\nso `toiljs dev` Just Works. **A deployment MUST replace all three secrets and pin its KEM key.** This page\nis the checklist.\n\n## The secrets\n\nAuth reads these from the tenant environment store (locally, `.env.secrets`; on the edge, the per-host\nsecure env). They resolve **lazily** the first time auth runs, so no startup wiring is needed.\n\n| Key | What it is | Dev fallback |\n| --- | --- | --- |\n| `AUTH_SESSION_SECRET` | HMAC-SHA256 key that signs the session cookie. Must be identical on every edge instance (a cookie minted anywhere must verify everywhere). | a public constant, **anyone can forge a session** |\n| `AUTH_OPRF_SEED` | Master seed for the per-user OPRF salt key. Rotating it invalidates every password (users must re-register). | a hashed public constant |\n| `AUTH_KEM_SK` | The server's ML-KEM-768 **secret** key (hex). Its public half is what the client encapsulates to. | a pinned dev key pair |\n\n```bash\n# .env.secrets (gitignored; mode 0600 on the edge, NEVER under hosts/, NEVER in the .wasm)\nAUTH_SESSION_SECRET=…64 hex chars (32 bytes)…\nAUTH_OPRF_SEED=…64 hex chars (32 bytes)…\nAUTH_KEM_SK=…hex of an ML-KEM-768 secret key…\n```\n\n### Generating them\n\n`AUTH_SESSION_SECRET` and `AUTH_OPRF_SEED` are just 32 random bytes each:\n\n```bash\nnode -e \"console.log(require('crypto').randomBytes(32).toString('hex'))\"\n```\n\n`AUTH_KEM_SK` is an ML-KEM-768 key pair. Generate it and keep BOTH halves, the secret goes in the env, the\npublic half is pinned in the client (next section):\n\n```ts\nimport { ml_kem768 } from '@dacely/noble-post-quantum/ml-kem';\nconst { secretKey, publicKey } = ml_kem768.keygen();\nconsole.log('AUTH_KEM_SK =', Buffer.from(secretKey).toString('hex'));\nconsole.log('client serverKemPublicKey =', Buffer.from(publicKey).toString('hex'));\n```\n\n## Pin the client's KEM public key\n\nThe browser must know the server's genuine KEM public key to run the mutual-auth handshake, this is the\nanti-phishing anchor. The `toiljs/client` `Auth` helper ships with the **dev** key pinned, so a deployment\nMUST pass its own:\n\n```ts\nimport { Auth } from 'toiljs/client';\n\nconst SERVER_KEM_PUBLIC_KEY = /* the publicKey bytes from AUTH_KEM_SK */;\n\nawait Auth.login(username, password, { serverKemPublicKey: SERVER_KEM_PUBLIC_KEY });\nawait Auth.register(username, password, { serverKemPublicKey: SERVER_KEM_PUBLIC_KEY });\n```\n\nShip the public key with your client bundle (it's public, safe to embed). If it doesn't match the server's\n`AUTH_KEM_SK`, login's `serverConfirm` check fails and the client aborts.\n\n## Optional: audience & domain\n\nBoth are optional and have sensible defaults; set them for stability across host aliases:\n\n| Key | Meaning | Default |\n| --- | --- | --- |\n| `TOIL_AUTH_AUDIENCE` | The service audience bound into the signed login message. | `\"toil\"` |\n| `TOIL_AUTH_DOMAIN` | The `domain` input of the stable `ToilUserId` (`sha256(pubkey ‖ username ‖ domain)`). | the request `Host` header, else `localhost` |\n\nSet `TOIL_AUTH_DOMAIN` explicitly if your site answers on multiple hostnames, otherwise the same user could\nget different `ToilUserId`s from different aliases. Once users exist, changing it changes everyone's id, so\npick it before launch.\n\n## Argon2id strength (known limitation)\n\nThe built-in controller currently uses **demo-light** Argon2id params (32 MiB, 2 iterations, 1 lane) so it\nstays responsive in a browser tab. These are baked into the shipped controller today; **config-driven\ntuning is a planned follow-up.** For a high-value production deployment you should either wait for the\nconfig knob or hand-write your own controller with `≥ 256 MiB / ≥ 3 iterations`. The OPRF still provides the\nprimary offline-attack resistance regardless, but raise these before protecting anything sensitive.\n\nThe client always derives against whatever params the server returns in `/login/start`, so when the config\nknob lands you can raise them server-side with **no client change**.\n\n## Deploy checklist\n\n- [ ] `AUTH_SESSION_SECRET` set (32 random bytes), identical on every edge instance.\n- [ ] `AUTH_OPRF_SEED` set (32 random bytes).\n- [ ] `AUTH_KEM_SK` set (an ML-KEM-768 secret key), and its **public** half pinned in the client via\n `serverKemPublicKey`.\n- [ ] `TOIL_AUTH_DOMAIN` set if you serve multiple hostnames (stable `ToilUserId`).\n- [ ] (Recommended) Argon2id params reviewed for your threat model.\n\nThe CLI doctor warns when `server.auth` is on and the secrets are missing, run it before you ship.\n",
|
|
10
10
|
"auth/extending.md": "# Extending & integrating auth\n\nBuilt-in auth is deliberately opinionated so the common case is one line. This page covers the identity\nyou build ON, `ToilUserId`, and how to go beyond the defaults: keying your own data on a user, a custom\nuser shape, and hand-writing auth from the same primitives.\n\n## `ToilUserId`\n\nThe stable, tenant-scoped user identity: `sha256(mldsaPublicKey ‖ identifier ‖ domain)`, a 256-bit value.\nIt's a **global** (no import), like `crypto`.\n\n```ts\n// Read the current user's id in any handler (gate on hasSession(), see the null note below).\nconst id: ToilUserId = AuthService.userId()!;\n\n// Or derive one yourself.\nconst id2 = ToilUserId.derive(mldsaPublicKey, 'alice@example.com', 'acme.dacely.com');\n```\n\n| Member | Description |\n| --- | --- |\n| `ToilUserId.derive(pk, identifier, domain)` | Derive from an ML-DSA public key + email/username + tenant domain. Deterministic. |\n| `ToilUserId.fromBytes(b)` | Rebuild from a 32-byte digest (from `toBytes()` or storage). |\n| `toBytes(): Uint8Array` | The 32 identity bytes. |\n| `toHex(): string` | Lowercase 64-char hex, a convenient string key. |\n| `isZero(): bool` | True for the unset / anonymous id. |\n| `equals(other): bool` | Value equality. |\n| `a == b` / `a != b` | Overloaded value comparison, **O(1)** (four `u64` word compares, no byte loop, no allocation). |\n\n```ts\nconst a = ToilUserId.derive(pk, 'alice', 'acme.com');\nconst b = ToilUserId.derive(pk, 'alice', 'acme.com');\nconst c = ToilUserId.derive(pk, 'bob', 'acme.com');\na == b; // true, same inputs, same id\na != c; // true, different user\n```\n\n> **Null-check gotcha:** because `ToilUserId` overloads `==`, `AuthService.userId() == null` does NOT\n> type-check (`==` expects a `ToilUserId`). Gate with `AuthService.hasSession()` and then `userId()!`, or\n> compare with `getUser()` (a plain nullable). `===` is reference identity in AssemblyScript and is not\n> overloadable, use `==` for value equality.\n\n## Keying your own data on the user\n\n`toilUserId` is the right key for per-user data, it's stable across sessions/devices and opaque. Use the\nhex as a string key, or the bytes in a `@data` key class:\n\n```ts\n@data\nclass UserKey {\n id: Uint8Array = new Uint8Array(0); // toilUserId bytes\n constructor(id: Uint8Array = new Uint8Array(0)) { this.id = id; }\n}\n\n@data class Profile { displayName: string = ''; bio: string = ''; }\n\n@database\nclass AppDb {\n @collection static profiles: Documents<UserKey, Profile>;\n}\n\n@rest('profile')\nclass ProfileApi {\n @auth\n @post('/')\n public save(ctx: RouteContext): Response {\n const key = new UserKey(AuthService.userId()!.toBytes());\n const p = Profile.decode(ctx.request.body);\n // Save this user's profile. create is insert-only, so the first save creates\n // the record and later saves overwrite the existing one with enqueue.\n if (!AppDb.profiles.create(key, p)) {\n AppDb.profiles.enqueue(key, p);\n }\n return Response.text('saved\\n');\n }\n}\n```\n\n## Extending the user: add your own fields\n\nBuilt-in auth reserves exactly two fields on the authenticated user: `toilUserId` and `username`. To carry\nmore (a role, a display name, a tenant), just declare your OWN `@user` in your server code while\n`server.auth` is on. The build detects it and **extends** it: it injects the reserved `toilUserId` +\n`username` as the first two fields and mints sessions for your shape automatically. You do not opt out of\nanything, and there is still exactly one `@user` per program.\n\n```ts\n@user\nclass Account {\n admin: bool = false;\n displayName: string = '';\n tenant: string = '';\n // toilUserId + username are INJECTED by the build; do not declare them here.\n}\n```\n\nRules:\n\n- Do **not** declare `toilUserId` or `username` yourself. They are reserved and injected; declaring either is\n a compile error (`'username' is reserved by built-in auth`).\n- Your `@user` must be default-constructible (give every field an initializer), like any `@data` class.\n\n`AuthService.getUser()` is now typed to YOUR shape:\n\n```ts\nconst user = AuthService.getUser(); // { toilUserId, username, admin, displayName, tenant } | null\n```\n\n**Populating your fields.** Login fills `toilUserId` + `username`; your extra fields start at their declared\ndefaults (`admin = false`, and so on). The built-in controller cannot know your business fields, so you set\nthem yourself on one of your own `@auth` routes by reading the user, updating it, and re-minting the session:\n\n```ts\n@rest('account')\nclass AccountApi {\n @auth @post('/promote')\n public promote(): Response {\n const user = AuthService.getUser()!;\n user.admin = true;\n const resp = Response.text('promoted\\n');\n resp.setCookie(AuthService.mintSession(user.encode())); // re-sign the session with the new fields\n resp.setCookie(AuthService.userCookie(user.encode())); // update the readable companion cookie\n return resp;\n }\n}\n```\n\nIf you would rather hand-write the whole controller instead, you still can: do **not** enable `server.auth`,\nand build your own `@user` + routes from the [AuthService primitives](#the-authservice-primitive-reference)\nbelow. But for most apps, extending is all you need.\n\n## Adding email verification / 2FA\n\nLayer a second factor on top of the session with `TwoFactor` (stateless email codes, no DB; see\n[email](../services/email.md)). Typical flow: after login, require a verified email before granting access to\nsensitive routes.\n\n```ts\n@rest('2fa')\nclass TwoFactorApi {\n // Step 1: email a code to the logged-in user, hand back the signed token.\n @auth @post('/send')\n public send(): Response {\n const email = /* the user's email, e.g. their username, or a stored profile field */;\n const ch = TwoFactor.send(email, 'login'); // emails the code, returns { token, status }\n return Response.bytes(new DataWriter().writeString(ch.token).toBytes());\n }\n\n // Step 2: verify the code the user typed against the token.\n @auth @post('/verify')\n public verify(ctx: RouteContext): Response {\n const r = new DataReader(ctx.request.body);\n const token = r.readString(); const email = r.readString(); const code = r.readString();\n if (!TwoFactor.verify(token, email, code)) return Response.text('bad code\\n', 401);\n // Mark this session 2FA-verified: re-mint the session with a flag in your own @user, or store a\n // per-user \"verified\" record keyed on AuthService.userId().\n return Response.text('verified\\n');\n }\n}\n```\n\n`TwoFactor` gives integrity + expiry but not single-use (a code re-verifies within its TTL); keep the TTL\nshort. For a branded email, use `TwoFactor.issue(...)` (returns the code without sending) + your own\n`Emails.*` template. Call `TwoFactor.setSecret(...)` once at startup in production.\n\n## The `AuthService` primitive reference\n\nEverything the built-in controller is built from, available for hand-written auth. All are ambient globals\n(no import).\n\n**Sessions & cookies**\n- `mintSession(userData: Uint8Array, ttlSecs?: u64): Cookie`: the signed `__Host-toil_sess` cookie.\n- `userCookie(userData, ttlSecs?): Cookie`: the readable `__Secure-toil_user` companion.\n- `clearSession(): Cookie` / `clearUserCookie(): Cookie`.\n- `hasSession(): bool`: the `@auth` predicate.\n- `getSessionBytes(): Uint8Array | null`: the verified `@user` payload bytes.\n- `getUser(): <your @user> | null`: decoded, typed to your `@user`.\n- `userId(): ToilUserId | null`: the stable id (built-in `@user` layout).\n- `setSecret(secret: Uint8Array)`: override the session HMAC key programmatically.\n\n**Post-quantum login crypto**\n- `oprfEvaluate(username, blinded): Uint8Array`: server-keyed OPRF eval.\n- `buildRegisterMessage(username, pk)` / `verifyRegister(pk, msg, sig): bool`: proof-of-possession.\n- `buildLoginMessage(sub, aud, cid, nonce, iat, exp, ct, memKiB, iterations, parallelism, serverKemKeyId)`\n / `verifyLogin(pk, msg, sig): bool`.\n- `mlkemDecapsulate(ct): Uint8Array`, `serverKemKeyId(): Uint8Array`.\n- `deriveSessionKey(sharedSecret, transcriptHash)`, `serverConfirmTag(sessionKey, transcriptHash)`: the\n mutual-auth confirmation.\n- `sha256(data): Uint8Array`.\n- `setOprfSeed(seed)`, `setServerKemSecretKey(sk)`, `setServerKemPublicKey(pk)`: override the seeds/keys.\n- Sizes: `PUBLIC_KEY_LEN`, `SIGNATURE_LEN`, `OPRF_ELEMENT_LEN`, `KEM_CIPHERTEXT_LEN`, `SHARED_SECRET_LEN`, …\n\n**Related globals**, `TwoFactor` (email codes; see [email](../services/email.md)), `RateLimitService` (used by\n`@ratelimit`), `Environment` (the secret store).\n\n## Two ways in, one behavior\n\nThe config flag and the import are identical at build time, both make the build append the shipped\n`@user` + `@rest('auth')` controller to the toilscript **entry** set, where their decorators weave and the\n`@rest` class self-mounts. (A framework decorator source only weaves as an entry; that's why a plain\n`import` of the controller isn't enough on its own, the marker import is detected by the build, which does\nthe entry injection.) This is why built-in auth needs no runtime registration call.\n",
|
|
11
11
|
"auth/how-it-works.md": "# How Toil auth works\n\nToil auth is a **post-quantum, password-based, mutually-authenticated** login. The design goal: the user\ntypes a password, but the server never sees it, never stores it, and can't be phished into leaking a\nverifier, and every primitive is quantum-resistant.\n\nThis page explains the protocol. You do not need to understand it to use auth (see [Usage](./usage.md)),\nbut you should understand the guarantees before you ship.\n\n## The building blocks\n\n| Primitive | Role |\n| --- | --- |\n| **OPRF** (RFC 9497, ristretto255-SHA512) | A *server-keyed* salt. The client blinds its password, the server evaluates it under a per-user key, the client unblinds. The result can't be computed without the server, so a stolen database can't be brute-forced offline. |\n| **Argon2id** | Stretches the OPRF output into key material (memory-hard, GPU/ASIC-resistant). Runs in the browser. |\n| **ML-DSA-44** (FIPS 204) | The user's identity key pair is *derived* from the Argon2id output. The server stores only the **public** key. Login is proved by an ML-DSA signature. |\n| **ML-KEM-768** (FIPS 203) | Key encapsulation on login: the server proves it holds its KEM secret by returning a confirmation tag only derivable from the decapsulated shared secret → **mutual** auth (anti-phishing). |\n| **HMAC-SHA256** | Signs the session cookie (`AUTH_SESSION_SECRET`). |\n\n> The whole thing is sometimes called an *aPAKE* (asymmetric password-authenticated key exchange). Do not\n> call it OPAQUE, this is Toil's own OPRF + KEM + signature construction.\n\n## What the server stores\n\nPer account, in ToilDB (`@database AuthDb`):\n\n- `username`, a deterministic `salt`, the **ML-DSA public key**, and the Argon2id params.\n\nThat's it. **No password, no password hash, no verifier that can be brute-forced without the OPRF key.**\nLogin challenges are a second collection, consumed exactly once (atomic `getDelete`).\n\n## Registration\n\n```\n Browser (client) Toil edge (server)\n ───────────────── ──────────────────\n blind(password) OPRF key = f(seed, username)\n │ username, blinded │\n │ ───────── POST /auth/register/start ───────────► │\n │ │ evaluated = OPRF(blinded)\n │ ◄──── {mem,iters,par, salt, evaluated} ───────── │ (salt is deterministic per user)\n │ │\n unblind → oprfOut │\n seed = Argon2id(oprfOut, salt, params) │\n (pk, sk) = ML-DSA-44.keygen(seed) │\n proof = ML-DSA.sign(sk, \"register|username|pk\") │\n │ username, pk, proof │\n │ ───────── POST /auth/register/finish ──────────► │\n │ │ verifyRegister(pk, msg, proof) ✓ proof-of-possession\n │ ◄──────────── {status: 0 ok | 1 taken} ──────── │ store AuthAccount{username, salt, pk, params}\n```\n\nThe server never learns the password or the secret key `sk`, only the public key `pk` and a proof the\nclient holds the matching `sk`. A duplicate username returns a **distinguishable** `status = 1` (so the UI\ncan say \"taken, log in instead\"); everything else fails generically.\n\n## Login (with mutual auth)\n\n```\n Browser (client) Toil edge (server)\n ───────────────── ──────────────────\n blind(password) │\n │ username, blinded │\n │ ───────── POST /auth/login/start ──────────────► │ evaluated = OPRF(blinded) (ALWAYS, even unknown user)\n │ │ if known: store Challenge{cid, nonce, iat, exp}\n │ ◄─ {cid, aud, params, salt, nonce, iat, exp, ── │ (identical response whether the user exists or not)\n │ evaluated} │\n unblind → Argon2id → (pk, sk) = ML-DSA.keygen │\n (ct, ssС) = ML-KEM-768.encapsulate(serverKemPublicKey) │\n msg = \"login|username|aud|cid|nonce|iat|exp|ct|params|kid\" │\n sig = ML-DSA.sign(sk, msg) │\n │ cid, ct, sig │\n │ ───────── POST /auth/login/finish ─────────────► │ ch = challenges.getDelete(cid) (consume once; check exp)\n │ │ rebuild msg from OUR stored values + ct\n │ │ verifyLogin(acct.pk, msg, sig) ✓ it's really this user\n │ │ ssS = ML-KEM.decapsulate(ct) ✓ we hold the KEM key\n │ │ K = deriveSessionKey(ssS, H(msg))\n │ ◄──── {0, sessionToken, serverConfirm} ──────── │ serverConfirm = tag(K, H(msg))\n │ + Set-Cookie: __Host-toil_sess=… │ mint session cookie\n verify serverConfirm using ssC ✓ the server is genuine │\n```\n\nTwo verifications, both required:\n1. **The server verifies the client**: the ML-DSA signature over a message bound to the challenge, the\n Argon2id params, and the server's KEM key id. Replays fail (the challenge is consumed).\n2. **The client verifies the server**: the `serverConfirm` tag is derivable only from the ML-KEM shared\n secret, which only the holder of the KEM secret key can decapsulate. A phishing site can't forge it.\n\n## Anti-enumeration\n\n`/login/start` behaves identically for a known and an unknown user: it always runs the OPRF (a decoy key\nfor unknown users), returns a **deterministic** per-user salt and constant params, and a fresh challenge.\nThe challenge is persisted only for a real account, and `/login/finish` fails generically at consume for an\nunknown user. So an attacker can't probe which usernames exist. Every failure path returns the same\n`401 auth: request failed`.\n\n## Sessions & cookies\n\nOn successful login the server mints **two** cookies:\n\n- **`__Host-toil_sess`**: the authoritative session. HMAC-SHA256 signed with `AUTH_SESSION_SECRET`,\n `HttpOnly`, `Secure`, `SameSite=Lax`. It carries the `@user` codec payload. `@auth` and\n `AuthService.getUser()` open + verify this cookie server-side; a forged or tampered cookie fails.\n- **`__Secure-toil_user`**: a **readable** companion carrying the same payload, so the browser can show\n \"logged in as …\" via the client's `getUser()`. The server **never trusts it**, it is display-only.\n\nEach request runs in a fresh wasm instance, but the signed cookie is self-contained, so no server-side\nsession store is needed. `AUTH_SESSION_SECRET` must be identical across every edge instance (it is, via the\nenv store) so a cookie minted anywhere verifies everywhere.\n\n## The stable user identity: `ToilUserId`\n\nAt login the server derives a stable, tenant-scoped id and stores it in the session (the first field of the\nbuilt-in `@user`):\n\n```\ntoilUserId = SHA-256( mldsaPublicKey ‖ username ‖ domain ) // 256 bits\n```\n\n- **Stable:** same login key + username on the same tenant `domain` → same id, forever, across sessions\n and devices. Key your own data on it.\n- **Opaque + one-way:** it's a hash: safe to store, log, or expose without leaking the key or the address.\n- Read it anywhere with `AuthService.userId()`. See [Extending](./extending.md#toiluserid) for the\n `ToilUserId` API (O(1) `==` / `!=`, `toHex()`, …).\n\n## Threat-model summary\n\n- **Server database stolen** → attacker gets public keys + salts, not passwords. Brute-forcing needs the\n OPRF key (server-side) *and* Argon2id work per guess.\n- **Server compromised / malicious** → still can't recover passwords (only public keys) and can't forge a\n past session without `AUTH_SESSION_SECRET`.\n- **Phishing site** → can't produce the `serverConfirm` tag (no KEM secret), so a correct client aborts.\n- **Quantum adversary** → ML-DSA + ML-KEM are post-quantum; the OPRF/Argon2id/HMAC pieces are classical but\n not the long-term identity or key-exchange.\n- **Replay** → challenges are single-use (`getDelete`) with a short TTL.\n\nResidual responsibilities are yours: set the [secrets](./configuration.md), pin your deployment's KEM\npublic key in the client, and raise the Argon2id params for production.\n",
|
|
@@ -19,6 +19,7 @@ export const TOIL_DOCS: Record<string, string> = {
|
|
|
19
19
|
"background/derive.md": "# Derived views (`@derive`)\n\nA `@derive` keeps a fast, precomputed **View** of your data in sync automatically. You write a method that reads your source data and publishes a summary; toiljs re-runs it whenever the source data changes, so your pages can read the summary with one cheap lookup instead of doing the work on every request.\n\n## The problem it solves\n\nSome reads are expensive. \"Show the 10 newest comments\" means scanning an [events](../database/events.md) log. \"Show the leaderboard\" means totalling up scores. A **scan** (walking many rows to build a result) can fan out across an unbounded amount of data, and that is too slow and too unpredictable to do while a user waits.\n\nSo toiljs **bars scans on the request path**. A [route](../backend/rest.md) handler runs under a restricted mode:\n\n- A `@get` runs as a **query** (reads only, no scans).\n- A `@post` / `@put` / `@patch` / `@del` runs as an **action** (keyed reads and writes, still no scans).\n\nIf you cannot scan in a route, how do you show \"the latest 10\"? You precompute it. A `@derive` does the scan **off** the request path, folds the result into a [View](../database/views.md) (a read-optimized snapshot stored by key), and your route reads that View with a single keyed lookup, which is not a scan and so is allowed.\n\n```mermaid\nflowchart LR\n W[\"POST /guestbook<br/>(action: append + count)\"] --> S[(Events + Counter<br/>source data)]\n S -->|write triggers| D[\"@derive recompute()<br/>scans, builds, publishes\"]\n D --> V[(View<br/>precomputed snapshot)]\n G[\"GET /guestbook<br/>(query: one keyed read)\"] --> V\n```\n\n## A worked example: a guestbook\n\nHere is the whole pattern in one database class: an [events](../database/events.md) log of signatures, a [counter](../database/counters.md) of how many there are, and a [View](../database/views.md) that holds the ready-to-serve page.\n\n```ts\n@data\nclass GuestKey {\n room: string = 'main';\n constructor(room: string = 'main') { this.room = room; }\n}\n\n@database\nclass GuestbookDb {\n @collection static entries: Events<GuestKey, GuestEntry>; // a source: the log\n @collection static totals: Counter<GuestKey>; // a source: the count\n @collection static book: View<GuestKey, GuestbookView>; // the view we publish\n\n // Recompute the view from the sources. It MAY scan and publish; a route may not.\n @derive\n recompute(): void {\n const key = new GuestKey('main');\n const view = new GuestbookView();\n view.total = GuestbookDb.totals.get(key); // a keyed read\n view.entries = GuestbookDb.entries.latest(key, 10); // a scan: allowed here\n GuestbookDb.book.publish(key, view); // publish the snapshot\n }\n}\n```\n\nThe route then writes to the sources and reads the view:\n\n```ts\n@rest('guestbook')\nclass Guestbook {\n @get('/')\n list(): GuestbookView {\n const key = new GuestKey('main');\n const view = GuestbookDb.book.get(key); // one keyed read, not a scan\n return view == null ? new GuestbookView() : view; // empty until first publish\n }\n\n @post('/')\n sign(input: NewMessage): GuestbookView {\n const key = new GuestKey('main');\n GuestbookDb.entries.append(key, new GuestEntry(input.author, input.message, 0));\n GuestbookDb.totals.add(key, 1);\n // The @derive republishes `book` right after this action returns, so GET\n // serves the new entry. The action just acks with the new total (a counter\n // read is allowed here; scanning the entries list is not).\n const view = new GuestbookView();\n view.total = GuestbookDb.totals.get(key);\n return view;\n }\n}\n```\n\nSign the guestbook twice and the total climbs across requests, because the data lives in the database (and its view), not in module memory.\n\n## What a `@derive` may do\n\nA `@derive` runs under a special **derive** mode that is more powerful than a route:\n\n| Ability | Query (`@get`) | Action (`@post` ...) | Derive |\n| -------------------------------------------- | :------------: | :------------------: | :----------: |\n| Keyed reads (`.get`, counter total) | yes | yes | yes |\n| Writes (`create`, `patch`, `add`, `append`) | no | yes | yes |\n| **Scans** (`events.latest`, membership list) | **no** | **no** | **yes** |\n| `view.publish` / `view.append` | no | no | yes |\n\nSo a derive is exactly the place to do the reads and scans a route cannot, and to publish the result.\n\n## Declaring a derive\n\nA `@derive` is a method on your [`@database`](../database/setup.md) class, next to the collections it reads and the View it writes.\n\n```ts\n@database\nclass MyDb {\n @collection static events: Events<Key, Fact>; // a source\n @collection static home: View<Key, HomePage>; // the materialized view\n\n @derive\n rebuild(): void {\n // read sources, build the value, publish it\n }\n}\n```\n\nRules:\n\n- A `@derive` method takes **no arguments and returns `void`**.\n- A database may declare **multiple** `@derive` methods; each runs independently.\n- The View value and its key are ordinary [`@data`](../backend/data.md) types, so they round-trip through the codec like any other stored value.\n\n## When a derive runs\n\nYou never call a derive yourself. The runtime runs it for you at two moments:\n\n1. **After a write to a source.** When a request writes one of the database's source collections (an `events.append`, a `counter.add`, a document `create` or `patch`), that database's derives run **right after the response is produced**, so the view reflects the new data on the next read. Many writes to one database in a single request are **coalesced** into one recompute (it does not run once per write).\n\n2. **On box load.** When a server box starts, hot-reloads, or notices the underlying data changed out of band, the views are rebuilt from their sources **before the first read is served**. This is also where a value type's [`@migrate`](../backend/data.md) runs against old stored events, as the derive re-reads and republishes them.\n\nA derive's own `view.publish` never re-triggers it, so there is no infinite loop.\n\nThe same code runs under `toiljs dev` (the in-process emulator) and on the production edge, with no flags or wiring to change.\n\n## Folding a growing log incrementally\n\nThe guestbook above uses `events.latest` and **recomputes** the view from scratch on every change. That is simple, correct, and the right default for a **bounded** read: the latest N, a counter total, a small set.\n\nSome views instead fold an **unbounded** log: a running total over every event ever, an activity rollup, an audit summary. Rescanning the whole log on every change gets slower as it grows. For those, read the source with [`events.since`](../database/events.md) instead of `latest`. `since` hands you only the events you have **not folded yet**, so the derive folds forward incrementally.\n\n```ts\n@derive\nrollup(): void {\n const key = new StatsKey('all');\n const view = StatsDb.summary.get(key) ?? new Summary(); // the running view so far\n let batch = StatsDb.events.since(key, 500);\n while (batch.length > 0) { // drain the new events in bounded batches\n for (let i = 0; i < batch.length; i++) view.apply(batch[i]);\n batch = StatsDb.events.since(key, 500);\n }\n StatsDb.summary.publish(key, view);\n}\n```\n\nThe host owns the cursor, you never manage it: it seeds `since` from a durable checkpoint, advances it as it hands you events, and saves it **only after** your `publish` lands (so a crash re-folds the batch instead of skipping it). On the next trigger the derive resumes exactly where it left off.\n\n**One rule: the fold must be idempotent per event.** A rare crash-recovery case (an event that \"heals\" at an older position after the derive already passed it) makes the host re-read the whole log that one run, so applying an event twice must not change the result. Use set-style updates keyed by the event's id, not blind `count += 1` accumulation. If your fold cannot be idempotent, stay on the simple `latest` recompute path.\n\n## Guarantees and limitations\n\n**Guarantees**\n\n- **It converges to a correct snapshot.** Publishes are last-writer-wins (the host versions each publish so a later one always supersedes an earlier one), and a derive recomputes from the source of truth, so the view always ends up matching its sources.\n- **Reads stay cheap.** A route serves the view with a single keyed lookup, never a scan.\n\n**Limitations (read these)**\n\n- **By default it recomputes from scratch each time.** A derive re-reads its sources and republishes on every trigger. That is the simple path and a great fit for a **bounded** read: the latest N, a counter total, a small set. To fold an **unbounded, ever-growing** log efficiently, read it with [`events.since`](../database/events.md) instead of `latest` (see [Folding a growing log incrementally](#folding-a-growing-log-incrementally) above); it folds only the new events since a checkpoint.\n- **It is eventually consistent, by a moment.** The view is republished right after the writing request finishes, so there is a tiny window where a reader could see the pre-write view. For most pages (a feed, a leaderboard) that is invisible and fine.\n- **`view.publish` is derive-only.** Routes cannot publish; they can only read the view. That is the whole point: the expensive build happens off the request path.\n\n## When not to use a derive\n\n- **When the read is already cheap.** If a route can answer with a plain keyed `.get`, you do not need a view at all.\n- **When you need it on a timer, not on a data change.** Recomputing \"yesterday's report\" at 2am is a [daemon](./daemons.md) job, not a derive (a derive is triggered by writes, not by the clock).\n- **When the source is an unbounded full-history fold.** See the limitation above; keep derives to bounded reads.\n\n## Related\n\n- [Views](../database/views.md): the `View<K, V>` family a derive publishes into, and how to read it.\n- [Events](../database/events.md): the append-only log a derive commonly folds into a view.\n- [Counters](../database/counters.md): running totals a derive can read.\n- [Background overview](./README.md): `@derive` versus `@daemon`, and which to reach for.\n- [Data types (`@data`)](../backend/data.md): the value and key types a view stores.\n",
|
|
20
20
|
"background/README.md": "# Background work\n\nBackground work is code that runs **without a user waiting for it**: on a timer, once globally, or automatically after your data changes. toiljs gives you two tools for it, `@daemon` and `@derive`, and this page helps you pick the right one.\n\n## Why background work\n\nMost of your server code runs **because a user asked**: a browser hits a [route](../backend/rest.md) or calls an [RPC](../backend/rpc.md), your handler runs, and a response goes back. But some work does not belong on that path:\n\n- It should happen **on a schedule** (every hour, every night at 2am), not when a request happens to arrive.\n- It would be **too slow** to do inside a request, so you want it precomputed and ready.\n- It must happen **exactly once across the whole world**, not once per user and not once per server.\n\nThat is background work. In toiljs there are two kinds, and they solve two different problems.\n\n## The two tools\n\n```mermaid\nflowchart TD\n Q{What do you need?}\n Q -->|Run on a timer, or once globally| D[\"@daemon + @scheduled<br/>a single background worker\"]\n Q -->|Keep a fast read-view in sync with your data| V[\"@derive<br/>maintains a materialized View\"]\n D --> DD[\"e.g. nightly cleanup, hourly poll,<br/>periodic rollup\"]\n V --> VV[\"e.g. a leaderboard, a 'latest 10' feed,<br/>a home-page summary\"]\n```\n\n### `@daemon`: a scheduled, global worker\n\nA [daemon](./daemons.md) is one long-lived background worker for your whole app. It runs on a **schedule** you set (an interval like every 5 minutes, or a cron time like \"9:15 on weekdays\"), and there is exactly **one** of it worldwide at any moment (a \"singleton\"). Use it for work that is driven by **time** or that must run **once globally**:\n\n- clean up stale rows every night;\n- poll an external API every few minutes;\n- send a daily digest email;\n- roll up yesterday's numbers into a summary.\n\n### `@derive`: keep a read-view up to date\n\nA [derive](./derive.md) is not on a timer. It runs **automatically whenever the data it depends on changes**, and its job is to keep a precomputed **View** (a read-optimized copy of your data) fresh. Use it when a page needs data that is **expensive to compute on every read**, like a leaderboard or a \"latest 10 comments\" list, so the read itself stays a single cheap lookup:\n\n- fold an [events](../database/events.md) log into a \"latest N\" list;\n- total up [counters](../database/counters.md) into a scoreboard;\n- assemble a home-page summary from several sources.\n\n## Which one do I reach for?\n\n| Question | Use |\n| ----------------------------------------------------- | ------------ |\n| \"Run this every hour / at midnight.\" | `@daemon` |\n| \"Do this once for the whole app, not per server.\" | `@daemon` |\n| \"Poll or call an outside service on a schedule.\" | `@daemon` |\n| \"This page's data is too slow to compute per request.\"| `@derive` |\n| \"Keep a leaderboard / feed in sync as data changes.\" | `@derive` |\n\nA simple rule of thumb: if the trigger is **the clock**, use a `@daemon`. If the trigger is **a change to your data** and the goal is a fast read, use a `@derive`.\n\nThey also combine well. A `@daemon` might do a heavy nightly aggregation and write a summary row, while a `@derive` keeps a small live view fresh on every write. They are different tiers of the edge and are covered on their own pages.\n\n## `@job`: the widest database surface for background work\n\n`@daemon` and `@derive` decide **where and when** background code runs. `@job` answers a different question: **what a function is allowed to do to the database.** It is one of the ToilDB **function kinds** (the same family as `@query`, `@action`, and `@derive`), and it grants the **widest** data surface of them all.\n\nA **function kind** is a label the compiler puts on a backend function to gate which database operations it may issue. This is a safety rail: a read-only endpoint physically cannot write, and an expensive scan cannot run on the hot request path. The kinds line up from narrowest to widest:\n\n| Kind | Typical trigger | Point reads | **Scan** (`latest`, membership `list`) | Writes | `publish` a View |\n| ---------- | ------------------------- | ----------- | -------------------------------------- | ----------------- | ---------------- |\n| `@query` | a `@get` / plain `@remote`| yes | no | no | no |\n| `@action` | a `@post` / `@action` | yes | no | yes (bounded) | no |\n| `@derive` | your data changed | yes | yes | append / counter add only | yes |\n| `@job` | you drive it (background) | yes | yes | yes (all) | yes |\n\nA **scan** is a read that can fan out across many rows (like \"the newest 50 events\" via `events.latest`, or \"every member of this set\" via `membership.list`). Scans are **barred from request handlers** because a request must stay fast and bounded. `@derive` and `@job` run **off the request path**, so they are the only kinds allowed to scan.\n\n### When to reach for `@job`\n\nUse `@job` when a piece of background work needs **more** database power than the default:\n\n- it must **scan** (fold `events.latest`, walk `membership.list`), and/or\n- it must **publish a View** *while also* doing arbitrary writes (`create`, `patch`, `delete`). A `@derive` can publish a View but cannot `patch` or `delete` a record; a `@job` can do everything.\n\nThe compiler accepts `@job` on a method, including a `@daemon`'s `@scheduled` method. Tag a scheduled task `@job` when it needs that full surface (say, a nightly repair that scans a log, fixes rows, and republishes a summary View). A plain, **untagged** `@scheduled` method runs with the **`@action`** surface instead: point reads plus bounded writes, which is all most rollups and cleanups need.\n\n```ts\n@daemon\nclass Jobs {\n // A plain scheduled method: point reads + bounded writes (the @action surface).\n @scheduled('1h')\n rollup(): void { /* get a counter, patch a summary row */ }\n\n // A scheduled method that also needs SCANS and to PUBLISH a View: tag it @job.\n @scheduled('1d')\n @job\n nightlyRepair(): void {\n // @job unlocks scan-class reads (events.latest / membership.list)\n // and publishing a View, on top of ordinary reads and writes.\n }\n}\n```\n\n### `@job` versus `@derive`\n\nThey overlap (both run off the request path, both may scan, both may publish a View), but they are triggered differently and sized differently:\n\n- **`@derive`** is **change-triggered**: it re-runs automatically whenever its source data changes, and its narrow job is to keep one **View** in sync. It cannot `create` / `patch` / `delete` records (only append, counter-add, and publish). Reach for it when a read is too slow and you want it kept fresh on every write.\n- **`@job`** is **you-drive-it** background work (typically a `@scheduled` daemon method) with the **full** write surface. Reach for it when the work is clock-driven or one-off and needs to both scan and mutate freely.\n\n> **Rule of thumb.** Clock-driven and needs the full database surface: a `@job` (usually inside a `@daemon`). Change-driven and only maintains a View: a `@derive`. Clock-driven with modest reads and writes: a plain `@scheduled` method (no `@job` needed).\n\nFor the complete permission grid see the [function-kind matrix](../database/setup.md#how-access-is-gated-query-action-and-friends), and for the decorator itself see [every decorator](../concepts/decorators.md#database-function-kinds-data-access-policy).\n\n## Related\n\n- [Daemons and scheduled jobs](./daemons.md): `@daemon`, `@scheduled`, interval vs cron, and how a single global worker fails over safely.\n- [Derived views (`@derive`)](./derive.md): keeping a materialized View in sync with its source data.\n- [Compute tiers (L1 to L4)](../concepts/tiers.md): where daemons and request handlers each run on the edge.\n- [Views](../database/views.md) and [Events](../database/events.md): the database families a `@derive` reads from and writes to.\n",
|
|
21
21
|
"cli/README.md": "# The toiljs CLI\n\nThe `toiljs` command is how you scaffold, run, build, self-host, and diagnose a toiljs app. This page lists every command and every flag, with copy-pasteable examples.\n\n## What it is\n\nWhen you install the `toiljs` package, it adds one executable to your project: `toiljs`. Everything you do day to day (start the dev server, produce a production build, check your setup) goes through it.\n\nYou almost never type the full path. A freshly scaffolded project already has npm scripts that call it, so you run `npm run dev` and `npm run build`. When you want a command that has no script (like `doctor` or `db`), run it with `npx`:\n\n```bash\nnpx toiljs doctor\n```\n\n`toiljs` needs **Node.js 24 or newer** (older versions will not run it).\n\n## Command overview\n\n| Command | What it does |\n| --- | --- |\n| `toiljs create [name]` | Scaffold a brand new toiljs app in a new folder. |\n| `toiljs dev` | Start the local development server with hot reload. |\n| `toiljs build` | Produce the optimized production build (client bundle + server WebAssembly). |\n| `toiljs start` | Self-host the built app on a fast production HTTP server. |\n| `toiljs configure` | Turn styling features (Sass/Less/Stylus, Tailwind) on or off in an existing project. |\n| `toiljs doctor` | Diagnose your project setup and (with `--fix`) repair common wiring. |\n| `toiljs update` | Check npm for newer dependency versions and apply the ones you pick. |\n| `toiljs db <action>` | Inspect, reset, snapshot, or restore the local dev database. |\n| `toiljs help` | Print the built-in help. Also `--help` or `-h`. |\n| `toiljs --version` | Print the installed toiljs version. Also `-v`. |\n\nRun `toiljs help` any time to see this list in your terminal.\n\n## Global behavior\n\nA few things apply to every command.\n\n- **`--root <dir>`**: run the command against a project in another directory instead of the current one. Every command accepts it.\n- **Automatic update check**: on every run (including `npm run dev`, which calls the CLI under the hood), toiljs quietly asks the npm registry whether a newer `toiljs` exists and prints a one-line notice on stderr if you are behind. It never blocks or slows the command in a meaningful way (the answer is cached for an hour and the network call is capped at two seconds). To turn it off, set the environment variable `TOILJS_NO_UPDATE_CHECK=1`. It also respects the common `NO_UPDATE_NOTIFIER` and `CI` variables.\n\n```bash\n# Run any command against a project in another folder.\ntoiljs build --root ./apps/marketing\n\n# Silence the \"newer version available\" notice.\nTOILJS_NO_UPDATE_CHECK=1 toiljs dev\n```\n\n## `toiljs create`\n\nScaffolds a new project into a new folder. By default it is interactive: it asks a short series of questions (project name, template, styling, and so on), then writes the files and installs dependencies. Pass flags to skip questions, or `-y` to accept every default and run with no questions at all.\n\n```bash\n# Interactive: answer the prompts.\nnpx toiljs create\n\n# Give it a name up front.\nnpx toiljs create my-app\n\n# Fully non-interactive (great for scripts and CI).\nnpx toiljs create my-app --yes --template app --style css\n```\n\n### What it sets up\n\nEvery new project comes wired for you: the enforced TypeScript, ESLint, and Prettier presets, file-based routing, a `toil.config.ts`, a `toilconfig.json` (the server compiler settings), a `.gitignore`, and the editor settings that make the toilscript language plugin work. It also scaffolds a `server/migrations/` folder (where ToilDB schema migrations live) and, unless you opt out, a set of AI assistant helper files.\n\nThe scaffolded `package.json` includes these scripts:\n\n| Script | Runs |\n| --- | --- |\n| `npm run dev` | `toiljs dev` |\n| `npm run build` | `toiljs build` |\n| `npm run build:server` | `toiljs build --server` |\n| `npm run lint` | `eslint client` |\n| `npm run typecheck` | `tsc --noEmit` |\n| `npm run format` | `prettier --write ...` |\n\n### Generated docs and AI-assistant pointers\n\nEvery project carries a full copy of this documentation set at `.toil/docs/`. You do not maintain it: toiljs regenerates it from the installed toiljs version on every `toiljs dev` and `toiljs build`, so it always matches the version you are on. Do not edit those files by hand (your changes are overwritten on the next dev or build).\n\nUnless you opt out, `toiljs create` also writes small **pointer files** at the project root that tell AI coding assistants to read `.toil/docs/` before touching the project: `CLAUDE.md` (Claude Code), `AGENTS.md` (Codex and others), `.cursor/rules/toiljs.mdc` (Cursor), and `.github/copilot-instructions.md` (GitHub Copilot). These are written once, committed, and yours to edit. Control them with `--ai` / `--no-ai` (when you pass neither, `create` asks).\n\n### create options\n\n| Flag | Meaning |\n| --- | --- |\n| `[name]` | The project folder name (a positional argument). If omitted, you are asked. |\n| `-t, --template <app\\|minimal>` | `app` is the full starter (landing page, layout, styles, demo routes). `minimal` is just a layout and a home route. Default `app`. |\n| `--style <css\\|sass\\|less\\|stylus>` | Which CSS flavor to set up. Default `css` (plain CSS). |\n| `--tailwind` / `--no-tailwind` | Add or skip Tailwind CSS (v4). Off by default. |\n| `--ai` / `--no-ai` | Include or skip AI assistant files (like `CLAUDE.md`). When omitted, you are asked. |\n| `--images` / `--no-images` | Enable or skip build-time image optimization. On by default. |\n| `--git` / `--no-git` | Initialize a git repository. When omitted, you are asked (default yes). |\n| `--install` / `--no-install` | Install dependencies after scaffolding. When omitted, you are asked (default yes). |\n| `--pm <npm\\|pnpm\\|yarn\\|bun>` | Which package manager to install with. Default `npm`. |\n| `-y, --yes` | Accept all defaults and skip every prompt. |\n\nFor a walkthrough, see [Create a project](../getting-started/create-project.md).\n\n## `toiljs dev`\n\nStarts the local development server with hot reload, so you edit a file and the browser updates in place. This is the command you leave running while you build.\n\n```bash\nnpm run dev\n# or, to pick a port:\nnpx toiljs dev --port 4000\n```\n\n### What the dev server emulates\n\nThe whole point of the dev server is to run your app locally the same way the real Dacely edge runs it in production, so what you see locally is what you ship. It emulates three things:\n\n1. **The edge.** For a project with a server (any project with a `toilconfig.json`), a small local HTTP server takes the public port and dispatches incoming requests into your compiled `server.wasm` using the exact same request envelope the production edge uses. Anything your server does not claim (page routes, static assets, the hot-reload websocket) is proxied to Vite behind the scenes. So your React frontend and your WebAssembly backend run together, on one URL, exactly like production.\n2. **The database.** ToilDB runs in-process as a local emulator. Every family (documents, views, unique, events, counters, membership, capacity) works, and the data is written to `.toil/devdata.json` so it survives restarts. You manage that file with [`toiljs db`](#toiljs-db).\n3. **The host functions.** The platform services your server code calls (email, environment variables and secrets, time, crypto, rate limiting, auth) are wired up locally so they behave like the edge. Email actually sends if you configure a provider (see [Email](../services/email.md)).\n\nA **client-only** project (no `toilconfig.json`) just gets the plain Vite dev server on your port, unchanged.\n\n### How hot reload works\n\n```mermaid\nflowchart TD\n E[\"You edit a file\"] --> Q{\"Which file?\"}\n Q -->|\"client/ (React, CSS)\"| V[\"Vite hot-swaps it in the browser\"]\n Q -->|\"server/ (a @rest / @data / @service file)\"| S[\"toilscript recompiles the server\"]\n S --> G[\"shared/server.ts is regenerated\"]\n G --> V2[\"Vite hot-swaps the new typed client\"]\n S --> W[\"the dev server hot-swaps the new server.wasm\"]\n V --> B[\"Browser updates, no full reload\"]\n V2 --> B\n W --> B\n```\n\nClient edits go straight through Vite's hot module replacement. Server edits trigger a toilscript rebuild: toiljs recompiles your backend, regenerates `shared/server.ts` (the typed client the browser imports to call your server), and hot-swaps the recompiled WebAssembly, all without you touching the browser. Rebuilds are debounced (grouped over about 150 milliseconds) so a \"save all\" or a formatter pass does not trigger a storm of builds.\n\nIf your project has an `emails/` folder, the dev server also prints an email-preview URL at `/__toil/emails`.\n\n### dev options\n\n| Flag | Meaning |\n| --- | --- |\n| `--port <n>` | Port to listen on. Default `3000` (or `client.port` from your config). |\n| `--root <dir>` | Run against a project in another directory. |\n\nPress `Ctrl+C` to stop. toiljs restores your terminal and force-exits even if a native listener is slow to close, so you never end up with an orphaned dev server rebuilding in the background.\n\n## `toiljs build`\n\nProduces the optimized production build. Run this before you deploy or before `toiljs start`.\n\n```bash\nnpm run build\n# or build only the server:\nnpx toiljs build --server\n```\n\n### What it produces\n\nA full build runs in a careful order so the pieces line up:\n\n```mermaid\nflowchart LR\n A[\"Your project\"] --> S[\"1. Compile the server<br/>(toilscript)\"]\n S --> W[\"build/server/release.wasm\"]\n S --> R[\"shared/server.ts<br/>(typed client, regenerated)\"]\n R --> C[\"2. Bundle the client<br/>(Vite)\"]\n C --> O[\"build/client/<br/>(HTML, JS, CSS, assets)\"]\n C --> X[\"3. Prerender + SEO<br/>(SSG pages, robots.txt,<br/>sitemap.xml, llms.txt)\"]\n```\n\n1. **The server is built first.** toilscript compiles every decorated server file (not just the entry) into `build/server/release.wasm`, and regenerates `shared/server.ts`. Doing this first means the client always bundles against a current, correct typed server client. A project that also declares `@stream` or `@daemon` surfaces compiles those into their own artifacts (`release-stream.wasm`, `release-cold.wasm`).\n2. **The client is bundled** by Vite into `build/client/` (your HTML, JavaScript, CSS, and optimized assets). The dev toolbar and error overlay are stripped out of the production bundle.\n3. **Static pages and SEO files are generated**: any route that opts into static generation is prerendered to HTML, and if you configured `client.seo`, toiljs writes `robots.txt`, `sitemap.xml`, and `llms.txt`. Routes that opt into server-side rendering get their HTML-with-holes templates baked for the edge.\n\n### build options\n\n| Flag | Meaning |\n| --- | --- |\n| `--server` | Build **only** the server (recompile the wasm and regenerate `shared/server.ts`), and skip the client bundle. Fast when you only touched backend code. This is what `npm run build:server` runs. |\n| `--root <dir>` | Run against a project in another directory. |\n\nA client-only project (no `toilconfig.json`) skips step 1 and just bundles the client.\n\n## `toiljs start`\n\nSelf-hosts the app you just built, on a fast production HTTP server (hyper-express, backed by uWebSockets.js). It serves your static client, runs your `server.wasm` for dynamic requests, does server-side rendering, supports daemons, and exposes a `/_toil` websocket channel. Use it to run your app on your own machine or server instead of deploying to the Dacely edge.\n\n```bash\nnpm run build # start needs a build to serve\nnpx toiljs start\nnpx toiljs start --port 8080 --host 0.0.0.0 --threads 4\n```\n\n`start` fails fast if there is no build yet (it looks for `build/client/index.html`), so run `toiljs build` first.\n\n### start options\n\n| Flag | Meaning |\n| --- | --- |\n| `--port <n>` | Port to listen on. Default `3000` (or `client.port`). |\n| `--host <host>` | Address to bind. Default `127.0.0.1` (loopback only). Pass `0.0.0.0` to accept connections from other machines. |\n| `--threads <n>` | Number of HTTP worker processes. Default is automatic (one per available CPU). Pass `1` to disable the worker pool. `--workers` is an accepted alias. This can also be set as `server.threads` in your config. |\n| `--root <dir>` | Run against a project in another directory. |\n\n## `toiljs configure`\n\nToggles a project's client styling features (the CSS preprocessor and Tailwind) after the fact, on an existing app. It detects your current setup, asks what you want, then rewrites the stylesheets and your app entry's imports, edits `package.json`, and syncs `node_modules` so removed packages are actually uninstalled.\n\n```bash\n# Interactive.\nnpx toiljs configure\n\n# Non-interactive: switch to Sass and turn Tailwind on.\nnpx toiljs configure --style sass --tailwind\n```\n\n### configure options\n\n| Flag | Meaning |\n| --- | --- |\n| `--style <css\\|sass\\|less\\|stylus>` | Switch the CSS preprocessor. |\n| `--tailwind` / `--no-tailwind` | Turn Tailwind on or off. |\n| `--images` / `--no-images` | Turn build-time image optimization on or off (sets `client.images` in your config). |\n| `--no-install` | Edit the files but do not run the package manager. You then run install yourself. |\n| `--root <dir>` | Run against a project in another directory. |\n\nPassing any of `--style`, `--tailwind`, or `--images` makes the command non-interactive (it skips the prompts you did not answer with a flag). See [Styling](../frontend/styling.md) for the full picture.\n\n## `toiljs doctor`\n\nRead-only project diagnostics. It gathers facts from disk (your `package.json`, lockfiles, the resolved config, your app entry, `index.html`, your routes, and the server target), runs a set of checks, and prints a grouped report. It never changes anything unless you pass `--fix`, and it never crashes on a partial or non-toiljs project (missing pieces just become warnings or failures). It exits with a non-zero status when any check **fails** (warnings do not fail), so it is safe to run in CI.\n\n```bash\n# Human-readable report.\nnpx toiljs doctor\n\n# Machine-readable, for CI.\nnpx toiljs doctor --json\n\n# Auto-repair the common wiring.\nnpx toiljs doctor --fix\n```\n\n### What it checks\n\nThe report is grouped:\n\n| Group | Example checks |\n| --- | --- |\n| **Environment** | Node.js version, that `toiljs` and its peer dependencies (React, TypeScript, and so on) are installed and new enough, that a lockfile exists, and that your scripts do not wrap `toiljs` in a stray `npx`. |\n| **Project + routing** | The `client/` and `routes/` folders exist, `index.html` has a `<div id=\"root\">`, your app entry calls `mount(...)` with the `slots` argument, at least one route exists, no two routes collide on the same URL, and no asset paths are written in a way that 404s on nested routes. |\n| **Config + assets** | Your `toil.config` loads, the base path is well formed, `client.seo` has a `url` if SEO is configured, and your styling packages are actually installed. |\n| **Server / WASM** | The `toilconfig.json` and its entry files exist, `toilscript` is installed, a compiled `.wasm` exists, the typed-RPC wiring is in place, your `@rest` controllers are actually dispatched, the Prettier and editor plugins are wired, and a `migrations/` folder exists. |\n| **Security** | If your server uses auth, whether `AUTH_SESSION_SECRET` is set (an unset secret means sessions fall back to a published dev key, which is forgeable). |\n\n### What `--fix` repairs\n\n`--fix` only touches a server project (one with a `toilconfig.json`), and it repairs the wiring that is easy to get wrong or that older projects predate:\n\n- adds `--rpcModule shared/server.ts` to your server build scripts,\n- adds `shared` and the `shared/*` path alias to `tsconfig.json`,\n- adds `shared/server.ts` to `.gitignore`,\n- lifts the `toilscript` version floor if it is too old,\n- adds the `toiljs/prettier-plugin` to your Prettier config (so Prettier does not choke on server decorators),\n- adds the toilscript language-service plugin to your server `tsconfig.json` and points VS Code at the workspace TypeScript (so the editor stops false-flagging `@database` collections and `@data` members),\n- refreshes the editor-only server globals declaration file.\n\nIt is idempotent: it only writes files it actually needs to change, and it tells you which ones changed and which need a manual edit (for example a `tsconfig.json` that contains comments). If it changed `package.json`, run your installer afterward.\n\n### doctor options\n\n| Flag | Meaning |\n| --- | --- |\n| `--json` | Emit machine-readable JSON instead of the human report (the banner is suppressed so stdout stays valid JSON). |\n| `--fix` | Repair the server wiring in place, as above. |\n| `--root <dir>` | Run against a project in another directory. |\n\n## `toiljs update`\n\nA friendly wrapper over `npm-check-updates`. It checks the registry for newer versions of your dependencies, groups them by how big the jump is (major, minor, patch), lets you pick which to apply (or `-y` to apply all), bumps `package.json`, and runs your package manager's install. It also makes sure your `server/migrations/` folder exists (older projects predate it). `npm-check-updates` runs via `npx`, so it never becomes a permanent dependency of your project.\n\n```bash\n# Interactive picker.\nnpx toiljs update\n\n# Apply everything, non-interactively.\nnpx toiljs update --yes\n\n# Only patch-level updates.\nnpx toiljs update --target patch\n```\n\n### update options\n\n| Flag | Meaning |\n| --- | --- |\n| `-y, --yes` | Apply all available updates without the picker. |\n| `--target <latest\\|minor\\|patch\\|newest\\|greatest>` | How far to bump. Default `latest`. |\n| `--root <dir>` | Run against a project in another directory. |\n\n## `toiljs db`\n\nManages the local dev database: the on-disk ToilDB store your dev server writes to `.toil/devdata.json`. Use it to inspect data, wipe a corrupt state, save a snapshot to share as a fixture, or restore one. The snapshot is exactly the JSON the dev database uses, so an exported file imports cleanly.\n\n```bash\n# See what is stored.\nnpx toiljs db status\n\n# Wipe all dev data.\nnpx toiljs db reset\n\n# Save a snapshot (to a file, or to stdout if you omit the file).\nnpx toiljs db export fixture.json\n\n# Restore a snapshot.\nnpx toiljs db import fixture.json\n\n# Print the on-disk path (scriptable).\nnpx toiljs db path\n```\n\n### db actions\n\n| Action | What it does |\n| --- | --- |\n| `status` (alias `info`) | Show the database path, its size, and per-family row counts. |\n| `reset` (alias `purge`) | Delete all dev data (removes `devdata.json`). |\n| `export [file]` | Write a formatted snapshot to `file`, or to stdout if you omit it (pipe-friendly). |\n| `import <file>` | Replace the dev database with the snapshot in `file`. It refuses a file that is not a valid snapshot. |\n| `path` | Print the `devdata.json` path and nothing else. |\n\nThe dev database is per-project and lives under `.toil/`, which is gitignored. See [Database setup](../database/setup.md) for what actually populates it.\n\n## The dev, build, and deploy flow\n\nHere is how the commands fit together across your workflow.\n\n```mermaid\nflowchart LR\n subgraph Develop\n D[\"toiljs dev<br/>(hot reload, local edge + DB)\"]\n end\n subgraph Ship\n B[\"toiljs build<br/>(client bundle + server.wasm)\"]\n end\n subgraph Run\n ST[\"toiljs start<br/>(self-host locally)\"]\n DP[\"deploy to the Dacely edge\"]\n end\n D --> B\n B --> ST\n B --> DP\n```\n\n- **Develop** with `toiljs dev`. Everything runs locally and reloads as you type.\n- **Ship** with `toiljs build`. This produces the artifacts: the client bundle in `build/client/` and the server WebAssembly in `build/server/`.\n- **Run** the build two ways: `toiljs start` self-hosts it on your own machine, or you deploy the same build to the Dacely edge to serve it worldwide.\n\n## Gotchas\n\n- **`start` needs a build.** `toiljs start` serves what is in `build/`. Run `toiljs build` first, or it exits with an error.\n- **Run `toiljs`, not `npx toiljs`, inside npm scripts.** Under `npm run`, `node_modules/.bin` is already on your PATH, so an extra `npx` layer is redundant and can leave your terminal in a broken input mode after `Ctrl+C`. `doctor` warns about this; the scaffold gets it right.\n- **`--host` and `--threads` are `start`-only.** `toiljs dev` always binds locally and does not take `--host` or `--threads`.\n- **`--server` builds the backend only.** Use it while iterating on server code, but run a full `toiljs build` before you deploy so the client bundle is current.\n- **The update notice is not an error.** The \"newer toiljs available\" line is informational and prints to stderr. Set `TOILJS_NO_UPDATE_CHECK=1` to hide it.\n\n## Related\n\n- [Installation](../getting-started/installation.md) and [Create a project](../getting-started/create-project.md)\n- [Project structure](../getting-started/project-structure.md)\n- [Configuration reference (`toil.config.ts`)](../concepts/config.md)\n- [Styling](../frontend/styling.md)\n- [Database setup](../database/setup.md) and the [database overview](../database/README.md)\n- [Compute tiers (L1 to L4)](../concepts/tiers.md)\n",
|
|
22
|
+
"concepts/ai-guide.md": "# Writing toiljs correctly\n\nThis is a fast, high-signal cheat-sheet for writing toiljs and toilscript code that compiles and behaves. It is aimed at AI assistants generating code (and at humans in a hurry): the patterns to follow, the mistakes to avoid, and a link to the deep page whenever you need more. It does not re-teach the framework; each section points you at the authoritative doc.\n\n## Project shape\n\nA toiljs project has three top-level folders, and knowing which one a file belongs to tells you which rules apply:\n\n- **`client/`**: your React app that runs in the browser. Pages live under `client/routes/`, one file per URL, and each route file `export default`s its component. See [Frontend](../frontend/README.md) and [Routing](../frontend/routing.md).\n- **`shared/`**: a generated typed bridge (`shared/server.ts`) that lets the browser call the backend with full types. It is regenerated on every build, so **never hand-edit it**.\n- **`server/`**: your backend, written in TypeScript but compiled by **toilscript** to WebAssembly. It runs on the Dacely edge, not in Node or the browser. See [Backend](../backend/README.md).\n\nMost code runs on the L1 request tier. Long-lived connections (`@stream`) and scheduled work (`@daemon`) are opt-in tiers in their own entry files. See [Compute tiers](./tiers.md).\n\n## Client rules (the browser, React side)\n\n**Call the backend only through the generated `Server` client.** This is the number one rule. Use `Server.REST.<controller>.<route>(args)` for `@rest` HTTP controllers, or `Server.<service>.<method>(args)` / `Server.<remote>(args)` for RPC. Raw `fetch` to your own backend throws away the type safety, the argument and result decoding, and the binary wire codec, and it silently breaks when a route is renamed. `fetch` is only for third-party URLs (an external API, a CDN). See [Fetching data](../frontend/data-fetching.md).\n\n```ts\n// WRONG: raw fetch to your own backend\nawait fetch('/account/session', { method: 'POST' });\n\n// RIGHT: the typed client (renames become compile errors; args + result typed)\nawait Server.REST.account.session();\n```\n\n**Use the ambient `Toil.*` globals with no import.** `Toil` is the `toiljs/client` package exposed as a global and typed via a generated `toil-env.d.ts`, so it autocompletes with no import: `Toil.Link` / `Toil.NavLink` for navigation, `Toil.useParams`, `Toil.useLoaderData`, `Toil.Image`, `Toil.useHead` / `Toil.Head`, `Toil.Form` / `Toil.useAction`, and more. `Server` and `parseError` are also bare globals; `FastMap`, `FastSet`, `DataWriter`, and `DataReader` are bare globals too (write `new DataWriter()`, not `new Toil.DataWriter()`). Full index: [The Toil global](../frontend/toil-global.md).\n\n**Load page data with a `loader`, not a `useEffect` fetch.** A route file exports a `loader`, and the component reads its result with `Toil.useLoaderData`. The loader runs on navigation in parallel with the route chunk, integrates with `loading.tsx`, caching, and SSR hydration. A `useEffect` fetch runs only after mount (slower, invisible to the server).\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n return Server.REST.blog.get({ params: { id: params.id } });\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return <article><h1>{post.title}</h1></article>;\n}\n```\n\n**Mutate with `Toil.useAction` or `<Toil.Form>`, then revalidate.** Both track pending and error state and refetch the affected loader data on success, so the page updates without a manual refetch. Use `<Toil.Form>` for form submits, `useAction` for anything else (a delete button, a toggle). Reach for `Toil.revalidate()` / `router.revalidate()` after a write that these do not cover.\n\n**Per-route SEO with `export const metadata`** (or `Toil.useHead` / `<Toil.Head>` from inside a component). Opt a route into edge SSR with `export const ssr = true`. See [Metadata and SEO](../frontend/metadata.md) and [Rendering and SSR](../frontend/rendering.md).\n\n**Typed hrefs.** `Toil.Link` `href`, `Toil.navigate`, and `router.push` are all type-checked against your real routes, so a typo is a compile error. Use `Toil.href(str)` only when a URL is assembled from data and TypeScript cannot prove it is a real route. See [Navigation](../frontend/navigation.md).\n\n**`getUser()` on the client is display-only.** It reads a readable session cookie with no network call, so it is instant but **forgeable**. Use it to render \"logged in as ...\", never to gate real access. The authoritative check is a server route guarded with `@auth`. See [Auth usage](../auth/usage.md).\n\n## Server rules (toilscript, compiles to WASM)\n\nThe decorators, each doing one job (full list in [Decorators](./decorators.md)):\n\n- **`@data`** on a class: generates the binary and JSON codec so the value can cross the wire and the database. Every field needs a default; field order **is** the binary layout, so add new fields at the end. See [Data types](../backend/data.md).\n- **`@rest('name')`** on a class mounts an HTTP controller at `/name`; **`@get`/`@post`/`@put`/`@del`/`@patch`** on its methods declare routes (`@del`, not `@delete`, which is reserved). See [REST](../backend/rest.md).\n- **`@service`** + **`@remote`** expose typed RPC callable from your own frontend as `Server.<service>.<method>()`. See [RPC](../backend/rpc.md).\n- **`@user`** declares the authenticated-user shape and enables `AuthService.getUser()`. **Exactly one `@user` per project.** **`@auth`** guards a route or a whole `@rest` class. See [Auth](../auth/README.md).\n- **`@ratelimit(strategy, limit, window)`** caps how often a caller may hit a route, rejected at the edge before your code runs. See [Security](./security.md).\n- **`@database`** + **`@collection`** declare a ToilDB schema (below).\n\n```ts\n@rest('players')\nclass Players {\n @get('/:id')\n public get(ctx: RouteContext): Response {\n const id = ctx.param('id');\n return Response.json(`{\"id\":\"${id}\"}`);\n }\n}\n```\n\n**`@auth` rejects with 401 before your handler runs.** So inside an `@auth`-guarded handler, `AuthService.getUser()!` is guaranteed non-null and safe to assert. Without `@auth`, `getUser()` can be null, so null-check it instead. Either way the server re-verifies the signed session, so it (not the client cookie) is the real authorization boundary.\n\n```ts\n@auth\n@get('/settings')\npublic settings(): Response {\n const user = AuthService.getUser()!; // safe: @auth guarantees a session\n return Response.text('hi ' + user.username);\n}\n```\n\n**Server code is a strict, WASM-targeted dialect (AssemblyScript), not full TypeScript.** The real constraints, verified against the toilscript standard library:\n\n- **Use explicit value types**, never `number`: `i32` / `u32` / `i64` / `u64` / `f64` (and `i8`/`u8`/`i16`/`u16`/`f32`, plus `u128`..`u256`), `bool`, `string`, `Uint8Array`. Plain `number` resolves to `f64`, which is wrong for ids and counts. Integer math **wraps** on overflow (it never throws), and integer `/` truncates. See [The type system](./types.md).\n- **No `any`.** Every value has a concrete type; that is what lets it compile to WASM.\n- **No arbitrary npm.** Only the toilscript standard library plus the toiljs host APIs.\n- **Use `null`, not `undefined`** for \"no value\" (`T | null`, narrowed with `if (x != null)` or a `!` assertion).\n- **No usable built-in `RegExp`** (it is a host stub that throws). Parse strings by hand.\n- **Structured values are `@data` classes, not object literals.** Encode and decode raw binary with `DataWriter` / `DataReader` (imported on the server from the `data` module: `import { DataWriter, DataReader } from 'data';`). For dynamic JSON, use the ambient `JSON` value tree.\n- **Read config with `Environment.get(key)` and secrets with `Environment.getSecure(key)`** (each returns `string | null`; the two buckets are disjoint so a secret can never leak through `get`). See [Environment](../services/environment.md).\n- **Return a runtime `Response`** (`Response.json` / `.text` / `.html` / `.bytes` / `.notFound` / ...), or return a `@data` value and let toiljs serialize it.\n\n**ToilDB has seven collection families; pick the one that matches the job** (do not force everything into one). Declare each as a `static` `@collection` field on a `@database` class, typed by its family, and reach it statically (`AppDb.users.get(...)`). See [The database](../database/README.md) and [Setup](../database/setup.md).\n\n- **`Documents<K,V>`**: the default record store, looked up by id (users, posts, orders).\n- **`Unique<K,V>`**: a globally one-of-a-kind claim (usernames, emails, slugs).\n- **`Counter<K>`**: a running total many callers bump at once (likes, views); `add` a delta only.\n- **`Events<K,V>`**: an append-only log kept in order (feeds, audit trails).\n- **`Capacity<K>`**: a limited quantity handed out without overselling (tickets, seats).\n- **`Membership<K,M>`**: sets of who belongs to what (followers, tags, room members).\n- **`View<K,V>`**: a precomputed read-optimized snapshot (leaderboards, home pages).\n\n**Data access is gated by function kind.** A `@get` route is a read-only **Query**; a `@post` route is an **Action** (may write); a plain `@remote` defaults to read-only Query, so add **`@action`** to let it write. Scans (`events.latest`, `membership.list`) are barred from request handlers: do them in a `@derive` and have the request read the resulting `View`. The compiler enforces this and the edge re-checks it.\n\n## Common mistakes\n\nA short do-not list. Every one of these is a real, avoidable error:\n\n- **Raw `fetch` to your own backend.** Go through `Server.*` instead.\n- **Importing `Toil.*`.** They are ambient globals; importing them is wrong.\n- **Fetching page data in a `useEffect`.** Use a route `loader` + `Toil.useLoaderData`.\n- **A plain `<a>` for in-app links.** Use `Toil.Link` (a bare `<a>` triggers a full reload).\n- **More than one `@user`.** Exactly one per project.\n- **Trusting client `getUser()` for authorization.** It is display-only and forgeable; enforce on the server with `@auth`.\n- **`any`, `number`, or `RegExp` in server code.** Use explicit value types; there is no usable `RegExp`.\n- **Reordering `@data` fields** (or changing a field type) on a stored type. Field order is the layout; add at the end and use `@migrate` to evolve. See [Data types](../backend/data.md).\n- **Writing from a plain `@remote` or `@get`.** Reads are the default; a write needs `@action`.\n- **Calling a scan (`events.latest`, `membership.list`) from a request handler.** Do it in a `@derive`.\n- **Hand-editing `shared/server.ts` or `.toil/`.** They are generated; rebuild instead.\n\n## Where to go deeper\n\n- Concepts: [Decorators](./decorators.md), [Type system](./types.md), [Security](./security.md), [Compute tiers](./tiers.md).\n- Backend: [Overview](../backend/README.md), [REST](../backend/rest.md), [RPC](../backend/rpc.md), [Data types](../backend/data.md).\n- Frontend: [Overview](../frontend/README.md), [Fetching data](../frontend/data-fetching.md), [Routing](../frontend/routing.md), [Navigation](../frontend/navigation.md), [The Toil global](../frontend/toil-global.md).\n- Database: [ToilDB overview](../database/README.md), [Setup](../database/setup.md).\n- Auth: [Overview](../auth/README.md), [Usage](../auth/usage.md).\n- Services: [Environment](../services/environment.md), [Rate limiting](../services/ratelimit.md).\n</content>\n</invoke>\n",
|
|
22
23
|
"concepts/config.md": "# Configuration (`toil.config.ts`)\n\n`toil.config.ts` is the one file that configures your whole toiljs app: the client (React/Vite) side and the server (WebAssembly) side. Every field is optional and has a sensible default, so most projects keep it tiny.\n\n## The shortest possible config\n\nA scaffolded project ships something like this:\n\n```ts\n// toil.config.ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n // Optimize images at build time (resize and compress imported images).\n images: true,\n },\n});\n```\n\n`defineConfig` does not do anything at runtime: it is an identity helper that gives you full editor autocomplete and type-checking for the config shape. Always wrap your config in it.\n\nAn empty config is valid too. `export default defineConfig({})` gives you a working app with all defaults.\n\n## Where the file lives\n\ntoiljs looks in your project root for the first file that matches, in this order:\n\n```\ntoil.config.ts toil.config.mts toil.config.js toil.config.mjs\ntoiljs.config.ts toiljs.config.mts toiljs.config.js toiljs.config.mjs\n```\n\nUse whichever you like. `toil.config.ts` is the convention.\n\n## Config vs. environment variables\n\nThere are two very different places settings live, and it matters which one you use.\n\n| | `toil.config.ts` (this page) | Environment variables |\n| --- | --- | --- |\n| **When it applies** | Build time and dev time | Runtime, per request |\n| **What it holds** | Framework and build options (routing, styling, SEO, which features to compile) | Values and secrets your running server reads (API keys, feature flags, connection info) |\n| **Committed to git?** | Yes, it is source | No. Local values go in `.env` / `.env.secrets` (both gitignored); production values live on your deploy target |\n| **Read in code with** | Not read from your app code | `Environment.get(...)` / `Environment.getSecure(...)` on the server |\n\nRule of thumb: if it is a **secret** (a password, an API key, a session key), it does **not** go in `toil.config.ts`. It goes in the environment. See [Environment and secrets](../services/environment.md) for the full story.\n\nThere is also a **third** file, `toilconfig.json`, which is a different thing entirely. See [`toil.config.ts` is not `toilconfig.json`](#toilconfigts-is-not-toilconfigjson) at the bottom.\n\n## The top-level shape\n\n```ts\ninterface ToilConfig {\n root?: string; // project root (defaults to the current working directory)\n client?: ClientConfig; // the React / Vite frontend\n server?: ServerConfig; // the toilscript / WebAssembly backend\n}\n```\n\nEverything else lives under `client` or `server`.\n\n## `client` reference\n\nConfigures the frontend: source folders, the dev server, and build-time optimizations.\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `srcDir` | `string` | `\"client\"` | Your frontend source directory, relative to the project root. |\n| `routesDir` | `string` | `\"routes\"` | Your file-based routes directory, relative to `srcDir`. |\n| `publicDir` | `string` | `\"<srcDir>/public\"` | Static assets directory. Holds `index.html` (which you own) plus files served as-is (favicons, images). |\n| `outDir` | `string` | `\"build/client\"` | Where the production client bundle is written. |\n| `base` | `string` | `\"/\"` | The public base path. Use this if you serve the app under a sub-path (a non-root base should start and end with `/`, like `\"/app/\"`). |\n| `port` | `number` | `3000` | The dev server port. `--port` on the CLI overrides it. |\n| `images` | `boolean` | `true` | Optimize imported images at build time (resize and convert). See [Images](../frontend/images.md). |\n| `fonts` | `boolean` | `true` | Preload bundled fonts (inject `<link rel=\"preload\">` for each `@font-face`) so text paints faster. |\n| `viewTransitions` | `boolean` | `false` | Animate page navigations with the browser View Transitions API (a crossfade by default). Respects `prefers-reduced-motion`. |\n| `transitions` | `boolean` | `false` | Wrap navigations in a React transition, keeping the current page visible while the next route's loader runs (instead of showing its loading state immediately). |\n| `devtools` | `boolean` or object | `true` | The floating dev toolbar (route/build info, errors, live controls). It is dev-only and never ships in production. Set `false` to disable, or pass an object to configure its AI integration. |\n| `seo` | object | (off) | Build-time SEO: bakes site-level metadata into the HTML and generates `robots.txt`, `sitemap.xml`, `llms.txt`. See [`client.seo`](#clientseo) below. |\n| `vite` | Vite `InlineConfig` | `{}` | An escape hatch: raw Vite options, deep-merged over the framework's own Vite setup. toiljs owns the Vite config; use this only to override specific options. |\n\n### Example\n\n```ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n images: true,\n fonts: true,\n viewTransitions: true,\n // Serve the app under https://example.com/app/\n base: '/app/',\n },\n});\n```\n\n### `client.devtools`\n\nThe dev toolbar is on by default. To turn it off, or to give it an AI provider (so its \"explain this error\" helpers can call a model), pass an object:\n\n```ts\nimport { defineConfig, AiProvider } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n devtools: {\n ai: {\n provider: AiProvider.Anthropic, // 'anthropic' or 'openai'\n model: 'claude-sonnet-4-6',\n // The name of the env var holding the API key. It is read\n // server-side by the dev process and never sent to the browser.\n apiKeyEnv: 'ANTHROPIC_API_KEY',\n // Optional: a custom POST endpoint ({ prompt } in, { text } out)\n // that overrides `provider` entirely.\n // endpoint: 'http://localhost:5000/ai',\n },\n },\n },\n});\n```\n\n| `devtools.ai` field | Type | What it does |\n| --- | --- | --- |\n| `provider` | `AiProvider` | Built-in provider: `AiProvider.Anthropic` (`'anthropic'`) or `AiProvider.OpenAI` (`'openai'`). |\n| `model` | `string` | The model id, for example `'claude-sonnet-4-6'` or `'gpt-4o'`. |\n| `apiKeyEnv` | `string` | The **name** of the environment variable that holds the API key. The key stays server-side and never reaches the browser. |\n| `endpoint` | `string` | A custom POST endpoint. When set, it takes precedence over `provider`. |\n\nThe toolbar always offers hand-off links to Claude and ChatGPT even without any AI config; this only enables the in-toolbar helpers.\n\n### `client.seo`\n\n`client.seo` turns on build-time search-engine and social metadata. It bakes tags into your HTML `<head>` (so crawlers and link-preview bots that do not run JavaScript still see real metadata) and generates `robots.txt`, `sitemap.xml`, and `llms.txt`. This is a large section with its own guide; see [Metadata and SEO](../frontend/metadata.md) for the complete field list and examples. The one thing worth knowing here: set `seo.url` to your absolute site URL (like `\"https://example.com\"`), because the sitemap and canonical links need it.\n\n```ts\nexport default defineConfig({\n client: {\n seo: {\n url: 'https://example.com',\n title: 'My App',\n description: 'A toiljs app.',\n },\n },\n});\n```\n\n## `server` reference\n\nConfigures the backend: which platform features to compile in, and how the dev server and self-host behave.\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `auth` | `boolean` | `false` | Opt into the framework's built-in post-quantum login. See [`server.auth`](#serverauth) below. |\n| `email` | object | (off) | The non-secret email backend config for the dev server and self-host. See [`server.email`](#serveremail) below. |\n| `daemon` | object | (defaults) | Background-job (L4 daemon) settings for dev and self-host. See [`server.daemon`](#serverdaemon) below. |\n| `nodeMode` | string | `\"all\"` | Which compute layer the single dev/self-host process emulates. See [`server.nodeMode`](#servernodemode) below. |\n| `threads` | `number` or `\"auto\"` | `\"auto\"` | HTTP worker count for `toiljs start`. `\"auto\"` uses one per CPU; `1` disables the worker pool. `--threads` on the CLI overrides it. |\n| `srcDir` | `string` | `\"server\"` | Declarative: your server source directory. See the note below. |\n| `outDir` | `string` | `\"build/server\"` | Declarative: the server build output directory. See the note below. |\n\n> **Note on `server.srcDir` / `server.outDir`.** These fields exist in the config type, but the actual location of your server source and its compiled output is driven by `toilconfig.json` (the toilscript compiler config), not by these two fields. Change your server paths in `toilconfig.json`. These fields are currently declarative and left for forward compatibility. See [not `toilconfig.json`](#toilconfigts-is-not-toilconfigjson) below.\n\n### `server.auth`\n\nSet `auth: true` to get a complete post-quantum login system with no boilerplate. The build appends a shipped `@rest('auth')` controller and its `@user` shape to your server, giving you `/auth/register`, `/auth/login`, `/auth/me`, and `/auth/logout` plus sessions.\n\n```ts\nexport default defineConfig({\n server: {\n auth: true,\n },\n});\n```\n\nTwo things to know:\n\n- If you opt in, your app must **not** declare its own `@user` type. The built-in auth owns the single per-program one.\n- There is an escape hatch: adding `import 'toiljs/server/auth'` in `server/main.ts` turns on the same built-in auth surface without this flag.\n\nAuth has its own configuration (session secrets, the OPRF and KEM keys) that lives in the **environment**, not here. See [Auth configuration](../auth/configuration.md).\n\n### `server.email`\n\nThe **non-secret** part of your email setup: which provider, the \"from\" address, and send caps. The dev server and self-host read it. The API key or SMTP password is a **secret** and never goes here; it comes from `.env.secrets` (`TOIL_EMAIL_API_KEY`). Any `TOIL_EMAIL_*` environment variable overrides the matching field here.\n\n```ts\nexport default defineConfig({\n server: {\n email: {\n provider: 'resend',\n from: 'hello@example.com',\n maxPerMin: 60,\n },\n },\n});\n```\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `provider` | `'resend'` \\| `'gmail'` \\| `'smtp'` | `'resend'` | Which email backend to use. |\n| `from` | `string` | (none) | The \"from\" address. Validated (single address, no line breaks). |\n| `maxPerMin` | `number` | `60` | Per-process send ceiling per minute (rolling). `0` means unlimited. |\n| `maxPerDay` | `number` | `0` | Per-process send ceiling per day (rolling). `0` means unlimited. |\n| `maxPerRecipientPerHour` | `number` | `5` | Per-recipient hourly cap (anti-abuse). |\n| `smtp` | object | (none) | Connection details for the `gmail` / `smtp` providers: `host`, `port` (defaults to 587 STARTTLS; 465 is implicit TLS), and `user` (defaults to `from`). |\n\nSee the full guide at [Email and 2FA](../services/email.md).\n\n### `server.daemon`\n\nSettings for the daemon (L4) background layer, used by the dev process and self-host. In dev, the local process is always the leader, so region fields are informational.\n\n```ts\nexport default defineConfig({\n server: {\n daemon: {\n defaultIntervalMs: 60000,\n maxTasks: 64,\n },\n },\n});\n```\n\n| Field | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `region` | `string` | (none) | Region the daemon is pinned to (informational in dev). |\n| `standbyRegion` | `string` | (none) | Warm standby region (informational in dev). |\n| `defaultIntervalMs` | `number` | `60000` | Default interval for a `@scheduled` task that declares none. Values below `1000` are clamped up to `1000` (a sub-second loop would flood the console). |\n| `tickBudgetMs` | `number` | `30000` | Per-tick wall-clock budget before the dev scheduler logs an overrun. |\n| `gasTick` | `number` | `0` | Per-tick gas cap (a dev stub: charged then ignored). |\n| `maxTasks` | `number` | `64` | Maximum number of `@scheduled` tasks. Clamped to the range 1 to 1024. |\n\nSee [Daemons and scheduled jobs](../background/daemons.md).\n\n### `server.nodeMode`\n\nWhich compute layer the single local process emulates. This is a dev and self-host knob only; in production the Dacely edge decides each server's role. Valid values are `hot`, `regional`, `continental`, `daemon`, and `all`. The default, `all`, runs every surface (requests, streams, and daemons) in one process, which is what you want for a full local run. An invalid value falls back to `all` with a warning rather than failing. See [Compute tiers (L1 to L4)](../concepts/tiers.md) for what each layer means.\n\n```ts\nexport default defineConfig({\n server: {\n nodeMode: 'all', // run everything locally (the default)\n },\n});\n```\n\n## Defaults at a glance\n\nIf you write nothing, this is what you get.\n\n| Setting | Default |\n| --- | --- |\n| `client.srcDir` | `\"client\"` |\n| `client.routesDir` | `\"routes\"` |\n| `client.publicDir` | `\"client/public\"` |\n| `client.outDir` | `\"build/client\"` |\n| `client.base` | `\"/\"` |\n| `client.port` | `3000` |\n| `client.images` | `true` |\n| `client.fonts` | `true` |\n| `client.viewTransitions` | `false` |\n| `client.transitions` | `false` |\n| `client.devtools` | on |\n| `client.seo` | off |\n| `server.auth` | `false` |\n| `server.email` | off |\n| `server.nodeMode` | `\"all\"` |\n| `server.threads` | `\"auto\"` |\n| `server.daemon.defaultIntervalMs` | `60000` |\n| `server.daemon.tickBudgetMs` | `30000` |\n| `server.daemon.maxTasks` | `64` |\n\n## A fuller example\n\n```ts\nimport { defineConfig, AiProvider } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n images: true,\n fonts: true,\n viewTransitions: true,\n seo: {\n url: 'https://example.com',\n title: 'Example',\n description: 'Built with toiljs.',\n },\n devtools: {\n ai: { provider: AiProvider.Anthropic, model: 'claude-sonnet-4-6', apiKeyEnv: 'ANTHROPIC_API_KEY' },\n },\n },\n server: {\n auth: true,\n email: { provider: 'resend', from: 'hello@example.com' },\n threads: 'auto',\n },\n});\n```\n\n## `toil.config.ts` is not `toilconfig.json`\n\nThese two look almost the same and are easy to confuse. They are not the same file.\n\n| File | What it is |\n| --- | --- |\n| `toil.config.ts` | **This page.** The framework config: client and server options, styling, SEO, which features to build. You edit it often. |\n| `toilconfig.json` | The **toilscript compiler** config: which server files are entry points, where the `.wasm` is written, and low-level WebAssembly options (optimization level, memory layout, enabled wasm features). It is scaffolded for you and you rarely touch it. Its presence is also what tells toiljs \"this project has a server.\" |\n\nIf you ever need to change where your server source lives or what the compiled artifact is named, that is `toilconfig.json`, not `toil.config.ts`.\n\n## The `toilconfig.json` reference\n\n`toilconfig.json` is the **toilscript compiler** config. toilscript is the compiler that turns your `server/` TypeScript into a `.wasm` file (WebAssembly, the sandboxed binary your backend ships as). This file tells toilscript which server files to compile, where to write the output, and which low-level WebAssembly options to use.\n\n`toiljs create` scaffolds it for you and most projects never touch it. You only edit it if you want to rename the compiled artifact, move your server entry, or hand-tune the WebAssembly codegen. Its presence at the project root is also the signal toiljs uses to decide \"this project has a server\" (a project with no `toilconfig.json` is a client-only app).\n\nA scaffolded file looks like this:\n\n```json\n{\n \"entries\": [\"server/main.ts\"],\n \"targets\": {\n \"release\": {\n \"outFile\": \"build/server/release.wasm\",\n \"textFile\": \"build/server/release.wat\"\n }\n },\n \"options\": {\n \"sourceMap\": false,\n \"optimizeLevel\": 3,\n \"shrinkLevel\": 1,\n \"converge\": true,\n \"noAssert\": false,\n \"enable\": [\n \"sign-extension\",\n \"mutable-globals\",\n \"nontrapping-f2i\",\n \"bulk-memory\",\n \"simd\",\n \"reference-types\",\n \"multi-value\"\n ],\n \"runtime\": \"stub\",\n \"lib\": [\"node_modules/toiljs/server/globals\"],\n \"memoryBase\": 65536,\n \"initialMemory\": 4,\n \"debug\": false,\n \"trapMode\": \"allow\"\n }\n}\n```\n\n### `entries`\n\nAn array of your server entry files: the toilscript starting points for the compile. The scaffold lists just `server/main.ts`, and `main.ts` imports your other surface modules so they all get pulled in. (Under `toiljs build`, toiljs compiles every decorated server file it finds, not only the entries, so a `@rest` or `@data` file you drop in is picked up even if `main.ts` does not import it.)\n\nA project that also has a streams tier or a daemon tier lists their entry files here too, so each tier can compile into its own artifact:\n\n```json\n\"entries\": [\"server/main.ts\", \"server/main.stream.ts\", \"server/main.daemon.ts\"]\n```\n\n### `targets`\n\nA map of named build targets. Each target names its output files. The scaffold has one target, `release`.\n\n| Field | Type | What it does |\n| --- | --- | --- |\n| `outFile` | `string` | Where the compiled `.wasm` is written. The scaffold uses `build/server/release.wasm`. |\n| `textFile` | `string` | Where the `.wat` (WebAssembly **text** format, the human-readable text form of the same module) is written. Handy for inspecting the output; not needed at runtime. |\n\n### `options`\n\nLow-level WebAssembly codegen options passed straight to toilscript. The defaults are already tuned for production, so change these only if you know you need to.\n\n| Field | Type | Scaffold value | What it controls |\n| --- | --- | --- | --- |\n| `sourceMap` | `boolean` | `false` | Emit a source map alongside the `.wasm` so a debugger can map machine code back to your TypeScript. Off by default (it makes the build bigger). |\n| `optimizeLevel` | `number` | `3` | How hard the optimizer works on **speed**, from `0` (none) to `3` (most). `3` is a production release setting. |\n| `shrinkLevel` | `number` | `1` | How hard the optimizer works on **size**, from `0` to `2`. `1` trades a little speed for a smaller binary. |\n| `converge` | `boolean` | `true` | Re-run the optimizer until the output stops getting better. Squeezes out a bit more at the cost of a slower build. |\n| `noAssert` | `boolean` | `false` | Strip `assert(...)` checks from the output (replace them with just their value, no trap). `false` keeps the safety checks in. |\n| `enable` | `string[]` | (see below) | Which modern WebAssembly features the compiled module is allowed to use. |\n| `runtime` | `string` | `\"stub\"` | The memory-management runtime baked into the module. `\"stub\"` is a minimal runtime that never frees memory, which fits toiljs's model exactly: the edge runs one fresh instance per request and throws its whole memory away when the request ends, so there is nothing to garbage-collect. |\n| `lib` | `string[]` | `[\"node_modules/toiljs/server/globals\"]` | Extra library paths whose top-level exports become **ambient globals** (usable with no `import`). This is what makes toiljs's server globals (like `crypto` and the auth primitives) available everywhere in `server/`. |\n| `memoryBase` | `number` | `65536` | The byte offset where your server's static data starts. toiljs reserves the first 64 KiB (`[0, 65536)`) for the **request envelope** the edge writes at offset 0, so a large request body can never overwrite your program's state. Raise it to accept larger request bodies (it costs a little more initial memory). |\n| `initialMemory` | `number` | `4` | How much linear memory the module starts with, in **pages**. One WebAssembly page is 64 KiB, so `4` is 256 KiB. It grows on demand past this. |\n| `debug` | `boolean` | `false` | Include debug information (names and the like) in the binary. Off for production. |\n| `trapMode` | `string` | `\"allow\"` | What happens on a trapping operation (like a bad float-to-int conversion). `\"allow\"` lets it trap (the default and correct choice); `\"clamp\"` replaces traps with clamping instead. |\n\nThe `enable` array turns on WebAssembly features that are off by default in the compiler. The scaffold enables the modern set the compiled server relies on: `sign-extension`, `mutable-globals`, `nontrapping-f2i` (non-trapping float-to-int conversions), `bulk-memory` (fast `memory.copy` / `memory.fill`), `simd` (vector operations), `reference-types`, and `multi-value` (functions that return more than one value). Leave this list as scaffolded unless you have a specific reason to change it; removing an entry can make the module fail to compile or run.\n\n### The `--rpcModule` build flag\n\nOne toilscript flag is worth knowing even though it lives in your npm scripts, not in `toilconfig.json`: `--rpcModule shared/server.ts`. It tells the compiler to also emit `shared/server.ts`, the fully typed client the browser imports to call your server (the `@data` codec plus the typed `Server` surface). toiljs adds this flag for you on the request build. If your `build:server` script is missing it (older projects predate it), `toiljs doctor --fix` injects it. See [the CLI reference](../cli/README.md#what---fix-repairs).\n\n## Gotchas\n\n- **Wrap the config in `defineConfig`.** Without it you lose autocomplete and type errors on typos.\n- **Secrets never go here.** API keys, session secrets, and passwords belong in the environment, not in `toil.config.ts` (which is committed to git). See [Environment and secrets](../services/environment.md).\n- **`server.srcDir` / `server.outDir` do not move your server.** The server source location is governed by `toilconfig.json` entries. Editing these two config fields has no effect today.\n- **`nodeMode` and `daemon` are dev/self-host only.** In production the edge assigns each server its role; these settings never override that.\n- **An invalid `nodeMode` does not crash the build.** It warns and falls back to `all`.\n\n## Related\n\n- [The CLI](../cli/README.md): the commands that read this config.\n- [Environment and secrets](../services/environment.md): runtime values and how they differ from build-time config.\n- [Auth configuration](../auth/configuration.md): the auth-related environment settings behind `server.auth`.\n- [Email and 2FA](../services/email.md): the full email setup behind `server.email`.\n- [Metadata and SEO](../frontend/metadata.md): the full `client.seo` field list.\n- [Daemons and scheduled jobs](../background/daemons.md): the background layer behind `server.daemon`.\n- [Compute tiers (L1 to L4)](../concepts/tiers.md): what `server.nodeMode` selects.\n- [Images](../frontend/images.md) and [Styling](../frontend/styling.md): the features `client.images` and `toiljs configure` control.\n",
|
|
23
24
|
"concepts/decorators.md": "# Decorators reference\n\nEvery feature of a toiljs backend, an HTTP route, an RPC method, a database collection, a scheduled job, is switched on by a **decorator**. This page lists them all, grouped by what they do, so you can find the right one at a glance and jump to the page that covers it in depth.\n\n## What a decorator is\n\nA **decorator** is the `@name` you write on the line just above a class, a method, or a field. It attaches meaning to that code without changing what the code itself does. You are labelling it so the compiler knows how to wire it up.\n\n```ts\n@rest('users') // <- a class decorator: \"this class is an HTTP controller\"\nclass Users {\n @get('/:id') // <- a method decorator: \"this method answers GET /users/:id\"\n public byId(): Response { /* ... */ }\n}\n```\n\nThree things to notice, because they decide where each decorator can go:\n\n- Some decorators apply to a **class** (`@rest`, `@service`, `@stream`, `@daemon`, `@database`, `@data`, `@user`).\n- Some apply to a **method** or a free **function** (`@get`, `@remote`, `@scheduled`, `@query`).\n- One applies to a **field** (`@collection`).\n\nSome take arguments, like `@get('/:id')` or `@cache(60)`; the bare ones, like `@rest` or `@daemon`, do not. You never register anything by hand: tagging the code is enough, and the build discovers it.\n\nEach decorator also belongs to a **tier** (where its code runs) or is **shared** (compiled into every tier). If tiers are new to you, read [Compute tiers](./tiers.md) first; the short version is L1 = per-request at the edge, L2 / L3 = long-lived stream connections, L4 = one global daemon.\n\n## Routing and HTTP (L1)\n\nTurn a class into an HTTP controller and its methods into routes. All run on the L1 request tier. Covered in [REST](../backend/rest.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@rest` | class | Marks the class an HTTP controller, mounted at a prefix (`@rest('users')` -> `/users`). |\n| `@route` | method | Declares a route with an explicit method + path: `@route({ method: Methods.GET, path: '/' })`. |\n| `@get` | method | Shorthand for a `GET` route: `@get('/:id')`. |\n| `@post` | method | Shorthand for a `POST` route. |\n| `@put` | method | Shorthand for a `PUT` route. |\n| `@del` | method | Shorthand for a `DELETE` route (named `del` because `delete` is a reserved word). |\n| `@patch` | method | Shorthand for a `PATCH` route. |\n| `@head` | method | Shorthand for a `HEAD` route. |\n| `@options` | method | Shorthand for an `OPTIONS` route. |\n\n## RPC (L1)\n\nExpose server functions that your own frontend calls like typed async functions. Run on L1. Covered in [RPC](../backend/rpc.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@service` | class | Marks an RPC service; its `@remote` methods are namespaced under the generated client as `Server.<service>.<method>()`. |\n| `@remote` | method / function | Marks a method (of a `@service`) or a top-level function as a client-callable RPC endpoint. |\n\n## Guards and policy (L1)\n\nAttach a rule to a route (or a whole controller). These stack above a route method. Run on L1.\n\n| Decorator | Applies to | What it does | Covered in |\n| --- | --- | --- | --- |\n| `@auth` | class / method | Requires a valid session; returns `401` otherwise. On a class it guards every route. | [Auth usage](../auth/usage.md) |\n| `@cache` | method | Caches the response at the edge and browser: `@cache(edgeMinutes, browserSeconds?, privateScope?, allowAuth?)`. | [Caching](../services/caching.md) |\n| `@ratelimit` | method | Rate-limits a route: `@ratelimit(strategy, limit, window)`. | [Rate limiting](../services/ratelimit.md) |\n\n## Realtime streams (L2 / L3)\n\nHandle a long-lived connection, keeping state per connected client. The `@stream` class runs on the L2 / L3 stream tier; its lifecycle-hook methods fire as events arrive. Covered in [Realtime streams](../realtime/streams.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@stream` | class | Marks a stream protocol handler. `@stream('name')` sets the mount name; `@stream({ scope })` picks Regional (L2) or Continental (L3). |\n| `@connect` | method | Lifecycle hook: fires when a client connects (returns a `StreamOutbound` accept/reject). |\n| `@message` | method | Lifecycle hook: fires on each inbound packet. |\n| `@close` | method | Lifecycle hook: fires on a graceful close. |\n| `@disconnect` | method | Lifecycle hook: fires on an abrupt transport loss. |\n\nA server-side broadcast hook, `@channel`, is planned but not live in the current runtime. See [Channels](../realtime/channels.md) for the status and the working client-side `useChannel` hook you can use today.\n\n## Background and daemon (L4)\n\nRecurring, run-once-globally background work. `@daemon` runs on the single L4 leader. Covered in [Daemons](../background/daemons.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@daemon` | class | Marks the single daemon class (at most one per project). May declare a zero-arg `onStart()` run once at boot. |\n| `@scheduled` | method | Runs the method on a cadence: an interval (`'30s'`, `'5m'`, `'1h'`, `'1d'`) or a 5-field cron string (`'15 9 * * 1-5'`). |\n\n## Database structure (shared)\n\nDeclare your database schema. These carry no tier of their own; they are compiled into every artifact so any tier can use them. Covered in the [database section](../database/README.md).\n\n| Decorator | Applies to | What it does |\n| --- | --- | --- |\n| `@database` | class | Marks a class as a ToilDB database; each `@collection` field becomes a typed handle (`App.users.get(...)`). |\n| `@collection` | field | Declares a field as a collection handle (`Documents` / `View` / `Unique` / `Counter` / `Events` / `Membership` / `Capacity`). |\n| `@data` | class | Marks a class serializable: the compiler generates a binary codec so it can cross the wire and the database. See [Data types](../backend/data.md). |\n| `@migrate` | function | Schema migration for a `@data` type: a free function that upgrades a record written under an old layout to the current shape (lives in `migrations/<Type>.migration.ts`). See [Data types](../backend/data.md). |\n\n## Database function kinds (data-access policy)\n\nA **function kind** labels a function or method with *which* database operations it is allowed to issue, and the compiler enforces it (a read-only `@query` that tries to write is a compile error). Covered across the [database section](../database/README.md).\n\n| Decorator | Applies to | What it does | Covered in |\n| --- | --- | --- | --- |\n| `@query` | function / method | Read-only data access. | [Database](../database/README.md) |\n| `@action` | function / method | Read plus bounded writes and claims. | [Database](../database/README.md) |\n| `@derive` | function / method | Publishes materialized views and rollups (a background materializer). | [@derive](../background/derive.md) |\n| `@job` | function / method | Background work. | [Background work](../background/README.md) |\n| `@admin` | function / method | Control-plane only operations. | [Database](../database/README.md) |\n\n## Auth and structure\n\n| Decorator | Applies to | What it does | Covered in |\n| --- | --- | --- | --- |\n| `@user` | class | Declares the authenticated-user shape; enables the typed `AuthService.getUser()`. At most one per project. | [Auth usage](../auth/usage.md) |\n| `@main` | function | Marks a single top-level function as the module entry point (exported as the WASM `main`). Rarely written by hand: toiljs supplies the entry glue. | [Project structure](../getting-started/project-structure.md) |\n\n## A note on low-level decorators\n\nAssemblyScript (the language toilscript is built on) has its own low-level decorators such as `@inline`, `@unsafe`, and `@operator`. They tune how a symbol compiles and are for advanced library authors, not everyday app code. You will not need them to build a normal toiljs app, and they are out of scope here.\n\n## Related\n\n- [Compute tiers](./tiers.md): where each decorator's code runs (L1 to L4).\n- [The type system](./types.md): the types (`u64`, `string`, `@data`) these decorators work with.\n- [REST](../backend/rest.md) and [RPC](../backend/rpc.md): the L1 surfaces.\n- [Realtime streams](../realtime/streams.md): the `@stream` surface.\n- [Daemons](../background/daemons.md) and [@derive](../background/derive.md): L4 and background work.\n- [The database (ToilDB)](../database/README.md): `@database`, `@collection`, and the function kinds.\n- [Auth](../auth/README.md): `@auth` and `@user`.\n",
|
|
24
25
|
"concepts/security.md": "# Security\n\nSecurity in toil is not a single feature you switch on. It is a set of defaults that are already on, spread across the whole stack, so that the safe path is also the default path.\n\n## The model\n\nToil is built for **defense in depth**: many independent layers, so that one weak spot is not one breach. The goal is the top of our internal security bar, the Resilience and Scale Grade (see [../../RSG.md](../../RSG.md)), where each of these holds at the same time:\n\n- All traffic is encrypted in transit with modern TLS (Transport Layer Security, the encryption every `https://` connection uses).\n- The login design never lets the server see a password it could reuse.\n- Secrets never live in your code.\n- The browser verifies every asset it loads before running it.\n- There is no implicit trust between the parts of the system (zero trust): each request is checked on its own merits.\n\nThe rest of this page walks through each layer and points to where it is documented in full.\n\n## The backend is a sandbox\n\nEvery site (every tenant) on toil runs inside a **sandbox**: a walled-off compartment that can only touch what the host explicitly hands it, and can never reach the host itself or another tenant. Your server code is compiled to **WebAssembly** (wasm, a portable binary instruction format that runs in a tightly controlled virtual machine), and the edge loads each tenant as its own isolated wasm module.\n\nThat isolation is backed by **hard per-host limits**, enforced by the host and not by your code:\n\n- **A committed-memory cap.** Each tenant may commit only a bounded amount of real memory (on the order of tens of MiB). A tenant cannot grow without bound and starve its neighbors.\n- **A compute cap.** Each request gets a bounded compute budget, and when a request exceeds it, the host stops it. A tenant cannot spin forever on the request path.\n- **Rate limits.** How often one client may call a route is capped (see [Transport and rate limiting](#transport-and-rate-limiting) below).\n- **Hostile-wasm containment.** The runtime is hardened to run untrusted, even deliberately malicious, wasm without letting it escape.\n\nThe point of all of this: a buggy or malicious tenant can crash or hang only itself. It cannot escape its sandbox, exhaust the host, or read or corrupt another tenant's data.\n\n## Passwords: post-quantum login\n\nToil ships a complete login system you turn on with one line, and it is built so the server never handles a usable password. See [../auth/README.md](../auth/README.md) for the overview and [../auth/how-it-works.md](../auth/how-it-works.md) for the mechanism.\n\nHere is why that matters. The pattern almost the whole internet uses is \"send the password over TLS, then hash it on the server.\" That is the accepted baseline, but it caps security below the top grade, because for a moment the server holds a live, replayable credential. Anything that reads server memory or a stray log line at the wrong instant walks away with real passwords, and that is one of the most common breach paths in practice.\n\nToil designs that surface away. The proof that you know your password happens without the password ever crossing the wire in a form anyone can replay, so a fully breached server still yields no usable credentials.\n\n## Subresource Integrity (SRI) on the client bundle\n\nThis is the layer that protects the code running in your users' browsers.\n\n### The risk\n\nYour app's JavaScript and CSS are static files served over the network, often through a CDN (Content Delivery Network, the fleet of edge caches that sits between your server and the browser). If any point in that path is compromised (the host, a cache, or the network itself), an attacker can serve **tampered** JavaScript or CSS: a modified bundle that steals sessions, skims form input, or rewrites the page. The browser, on its own, has no way to know the bytes it received are not the bytes you shipped.\n\n### What toil does at build time\n\n**Subresource Integrity (SRI)** is a browser feature that closes this gap. You attach a cryptographic fingerprint of a file to the tag that loads it, and the browser refuses to use the file unless its bytes match that fingerprint.\n\nAt build time (verified in [../../src/compiler/sri.ts](../../src/compiler/sri.ts)), toil adds that fingerprint to every **local** asset tag in the built HTML:\n\n- every `<script src=...>`,\n- every `<link rel=\"modulepreload\" href=...>`,\n- every `<link rel=\"stylesheet\" href=...>`.\n\nTo each it adds two attributes: `integrity=\"sha384-<base64>\"` and `crossorigin=\"anonymous\"`.\n\n- **`sha384`** is the hash algorithm: the file's bytes are run through SHA-384 (a member of the SHA-2 family), producing a fixed fingerprint that changes if even one byte changes. The `integrity` value is that fingerprint, base64-encoded, with a `sha384-` prefix.\n- **`crossorigin=\"anonymous\"`** is required for the browser to actually verify the fingerprint. It is a no-op when the asset is same-origin, so adding it is always safe.\n\nTags that are not toil's to hash are left completely untouched: external URLs (anything with a scheme like `https:`, a protocol-relative `//cdn/...`, or a `data:` URI), inline scripts (a `<script>` with no `src`), and any tag that already carries its own `integrity`.\n\n### Why an import map is also emitted\n\nA per-tag `integrity` only protects the **entry** script: the one file named directly in the tag. But a modern app is an ES module graph (ECMAScript modules, the browser's native `import` system). The entry script statically imports other chunks (for example the shared React vendor chunk), and it lazily pulls in each route with a dynamic `import()` as the user navigates. Those follow-on files are fetched by the browser's **module loader**, which does **not** inherit the entry tag's `integrity`. Without more, everything past the entry point would be unverified.\n\nSo toil also emits an **import map**: a `<script type=\"importmap\">` tag (a standard browser mechanism for describing module resolution). Its `integrity` section maps every emitted JavaScript chunk to its own `sha384` fingerprint. This is the platform mechanism that extends verification to the module loader's fetches, so the **full** dynamic `import()` graph is covered, not just the entry tag. Browsers that do not yet support the `integrity` key simply ignore it (progressive enhancement: nothing breaks on older browsers, they just get one fewer check).\n\n### Why it stays stable\n\nA fingerprint is only useful if it keeps matching the bytes actually served. Toil guarantees that by **content-hashing** every asset it puts SRI on: the build tool (Vite) names each JavaScript chunk and each CSS file with a hash of its own contents baked into the URL (verified in [../../src/compiler/vite.ts](../../src/compiler/vite.ts)). Because the URL is pinned to the exact bytes, the fingerprint baked into the HTML can never falsely mismatch.\n\nThis prevents a real failure mode. Imagine a stylesheet kept a stable URL but its bytes changed on a new deploy. A user still holding an older HTML page (with the old baked fingerprint) would fetch the new bytes, the hashes would disagree, and the browser would **block** the stylesheet, breaking the page for that user. Content-hashing sidesteps this: new bytes get a new URL, so old HTML keeps pointing at the old bytes, and every fingerprint always matches. (Images and fonts keep stable names on purpose: they carry no SRI and are often referenced by fixed path.)\n\n### What the browser does\n\nFor each asset that carries an `integrity` value (from a tag or from the import map), the browser hashes the bytes it received and compares them to the expected fingerprint. If they match, it runs the script or applies the stylesheet. If they do not match, it refuses: the asset is blocked and tampered code never runs.\n\n```mermaid\nflowchart TD\n A[\"Browser loads the built page HTML\"] --> B[\"Reads integrity tags plus the importmap integrity section\"]\n B --> C[\"Fetches an asset: entry script, vendor chunk, route chunk, or stylesheet\"]\n C --> D[\"Computes the sha384 hash of the received bytes\"]\n D --> E{\"Hash matches the baked fingerprint?\"}\n E -->|yes| F[\"Runs the script or applies the stylesheet\"]\n E -->|no| G[\"Refuses: blocks the asset, tampered code never runs\"]\n```\n\n### When it applies\n\nSRI is part of the **production build output**. The dev server hands the browser un-hashed modules straight from Vite, where SRI would be meaningless, so it is not applied there. You get the protection automatically in what you ship, with nothing to configure.\n\n## Secrets\n\nSecrets (API keys, signing keys, and the like) are **never compiled into the wasm**. They are provided per-host at runtime through the secure environment store and read with `Environment.getSecure(key)`, which is kept strictly separate from the plain-variable `Environment.get(key)`: a secret can never leak out through a code path that only reads plain config. Because secrets live outside your code, they are never in your repository or its history. See [../services/environment.md](../services/environment.md) for the full model.\n\n## Transport and rate limiting\n\nAll traffic runs over modern **TLS**, so nothing sensitive travels in the clear. The edge speaks **HTTP/3** (built on QUIC, a faster, head-of-line-blocking-free transport) with graceful fallback to HTTP/2 and HTTP/1.1, so newer clients get the modern path and older clients still connect and work.\n\nOn top of that, you can cap how often a client may hit any given route. Adding the `@ratelimit` decorator to a route makes the edge reject a client that goes over the limit **before your code runs**, which blunts brute-force and abuse and helps absorb bursts and denial-of-service (DDoS) pressure at the edge rather than inside your handler. Only routes that opt in pay any cost; everything else stays on the untouched fast path. See [../services/ratelimit.md](../services/ratelimit.md) for the strategies and per-user (keyed) limits.\n\n## Related\n\n- [Authentication](../auth/README.md): the post-quantum login system and `@auth`-guarded routes.\n- [Environment](../services/environment.md): plain variables versus secrets, and where they live.\n- [Rate limiting](../services/ratelimit.md): the `@ratelimit` decorator, strategies, and keyed limits.\n- [Design principles](../introduction/design-principles.md): the ideas the framework is built on.\n",
|
|
@@ -34,12 +35,12 @@ export const TOIL_DOCS: Record<string, string> = {
|
|
|
34
35
|
"database/unique.md": "# Unique\n\nThe **Unique** family enforces that a value is claimed by only one owner across the entire world. It is how you make usernames, email addresses, and URL slugs one-of-a-kind, safely, even when two people try to grab the same one at the same instant.\n\n## What and why\n\nA **Unique collection** maps a `@data` **claim key** (the thing that must be unique, like a username) to a `@data` **owner value** (who or what claimed it, like a user id). At any moment a claim key is either **unclaimed** or **owned by exactly one owner**. The family gives you three operations: look up who owns a key, claim a key, and release it.\n\nReach for Unique whenever a value must be globally singular:\n\n- usernames or handles\n- email addresses\n- URL slugs or workspace names\n- any \"reserve this name for me\" scenario\n\nWhy not just check a Documents record first? Because a check-then-write has a race: two requests can both check \"is `alice` free?\", both see yes, and both create it. Unique closes that gap. The claim is decided at the key's single **home** (see [consistency](./README.md#eventual-consistency-in-plain-words)), where claims are processed one at a time, so exactly one of the two racers wins.\n\nDeclare one by typing a `@collection` as `Unique<ClaimKey, OwnerValue>`:\n\n```ts\n@data\nclass Username {\n name: string = '';\n constructor(name: string = '') { this.name = name; }\n}\n\n@data\nclass OwnerId {\n userId: string = '';\n constructor(userId: string = '') { this.userId = userId; }\n}\n\n@database\nclass AppDb {\n @collection static usernames: Unique<Username, OwnerId>;\n}\n```\n\n## The operations\n\n`K` is the claim-key type, `V` the owner value type.\n\n| Operation | Signature | Returns | Use it to |\n| --- | --- | --- | --- |\n| `lookup` | `lookup(key: K): V \\| null` | the current owner, or `null` if unclaimed | find out who owns a name |\n| `claim` | `claim(key: K, value: V): ClaimResult<V>` | a `ClaimResult` (see below) | try to take a name for an owner |\n| `release` | `release(key: K, value: V): void` | nothing; **traps** if you are not the owner | give a name back |\n\n`lookup` is a read, so it works in any function. `claim` and `release` are writes, so they need an **Action** (a `@post` route or an `@action`); see [Setup](./setup.md#how-access-is-gated-query-action-and-friends).\n\n### `ClaimResult`\n\n`claim` returns a small object that tells you what happened:\n\n```ts\nclass ClaimResult<V> {\n claimed: bool; // true if YOU own the key now\n owner: V | null; // when claimed is false, who owns it instead\n}\n```\n\n- **`claimed == true`**: you own the key. This covers both a fresh claim and an **idempotent re-claim**: if you claim a key you already own (same owner value), you still get `true`, so retrying a claim is safe. `owner` is `null` in this case.\n- **`claimed == false`**: someone else got there first. `owner` is their value, so you can tell the user \"that name is taken.\"\n\n### `lookup`\n\n`lookup` just reads the current owner without changing anything:\n\n```ts\nconst owner = AppDb.usernames.lookup(new Username('alice'));\nif (owner == null) {\n // 'alice' is free\n} else {\n // owner.userId currently holds 'alice'\n}\n```\n\nNote that `lookup` is subject to [eventual consistency](./README.md#eventual-consistency-in-plain-words): a claim made moments ago in another region may not show up in a far-away `lookup` yet. Do not use `lookup` as your uniqueness guard. `lookup` is for display (\"this name is taken by ...\"); the real guarantee comes from `claim`, which is decided at the home and cannot race.\n\n### `claim`\n\n`claim` is the operation that actually enforces uniqueness. You pass the key and the owner value:\n\n```ts\nconst result = AppDb.usernames.claim(new Username('alice'), new OwnerId('u_123'));\nif (result.claimed) {\n // 'alice' is now yours\n} else {\n // taken; result.owner is the current owner\n}\n```\n\nBecause claims are serialized at the key's home, this is race-safe. If two requests anywhere in the world call `claim('alice', ...)` at the same moment, the home applies them in order: the first gets `claimed: true`, the second gets `claimed: false` with the first as `owner`. There is no window where both win.\n\n### `release`\n\n`release` gives a claim back so the key becomes available again. Only the **current owner** may release: you pass both the key and the owner value, and if that value is not the current owner, `release` **traps** (aborts the request). This prevents one user from releasing another user's name.\n\n```ts\nAppDb.usernames.release(new Username('alice'), new OwnerId('u_123'));\n```\n\nRelease when a name is being changed or an account is deleted, so the name returns to the pool.\n\n## The claim / release lifecycle\n\nA claim is a small state machine: unclaimed, owned, and back to unclaimed.\n\n```mermaid\nstateDiagram-v2\n [*] --> Unclaimed\n Unclaimed --> Owned: claim(key, me) -> claimed: true\n Owned --> Owned: claim(key, me) again -> claimed: true (idempotent)\n Owned --> Owned: claim(key, someoneElse) -> claimed: false, owner: me\n Owned --> Unclaimed: release(key, me)\n```\n\nThe key insight: **claiming is the guard, releasing is cleanup.** You claim to reserve, you release to free. A claim that is never released stays owned forever (there is no automatic expiry; if you want time-limited holds, that is what the [Capacity](./capacity.md) family's TTL reservations are for).\n\n## Worked example: reserving a username on signup\n\nThe safe signup pattern is: claim the username first, then create the account, and if creating the account fails, release the claim so the name is not stranded.\n\n```ts\nimport { Response } from 'toiljs/server/runtime';\n\n@data\nclass Username {\n name: string = '';\n constructor(name: string = '') { this.name = name; }\n}\n\n@data\nclass OwnerId {\n userId: string = '';\n constructor(userId: string = '') { this.userId = userId; }\n}\n\n@data\nclass UserId {\n id: string = '';\n constructor(id: string = '') { this.id = id; }\n}\n\n@data\nclass User {\n id: string = '';\n username: string = '';\n}\n\n@data\nclass SignupInput {\n username: string = '';\n userId: string = '';\n}\n\n@database\nclass AppDb {\n @collection static usernames: Unique<Username, OwnerId>;\n @collection static users: Documents<UserId, User>;\n}\n\n@rest('signup')\nclass Signup {\n // POST /signup (Action: may claim and write)\n @post('/')\n public signup(input: SignupInput): Response {\n const owner = new OwnerId(input.userId);\n\n // 1. Claim the username. This is the uniqueness guard.\n const claim = AppDb.usernames.claim(new Username(input.username), owner);\n if (!claim.claimed) {\n return Response.text('username taken', 409);\n }\n\n // 2. Create the account. If this fails, undo the claim so the name is free.\n const user = new User();\n user.id = input.userId;\n user.username = input.username;\n if (!AppDb.users.create(new UserId(input.userId), user)) {\n AppDb.usernames.release(new Username(input.username), owner);\n return Response.text('could not create account', 409);\n }\n\n return Response.json(user.toJSON().toString());\n }\n}\n```\n\nTwo things make this correct:\n\n- **The claim happens before the account write**, so uniqueness is decided up front.\n- **The claim is released if the follow-up write fails**, so a failed signup does not permanently burn a username.\n\nBecause `claim` is idempotent for the same owner, a client that retries the whole request after a network hiccup does not get a false \"taken\" error: the second `claim('alice', u_123)` returns `claimed: true` again.\n\n## Consistency notes\n\n- **`claim` and `release` are strongly consistent at the key's home.** Two callers can never both own the same key; the home serializes claims. This is the whole point of the family.\n- **`lookup` is eventually consistent.** It reads a possibly-nearby copy, so a very recent claim from elsewhere may not appear yet. Never gate uniqueness on `lookup`; gate it on the result of `claim`.\n- **Claims do not expire.** A claim stays until someone releases it. Use [Capacity](./capacity.md) if you need holds that auto-release after a timeout.\n\n## Gotchas\n\n- **Do not use `lookup` as the uniqueness check.** Its answer can be stale. Call `claim` and trust its `claimed` flag.\n- **Release on failure and on account deletion.** A claim you forget to release stays owned forever, quietly blocking that name.\n- **`release` traps for a non-owner.** Pass the correct owner value, or you abort the request. If you are not sure you own it, `lookup` first (for display) but expect `release` to enforce ownership regardless.\n- **The owner value is data you choose.** Store enough in it (a user id, say) to know who holds the claim, so you can display and release it later.\n\n## Related\n\n- [ToilDB overview](./README.md): the seven families and how to choose.\n- [Setup](./setup.md): declaring the collection and which function kinds may claim.\n- [Documents](./documents.md): the account record you create alongside a claim.\n- [Capacity](./capacity.md): time-limited holds that auto-release (a different kind of \"reserve\").\n- [Data types (`@data`)](../backend/data.md): the claim key and owner value.\n",
|
|
35
36
|
"database/views.md": "# Views (materialized views)\n\nA `View` collection is a **precomputed, read-optimized result** that you publish\nonce and read many times. Reads are a single cheap keyed lookup, no scanning, no\nrecomputing.\n\n## What and why\n\nA \"materialized view\" is a fancy name for a simple idea: instead of computing an\nanswer every time someone asks, you compute it **once**, store the finished\nanswer, and just hand it out on each request. \"Materialized\" means the result is\nmade real and saved (as opposed to computed on the fly). \"View\" means it is a\nread-friendly projection of your underlying data.\n\nThink of a leaderboard. Computing \"the top 10 players by score\" means scanning\nevery player and sorting. You do not want to do that on every page load. So you\ncompute it in the background, save the finished top-10 list as a view, and every\npage load just reads that saved list. Fast, cheap, and it does not get slower as\nyou add players.\n\nUse a `View` when:\n\n- A page needs a result that is **expensive to compute** (a scan, a sort, a fold\n over many rows).\n- That result is **read far more often than it changes** (a home page, a feed, a\n leaderboard, a \"latest N\" list, a rendered summary).\n\nThe rule of thumb: if a route wants \"the newest N of something\" or \"the top N of\nsomething\", that is a scan, and scans are barred on the request path. A `View`\nis how you serve that result without scanning per request.\n\n```mermaid\nflowchart LR\n subgraph off[\"Off the request path (@derive)\"]\n S[(\"Source data<br/>events / counters / records\")] --> D[\"@derive: scan + compute\"]\n D --> P[\"publish(key, result)\"]\n end\n P --> V[(\"View\")]\n subgraph on[\"On the request path (@get)\"]\n G[\"view.get(key)\"] --> V\n end\n```\n\n## The type\n\nA `View` has two type parameters: the **key** (which view) and the **value**\n(the precomputed result).\n\n```ts\nView<K, V>\n```\n\n- `K` is the key type: it picks *which* view to read or publish. For a single\n global leaderboard, the key can be a fixed constant like `'main'`.\n- `V` is the value type: the finished result you serve. Both are\n [`@data`](../concepts/types.md) classes.\n\nDeclare it as a `@collection` field inside a `@database` class, alongside the\nsources it is built from:\n\n```ts\n@database\nclass BoardDb {\n @collection static scores: Events<GameKey, ScoreEvent>; // a source\n @collection static board: View<GameKey, Leaderboard>; // the view\n}\n```\n\n## Operations\n\nA `View` has exactly two operations you use directly (plus `require`, a\nconvenience wrapper). Exact signatures:\n\n| Operation | Signature | What it does |\n| --- | --- | --- |\n| `get` | `get(key: K): V \\| null` | Read the published view, or `null` if nothing has been published yet. |\n| `require` | `require(key: K): V` | Like `get`, but traps (aborts the request) if nothing is published. |\n| `publish` | `publish(key: K, value: V): void` | Overwrite the view for `key` with a fresh result. |\n\n### `get`\n\nA plain keyed read. It is cheap and is allowed from **any** handler, including a\nread-only `@get` route.\n\n```ts\nconst board = BoardDb.board.get(new GameKey('main'));\nif (board == null) {\n // Nothing published yet (e.g. brand-new game). Serve an empty default.\n return new Leaderboard();\n}\nreturn board;\n```\n\nAlways handle the `null` case: until a `@derive` (or `@job`) has published at\nleast once, `get` returns `null`.\n\n### `publish`\n\nOverwrites the stored view with a new value. This is how the view gets its\ncontent.\n\n```ts\nBoardDb.board.publish(new GameKey('main'), freshBoard);\n```\n\n`publish` is **restricted**: you may only call it from a\n[`@derive`](../background/derive.md) or a `@job` (a background task), never from a\nrequest handler (`@get`/`@post`/...). The compiler enforces this, and so does the\nedge at runtime. The reason: a view is meant to be maintained off the request\npath from the source of truth, not written ad hoc by whichever request happens to\nrun.\n\n### How `publish` differs from a Documents write\n\nA `View`'s `publish` looks like writing a value, but it is not the same as a\n[Documents](./documents.md) `patch`/`create`. The differences matter:\n\n| | `Documents` write (`create` / `patch`) | `View` `publish` |\n| --- | --- | --- |\n| Who may call it | Any action handler (`@post`, ...) | Only a `@derive` or `@job` |\n| Meaning | The record IS the source of truth | The view is a **copy** derived from a source |\n| Version control | You may version-check (optimistic concurrency) | The host assigns the version; a later publish always wins (last writer wins) |\n| Read path | Standard keyed read | A read-optimized fast path (views are heavily read) |\n| Who owns correctness | You (each writer edits the true value) | The derive (it recomputes from the source, so the view always converges) |\n\nIn short: a Document is the truth; a View is a saved snapshot of a computation\nover the truth. If the view is ever wrong or stale, the derive just recomputes and\nrepublishes it. You never hand-edit a view from a route.\n\n## Automatic maintenance with `@derive`\n\nYou rarely call `publish` by hand. The normal pattern is a\n[`@derive`](../background/derive.md): a method on your `@database` class that\nreads the sources, builds the value, and publishes it. The runtime runs it for\nyou:\n\n- **Right after a write to a source.** When a request writes one of the\n database's source collections (an `append`, a `counter.add`, a record write),\n the database's derives run right after the response is produced. So the view\n reflects the new data on the next read.\n- **On box load.** When the server starts or reloads, the views are rebuilt from\n their sources before the first read is served.\n\nYou never call a derive yourself, and a derive's own `publish` never re-triggers\nit. See [`@derive`](../background/derive.md) for the full rules.\n\n## Worked example: a leaderboard\n\nA game appends score events; a derive folds them into a top-10 list; a route\nserves that list with a single `get`.\n\n```ts\nimport { ScoreEvent } from '../models/ScoreEvent';\nimport { Leaderboard } from '../models/Leaderboard';\nimport { GameKey } from '../models/GameKey';\nimport { NewScore } from '../models/NewScore';\n\n@database\nclass BoardDb {\n // The source of truth: every score, appended as it happens.\n @collection static scores: Events<GameKey, ScoreEvent>;\n // The materialized view: the current top entries, ready to serve.\n @collection static board: View<GameKey, Leaderboard>;\n\n // Off the request path: scan the recent scores, sort, publish the top 10.\n @derive\n rebuild(): void {\n const key = new GameKey('main');\n const recent = BoardDb.scores.latest(key, 500); // a scan, allowed in a derive\n const board = new Leaderboard();\n board.entries = topTen(recent); // your own sort/aggregate\n BoardDb.board.publish(key, board);\n }\n}\n\n@rest('leaderboard')\nclass LeaderboardRoutes {\n // GET reads the precomputed view: one keyed read, no scan.\n @get('/')\n public top(): Leaderboard {\n const board = BoardDb.board.get(new GameKey('main'));\n return board == null ? new Leaderboard() : board;\n }\n\n // POST records a score. The @derive rebuilds `board` right after.\n @post('/')\n public submit(input: NewScore): Leaderboard {\n BoardDb.scores.append(new GameKey('main'), new ScoreEvent(input.player, input.score));\n return new Leaderboard(); // ack; the GET serves the updated board from the view\n }\n}\n```\n\nThe models:\n\n```ts\n@data\nexport class ScoreEvent {\n player: string = '';\n score: u64 = 0;\n}\n\n@data\nexport class Leaderboard {\n entries: ScoreEvent[] = [];\n}\n```\n\nA \"latest N\" view is the same shape: the derive calls `latest(key, N)` and\npublishes the list. See the [Events](./events.md) page for that variant end to\nend.\n\n## Consistency\n\n- **Last writer wins.** The host assigns each `publish` a version, and a later\n publish always supersedes an earlier one. Because a derive recomputes the whole\n view from the source of truth, the view converges to a correct snapshot; you do\n not need to coordinate concurrent publishes.\n- **A view can be briefly stale.** ToilDB is worldwide, and a view is published\n at its home then copied to other regions in the background (asynchronous\n replication). A read from a far region can lag the newest publish by a moment.\n This is usually fine for the things views hold (feeds, leaderboards, summaries):\n a leaderboard that is a second behind is still a good leaderboard.\n- **`get` returns `null` until the first publish.** Always default the empty case.\n\n## Gotchas\n\n- **`publish` is derive/job only.** You cannot publish from a route. If you need\n to update a view in response to a request, write the *source* (append an event,\n add to a counter) from the action and let the `@derive` republish.\n- **Handle `null` from `get`.** A never-published view reads as `null`, not as an\n empty value. Return a sensible default.\n- **A view is a copy, not the truth.** Never store data *only* in a view. Keep\n the real data in a source family (Documents, Events, Counters) and treat the\n view as a disposable, recomputable projection.\n- **Keep views bounded.** A view value is read whole on every request, so build\n it from a bounded read (top N, latest N, a total), not the entire history.\n\n## Related\n\n- [`@derive`](../background/derive.md): the normal way a view is maintained.\n- [Events](./events.md): the append-only log a view is often folded from.\n- [Counters](./counters.md): another common source for a view (totals).\n- [Documents](./documents.md): mutable source-of-truth records (and how a write\n differs from a publish).\n- [Data types (`@data`)](../concepts/types.md): how view keys and values are stored.\n",
|
|
36
37
|
"frontend/components.md": "# Components\n\nA toiljs frontend is a normal React app, so the components you write are just React components: functions that return JSX, hold state with hooks, and compose however you like. There is no toiljs-specific base class, no decorator, and no registration step. On top of your own components, toiljs ships a small set of ready-made ones on the `Toil` global (an image, a script loader, a form, and a few more) for the jobs a plain React app makes you wire up by hand. This page covers both: how your components fit in, and a reference for every `Toil.*` component.\n\n## Writing your own components\n\nAnything you already know about writing React components applies unchanged. You write ordinary functions, use `useState` / `useEffect` / `useMemo` and any hooks you like, and return JSX:\n\n```tsx\n// client/components/Counter.tsx\nimport { useState } from 'react';\n\nexport default function Counter({ start = 0 }: { start?: number }) {\n const [n, setN] = useState(start);\n return <button onClick={() => setN(n + 1)}>Clicked {n} times</button>;\n}\n```\n\nYour reusable components live in `client/components/`, and you import them the normal way from anywhere in `client/`:\n\n```tsx\n// client/routes/index.tsx\nimport Counter from '../components/Counter';\n\nexport default function Home() {\n return (\n <main>\n <h1>Welcome</h1>\n <Counter start={10} />\n </main>\n );\n}\n```\n\nA few things are worth spelling out, because they are the parts that are handled for you rather than by you:\n\n- **Route files must `export default` a component.** A file under `client/routes/` becomes a page only if it default-exports a component (see [Routing](./routing.md)). Files under `client/components/` have no such rule; export them however you please (default or named).\n- **The `Toil.*` globals need no import.** `Toil.Link`, `Toil.useParams()`, `Toil.Image`, and everything else on `Toil` are ambient. The compiler generates a `toil-env.d.ts` that types the global, so your editor autocompletes `Toil.` and type-checks it with no `import` line. The same goes for `Server` (the typed backend client) and the `FastMap` / `DataWriter` data utilities. See the [Frontend overview](./README.md) for the whole ambient surface.\n- **The JSX runtime is configured for you.** You do not write `import React from 'react'` at the top of every file. The build sets up the automatic JSX runtime, so JSX just works. Import named hooks and types from `react` when you need them (`import { useState, type ReactNode } from 'react'`), but the bare React import for JSX is unnecessary.\n\nIn other words, a component in a toiljs app is indistinguishable from a component in any React app. Two things are genuinely special, and both are opt-in.\n\n### Special case 1: components inside an SSR route\n\nIf a route opts into server rendering with `export const ssr = true`, its component tree is rendered once at build time into a template, then filled per request on the edge (see [Rendering and SSR](./rendering.md)). For that to work, any part of the JSX that changes per request (a value from the URL, a list from a loader, a block of user HTML) has to be wrapped in one of the [SSR marker primitives](#ssr-marker-primitives) below, or isolated in a `Toil.Island`. Anything you leave unwrapped gets frozen into the template at its build-time value.\n\nYou do not have to get this perfect: if an SSR route (or a layout above it) cannot render on the server, toiljs skips SSR for that route at build, prints a warning telling you what to move, and falls back to plain client rendering. So the route still works, it just loses its server first paint until you address the warning.\n\n### Special case 2: importing images and other assets\n\nImporting an image with the `?toil` suffix gives you an object carrying the resolved URL, the intrinsic width and height, and an auto-generated blur placeholder (a `blurDataURL`), ready to hand straight to `Toil.Image`:\n\n```tsx\nimport hero from './hero.webp?toil';\n// hero is { src, width, height, blurDataURL }\n\n<Toil.Image src={hero} alt=\"Our office\" />;\n```\n\nPassing the whole object lets `Toil.Image` reserve the correct aspect ratio and paint the blur while the real image loads, with no extra props. See [Images](./images.md) for the full treatment.\n\nPlain asset imports are typed for you as well. A bare `import logo from './logo.svg'` (or `.png`, `.webp`, and so on) resolves to the hashed URL string, and the vite-imagetools query forms (`?url`, `?as=srcset`, `?as=metadata`) are typed too, so you get autocomplete and type-checking on all of them without declaring any modules yourself.\n\n## The toiljs component primitives\n\nThese are the components toiljs provides on the `Toil` global. They cover the framework-level jobs a plain React app leaves to you. All are ambient (no import), and all are fully typed.\n\n| Component | What it is | Renders |\n| --- | --- | --- |\n| `Toil.Image` | Layout-shift-free `<img>` replacement with lazy-load and blur placeholder. | An `<img>` (optionally wrapped in a sizing box). |\n| `Toil.Script` | One-time external or inline `<script>` loader with a load strategy. | Nothing (`null`). |\n| `Toil.Form` | A `<form>` that runs an action on submit and revalidates loader data. | A `<form>`. |\n| `Toil.Slot` | Renders a named parallel-route slot for the current URL. | The slot's route tree, or a fallback. |\n| `Toil.Head` | Declarative `<head>` contribution (title, meta, link). | Nothing (`null`). |\n| `Toil.Metadata` | Declarative route-style metadata applied from any component. | Nothing (`null`). |\n\nThe [SSR marker primitives](#ssr-marker-primitives) (`Toil.Hole`, `Toil.Repeat`, `Toil.RawHtml`, `Toil.attr`, `Toil.Island`) are a separate group, covered in their own section below.\n\n### `Toil.Image`\n\nA drop-in `<img>` replacement that prevents layout shift and lazy-loads by default. You give it `width` and `height` (or `fill`) so it reserves space before the image arrives, it decodes asynchronously, it lazy-loads unless you mark it `priority`, and it can fade in from a blur placeholder. It accepts either a string URL or a `?toil` import object (which auto-fills the size and blur):\n\n```tsx\nimport hero from './hero.webp?toil';\n\nexport default function Home() {\n return (\n <>\n <Toil.Image src={hero} alt=\"Our office\" priority placeholder=\"blur\" />\n <Toil.Image src=\"/team.jpg\" alt=\"The team\" width={800} height={600} />\n </>\n );\n}\n```\n\nKey props are `src`, `alt` (required, pass `alt=\"\"` for a decorative image), `width` / `height` or `fill`, `priority` (for an above-the-fold hero), and `placeholder` (`'empty'` or `'blur'`). It also accepts any standard `<img>` attribute. This is kept brief on purpose: [Images](./images.md) covers sizing, `fill`, blur placeholders, and how it stops layout shift in full.\n\n### `Toil.Script`\n\nLoads an external or inline `<script>` exactly once for the whole life of the app (even across client-side navigations), and lets you choose when it runs with a `strategy` prop. Use it instead of a hand-written `<script>` tag for third-party snippets like analytics or chat widgets, which a plain `<script>` in a single-page app runs unreliably or twice:\n\n```tsx\n// client/layout.tsx\nexport default function Layout({ children }: { children?: React.ReactNode }) {\n return (\n <div className=\"app\">\n <Toil.Script src=\"https://cdn.example-analytics.com/analytics.js\" />\n {children}\n </div>\n );\n}\n```\n\nThe `strategy` is `afterInteractive` (default), `lazyOnload`, or `beforeInteractive`, and there are `onLoad` / `onReady` / `onError` callbacks. `Toil.Script` renders nothing. See [Scripts](./scripts.md) for the strategies, inline scripts, dedup rules, and the full prop table.\n\n### `Toil.Form`\n\nA `<form>` that submits without reloading the page. On submit it runs your `action` (which receives the form's `FormData`), tracks pending and error state, and on success revalidates the current route's loader data so the page reflects the write. It is the convenient front end of the loader/action data loop:\n\n```tsx\n// A guestbook that refreshes its list after a successful sign.\nexport default function Guestbook() {\n const entries = Toil.useLoaderData<typeof loader>();\n\n const sign = async (data: FormData) => {\n const author = String(data.get('author'));\n const message = String(data.get('message'));\n await Server.REST.guestbook.sign({ body: new NewMessage(author, message) });\n };\n\n return (\n <Toil.Form action={sign} resetOnSuccess>\n {({ pending, error }) => (\n <>\n <input name=\"author\" placeholder=\"Your name\" />\n <textarea name=\"message\" />\n <button disabled={pending}>{pending ? 'Signing...' : 'Sign'}</button>\n {error ? <p className=\"err\">Could not sign, try again.</p> : null}\n </>\n )}\n </Toil.Form>\n );\n}\n```\n\nPass a render function as the child to read the live submit state: it receives `{ pending, error, data }`, which is how you disable the button while pending or show an error. The props:\n\n| Prop | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `action` | `(data: FormData) => void \\| Promise<void>` | (required) | Runs on submit, receiving the form's `FormData`. May be async. |\n| `revalidate` | `RevalidateTarget` | `true` | Which loader data to refetch after a successful submit. `true` is the current route. |\n| `onSuccess` | `() => void` | (none) | Called after a successful submit. |\n| `onError` | `(error: unknown) => void` | (none) | Called when the action throws. |\n| `resetOnSuccess` | `boolean` | `false` | Reset the form fields after a successful submit. |\n| `className` | `string` | (none) | Class on the `<form>` element. |\n| `children` | `ReactNode` or `(state) => ReactNode` | (none) | Form contents. A function child receives `{ pending, error, data }`. |\n\nFor writes that are not form submits (a delete button, a like toggle), reach for the underlying `Toil.useAction` hook instead. Both are covered in [Fetching data](./data-fetching.md).\n\n### `Toil.Slot`\n\nRenders the named parallel-route slot for the current URL. A folder starting with `@` (like `@modal`) under `client/routes/` is a whole second route tree that matches the URL independently of the main page; placing `<Toil.Slot name=\"modal\" />` is where that tree renders. If no slot route matches the current URL, it renders the `fallback` (nothing by default):\n\n```tsx\n// client/routes/gallery/layout.tsx\nexport default function GalleryLayout({ children }: { children?: React.ReactNode }) {\n return (\n <div>\n {children}\n <Toil.Slot name=\"modal\" fallback={null} />\n </div>\n );\n}\n```\n\n| Prop | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `name` | `string` | (required) | The slot name: the `@name` directory under `routes/`, without the `@`. |\n| `fallback` | `ReactNode` | `null` | Rendered when no slot route matches the current URL. |\n\nParallel slots and the intercepting routes that fill them (the \"click a photo, open it in a modal\" pattern) are explained in [Routing](./routing.md).\n\n### `Toil.Head`\n\nA declarative way for any component (a page, a layout, a deep child) to contribute to the document `<head>`: a title, `<meta>` tags, and `<link>` tags. It renders nothing; it just applies its head entries for the lifetime of the component and reverts them on unmount. Entries compose across the tree, with later or deeper ones winning per key:\n\n```tsx\nexport default function ArticlePage() {\n const post = Toil.useLoaderData<typeof loader>();\n return (\n <>\n <Toil.Head\n title={post.title}\n meta={[\n { name: 'description', content: post.summary },\n { property: 'og:title', content: post.title },\n ]}\n link={[{ rel: 'canonical', href: `https://example.com/blog/${post.id}` }]}\n />\n <article>{/* ... */}</article>\n </>\n );\n}\n```\n\n`<Toil.Head>` takes `title`, `meta` (an array of `{ name | property, content }` tags), and `link` (an array of `{ rel, href }` tags). The hook form `Toil.useHead(spec)` and the shorthand `Toil.useTitle(title)` do the same job imperatively. See [Metadata and SEO](./metadata.md).\n\n### `Toil.Metadata`\n\nThe declarative, convenience-shaped cousin of `Toil.Head`. Instead of raw meta and link arrays, you pass a route-style metadata object (the same shape a route file's `export const metadata` uses), and toiljs expands the convenience fields (`description`, `keywords`, `canonical`, `openGraph`, and so on) into the right tags. It renders nothing, and applies for the component's lifetime. Use it to set metadata from a component that is not itself a route file (a rendered article, a widget):\n\n```tsx\n<Toil.Metadata\n title=\"Our pricing\"\n description=\"Simple, flat pricing for teams of any size.\"\n openGraph={{ title: 'Pricing', image: 'https://example.com/og/pricing.png' }}\n/>\n```\n\nBecause a route's own `metadata` export is applied last (highest priority), `Toil.Metadata` fills in for routes that declare none and yields to a route that sets the same key. The full field list lives in [Metadata and SEO](./metadata.md).\n\n## SSR marker primitives\n\nThese five primitives matter only for routes with `export const ssr = true`. They are how you tell the build which parts of a server-rendered page are dynamic (filled per request) versus static (baked into the template once). On a client-only route they do nothing special, so you never need them unless you opt a route into SSR.\n\n**The mental model.** In the browser these markers are transparent: `<Toil.Hole>` renders its children, `<Toil.Repeat>` maps its rows, `<Toil.RawHtml>` renders a raw-HTML wrapper, and `Toil.attr(id, value)` returns the value unchanged. Your client-side app runs exactly as written. But under the build-time SSR extractor, each marker instead emits a sentinel token that marks an insertion point. The extractor strips those tokens and records their positions, producing a static template with numbered holes. Per request, your compiled backend fills only the hole values, and the edge splices them into the template. Because the static scaffold around each hole is React's own rendering output and the hole values are escaped exactly as React escapes them, the browser hydrates the result byte-for-byte, with no mismatch.\n\n| Marker | Use it for | Shape |\n| --- | --- | --- |\n| `Toil.Hole` | A single dynamic **text** value. | `<Toil.Hole id=\"...\">{value}</Toil.Hole>` |\n| `Toil.Repeat` | A **list**: one row template repeated over an `each` array. | `<Toil.Repeat id=\"...\" each={rows}>{(item, i) => ...}</Toil.Repeat>` |\n| `Toil.RawHtml` | A block of **trusted, pre-rendered HTML**. | `<Toil.RawHtml id=\"...\" html={s} as=\"section\" />` |\n| `Toil.attr` | A dynamic value in **attribute position** (an `href`, a `class`). | `href={Toil.attr('id', value)}` (a function call) |\n| `Toil.Island` | Content that must render **only in the browser** (escape hatch). | `<Toil.Island>{children}</Toil.Island>` |\n\nA few rules that keep the template valid:\n\n- **Every marker needs a stable `id`**, a short name unique within the page. The build maps each id to a numbered slot, so keep the ids constant across builds.\n- **`Toil.Repeat` needs at least one row at build time.** It captures that first row as the sub-template for every row, so the build render must see a sample with one or more items. An empty `each` gives it nothing to capture.\n- **`Toil.RawHtml` renders inside a wrapper element** (a `<div>` by default, `as=\"section\"` to change the tag), and you own sanitising that HTML, exactly like React's `dangerouslySetInnerHTML`.\n- **`Toil.attr` is a function, not an element.** An attribute is not a child node, so it cannot be a JSX element. You call `Toil.attr(id, value)` right where the attribute value goes, and it composes with literal text around it (`` className={`btn ${Toil.attr('kind', d.kind)}`} ``).\n- **`Toil.Island` renders nothing on the server and on the first (hydration) render**, then reveals its children after mount. So an island gets no server first paint and no SEO, by design. It is the place for anything that genuinely cannot run on the server (reads `window`, calls `Date.now()`, depends on the live URL).\n\nHere is a compact SSR route wiring several of them together. The title, tag list, and author link all come from the route's `loader`:\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const ssr = true;\n\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n // Illustrative shape: { title, tags, authorUrl }.\n return Server.REST.blog.get({ params: { id: params.id } });\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return (\n <article>\n <h1>\n <Toil.Hole id=\"title\">{post.title}</Toil.Hole>\n </h1>\n\n {/* A dynamic attribute: call attr() in attribute position. */}\n <a href={Toil.attr('authorUrl', post.authorUrl)}>By the author</a>\n\n {/* A list: one row template, stamped once per item on the server. */}\n <ul>\n <Toil.Repeat id=\"tags\" each={post.tags}>\n {(tag) => <li key={tag}>{tag}</li>}\n </Toil.Repeat>\n </ul>\n </article>\n );\n}\n```\n\nFor the whole SSR story (the template extraction flow, keeping a route SSR-safe, and the current SSR limitations), see [Rendering and SSR](./rendering.md).\n\n## Related\n\n- [Rendering and SSR](./rendering.md): how SSR routes render, hydrate, and use the marker primitives.\n- [Images](./images.md): the full `Toil.Image` reference, blur placeholders, and layout shift.\n- [Scripts](./scripts.md): `Toil.Script` strategies, inline scripts, and dedup.\n- [Fetching data](./data-fetching.md): `Toil.Form`, `useAction`, loaders, and the typed backend clients.\n- [Metadata and SEO](./metadata.md): `Toil.Head`, `Toil.Metadata`, and per-route metadata.\n- [Routing](./routing.md): pages, layouts, and the parallel slots `Toil.Slot` renders.\n",
|
|
37
|
-
"frontend/data-fetching.md": "# Fetching data\n\nYour React frontend and your toiljs backend live in one project, so toiljs generates a **typed client** that lets the browser call your server with full type safety and no hand-written `fetch` boilerplate. This page covers loading data for a page, calling the backend directly, submitting forms, and reading who is logged in.\n\n## The generated `Server` client\n\nWhen you build the server, toiljs writes a file at `shared/server.ts` that contains your `@data` classes and a typed description of every backend endpoint. Importing anything from `shared/server` attaches the runtime clients to a global called `Server`. From then on you call your backend through `Server`, fully typed, with editor autocomplete.\n\nThere are two surfaces under `Server`:\n\n- **`Server.REST.<controller>.<route>(args)`**: a real, typed `fetch` client for your `@rest` HTTP controllers. This is the working, recommended way to call the backend today.\n- **`Server.<service>.<method>(args)` and `Server.<remote>(args)`**: the typed RPC surface for `@service` / `@remote` functions.\n\nBoth are generated from your server code, so if you rename a route or change an argument type, the call site is a compile error until you fix it.\n\n## Calling a REST endpoint\n\n`Server.REST` mirrors your `@rest` controllers. If your backend has a `players` controller with routes on it, you call them like this:\n\n```tsx\nimport { NewPlayer, ScoreDelta } from 'shared/server';\n\n// POST /players with a typed @data body -> typed Promise<Player>\nconst player = await Server.REST.players.create({ body: new NewPlayer('Ada') });\n\n// POST /players/:id/score with a path param AND a body\nconst updated = await Server.REST.players.addScore({\n params: { id: 1 },\n body: new ScoreDelta(5n),\n});\n\n// GET /leaderboard -> typed Promise<Standings>\nconst board = await Server.REST.leaderboard.top();\n```\n\nThe single argument is an object with up to four optional parts:\n\n| Key | What it is |\n| --- | --- |\n| `params` | Path parameters, e.g. `{ id: 1 }` for a `/players/:id` route. |\n| `body` | The request body, usually a `@data` class instance. |\n| `query` | Query-string values. |\n| `headers` | Extra request headers. |\n\nReturn values are typed and decoded for you. A route that returns a `@data` type hands you the parsed class instance. A route that returns a raw `Response` hands you the raw fetch `Response`, so you can inspect the status and headers yourself:\n\n```tsx\n// This route returns a Response, so you get the raw fetch Response.\nconst res = await Server.REST.players.get({ params: { id: 1 } });\nif (!res.ok) {\n console.log('status', res.status);\n} else {\n const p = await res.json();\n}\n```\n\n`@data` classes are the typed values that cross the wire. You import them from `shared/server` and construct them normally (`new NewPlayer('Ada')`). See [Data types](../backend/data.md) for how they are defined on the server.\n\n### Handling errors\n\nA failed call throws. The global `parseError` helper turns any caught value into a readable message, which is handy in a `catch`:\n\n```tsx\ntry {\n const board = await Server.REST.leaderboard.top();\n} catch (err) {\n console.error(parseError(err));\n}\n```\n\n## Loading data for a page (loaders)\n\nCalling the backend from a button handler (as above) is fine for actions. For the data a page needs to *render*, use a route **loader** instead of a `useEffect`. A loader runs on navigation, in parallel with loading the page's code, and the page suspends (showing its `loading.tsx`) until the data is ready:\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n return Server.REST.blog.get({ params: { id: params.id } });\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return <article><h1>{post.title}</h1></article>;\n}\n```\n\n`useLoaderData<typeof loader>()` is fully typed from the loader's return. This keeps data fetching declarative and out of effects, and it is what lets server-rendered pages seed the browser with the server's data so hydration stays clean.\n\n### Caching and revalidating loader data\n\nLoader results are cached by URL. You control how long with `export const revalidate`:\n\n| `revalidate` value | Behavior |\n| --- | --- |\n| `0` (default) | Re-run the loader on every navigation to the route. |\n| a number `n` | Reuse cached data for `n` seconds, then refetch. |\n| `false` | Cache until you invalidate it manually. |\n\nAfter a mutation (say you just POSTed a new comment), refresh the current page's data with `revalidate()` or the router:\n\n```tsx\nconst router = Toil.useRouter();\nawait Server.REST.comments.add({ body: new NewComment(text) });\nrouter.revalidate(); // refetch the active route's loader\n// or target another route: router.revalidate('/posts');\n```\n\n`router.refresh()` re-runs the current loader and clears the cache; `router.revalidate(href)` invalidates a specific route.\n\n### Invalidating loader data by hand\n\n`revalidate()` and the `<Toil.Form>` `revalidate` prop are both built on one lower-level primitive: `Toil.invalidateLoaderData(href?)`. It drops cached loader data so the next render refetches it, but it does **not** trigger the re-render itself (that is the difference from `revalidate()`, which does both):\n\n- `invalidateLoaderData()` with no argument clears every route's cached data.\n- `invalidateLoaderData('/posts')` clears just that one route's entry.\n\nYou rarely need it directly; `revalidate()` (which pairs it with a re-render) is what you usually want. Reach for the primitive when you want to mark data stale *now* but let a later navigation do the actual refetch:\n\n```tsx\n// After a background sync finished, mark the dashboard stale so its\n// next visit refetches, without disturbing the page the user is on.\nToil.invalidateLoaderData('/dashboard');\n```\n\nSignature: `invalidateLoaderData(href?: string): void`.\n\n## Forms\n\nFor the common \"submit a form, then refresh the page's data\" loop, use `Toil.Form`. It runs an async action on submit (no page reload), tracks pending and error state, and revalidates the route's loader data on success:\n\n```tsx\nimport { NewMessage } from 'shared/server';\n\nexport default function Guestbook() {\n const entries = Toil.useLoaderData<typeof loader>();\n\n const sign = async (data: FormData) => {\n const author = String(data.get('author'));\n const message = String(data.get('message'));\n await Server.REST.guestbook.sign({ body: new NewMessage(author, message) });\n };\n\n return (\n <Toil.Form action={sign} resetOnSuccess>\n {({ pending }) => (\n <>\n <input name=\"author\" />\n <input name=\"message\" />\n <button disabled={pending}>Sign</button>\n </>\n )}\n </Toil.Form>\n );\n}\n```\n\nKey `Toil.Form` props:\n\n| Prop | Default | What it does |\n| --- | --- | --- |\n| `action` | (required) | Runs on submit, receiving the form's `FormData`. May be async. |\n| `revalidate` | `true` (current route) | Which loader data to refetch after a successful submit. |\n| `resetOnSuccess` | `false` | Clear the form fields after success. |\n| `onSuccess` / `onError` | | Callbacks for the two outcomes. |\n\nPassing a **render function** as `children` gives you live submit state (`pending`, `error`), so you can disable the button while the request is in flight. On success, `Form` revalidates the loader, so the page's data updates automatically without a manual refetch.\n\n## Mutations without a form: `useAction`\n\n`Toil.Form` is convenient sugar over a more general primitive, `Toil.useAction`. Use `useAction` for any write that is not a form submit: a delete button, a \"like\" toggle, a \"mark as read\" action. It runs your async function on demand and tracks the whole lifecycle (`pending`, `error`, `data`), then revalidates the affected loader data on success so the page reflects the change, exactly like `Form` does.\n\nYou call it with the function to run and an optional options object, and it hands back a handle:\n\n```tsx\nexport default function PostRow({ id }: { id: number }) {\n const del = Toil.useAction(\n (postId: number) => Server.REST.posts.remove({ params: { id: postId } }),\n { revalidate: true, onError: (e) => alert(parseError(e)) },\n );\n\n return (\n <button disabled={del.pending} onClick={() => void del.run(id)}>\n {del.pending ? 'Deleting...' : 'Delete'}\n </button>\n );\n}\n```\n\nThe handle it returns:\n\n| Field | What it is |\n| --- | --- |\n| `run(input)` | Runs the action. Resolves to the result, or `undefined` if it threw. The error is captured in `error` rather than rejected, so a fire-and-forget `onClick` cannot leak an unhandled rejection. |\n| `pending` | `true` while a run is in flight. Drive a spinner or a disabled button off this. |\n| `error` | The error from the last failed run, or `undefined`. |\n| `data` | The value the last successful run returned, or `undefined`. |\n| `reset()` | Clear `pending` / `error` / `data` back to idle. |\n\nThe options mirror `Toil.Form`:\n\n| Option | Default | What it does |\n| --- | --- | --- |\n| `revalidate` | `true` (current route) | Which loader data to refetch after a successful run: `true` (the active route), an `href` or array of hrefs (those routes), or `false` (none). |\n| `onSuccess` | | Called with the action's return value after a successful run. |\n| `onError` | | Called with the thrown value when the action fails. |\n\n`Toil.Form` is just this hook wired to a `<form>`'s submit event, with the form's `FormData` passed as the input. Anything a form does, you can do by hand with `useAction`.\n\n## Reading who is logged in\n\nTo render \"logged in as ...\" you need the current user. toiljs generates a `getUser()` helper (from the backend's `@user` surface) that reads the current user from a readable session cookie with no network round-trip. It is instant, which makes it perfect for display, but it is **untrusted**: a value read on the client can be forged, so never gate anything security-sensitive on it. For a trusted check, call a guarded backend route that re-verifies the signed session.\n\nThe full auth flow (post-quantum login, sessions, `getUser()`, and guarding routes) is its own guide:\n\n- [Auth usage](../auth/usage.md): reading the session, `getUser()`, and guarding pages.\n\n## RPC (`@service` / `@remote`)\n\nAlongside REST, toiljs generates a typed RPC surface for `@service` methods and free `@remote` functions. These read like plain function calls with no URL:\n\n```tsx\nconst n = await Server.ping(10); // a free @remote\nconst count = await Server.stats.playerCount(); // a @service method\n```\n\nThe types come straight from your server, so `Server.ping` knows it takes a number and returns a number. Under the hood each call encodes its arguments and POSTs them to a single reserved endpoint (`/__toil_rpc`) with a compact method id, then decodes the typed result. Both the local dev server and the production edge dispatch this endpoint, so RPC works end to end.\n\nIf a call throws an \"unavailable\" error, it means the generated client has not attached yet: build the server (`npm run build:server`) to regenerate `shared/server.ts`, and import from `shared/server` so the client loads (see the gotchas below). See [Typed RPC](../backend/rpc.md) for the backend side and when to choose RPC over REST.\n\n## Gotchas\n\n- **Import from `shared/server` to attach the clients.** `Server.REST` (and the RPC client) attach when you import from `shared/server`. Importing your `@data` classes from there does it naturally; if `Server.REST.foo()` throws an \"unavailable\" error, make sure the server has been built (`npm run build:server`) and that `shared/server` is imported.\n- **The server runs a fresh instance per request.** In the examples, in-memory writes are previews that do not persist. To keep data, write it to the database (see [Database](../database/README.md)). This is a backend property, but it explains why a create returns a value that is not there on the next request.\n- **Fetch page data in a `loader`, not `useEffect`.** A loader runs in parallel with the route chunk and integrates with `loading.tsx`, suspense, caching, and SSR hydration. A `useEffect` fetch runs only after the page mounts (slower, and invisible to the server).\n- **`getUser()` is display-only.** It is fast because it does not verify anything. Do real authorization on the server.\n\n## Related\n\n- [Routing](./routing.md): loaders, `loading.tsx`, and navigation.\n- [Backend HTTP routes](../backend/rest.md): the `@rest` controllers behind `Server.REST`.\n- [Typed RPC](../backend/rpc.md): the `@service` / `@remote` surface behind `Server`.\n- [Data types](../backend/data.md): the `@data` classes that cross the wire.\n- [Auth usage](../auth/usage.md): sessions and `getUser()`.\n",
|
|
38
|
+
"frontend/data-fetching.md": "# Fetching data\n\nYour React frontend and your toiljs backend live in one project, so toiljs generates a **typed client** that lets the browser call your server with full type safety and no hand-written `fetch` boilerplate. This page covers loading data for a page, calling the backend directly, submitting forms, and reading who is logged in.\n\n## The generated `Server` client\n\nWhen you build the server, toiljs writes a file at `shared/server.ts` that contains your `@data` classes and a typed description of every backend endpoint. Importing anything from `shared/server` attaches the runtime clients to a global called `Server`. From then on you call your backend through `Server`, fully typed, with editor autocomplete.\n\nThere are two surfaces under `Server`:\n\n- **`Server.REST.<controller>.<route>(args)`**: a real, typed `fetch` client for your `@rest` HTTP controllers. This is the working, recommended way to call the backend today.\n- **`Server.<service>.<method>(args)` and `Server.<remote>(args)`**: the typed RPC surface for `@service` / `@remote` functions.\n\nBoth are generated from your server code, so if you rename a route or change an argument type, the call site is a compile error until you fix it.\n\n> **Rule: never call your own backend with raw `fetch`. Always go through `Server`.**\n> Use `Server.REST.<controller>.<route>(args)` for `@rest` HTTP controllers, or\n> `Server.<service>.<method>(args)` / `Server.<remote>(args)` for `@service` / `@remote`\n> RPC. Raw `fetch` throws away the type safety, the argument/return decoding, and the\n> binary wire format that the generated client handles for you, and it silently breaks\n> when a route is renamed. This applies to AI assistants generating code too.\n>\n> ```ts\n> // WRONG: raw fetch to your backend\n> await fetch('/account/session', { method: 'POST', credentials: 'same-origin' });\n>\n> // RIGHT: the typed client (renames become compile errors, args + result are typed)\n> await Server.REST.account.session();\n> ```\n>\n> `fetch` is only for URLs that are NOT your toiljs backend (a third-party API, a CDN).\n\n## Calling a REST endpoint\n\n`Server.REST` mirrors your `@rest` controllers. If your backend has a `players` controller with routes on it, you call them like this:\n\n```tsx\nimport { NewPlayer, ScoreDelta } from 'shared/server';\n\n// POST /players with a typed @data body -> typed Promise<Player>\nconst player = await Server.REST.players.create({ body: new NewPlayer('Ada') });\n\n// POST /players/:id/score with a path param AND a body\nconst updated = await Server.REST.players.addScore({\n params: { id: 1 },\n body: new ScoreDelta(5n),\n});\n\n// GET /leaderboard -> typed Promise<Standings>\nconst board = await Server.REST.leaderboard.top();\n```\n\nThe single argument is an object with up to four optional parts:\n\n| Key | What it is |\n| --- | --- |\n| `params` | Path parameters, e.g. `{ id: 1 }` for a `/players/:id` route. |\n| `body` | The request body, usually a `@data` class instance. |\n| `query` | Query-string values. |\n| `headers` | Extra request headers. |\n\nReturn values are typed and decoded for you. A route that returns a `@data` type hands you the parsed class instance. A route that returns a raw `Response` hands you the raw fetch `Response`, so you can inspect the status and headers yourself:\n\n```tsx\n// This route returns a Response, so you get the raw fetch Response.\nconst res = await Server.REST.players.get({ params: { id: 1 } });\nif (!res.ok) {\n console.log('status', res.status);\n} else {\n const p = await res.json();\n}\n```\n\n`@data` classes are the typed values that cross the wire. You import them from `shared/server` and construct them normally (`new NewPlayer('Ada')`). See [Data types](../backend/data.md) for how they are defined on the server.\n\n### Handling errors\n\nA failed call throws. The global `parseError` helper turns any caught value into a readable message, which is handy in a `catch`:\n\n```tsx\ntry {\n const board = await Server.REST.leaderboard.top();\n} catch (err) {\n console.error(parseError(err));\n}\n```\n\n## Loading data for a page (loaders)\n\nCalling the backend from a button handler (as above) is fine for actions. For the data a page needs to *render*, use a route **loader** instead of a `useEffect`. A loader runs on navigation, in parallel with loading the page's code, and the page suspends (showing its `loading.tsx`) until the data is ready:\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n return Server.REST.blog.get({ params: { id: params.id } });\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return <article><h1>{post.title}</h1></article>;\n}\n```\n\n`useLoaderData<typeof loader>()` is fully typed from the loader's return. This keeps data fetching declarative and out of effects, and it is what lets server-rendered pages seed the browser with the server's data so hydration stays clean.\n\n### Caching and revalidating loader data\n\nLoader results are cached by URL. You control how long with `export const revalidate`:\n\n| `revalidate` value | Behavior |\n| --- | --- |\n| `0` (default) | Re-run the loader on every navigation to the route. |\n| a number `n` | Reuse cached data for `n` seconds, then refetch. |\n| `false` | Cache until you invalidate it manually. |\n\nAfter a mutation (say you just POSTed a new comment), refresh the current page's data with `revalidate()` or the router:\n\n```tsx\nconst router = Toil.useRouter();\nawait Server.REST.comments.add({ body: new NewComment(text) });\nrouter.revalidate(); // refetch the active route's loader\n// or target another route: router.revalidate('/posts');\n```\n\n`router.refresh()` re-runs the current loader and clears the cache; `router.revalidate(href)` invalidates a specific route.\n\n### Invalidating loader data by hand\n\n`revalidate()` and the `<Toil.Form>` `revalidate` prop are both built on one lower-level primitive: `Toil.invalidateLoaderData(href?)`. It drops cached loader data so the next render refetches it, but it does **not** trigger the re-render itself (that is the difference from `revalidate()`, which does both):\n\n- `invalidateLoaderData()` with no argument clears every route's cached data.\n- `invalidateLoaderData('/posts')` clears just that one route's entry.\n\nYou rarely need it directly; `revalidate()` (which pairs it with a re-render) is what you usually want. Reach for the primitive when you want to mark data stale *now* but let a later navigation do the actual refetch:\n\n```tsx\n// After a background sync finished, mark the dashboard stale so its\n// next visit refetches, without disturbing the page the user is on.\nToil.invalidateLoaderData('/dashboard');\n```\n\nSignature: `invalidateLoaderData(href?: string): void`.\n\n## Forms\n\nFor the common \"submit a form, then refresh the page's data\" loop, use `Toil.Form`. It runs an async action on submit (no page reload), tracks pending and error state, and revalidates the route's loader data on success:\n\n```tsx\nimport { NewMessage } from 'shared/server';\n\nexport default function Guestbook() {\n const entries = Toil.useLoaderData<typeof loader>();\n\n const sign = async (data: FormData) => {\n const author = String(data.get('author'));\n const message = String(data.get('message'));\n await Server.REST.guestbook.sign({ body: new NewMessage(author, message) });\n };\n\n return (\n <Toil.Form action={sign} resetOnSuccess>\n {({ pending }) => (\n <>\n <input name=\"author\" />\n <input name=\"message\" />\n <button disabled={pending}>Sign</button>\n </>\n )}\n </Toil.Form>\n );\n}\n```\n\nKey `Toil.Form` props:\n\n| Prop | Default | What it does |\n| --- | --- | --- |\n| `action` | (required) | Runs on submit, receiving the form's `FormData`. May be async. |\n| `revalidate` | `true` (current route) | Which loader data to refetch after a successful submit. |\n| `resetOnSuccess` | `false` | Clear the form fields after success. |\n| `onSuccess` / `onError` | | Callbacks for the two outcomes. |\n\nPassing a **render function** as `children` gives you live submit state (`pending`, `error`), so you can disable the button while the request is in flight. On success, `Form` revalidates the loader, so the page's data updates automatically without a manual refetch.\n\n## Mutations without a form: `useAction`\n\n`Toil.Form` is convenient sugar over a more general primitive, `Toil.useAction`. Use `useAction` for any write that is not a form submit: a delete button, a \"like\" toggle, a \"mark as read\" action. It runs your async function on demand and tracks the whole lifecycle (`pending`, `error`, `data`), then revalidates the affected loader data on success so the page reflects the change, exactly like `Form` does.\n\nYou call it with the function to run and an optional options object, and it hands back a handle:\n\n```tsx\nexport default function PostRow({ id }: { id: number }) {\n const del = Toil.useAction(\n (postId: number) => Server.REST.posts.remove({ params: { id: postId } }),\n { revalidate: true, onError: (e) => alert(parseError(e)) },\n );\n\n return (\n <button disabled={del.pending} onClick={() => void del.run(id)}>\n {del.pending ? 'Deleting...' : 'Delete'}\n </button>\n );\n}\n```\n\nThe handle it returns:\n\n| Field | What it is |\n| --- | --- |\n| `run(input)` | Runs the action. Resolves to the result, or `undefined` if it threw. The error is captured in `error` rather than rejected, so a fire-and-forget `onClick` cannot leak an unhandled rejection. |\n| `pending` | `true` while a run is in flight. Drive a spinner or a disabled button off this. |\n| `error` | The error from the last failed run, or `undefined`. |\n| `data` | The value the last successful run returned, or `undefined`. |\n| `reset()` | Clear `pending` / `error` / `data` back to idle. |\n\nThe options mirror `Toil.Form`:\n\n| Option | Default | What it does |\n| --- | --- | --- |\n| `revalidate` | `true` (current route) | Which loader data to refetch after a successful run: `true` (the active route), an `href` or array of hrefs (those routes), or `false` (none). |\n| `onSuccess` | | Called with the action's return value after a successful run. |\n| `onError` | | Called with the thrown value when the action fails. |\n\n`Toil.Form` is just this hook wired to a `<form>`'s submit event, with the form's `FormData` passed as the input. Anything a form does, you can do by hand with `useAction`.\n\n## Reading who is logged in\n\nTo render \"logged in as ...\" you need the current user. toiljs generates a `getUser()` helper (from the backend's `@user` surface) that reads the current user from a readable session cookie with no network round-trip. It is instant, which makes it perfect for display, but it is **untrusted**: a value read on the client can be forged, so never gate anything security-sensitive on it. For a trusted check, call a guarded backend route that re-verifies the signed session.\n\nThe full auth flow (post-quantum login, sessions, `getUser()`, and guarding routes) is its own guide:\n\n- [Auth usage](../auth/usage.md): reading the session, `getUser()`, and guarding pages.\n\n## RPC (`@service` / `@remote`)\n\nAlongside REST, toiljs generates a typed RPC surface for `@service` methods and free `@remote` functions. These read like plain function calls with no URL:\n\n```tsx\nconst n = await Server.ping(10); // a free @remote\nconst count = await Server.stats.playerCount(); // a @service method\n```\n\nThe types come straight from your server, so `Server.ping` knows it takes a number and returns a number. Under the hood each call encodes its arguments and POSTs them to a single reserved endpoint (`/__toil_rpc`) with a compact method id, then decodes the typed result. Both the local dev server and the production edge dispatch this endpoint, so RPC works end to end.\n\nIf a call throws an \"unavailable\" error, it means the generated client has not attached yet: build the server (`npm run build:server`) to regenerate `shared/server.ts`, and import from `shared/server` so the client loads (see the gotchas below). See [Typed RPC](../backend/rpc.md) for the backend side and when to choose RPC over REST.\n\n## Gotchas\n\n- **Import from `shared/server` to attach the clients.** `Server.REST` (and the RPC client) attach when you import from `shared/server`. Importing your `@data` classes from there does it naturally; if `Server.REST.foo()` throws an \"unavailable\" error, make sure the server has been built (`npm run build:server`) and that `shared/server` is imported.\n- **The server runs a fresh instance per request.** In the examples, in-memory writes are previews that do not persist. To keep data, write it to the database (see [Database](../database/README.md)). This is a backend property, but it explains why a create returns a value that is not there on the next request.\n- **Fetch page data in a `loader`, not `useEffect`.** A loader runs in parallel with the route chunk and integrates with `loading.tsx`, suspense, caching, and SSR hydration. A `useEffect` fetch runs only after the page mounts (slower, and invisible to the server).\n- **`getUser()` is display-only.** It is fast because it does not verify anything. Do real authorization on the server.\n\n## Related\n\n- [Routing](./routing.md): loaders, `loading.tsx`, and navigation.\n- [Backend HTTP routes](../backend/rest.md): the `@rest` controllers behind `Server.REST`.\n- [Typed RPC](../backend/rpc.md): the `@service` / `@remote` surface behind `Server`.\n- [Data types](../backend/data.md): the `@data` classes that cross the wire.\n- [Auth usage](../auth/usage.md): sessions and `getUser()`.\n",
|
|
38
39
|
"frontend/images.md": "# Images\n\n`Toil.Image` is a drop-in replacement for `<img>` that stops layout shift, lazy-loads by default, and can fade in from a blurred placeholder. Use it for content images instead of a raw `<img>`.\n\n## Why not just use `<img>`?\n\nTwo problems with a plain `<img>`:\n\n1. **Layout shift.** Before the image loads, the browser does not know how tall it is, so the page has no space reserved. When the image arrives, everything below it jumps down. That jump hurts the user experience and your Core Web Vitals score (specifically CLS, Cumulative Layout Shift).\n2. **Loading cost.** Every image loads eagerly by default, competing for bandwidth with the things the user actually needs first.\n\n`Toil.Image` fixes both: it reserves the right amount of space up front, and it lazy-loads everything except the images you mark as high priority.\n\n## The simplest usage\n\nGive it a `src`, an `alt`, and a `width` + `height`. The width and height are the image's real pixel dimensions, and they are what reserve space so nothing jumps:\n\n```tsx\nexport default function Post() {\n return (\n <Toil.Image\n src=\"/images/diagram.png\"\n alt=\"How the request flows\"\n width={800}\n height={400}\n />\n );\n}\n```\n\n`alt` is required (toiljs enforces it for accessibility). For a purely decorative image, pass `alt=\"\"`.\n\nThere is no server-side resizing here: `Toil.Image` is a client component. Point `src` at an already-optimized image. When you import an image (see below), Vite hashes and optimizes the file for you.\n\n## Automatic blur placeholders with `?toil`\n\nFor a nicer loading experience you can show a tiny blurred preview of the image while the full one downloads. toiljs generates that preview automatically when you import the image with a `?toil` flag:\n\n```tsx\nimport hero from './hero.webp?toil';\n\nexport default function Landing() {\n return <Toil.Image src={hero} alt=\"Product hero\" placeholder=\"blur\" />;\n}\n```\n\nThe `?toil` import does not give you a plain URL string. It gives you an object:\n\n```ts\n{\n src: string; // the optimized, hashed image URL\n width: number; // the image's intrinsic width\n height: number; // the image's intrinsic height\n blurDataURL: string; // a tiny base64 blurred preview, inlined\n}\n```\n\n`Toil.Image` unpacks that object for you. So a `?toil` import auto-fills `width`, `height`, and the blur placeholder with **no extra props**: you do not repeat the dimensions, and `placeholder=\"blur\"` just works. Explicit props still win if you pass them.\n\nBehind the scenes, at import time toiljs uses `sharp` to downscale the image to about 24 pixels on its longest edge, blur it, and encode it as a small WebP data URI (a few hundred bytes) inlined right into your bundle. This runs in dev and in the build. Only raster images (`.png`, `.jpg`, `.webp`, `.avif`, `.gif`, `.tiff`) get a blur; SVGs and animated images are left as-is.\n\n### The skeleton fallback\n\n`placeholder=\"blur\"` is never a silent no-op. If there is no `blurDataURL` (you used a string `src` and did not pass one), `Toil.Image` shows a neutral animated \"skeleton shimmer\" instead of a blur. Either way the placeholder needs a reserved box to paint into, so give it `width` + `height` (or use `fill`, below).\n\nYou can also pass a `blurDataURL` by hand for a plain string `src`:\n\n```tsx\n<Toil.Image\n src=\"/images/photo.jpg\"\n alt=\"A photo\"\n width={1200}\n height={800}\n placeholder=\"blur\"\n blurDataURL=\"data:image/webp;base64,UklGR...\"\n/>\n```\n\n## The real layout-shift fix: aspect ratio\n\nThe important detail: the thing that actually prevents layout shift is not the blur, it is the **reserved aspect ratio**. When you pass both `width` and `height`, `Toil.Image` sets a CSS `aspect-ratio` derived from them. That reservation survives responsive CSS like `width: 100%` (a bare `width`/`height` attribute does not survive that). So the box holds its shape while the image loads, and nothing jumps. The blur is cosmetic; the reserved size is what holds.\n\nThe takeaway: always give `width` + `height` (or `fill`). A `?toil` import supplies them for you.\n\n## `fill`: sizing from the container\n\nSometimes you do not know the image's size, you want it to fill a box you control (a card, a banner). Pass `fill`:\n\n```tsx\n<div style={{ maxWidth: 480 }}>\n <Toil.Image src={photo} alt=\"Cover\" fill />\n</div>\n```\n\nWith `fill`, `Toil.Image` wraps the image in a block-level box and the image fills that box's width, scaling to its natural height. If you size the box yourself (with `width`/`height`, or a `style` like `aspectRatio: '16 / 9'`), the image *covers* that box, cropped according to `objectFit` (default `cover`):\n\n```tsx\n<Toil.Image\n src={photo}\n alt=\"Cover\"\n fill\n style={{ aspectRatio: '16 / 9' }}\n objectFit=\"cover\"\n/>\n```\n\nThe `fill` image flows in the normal document layout (it is never absolutely positioned), so it cannot escape to cover the whole page nor collapse to zero height, two common bugs with hand-rolled fill images. With `fill`, your `className` and `style` apply to the wrapper box, not the raw `<img>`.\n\nUse fixed `width`/`height` when you know the image's size; use `fill` when the layout decides the size.\n\n## Priority images (above the fold)\n\nBy default every `Toil.Image` lazy-loads: the browser fetches it only as it nears the viewport. That is right for most images, but wrong for the one big image at the top of the page (your hero, your LCP element). Mark that one `priority`:\n\n```tsx\n<Toil.Image src={hero} alt=\"Hero\" width={1200} height={630} priority />\n```\n\n`priority` loads the image eagerly and sets `fetchPriority=\"high\"`, telling the browser to fetch it right away. Use it only for the important above-the-fold image; everything else should stay lazy.\n\n## Prop reference\n\n| Prop | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `src` | `string \\| ToilImageSource` | (required) | A URL, or a `?toil` import object. |\n| `alt` | `string` | (required) | Alt text. Use `\"\"` for decorative images. |\n| `width` / `height` | `number` | from `?toil` | Intrinsic size, reserves space. |\n| `fill` | `boolean` | `false` | Fill the container instead of using fixed size. |\n| `objectFit` | CSS `object-fit` | `cover` (with fill) | How the image fits its box. |\n| `priority` | `boolean` | `false` | Eager load + high fetch priority (above-the-fold). |\n| `placeholder` | `'empty' \\| 'blur'` | `'empty'` | Show a blur / skeleton while loading. |\n| `blurDataURL` | `string` | from `?toil` | The tiny preview for `placeholder=\"blur\"`. |\n\nEvery other standard `<img>` attribute (`className`, `style`, `onLoad`, `sizes`, `srcSet`, `id`, `data-*`) passes straight through.\n\n## Gotchas\n\n- **No server-side resizing.** `Toil.Image` does not shrink or convert your image at request time. Ship an appropriately sized `src` (importing it lets Vite optimize and hash it). The `?toil` blur is only a placeholder, not a resized copy.\n- **A placeholder needs a reserved box.** `placeholder=\"blur\"` only shows if there is a size to paint into. Give `width` + `height`, or use `fill` inside a sized parent.\n- **`?toil` is for raster images.** SVGs and animated formats do not get a generated blur (importing them with `?toil` skips the blur step). Import an SVG normally.\n- **`alt` is mandatory.** This is intentional. For decorative images that add no meaning, pass `alt=\"\"` so screen readers skip them.\n- **Do not double up dimensions with a `?toil` import.** The import already carries `width`/`height`; passing them again is redundant (though harmless, explicit props win).\n\n## Related\n\n- [Styling](./styling.md): importing images and CSS in general.\n- [Metadata and SEO](./metadata.md): setting `og:image` for social-share previews.\n- [Rendering and SSR](./rendering.md): why first-paint layout stability matters.\n",
|
|
39
40
|
"frontend/metadata.md": "# Metadata and SEO\n\nMetadata is the information in a page's `<head>`: its title, its description, and the tags that control how it looks when shared on social media or listed by a search engine. toiljs lets you set all of it per route, and bakes it into real HTML so crawlers see it even without running your JavaScript.\n\n## The quick version\n\nFor most pages, `export const metadata` from the route file. That is it:\n\n```tsx\n// client/routes/features/seo.tsx\nexport const metadata: Toil.Metadata = {\n title: 'useReducer | React Hooks',\n description: 'Manage complex state transitions with a reducer function.',\n keywords: ['react', 'hooks', 'useReducer'],\n canonical: 'https://example.com/features/seo',\n openGraph: {\n title: 'useReducer | React Hooks',\n description: 'Manage complex state transitions with a reducer.',\n type: 'website',\n },\n};\n\nexport default function SeoDemo() {\n return <main><h1>Route metadata</h1></main>;\n}\n```\n\nThe router applies this before the page paints (so the tab title updates with no flicker), and the build bakes it into the page's static HTML so search engines and link-preview bots read it directly.\n\n## The `Metadata` fields\n\n`Toil.Metadata` maps friendly fields onto the right `<meta>` and `<link>` tags for you:\n\n| Field | Becomes |\n| --- | --- |\n| `title` | The document `<title>`. |\n| `description` | `<meta name=\"description\">`. |\n| `keywords` | `<meta name=\"keywords\">` (an array is joined with commas). |\n| `canonical` | `<link rel=\"canonical\">`. |\n| `robots` | `<meta name=\"robots\">`, e.g. `'noindex, nofollow'`. |\n| `themeColor` | `<meta name=\"theme-color\">` (the accent color of some link embeds). |\n| `openGraph` | The `og:*` tags (title, description, type, url, image, siteName). |\n| `meta` | Escape hatch: extra raw `<meta>` tags. |\n| `link` | Escape hatch: extra raw `<link>` tags. |\n\nOpen Graph (the `og:*` tags) is the shared standard that Facebook, Discord, Slack, LinkedIn, and iMessage read to build a link preview card. Set `openGraph.image` (an absolute URL, ideally at least 1200 by 630 pixels) to control the preview picture:\n\n```tsx\nexport const metadata: Toil.Metadata = {\n title: 'Our launch',\n description: 'Read the announcement.',\n openGraph: {\n title: 'Our launch',\n description: 'Read the announcement.',\n type: 'article',\n image: 'https://example.com/og/launch.png',\n },\n};\n```\n\n## Dynamic metadata: `generateMetadata`\n\nWhen the title depends on the URL or on fetched data (a blog post's title, a product name), export `generateMetadata` instead of a static object. It receives the route params, the query, and the route loader's data, and returns a `Metadata`:\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const generateMetadata: Toil.GenerateMetadata = ({ params }) => ({\n title: `Blog post ${params.id}`,\n description: `Reading blog post ${params.id}.`,\n});\n```\n\nNow `/blog/42` sets the tab to \"Blog post 42\". If your route has a `loader`, its data is passed in as `data`, so you can title a page from the content it loaded:\n\n```tsx\nexport const loader = async ({ params }: Toil.LoaderArgs) =>\n Server.REST.blog.get({ params: { id: params.id } });\n\nexport const generateMetadata: Toil.GenerateMetadata = ({ data }) => ({\n title: data.title,\n description: data.excerpt,\n});\n```\n\n## Imperative and stateful head: `useHead`, `useTitle`, `<Head>`\n\nSometimes the head depends on component state, not on the route. For that, set it from inside a component with `Toil.useHead`, `Toil.useTitle`, or the declarative `<Toil.Head>`. These apply for the component's lifetime and revert when it unmounts:\n\n```tsx\nexport default function HeadDemo() {\n const [count, setCount] = useState(0);\n\n // The tab title updates every render as count changes.\n Toil.useTitle(`Clicked ${count} times`);\n\n Toil.useHead({\n meta: [{ name: 'description', content: `Clicked ${count} times.` }],\n });\n\n return <button onClick={() => setCount((c) => c + 1)}>Clicked {count}</button>;\n}\n```\n\nThe declarative form renders nothing and is equivalent:\n\n```tsx\n<Toil.Head\n title=\"Blog\"\n meta={[{ name: 'description', content: 'Latest posts' }]}\n/>\n```\n\nUse `useHead`/`useTitle` when the value is dynamic or lives in component state; use the `metadata` export when it is a static property of the route.\n\n### Applying a whole `Metadata` object from a component: `useMetadata`\n\n`useHead` takes raw `<meta>` and `<link>` tags. When you would rather pass the same friendly `Metadata` shape you use in a route's `metadata` export (with `title`, `openGraph`, `keywords`, and the rest), use `Toil.useMetadata` from inside any component. It applies for that component's lifetime and reverts on unmount, exactly like `useHead`. This is the tool for content that is not itself a route file: a reusable article component, a widget, a search-result view.\n\n```tsx\nexport default function Article({ post }: { post: Post }) {\n Toil.useMetadata({\n title: post.title,\n description: post.excerpt,\n openGraph: { type: 'article', title: post.title, image: post.cover },\n });\n\n return <article>{/* ... */}</article>;\n}\n```\n\nThere is a declarative twin, `<Toil.Metadata title=\"...\" openGraph={...} />`, which renders nothing (the component-level counterpart of a route's `metadata` export). A route's own `metadata` export is still merged last (highest priority), so it wins for the keys it sets, and `useMetadata` fills in for routes that declare none.\n\n> **Advanced.** Under the hood, `Toil.resolveMetadata(metadata)` is the pure function that expands a `Metadata` object into concrete `<meta>`/`<link>` tags (a `HeadSpec`), and `Toil.mergeHead(specs)` is the pure merge that resolves all active head contributions into the final result (the last `title` wins, `meta` dedupes by `name`/`property`, `link` by `rel` + `href`). You rarely call these directly, but they are exported for tooling and tests.\n\n## How the pieces combine\n\nMultiple things can contribute to the head at once: your site-wide defaults, a layout, and the page. They merge by key, with a clear priority. Later, more specific contributions win per key, and anything left unset falls through to a broader default:\n\n```mermaid\nflowchart TB\n A[\"Site-wide defaults<br/>client.seo config\"] --> M{\"merge by key\"}\n B[\"Root layout <Toil.Head><br/>(fallback title, description)\"] --> M\n C[\"Component useHead / useTitle\"] --> M\n D[\"Route metadata / generateMetadata<br/>(highest priority)\"] --> M\n M --> R[\"The page's final <head>\"]\n```\n\n- A **root layout** is the natural home for site-wide fallbacks (a default title and description for any page that sets none):\n\n ```tsx\n // client/layout.tsx\n <Toil.Head\n title=\"My Site\"\n meta={[{ name: 'description', content: 'Planet-scale apps.' }]}\n />\n ```\n\n- A **route's `metadata`** overrides those defaults for the keys it sets, while the layout still fills in everything the route leaves unset. So a page can set just a `title` and inherit the site description.\n\nThe rule of thumb: put fallbacks in the layout, put the specifics on the route.\n\n## Build-time SEO for the whole site\n\nBeyond per-route metadata, toiljs generates site-level SEO assets at build time from a `client.seo` block in `toil.config.ts`. These are baked into the HTML `<head>` and written as files, so JavaScript-less crawlers and AI bots get correct information:\n\n```ts\n// toil.config.ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n seo: {\n url: 'https://example.com', // required for canonical/OG urls, sitemap\n title: 'My Site',\n description: 'Planet-scale apps from a single repo.',\n openGraph: {\n type: 'website',\n siteName: 'My Site',\n image: 'https://example.com/og.png',\n },\n twitter: { card: 'summary_large_image', site: '@mysite' },\n robots: { ai: 'allow' }, // allow or disallow known AI crawlers\n llms: { instructions: 'Docs live at /docs.' },\n jsonLd: { '@context': 'https://schema.org', '@type': 'WebSite', name: 'My Site' },\n },\n },\n});\n```\n\nFrom this one block, the build:\n\n- bakes the default `<title>`, `description`, Open Graph, Twitter card, and JSON-LD structured data into every page's HTML;\n- overlays each route's own `metadata` on top and points that route's canonical and `og:url` at its own URL;\n- generates `robots.txt` (with directives for AI crawlers like GPTBot and ClaudeBot), `sitemap.xml` (from your static routes), and `llms.txt` (a guidance file for AI crawlers).\n\nYou get correct, per-page SEO in the raw HTML with almost no manual tag writing. Confirm it with \"View source\" on a built page: the real title and tags are right there, not injected later by JavaScript.\n\n### The complete `client.seo` reference\n\nEvery field `client.seo` accepts is below. All of them are optional, and everything you set becomes a baked-in `<head>` tag or a generated file (`robots.txt`, `sitemap.xml`, `llms.txt`).\n\n**Top level:**\n\n| Field | Type | What it does |\n| --- | --- | --- |\n| `url` | `string` | Absolute site base URL, e.g. `'https://example.com'`. Unlocks `sitemap.xml`, the canonical `<link>`, and absolute Open Graph / Twitter URLs. |\n| `title` | `string` | Default document `<title>`. |\n| `description` | `string` | Default `<meta name=\"description\">`. |\n| `robotsMeta` | `string` | Default `<meta name=\"robots\">`, e.g. `'index, follow'`. (This is the meta tag; the `robots` field below is the separate `robots.txt` file.) |\n| `themeColor` | `string` | `<meta name=\"theme-color\">`, also the accent color of some Discord / Slack link embeds. |\n| `openGraph` | object | Open Graph (`og:*`) defaults, see below. |\n| `twitter` | object | Twitter / X card, see below. |\n| `facebook` | `{ appId?: string }` | `appId` renders `<meta property=\"fb:app_id\">`. Open Graph covers the rest of the Facebook card. |\n| `preconnect` | `string[]` | Origins to `<link rel=\"preconnect\">` (early connection hints). |\n| `dnsPrefetch` | `string[]` | Origins to `<link rel=\"dns-prefetch\">`. |\n| `jsonLd` | object or object[] | JSON-LD structured data injected as `<script type=\"application/ld+json\">`. Pass an array to include several nodes (they are serialized into one `<script>` as a JSON array). |\n| `robots` | object or `false` | `robots.txt` generation, see below. `false` skips the file. |\n| `sitemap` | `boolean` | `sitemap.xml` generation. On by default when `url` is set; `false` skips it. |\n| `llms` | object or `boolean` | `llms.txt` (AI-crawler guidance) generation. `false` skips it, `true` or an object configures it. |\n\n**`openGraph`** (the `og:*` tags; `title` and `description` fall back to the top-level values):\n\n| Field | Type | Renders |\n| --- | --- | --- |\n| `title` | `string` | `og:title` |\n| `description` | `string` | `og:description` |\n| `type` | `string` | `og:type`, e.g. `'website'` or `'article'` (defaults to `'website'`). |\n| `siteName` | `string` | `og:site_name` |\n| `locale` | `string` | `og:locale`, e.g. `'en_US'`. |\n| `image` | `string` | `og:image`, the preview picture (absolute URL, ideally at least 1200 by 630 pixels). |\n| `imageAlt` | `string` | `og:image:alt` |\n| `imageWidth` | `number` | `og:image:width` in pixels (lets Facebook / LinkedIn render without a re-fetch). |\n| `imageHeight` | `number` | `og:image:height` in pixels. |\n| `imageType` | `string` | `og:image:type`, e.g. `'image/png'`. |\n\n**`twitter`** (the Twitter / X card; unset fields fall back to the Open Graph or top-level values):\n\n| Field | Type | Renders |\n| --- | --- | --- |\n| `card` | `string` | `twitter:card`: `'summary'` or `'summary_large_image'` (defaults by whether an image is present). |\n| `site` | `string` | `twitter:site`, the site's `@handle`. |\n| `creator` | `string` | `twitter:creator`, the author's `@handle`. |\n| `title` | `string` | `twitter:title` (falls back to `openGraph.title` / `title`). |\n| `description` | `string` | `twitter:description` (falls back to `openGraph.description` / `description`). |\n| `image` | `string` | `twitter:image` (falls back to `openGraph.image`). |\n| `imageAlt` | `string` | `twitter:image:alt` (falls back to `openGraph.imageAlt`). |\n\n**`robots`** (the `robots.txt` file; set `robots: false` to skip it):\n\n| Field | Type | What it does |\n| --- | --- | --- |\n| `rules` | `RobotsRule[]` | Custom `User-agent` groups. Each rule is `{ userAgent?: string \\| string[], allow?: string[], disallow?: string[] }`. Defaults to one group allowing everything (`User-agent: *`, `Allow: /`). |\n| `ai` | `'allow'` or `'disallow'` | How to treat known AI crawlers (GPTBot, ClaudeBot, Google-Extended, and more). Default `'allow'`. |\n| `sitemap` | `string` | Explicit `Sitemap:` line (defaults to `<url>/sitemap.xml` when `url` is set). |\n\n**`llms`** (the `llms.txt` guidance file; set `llms: false` to skip it, `llms: true` for defaults):\n\n| Field | Type | What it does |\n| --- | --- | --- |\n| `title` | `string` | The file's heading (falls back to `seo.title`). |\n| `summary` | `string` | The summary blockquote (falls back to `seo.description`). |\n| `instructions` | `string` | Free-form guidance for AI / LLM crawlers. |\n| `pages` | `LlmsPage[]` | Key pages, each `{ title: string, url: string, description?: string }`. Defaults to the site's static routes. |\n\nA fully worked config touching most of these fields:\n\n```ts\n// toil.config.ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n client: {\n seo: {\n url: 'https://example.com', // base URL: unlocks sitemap, canonical, absolute OG/Twitter URLs\n title: 'My Site', // default <title>\n description: 'Planet-scale apps.', // default <meta name=\"description\">\n robotsMeta: 'index, follow', // default <meta name=\"robots\">\n themeColor: '#0b0b0f', // <meta name=\"theme-color\">\n\n openGraph: {\n type: 'website', // og:type\n siteName: 'My Site', // og:site_name\n locale: 'en_US', // og:locale\n image: 'https://example.com/og.png', // og:image (absolute, ideally 1200x630)\n imageAlt: 'My Site preview', // og:image:alt\n imageWidth: 1200, // og:image:width\n imageHeight: 630, // og:image:height\n imageType: 'image/png', // og:image:type\n },\n\n twitter: {\n card: 'summary_large_image', // twitter:card\n site: '@mysite', // twitter:site\n creator: '@ada', // twitter:creator\n // title / description / image / imageAlt fall back to openGraph + top level\n },\n\n facebook: { appId: '1234567890' }, // <meta property=\"fb:app_id\">\n\n preconnect: ['https://cdn.example.com'], // <link rel=\"preconnect\">\n dnsPrefetch: ['https://analytics.example.com'], // <link rel=\"dns-prefetch\">\n\n robots: {\n ai: 'allow', // known AI crawlers: 'allow' | 'disallow'\n rules: [ // custom User-agent groups for robots.txt\n { userAgent: '*', allow: ['/'], disallow: ['/admin'] },\n ],\n // sitemap: 'https://example.com/sitemap.xml', // explicit line (auto from url otherwise)\n },\n sitemap: true, // generate sitemap.xml (on by default when url is set)\n\n llms: { // llms.txt (AI-crawler guidance)\n title: 'My Site',\n summary: 'Planet-scale apps from a single repo.',\n instructions: 'Docs live at /docs.',\n // pages: [{ title: 'Docs', url: 'https://example.com/docs', description: 'Guides' }],\n },\n\n jsonLd: [ // array = several nodes in one <script>\n { '@context': 'https://schema.org', '@type': 'WebSite', name: 'My Site' },\n { '@context': 'https://schema.org', '@type': 'Organization', name: 'My Site' },\n ],\n },\n },\n});\n```\n\n## Per-request titles on server-rendered pages\n\nFor a page that is server-rendered (`export const ssr = true`), the backend can set a fresh `<title>` for each individual request (for example, a search page titled with the query the visitor typed). The edge splices that per-request title into the document before sending it, so the correct title is in the very first byte of HTML. This is a server-side API used in your `render` function; the frontend `metadata` above covers everything else. See [Rendering and SSR](./rendering.md) for how SSR pages are assembled.\n\n## Gotchas\n\n- **`generateMetadata` needs the data available.** It runs with the route loader's data, so it can only use what the loader returns. Fetch what the title needs in the loader.\n- **Open Graph images must be absolute URLs.** A relative `/og.png` will not resolve for an external crawler. Use the full `https://...` URL (set `seo.url` and it can build them for you).\n- **`seo.url` unlocks the site-level assets.** `sitemap.xml`, canonical links, and absolute OG urls all need the site's base `url`. Set it once in the config.\n- **Static export vs runtime.** The `metadata` export is baked into HTML at build (great for crawlers) *and* applied at runtime on navigation. `useHead` runs only at runtime; a crawler that does not execute JS will not see it. Prefer the `metadata` export for anything that matters for SEO.\n\n## Related\n\n- [Rendering and SSR](./rendering.md): how the baked head and SSR title reach the browser.\n- [Images](./images.md): producing the `og:image` for share previews.\n- [Routing](./routing.md): where `metadata` and `generateMetadata` live in a route file.\n- [Configuration](../concepts/config.md): the full `toil.config.ts` reference.\n",
|
|
40
41
|
"frontend/navigation.md": "# Navigation\n\nOnce your files are turned into URLs (see [Routing](./routing.md)), you need a way to move between them. This page is about that half: the links, hooks, and functions that navigate a user from one page to the next without a full reload, keep the current section highlighted, prefetch what is coming, and restore scroll. Everything here lives on the global `Toil` object, so route files need no imports for the common cases.\n\nRouting is \"file to URL\"; navigation is \"get me to that URL\". If you are looking for how a file becomes a page, or for dynamic params and layouts, that is the [Routing](./routing.md) page.\n\n## `Toil.Link`\n\n`Toil.Link` is the client-side replacement for a plain `<a>`. It navigates in place (no full page reload), and it prefetches the target route's chunk on hover or focus, so the click feels instant:\n\n```tsx\n<Toil.Link href=\"/about\">About</Toil.Link>\n```\n\n`Link` accepts every standard anchor attribute (`className`, `style`, `target`, `rel`, `download`, `referrerPolicy`, `ref`, `data-*`, `aria-*`, event handlers, and so on), plus a few toiljs controls:\n\n| Prop | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `href` | `Href` | (required) | Destination, typed to your project's real routes (a typo is a compile error). |\n| `replace` | `boolean` | `false` | Replace the current history entry instead of pushing a new one. |\n| `scroll` | `boolean` | `true` | Scroll to the top after navigating. `false` keeps the current position. |\n| `prefetch` | `boolean` | `true` | Prefetch the route chunk on hover/focus. `false` opts this link out. |\n\n### When `Link` does not intercept\n\n`Link` is deliberate about when to hand a click back to the browser. It only intercepts a plain, same-origin, left-click. All of these fall through to native browser behavior instead:\n\n- **External URLs** (a different origin), and opaque targets like `mailto:` or `tel:`.\n- **`target` other than `_self`** (for example `target=\"_blank\"`, a new tab).\n- **`download`** links.\n- **In-page `#hash`-only** links.\n- **Modified clicks**: middle-click or any click with Ctrl, Cmd (Meta), Shift, or Alt held (so \"open in new tab\" keeps working).\n\nBecause of this, you can point a `Link` at an external site or a download and it behaves exactly like an `<a>` would, no special-casing on your part. For genuinely external links a plain `<a>` is still fine; reach for `Link` when the target is one of your own routes.\n\n## `Toil.NavLink` and active state\n\n`Toil.NavLink` is a `Link` that knows whether it points at the current page. When active it adds a class (`active` by default) and sets `aria-current=\"page\"`, which is exactly what a navigation bar wants for highlighting the current section. It inherits Link's full anchor API and its prefetching.\n\nOn top of `LinkProps` it adds:\n\n| Prop | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `end` | `boolean` | `false` | Require an exact match. Without it, a parent link is active for its sub-paths. |\n| `activeClassName` | `string` | `'active'` | The class added when active (used with a string `className`). |\n| `className` | `string \\| (state) => string \\| undefined` | (none) | A string, or a function of `{ isActive }`. |\n| `style` | `CSSProperties \\| (state) => CSSProperties \\| undefined` | (none) | A style object, or a function of `{ isActive }`. |\n| `children` | `ReactNode \\| (state) => ReactNode` | (none) | Content, or a function of `{ isActive }`. |\n\nA simple nav bar with string classes:\n\n```tsx\n// client/components/Nav.tsx\nexport default function Nav() {\n return (\n <nav>\n <Toil.NavLink href=\"/\" end>Home</Toil.NavLink>\n <Toil.NavLink href=\"/blog\">Blog</Toil.NavLink>\n <Toil.NavLink href=\"/about\" activeClassName=\"is-current\">About</Toil.NavLink>\n </nav>\n );\n}\n```\n\nBy default a parent link stays active for its sub-paths, so `/blog` is active on `/blog`, `/blog/42`, and `/blog/42/edit`. Pass `end` to require an exact match. This matters most for the home link: `/` would otherwise be active on every page, so `Home` almost always wants `end`.\n\nThe function forms let the active state drive `className`, `style`, or `children` directly. Each receives a `{ isActive }` object:\n\n```tsx\n<Toil.NavLink href=\"/blog\" className={({ isActive }) => (isActive ? 'tab on' : 'tab')}>\n {({ isActive }) => <span>{isActive ? '• ' : ''}Blog</span>}\n</Toil.NavLink>\n```\n\n## Navigating in code\n\nNot every navigation is a click. After a form submit, a login, or a `fetch`, you often want to move the user yourself. There are three ways, from smallest to fullest.\n\n### `Toil.useNavigate()`\n\nThe hook returns the bare `navigate(href, options)` function:\n\n```tsx\nexport default function Login() {\n const navigate = Toil.useNavigate();\n return (\n <button onClick={() => navigate('/dashboard', { replace: true })}>\n Continue\n </button>\n );\n}\n```\n\nThe options are `NavigateOptions`, the same two controls `Link` exposes:\n\n| Option | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `replace` | `boolean` | `false` | Replace the current history entry instead of pushing. |\n| `scroll` | `boolean` | `true` | Scroll to top after navigating. `false` keeps the position. |\n\n### `Toil.navigate` (outside React)\n\nThe exact same function is available free of any hook as `Toil.navigate(href, options)`. Use it in code that is not a React component: a plain event handler, a utility module, or right after a `fetch` resolves:\n\n```tsx\nasync function saveDraft(draft: Draft) {\n await Server.REST.posts.create({ body: draft });\n Toil.navigate('/posts');\n}\n```\n\n`Toil.back()`, `Toil.forward()`, and `Toil.refresh()` are the history counterparts, also callable from anywhere: `back` and `forward` step through history, and `refresh` re-renders the current route (re-running its loader for the current URL).\n\n### `Toil.useRouter()`\n\nFor a full imperative handle, `useRouter()` returns a `RouterInstance`:\n\n```tsx\nexport default function PostActions() {\n const router = Toil.useRouter();\n return (\n <>\n <button onClick={() => router.push('/blog/new')}>New post</button>\n <button onClick={() => router.back()}>Back</button>\n <button onClick={() => router.revalidate()}>Refresh data</button>\n </>\n );\n}\n```\n\n| Method | What it does |\n| --- | --- |\n| `push(href, options?)` | Navigate to `href`, pushing a new history entry (or replacing with `{ replace: true }`). |\n| `replace(href)` | Navigate to `href`, replacing the current history entry. |\n| `back()` | Go back one history entry. |\n| `forward()` | Go forward one history entry. |\n| `refresh()` | Re-render the current route and re-run its loader (clears all cached loader data). |\n| `revalidate(href?)` | Invalidate cached loader data and re-render so it refetches. No argument targets the active route; pass an `href` for a specific route. Use after a mutation. |\n| `prefetch(href)` | Warm a route's chunk ahead of navigation. |\n\nReach for `revalidate()` after a write (you changed some data and want the current page's loader to refetch), and `refresh()` when you want a clean re-run of everything. `push`/`replace` are the imperative twins of a `Link` click.\n\n## Typed hrefs and the `href()` escape hatch\n\nEvery `href` you pass to `Toil.Link`, `NavLink`, `navigate`, or `router.push` is type-checked against your project's real routes. The compiler scans `client/routes/` and generates a `toil-routes.d.ts` that narrows the `Href` type to the union of your actual paths, so a typo is a compile error before you run the app:\n\n```tsx\n<Toil.Link href=\"/abuot\">About</Toil.Link>\n// ^ Type error: \"/abuot\" is not assignable to type 'Href'. (No such route.)\n```\n\nDynamic routes appear in that union as template-literal types, so a file at `client/routes/blog/[id].tsx` contributes `` `/blog/${string}` `` and a single interpolation still checks:\n\n```tsx\n<Toil.Link href={`/blog/${post.id}`}>Read</Toil.Link> // fine\n```\n\nWhen a URL is assembled from several data pieces (or from values TypeScript cannot prove the shape of), it is typed as a plain `string`, and `string` is not assignable to `Href`. That is when you reach for `Toil.href()`, the escape hatch that asserts a runtime string is a valid href:\n\n```tsx\nconst path = `/${product.category}/${product.slug}`; // string\nToil.navigate(Toil.href(path)); // asserted valid\n<Toil.Link href={Toil.href(path)}>Open</Toil.Link>;\n```\n\n`href()` is a pure type assertion (it returns the string unchanged), so use it only when the type-check is genuinely in your way, not to paper over a real typo. Before the routes are generated (a fresh project, the first build), `Href` is just `string`, so nothing complains until `toil-routes.d.ts` exists.\n\n## Reading the current location\n\nA handful of hooks let a component read where it is. Each re-reads on every navigation, so a component using one re-renders when the location changes:\n\n| Hook | Returns |\n| --- | --- |\n| `Toil.usePathname()` | The current pathname, e.g. `\"/blog/42\"`. |\n| `Toil.useLocation()` | The current pathname (an alias of `usePathname()`). |\n| `Toil.useParams<T>()` | The dynamic route params, e.g. `{ id }` for `/blog/[id]`. |\n| `Toil.useSearchParams()` | The query string as a `URLSearchParams`. |\n| `Toil.useNavigationPending()` | `true` while a navigation is in flight (started but not committed). |\n\n`useNavigationPending()` is what you wire a top loading bar to. It flips to `true` when a navigation begins and back to `false` once the new route commits:\n\n```tsx\n// client/components/ProgressBar.tsx\nexport default function ProgressBar() {\n const pending = Toil.useNavigationPending();\n return <div className=\"progress\" data-active={pending} />;\n}\n```\n\n`useSearchParams()` gives you a live `URLSearchParams`, so a filtered list reads its state from the URL:\n\n```tsx\nexport default function Results() {\n const params = Toil.useSearchParams();\n const q = params.get('q') ?? '';\n return <p>Results for {q}</p>;\n}\n```\n\n## Prefetching\n\ntoiljs prefetches routes before you navigate to them, so a click resolves with nothing left to download. There are two intents, both automatic:\n\n- **Hover / focus intent.** When you hover or focus a link that points at a known internal route, toiljs warms both its route chunk and its loader data, so the actual click can commit right away.\n- **Viewport intent.** As a link scrolls into view (or within about 200px of it), its route chunk is warmed. Links added later by client navigation are picked up automatically.\n\nPrefetching is best-effort and cheap: each route loads at most once, a failed prefetch is forgotten (so the real navigation can retry and surface the error), and new-tab / download / opted-out links are skipped. It is also **skipped entirely when the browser signals data-saver** (or reports a 2g-class connection), so you never spend a metered user's bandwidth on speculation.\n\nOpt a single link out with `prefetch={false}`, which emits a `data-no-prefetch` attribute the prefetcher respects:\n\n```tsx\n<Toil.Link href=\"/huge-report\" prefetch={false}>Annual report</Toil.Link>\n```\n\nYou can also warm a route yourself, for example right before an imperative `navigate`, with the standalone `Toil.prefetch(href)`:\n\n```tsx\n<button\n onPointerEnter={() => Toil.prefetch('/dashboard')}\n onClick={() => Toil.navigate('/dashboard')}>\n Open dashboard\n</button>\n```\n\n## Scroll restoration\n\ntoiljs manages scroll for you (it switches off the browser's automatic restoration and does the intuitive thing per navigation type):\n\n- **A push navigation** (a `Link` click, `navigate`, `router.push`) **scrolls to the top** of the new page.\n- **Back and forward restore** the scroll position you had on that entry, so returning to a long list lands you where you were.\n- **A `#hash` target** scrolls that element into view instead of jumping to the top.\n\nTo keep the current scroll on a push navigation, set `scroll={false}` on the `Link` (or `{ scroll: false }` in `NavigateOptions`). This is handy for tab bars or filters that change the URL but should not yank the viewport:\n\n```tsx\n<Toil.Link href=\"/settings/billing\" scroll={false}>Billing</Toil.Link>\n```\n\n```tsx\nnavigate('/settings/billing', { scroll: false });\n```\n\n## Animated transitions\n\nTwo optional effects can animate navigations. Both are **off by default** and are normally enabled from `toil.config.ts`, not in code:\n\n```ts\n// toil.config.ts\nexport default {\n client: {\n viewTransitions: true, // browser View Transitions API (a crossfade between pages)\n transitions: true, // React transition: keep the old page visible while the next loads\n },\n};\n```\n\n`viewTransitions` uses the browser's View Transitions API to crossfade the old and new page (and it respects `prefers-reduced-motion`, animating nothing for users who ask for less motion). `transitions` wraps each navigation in a React transition, keeping the current page on screen while the next route's loader runs, instead of showing its `loading.tsx` right away (smoother, but you trade away the immediate loading state).\n\nConfig is the normal way to turn these on. For a manual override at runtime, the setters are `Toil.setViewTransitions(enabled)` and `Toil.setTransitions(enabled)`:\n\n```tsx\nToil.setViewTransitions(true);\nToil.setTransitions(false);\n```\n\nYou rarely need the setters; prefer the config keys unless you are toggling an effect dynamically.\n\n## Types\n\nAlmost everything on this page is a value on the global `Toil` object, so you call it with no import (`Toil.Link`, `Toil.navigate`, `Toil.useRouter`, and so on). A few of the **types**, though, are not namespaced under `Toil`, so if you want to annotate a prop or a variable you import them from `toiljs/client`:\n\n```tsx\nimport type {\n LinkProps,\n NavLinkProps,\n NavLinkState,\n NavigateOptions,\n RouterInstance,\n} from 'toiljs/client';\n```\n\nFor example, a component that forwards `Link` props, or a helper typed against the router handle:\n\n```tsx\nimport type { RouterInstance } from 'toiljs/client';\n\nfunction logoutThenGo(router: RouterInstance) {\n router.replace('/login');\n}\n```\n\n## Related\n\n- [Routing](./routing.md): how files become URLs, dynamic params, layouts, and templates.\n- [Fetching data](./data-fetching.md): loaders, the typed backend clients, forms, and revalidation.\n- [Rendering and SSR](./rendering.md): what renders on the server versus the browser, and how hydration fits with client navigation.\n",
|
|
41
42
|
"frontend/README.md": "# Frontend\n\nYour toiljs frontend is a React app with file-based routing that runs in the browser, and can also be rendered ahead of time on the server for a fast first paint and good SEO.\n\nIf you have written React before, everything here is familiar React: components, hooks, JSX. toiljs adds the parts a plain React app makes you wire up yourself: a router, data loading, `<head>` and SEO management, an image component, and a typed client for calling your backend. You get them for free and you do not import most of them, they live on a global called `Toil`.\n\n## What \"frontend\" means here\n\nA toiljs project has three top-level folders. The frontend is the first two:\n\n- **`client/`** is your React app: pages, components, and styles. This is what runs in the user's browser.\n- **`shared/`** is a typed bridge that toiljs generates for you. It lets the browser call your backend with full type safety (see [Fetching data](./data-fetching.md)).\n- **`server/`** is your backend. It compiles to WebAssembly and runs on the edge. That is a separate section (see [Backend](../backend/README.md)).\n\nInside `client/`, the important pieces are:\n\n| Path | What it is |\n| --- | --- |\n| `client/toil.tsx` | The entry file. It imports your routes and global styles and mounts the app. |\n| `client/routes/` | Your pages. One file per URL (file-based routing). See [Routing](./routing.md). |\n| `client/layout.tsx` | The root layout that wraps every page (a header, a footer, and so on). |\n| `client/components/` | Your own reusable React components. |\n| `client/styles/` | Your CSS. See [Styling](./styling.md). |\n| `client/public/` | Static files served as-is (`favicon.ico`, images, `robots.txt`). |\n\nThe entry file is tiny, and you rarely touch it:\n\n```tsx\n// client/toil.tsx\nimport { routes, layout, notFound, globalError, slots } from 'toiljs/routes';\nimport './styles/main.css';\n\nToil.mount(routes, layout, notFound, globalError, slots);\n```\n\n`toiljs/routes` is a virtual module: the compiler scans `client/routes/` and generates the route table for you, so you never hand-maintain a list of pages. Adding a file under `client/routes/` adds a page.\n\n## SPA, but with a server head start\n\nBy default a toiljs app is a single-page application (SPA). The word \"SPA\" means the browser loads one HTML shell, then JavaScript builds every page and swaps between them without a full reload. That makes navigation instant, but it has a classic weakness: the very first load shows a blank page until the JavaScript runs, and simple crawlers that do not run JavaScript see nothing useful.\n\ntoiljs closes that gap in two ways, both optional and both explained in [Rendering and SSR](./rendering.md):\n\n1. **Build-time prerendering.** At build time toiljs bakes each route's `<head>` (title, description, Open Graph, and so on) into real HTML, so crawlers and link-preview bots see correct metadata even without running your code.\n2. **Edge server-side rendering (SSR).** For routes you opt in with `export const ssr = true`, the edge fills in real first-paint HTML for the page body, then the browser \"hydrates\" it (attaches React to the already-drawn markup instead of redrawing it).\n\nHere is the full life of one request, from a cold link click to a warm client-side app:\n\n```mermaid\nsequenceDiagram\n participant U as User's browser\n participant E as Dacely edge\n U->>E: GET /some-page\n E-->>U: HTML (baked head + first-paint body if ssr=true)\n Note over U: First paint appears immediately\n U->>E: fetch the JS bundle + route chunk\n Note over U: React \"hydrates\": attaches to the existing HTML\n Note over U: Page is now interactive\n U->>U: Click a link -> client-side navigation (no reload)\n Note over U: Only the new route's data + chunk are fetched\n```\n\nThe key idea: the server gets you a correct first paint fast, and from then on the app runs entirely in the browser, fetching only small route chunks and data as you navigate.\n\n## The `Toil` global\n\nMost of the client API lives on a global object called `Toil`, so route files need no imports for the common things. A few examples you will meet across these pages:\n\n```tsx\nToil.Link // a client-side navigation link\nToil.NavLink // a Link that knows when it is \"active\"\nToil.useParams() // read dynamic URL params, e.g. { id } for /blog/[id]\nToil.useLoaderData // read data your route's loader fetched\nToil.Image // an <img> that avoids layout shift and lazy-loads\nToil.useHead // set the <title> and <meta> tags\nServer.REST.* // the typed fetch client for your backend\n```\n\n`Server` is also global (it is the typed backend client, see [Fetching data](./data-fetching.md)). Everything on `Toil` is fully typed: your editor autocompletes it, because toiljs generates a `toil-env.d.ts` that maps `Toil` onto the `toiljs/client` package.\n\nThe same fast data utilities your backend uses are available in client code too, as bare globals with no import: `FastMap` and `FastSet` (high-performance map and set collections), and `DataWriter` / `DataReader` (a compact binary codec for encoding and decoding buffers). They are handed to you the same way `Toil` and `Server` are, so you can write `new DataWriter()` straight in a component. See [Data types](../backend/data.md) for the codec and when to reach for it.\n\n## The frontend pages\n\nRead them in roughly this order:\n\n- **[Routing](./routing.md)**: turn files into URLs. Index, nested, and dynamic pages; layouts and templates.\n- **[Navigation](./navigation.md)**: move between pages. Links and active state, navigating in code, typed hrefs, prefetching, scroll restoration, and animated transitions.\n- **[Components](./components.md)**: use your own React components, plus the toiljs primitives (`Image`, `Script`, `Form`, `Slot`, `Head`, and the SSR markers).\n- **[Rendering and SSR](./rendering.md)**: what renders on the server versus in the browser, how hydration works, and the current SSR limitations.\n- **[Styling](./styling.md)**: plain CSS, preprocessors (Sass / Less / Stylus), and Tailwind.\n- **[Images](./images.md)**: the `Toil.Image` component, automatic blur placeholders, and how it stops layout shift.\n- **[Metadata and SEO](./metadata.md)**: set the page title, description, and social-share tags per route.\n- **[Fetching data](./data-fetching.md)**: call your backend with the generated typed clients, submit forms, and read who is logged in.\n- **[Scripts](./scripts.md)**: load external or inline `<script>` tags with a loading strategy, using `Toil.Script`.\n- **[Search](./search.md)**: the built-in, statically-baked page search and command palette (`usePageSearch`).\n- **[The Toil global (reference)](./toil-global.md)**: a complete, grouped list of everything on the `Toil` object.\n\n## Related\n\n- [Getting started](../getting-started/README.md): install toiljs and create a project.\n- [Project structure](../getting-started/project-structure.md): the full folder layout.\n- [Backend overview](../backend/README.md): the `server/` side your frontend talks to.\n- [The CLI](../cli/README.md): `toiljs dev`, `toiljs build`, and every flag.\n",
|
|
42
|
-
"frontend/rendering.md": "# Rendering and SSR\n\nThis page explains where your pages are built: in the browser, ahead of time at build, or on the server for each request. Getting this right is what makes a page paint fast and rank well.\n\n## The three ways a page can render\n\nA toiljs page can reach the user in three ways. You mostly get all three for free; the only one you opt into per page is edge SSR.\n\n| Mode | Who builds the HTML | When | Good for |\n| --- | --- | --- | --- |\n| **Client rendering** | The browser, from JavaScript | On every visit | Interactive, per-user pages (a dashboard). The default. |\n| **Build-time prerender** | The build, once | When you run `toiljs build` | Baking each route's `<head>` (SEO) into real HTML. Automatic. |\n| **Edge SSR** | The edge server, per request | When you set `ssr = true` | A real first-paint page body plus SEO, for landing and content pages. |\n\nLet us define the two words that trip people up:\n\n- **Rendering** means turning your React components into HTML.\n- **Hydration** means React attaching to HTML that already exists on the page (from the server) instead of throwing it away and redrawing it. Hydration is what makes a server-rendered page interactive without a flash.\n\n## Client rendering (the default)\n\nBy default, toiljs ships a small HTML shell with an empty `<div id=\"root\">`, plus your JavaScript. The browser downloads the JS, React runs, and it builds the page into `#root`. From then on, navigating between pages is pure JavaScript: only the next route's small code chunk and its data are fetched, and the page swaps in place with no reload.\n\nThis is fast to navigate and simple to reason about. Its one weakness is the *first* paint: until the JavaScript runs, `#root` is empty. For an app behind a login (a dashboard) that is fine, nobody is trying to index it. For a public landing page, you usually want one of the two server-assisted modes below.\n\n## Build-time prerender (automatic SEO)\n\nEvery time you build, toiljs renders each static route once and bakes its resolved `<head>` (title, description, canonical link, Open Graph tags, and so on) into that route's HTML file. This happens for all routes with no extra work from you, and it is driven by the `metadata` you export from a route plus the site-wide `seo` config (see [Metadata and SEO](./metadata.md)).\n\nThe payoff: a crawler or a link-preview bot (Slack, Discord, iMessage) that fetches your page sees correct tags immediately, even though it does not run your JavaScript. \"View source\" on a built page shows the real title and meta tags, not an empty shell.\n\nIn production, `toiljs build` writes one prerendered HTML file per route (for example `about/index.html`), and the production static server (`npm start`) serves each route its own prerendered file rather than a single shared shell. That is how each page gets its own metadata in the raw HTML.\n\nBuild-time prerender covers the `<head>`. It does not, by itself, fill in the page *body*: for a client-rendered route the body is still built by React in the browser. To get real first-paint body HTML, opt the route into edge SSR.\n\n### Prerendering dynamic routes with `generateStaticParams`\n\nBuild-time prerender bakes a `<head>` for every **static** route on its own. A **dynamic** route (`client/routes/blog/[id].tsx`, one file that serves `/blog/1`, `/blog/2`, and so on) has no single URL to prerender, so by default it gets none of that per-URL HTML. If you know the concrete URLs ahead of time (a fixed set of blog posts, product pages, or docs), you can opt the route into **static site generation (SSG)**, which means the build renders one HTML file per known URL: it enumerates each URL and writes a real `<url>/index.html` with that page's resolved metadata, plus a `sitemap.xml` entry. This is the toiljs analog of Next.js `generateStaticParams`.\n\nYou opt in by exporting `generateStaticParams` from the dynamic route. It returns one object per URL, keyed by the route's param names:\n\n```tsx\n// client/routes/blog/[id].tsx\n\n// One entry per URL to prerender. `id` matches the [id] segment.\nexport const generateStaticParams: Toil.GenerateStaticParams = () => {\n return [{ id: '1' }, { id: '2' }, { id: '3' }];\n};\n\n// The build runs this once per URL, so each baked page gets its own <head>.\nexport const generateMetadata: Toil.GenerateMetadata = ({ params }) => ({\n title: `Blog post ${params.id}`,\n description: `Reading blog post ${params.id}.`,\n});\n\nexport default function BlogPost() {\n const { id } = Toil.useParams();\n return <h1>Blog post {id}</h1>;\n}\n```\n\nAt build, that writes `blog/1/index.html`, `blog/2/index.html`, and `blog/3/index.html`, each with its own title and description in the raw HTML (so a crawler that does not run JavaScript sees them), and all three land in `sitemap.xml`. The param values also feed the route's `loader`, so per-URL metadata can depend on real data.\n\n`generateStaticParams` is async-friendly (return a promise if you fetch the id list first) and completely opt-in: a dynamic route without it is untouched, and the whole pass is skipped when your project has no `seo` config. A catch-all segment (`[...slug]`) takes an array value: `{ slug: ['2024', 'hello'] }` fills the URL as `2024/hello`.\n\n> Two things this is **not**: it is build-time prerender of the `<head>`, not edge SSR of the body. To also serve real body HTML for these URLs on first paint, add `export const ssr = true` as well (see below). And to make a dynamic route show up in the on-site [search index](./search.md) even though its title is dynamic, export `searchHints` (covered there), which is separate from `generateStaticParams`.\n\n## Edge SSR (`ssr = true`)\n\nFor a route where you want the body content visible on first paint (a marketing page, an article), add one line:\n\n```tsx\n// client/routes/index.tsx\nexport const ssr = true;\n\nexport default function Home() {\n return (\n <section className=\"hero\">\n <h1>Welcome</h1>\n </section>\n );\n}\n```\n\nNow the Dacely edge sends a real, filled-in first paint for that page, and the browser hydrates it. The user sees content immediately, and React takes over without redrawing anything.\n\n### How it works, in brief\n\ntoiljs does something clever to keep SSR cheap. It does not re-run React on the server for every request. Instead, at build time it renders the page once into a **template**: the static HTML with the dynamic bits punched out into named holes. Then, per request, your compiled backend fills only the hole values (a small list of \"slot 3 = this text\"), and the edge splices those values into the pre-baked template. The result is real first-paint HTML produced about as fast as serving a static file.\n\n```mermaid\nsequenceDiagram\n participant B as Build\n participant Edge as Dacely edge\n participant U as Browser\n B->>Edge: prebuilt template (HTML with holes) + a coherence hash\n U->>Edge: GET / (ssr route)\n Edge->>Edge: run the wasm render -> small \"hole values\" list\n Edge->>Edge: splice values into the template\n Edge-->>U: real first-paint HTML\n Note over U: Content is visible immediately\n U->>Edge: fetch JS + route chunk\n Note over U: React hydrates: attaches to the existing HTML\n Note over U: Page is interactive; no redraw, no flash\n```\n\nFor SSR to hydrate cleanly, the HTML the server produced and the HTML the browser would produce must match byte-for-byte. toiljs guarantees this by escaping hole values exactly as React does and by carrying a hash that ties the running backend to the exact template it was built against. Authoring the server side of an SSR route (the hole markers in the page and the matching `render` function in `server/`) is a deeper topic that lives with the [backend](../backend/README.md). For most pages you only need `export const ssr = true` and to keep the page \"SSR-safe\" (below).\n\n### Authoring an SSR route\n\n`export const ssr = true` is all you need for a page whose body is fully static. The moment the body has a **dynamic bit** (a value that changes per request: a name from the URL, a list from a loader, a chunk of user HTML), you have to tell the build *where* that dynamic bit lives, so it can be punched out into a hole. You do that by wrapping the dynamic value in a **hole marker**.\n\nWhy is this necessary? The build renders your page once into a template. It cannot guess which `{expression}` in your JSX is a per-request value and which is a constant, so it does not try: any dynamic content you leave unwrapped gets frozen into the template as whatever value it happened to have during that one build render, and it will never update per request. The markers are the explicit \"fill this in later\" signal.\n\nThe markers live on the `Toil` global (so you do not import them), and they are **transparent in the browser**: `<Toil.Hole>` just renders its children, `<Toil.Repeat>` just maps its rows. They only behave differently during the build render, so your client-side app runs exactly as written.\n\n| Marker | Use it for | Shape |\n| --- | --- | --- |\n| `Toil.Hole` | A single dynamic **text** value. | JSX element with `id` + children |\n| `Toil.Repeat` | A **list**: a row template repeated over an `each` array. | JSX element with `id` + `each` + a render function |\n| `Toil.RawHtml` | A block of **pre-rendered HTML** you trust (Markdown you rendered, say). | JSX element with `id` + `html` (+ optional `as`) |\n| `Toil.attr` | A dynamic value in **attribute position** (an `href`, a `class`). | a **function** you call inside the attribute |\n| `Toil.Island` | Content that must render **only in the browser** (the escape hatch). | JSX element with children |\n\n`Toil.attr` is a function rather than an element because an attribute is not a child node, so it cannot be a JSX element. You call it right where the value goes.\n\nHere is a full SSR route: a blog post whose title, body HTML, tag list, and author link all come from the route's `loader`.\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const ssr = true;\n\n// Runs on the server for the first paint, then again on the client to reproduce\n// the same data so hydration matches (see \"Keeping a route SSR-safe\" below).\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n // Illustrative shape: { title, bodyHtml, tags, authorUrl }.\n return Server.REST.blog.get({ params: { id: params.id } });\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return (\n <article>\n <h1>\n <Toil.Hole id=\"title\">{post.title}</Toil.Hole>\n </h1>\n\n {/* A dynamic attribute: call attr() in attribute position. */}\n <a href={Toil.attr('authorUrl', post.authorUrl)}>By the author</a>\n\n {/* A block of trusted, pre-rendered HTML. */}\n <Toil.RawHtml id=\"body\" html={post.bodyHtml} />\n\n {/* A list: one row template, stamped once per item on the server. */}\n <ul>\n <Toil.Repeat id=\"tags\" each={post.tags}>\n {(tag) => <li key={tag}>{tag}</li>}\n </Toil.Repeat>\n </ul>\n </article>\n );\n}\n```\n\nA few rules that keep the template valid:\n\n- **Every marker needs a stable `id`**: a short name unique within the page. The build maps each id to a numbered slot, so keep the ids constant across builds.\n- **`Toil.Repeat` needs at least one row at build time.** It captures that first row as the sub-template for every row, so the build render must see sample data with one or more items (an empty `each` gives it nothing to capture).\n- **`Toil.RawHtml` renders inside a wrapper element** (a `<div>` by default; pass `as=\"section\"` to change the tag), and you own sanitising that HTML, exactly like React's `dangerouslySetInnerHTML`.\n\nAnything that genuinely cannot run on the server (it reads `window`, calls `Date.now()`, or depends on the live URL) goes inside a `Toil.Island`, which renders nothing on the server and reveals its children only after hydration:\n\n```tsx\n<Toil.Island>\n <LiveClock />\n</Toil.Island>\n```\n\nThis is the flow from your marked-up JSX to the first-paint HTML:\n\n```mermaid\nflowchart TD\n A[\"Your SSR route JSX<br/>dynamic bits wrapped in Toil.Hole / Repeat / RawHtml / attr\"] --> B[\"Build render (once, in sentinel mode)\"]\n B --> C[\".tmpl: static HTML with numbered holes\"]\n B --> D[\".slots: which hole is text / raw / attr / repeat\"]\n B --> E[\"Slot enum + coherence HASH\"]\n C --> F[\"Deployed to the Dacely edge\"]\n E --> G[\"Compiled backend render(req)\"]\n G -->|\"per request: fill only the hole values\"| F\n D --> G\n F -->|\"splice values into the template\"| H[\"First-paint HTML to the browser\"]\n```\n\n#### Advanced: hand-writing the server `render`\n\nYou almost never do this. The compiler generates the server `render(req)` for an SSR route from the JSX above, so the hole ids line up automatically. But the server side is backed by a plain, hand-writable API for the rare case you need full control (an unusual template, or a value the compiler cannot derive). Your `render` returns a `SlotValues` object, filled with:\n\n- `setText(slot, value)`: a text hole (React-escaped for you).\n- `setRaw(slot, html)`: a raw-HTML hole (you own sanitising).\n- `setAttr(slot, value)`: an attribute hole.\n- `setRepeat(slot, rows)`: a repeat region, with the rows assembled through an `HtmlBuilder` (chain `.raw(...)`, `.text(...)`, `.attr(...)`).\n- `setHeader(name, value)`, `setTitle(title)`, and `setStatus(code)`: response headers, a per-request `<title>`, and the status code.\n\n`SlotValues` and `HtmlBuilder` live in `server/runtime/ssr/slots.ts`. The `setTitle` helper is the supported way to give an SSR page a data-driven `<title>` (a blog post's real title from its loader), overriding the one baked into the template. This is server (backend) code, so it belongs with the [backend](../backend/README.md).\n\n### Keeping a route SSR-safe\n\nServer rendering happens where there is no browser: no `window`, no `document`, no mouse. So an SSR route (and every layout above it) must render without touching browser-only APIs during that first render. Anything that must run only in the browser (reading `window`, using `Date.now()`, or router hooks that need the live URL) goes inside an **island**, a marker that renders nothing on the server and appears only after hydration.\n\nIf a route or one of its layouts throws while rendering on the server, toiljs does not ship a broken page. It **skips SSR for that route at build with a warning** and falls back to plain client rendering. So adding `ssr = true` is always safe: worst case you get client rendering plus a build warning telling you what to move into an island.\n\n### Suspense markers and self-healing hydration\n\nUnder the hood the client wraps each route and layout in React `Suspense` boundaries that line up with what the server emitted, so hydration matches. If hydration ever does mismatch (the server HTML and the client's idea of the page disagree), React does the safe thing: it discards the server markup for that part and re-renders it on the client. You get a correct page either way. The cost of a mismatch is a small flash and some wasted work, not a broken page, which is why the guidance above (keep it SSR-safe, put browser-only bits in islands) is about smoothness, not correctness.\n\n## Known SSR limitations\n\nBe aware of these honest gaps as of today:\n\n- **`template.tsx` is not server-rendered.** A `template.tsx` wrapper (the re-mounting cousin of a layout) is not part of the SSR output. A route under one still works: hydration self-heals to client rendering for that part.\n- **Parallel slots (`@slot`) are not server-rendered.** Slot content (including intercepted modals) renders on the client after hydration, not in the first paint. Since slots are typically modals and overlays that appear on interaction, this is rarely a problem.\n- **Islands have no first paint or SEO.** That is by design: an island is your \"client only\" escape hatch, so anything inside it is intentionally absent from the server HTML and from what crawlers see.\n- **The client loader must reproduce the server's data.** For a hole whose value comes from the request (a query param), the route's client `loader` has to derive the same value the server used, or hydration will re-render that part. If the client cannot reproduce a value, put that content in an island.\n\n## Which mode should I use?\n\n- **Interactive, per-user page** (dashboard, settings): client rendering. Do nothing.\n- **Public page that needs correct link previews and titles**: you already have build-time prerender. Do nothing extra.\n- **Public page that should also show its content instantly on first load** (landing page, blog post, docs): add `export const ssr = true` and keep it SSR-safe.\n\n## Related\n\n- [Backend overview](../backend/README.md): where the server-side `render` for an SSR route lives.\n- [Metadata and SEO](./metadata.md): what gets baked into the `<head>`.\n- [Routing](./routing.md): layouts, templates, and slots.\n- [Fetching data](./data-fetching.md): loaders and how their data seeds hydration.\n",
|
|
43
|
+
"frontend/rendering.md": "# Rendering and SSR\n\nThis page explains where your pages are built: in the browser, ahead of time at build, or on the server for each request. Getting this right is what makes a page paint fast and rank well.\n\n## The three ways a page can render\n\nA toiljs page can reach the user in three ways. You mostly get all three for free; the only one you opt into per page is edge SSR.\n\n| Mode | Who builds the HTML | When | Good for |\n| --- | --- | --- | --- |\n| **Client rendering** | The browser, from JavaScript | On every visit | Interactive, per-user pages (a dashboard). The default. |\n| **Build-time prerender** | The build, once | When you run `toiljs build` | Baking each route's `<head>` (SEO) into real HTML. Automatic. |\n| **Edge SSR** | The edge server, per request | When you set `ssr = true` | A real first-paint page body plus SEO, for landing and content pages. |\n\nLet us define the two words that trip people up:\n\n- **Rendering** means turning your React components into HTML.\n- **Hydration** means React attaching to HTML that already exists on the page (from the server) instead of throwing it away and redrawing it. Hydration is what makes a server-rendered page interactive without a flash.\n\n## Client rendering (the default)\n\nBy default, toiljs ships a small HTML shell with an empty `<div id=\"root\">`, plus your JavaScript. The browser downloads the JS, React runs, and it builds the page into `#root`. From then on, navigating between pages is pure JavaScript: only the next route's small code chunk and its data are fetched, and the page swaps in place with no reload.\n\nThis is fast to navigate and simple to reason about. Its one weakness is the *first* paint: until the JavaScript runs, `#root` is empty. For an app behind a login (a dashboard) that is fine, nobody is trying to index it. For a public landing page, you usually want one of the two server-assisted modes below.\n\n## Build-time prerender (automatic SEO)\n\nEvery time you build, toiljs renders each static route once and bakes its resolved `<head>` (title, description, canonical link, Open Graph tags, and so on) into that route's HTML file. This happens for all routes with no extra work from you, and it is driven by the `metadata` you export from a route plus the site-wide `seo` config (see [Metadata and SEO](./metadata.md)).\n\nThe payoff: a crawler or a link-preview bot (Slack, Discord, iMessage) that fetches your page sees correct tags immediately, even though it does not run your JavaScript. \"View source\" on a built page shows the real title and meta tags, not an empty shell.\n\nIn production, `toiljs build` writes one prerendered HTML file per route (for example `about/index.html`), and the production static server (`npm start`) serves each route its own prerendered file rather than a single shared shell. That is how each page gets its own metadata in the raw HTML.\n\nBuild-time prerender covers the `<head>`. It does not, by itself, fill in the page *body*: for a client-rendered route the body is still built by React in the browser. To get real first-paint body HTML, opt the route into edge SSR.\n\n### Prerendering dynamic routes with `generateStaticParams`\n\nBuild-time prerender bakes a `<head>` for every **static** route on its own. A **dynamic** route (`client/routes/blog/[id].tsx`, one file that serves `/blog/1`, `/blog/2`, and so on) has no single URL to prerender, so by default it gets none of that per-URL HTML. If you know the concrete URLs ahead of time (a fixed set of blog posts, product pages, or docs), you can opt the route into **static site generation (SSG)**, which means the build renders one HTML file per known URL: it enumerates each URL and writes a real `<url>/index.html` with that page's resolved metadata, plus a `sitemap.xml` entry. This is the toiljs analog of Next.js `generateStaticParams`.\n\nYou opt in by exporting `generateStaticParams` from the dynamic route. It returns one object per URL, keyed by the route's param names:\n\n```tsx\n// client/routes/blog/[id].tsx\n\n// One entry per URL to prerender. `id` matches the [id] segment.\nexport const generateStaticParams: Toil.GenerateStaticParams = () => {\n return [{ id: '1' }, { id: '2' }, { id: '3' }];\n};\n\n// The build runs this once per URL, so each baked page gets its own <head>.\nexport const generateMetadata: Toil.GenerateMetadata = ({ params }) => ({\n title: `Blog post ${params.id}`,\n description: `Reading blog post ${params.id}.`,\n});\n\nexport default function BlogPost() {\n const { id } = Toil.useParams();\n return <h1>Blog post {id}</h1>;\n}\n```\n\nAt build, that writes `blog/1/index.html`, `blog/2/index.html`, and `blog/3/index.html`, each with its own title and description in the raw HTML (so a crawler that does not run JavaScript sees them), and all three land in `sitemap.xml`. The param values also feed the route's `loader`, so per-URL metadata can depend on real data.\n\n`generateStaticParams` is async-friendly (return a promise if you fetch the id list first) and completely opt-in: a dynamic route without it is untouched, and the whole pass is skipped when your project has no `seo` config. A catch-all segment (`[...slug]`) takes an array value: `{ slug: ['2024', 'hello'] }` fills the URL as `2024/hello`.\n\n> Two things this is **not**: it is build-time prerender of the `<head>`, not edge SSR of the body. To also serve real body HTML for these URLs on first paint, add `export const ssr = true` as well (see below). And to make a dynamic route show up in the on-site [search index](./search.md) even though its title is dynamic, export `searchHints` (covered there), which is separate from `generateStaticParams`.\n\n## Edge SSR (`ssr = true`)\n\nFor a route where you want the body content visible on first paint (a marketing page, an article), add one line:\n\n```tsx\n// client/routes/index.tsx\nexport const ssr = true;\n\nexport default function Home() {\n return (\n <section className=\"hero\">\n <h1>Welcome</h1>\n </section>\n );\n}\n```\n\nNow the Dacely edge sends a real, filled-in first paint for that page, and the browser hydrates it. The user sees content immediately, and React takes over without redrawing anything.\n\n### How it works, in brief\n\ntoiljs does something clever to keep SSR cheap. It does not re-run React on the server for every request. Instead, at build time it renders the page once into a **template**: the static HTML with the dynamic bits punched out into named holes. Then, per request, your compiled backend fills only the hole values (a small list of \"slot 3 = this text\"), and the edge splices those values into the pre-baked template. The result is real first-paint HTML produced about as fast as serving a static file.\n\n```mermaid\nsequenceDiagram\n participant B as Build\n participant Edge as Dacely edge\n participant U as Browser\n B->>Edge: prebuilt template (HTML with holes) plus a coherence hash\n U->>Edge: GET / (ssr route)\n Edge->>Edge: run the wasm render to get a small hole-values list\n Edge->>Edge: splice values into the template\n Edge-->>U: real first-paint HTML\n Note over U: Content is visible immediately\n U->>Edge: fetch JS plus route chunk\n Note over U: React hydrates and attaches to the existing HTML\n Note over U: Page is interactive with no redraw or flash\n```\n\nFor SSR to hydrate cleanly, the HTML the server produced and the HTML the browser would produce must match byte-for-byte. toiljs guarantees this by escaping hole values exactly as React does and by carrying a hash that ties the running backend to the exact template it was built against. Authoring the server side of an SSR route (the hole markers in the page and the matching `render` function in `server/`) is a deeper topic that lives with the [backend](../backend/README.md). For most pages you only need `export const ssr = true` and to keep the page \"SSR-safe\" (below).\n\n### Authoring an SSR route\n\n`export const ssr = true` is all you need for a page whose body is fully static. The moment the body has a **dynamic bit** (a value that changes per request: a name from the URL, a list from a loader, a chunk of user HTML), you have to tell the build *where* that dynamic bit lives, so it can be punched out into a hole. You do that by wrapping the dynamic value in a **hole marker**.\n\nWhy is this necessary? The build renders your page once into a template. It cannot guess which `{expression}` in your JSX is a per-request value and which is a constant, so it does not try: any dynamic content you leave unwrapped gets frozen into the template as whatever value it happened to have during that one build render, and it will never update per request. The markers are the explicit \"fill this in later\" signal.\n\nThe markers live on the `Toil` global (so you do not import them), and they are **transparent in the browser**: `<Toil.Hole>` just renders its children, `<Toil.Repeat>` just maps its rows. They only behave differently during the build render, so your client-side app runs exactly as written.\n\n| Marker | Use it for | Shape |\n| --- | --- | --- |\n| `Toil.Hole` | A single dynamic **text** value. | JSX element with `id` + children |\n| `Toil.Repeat` | A **list**: a row template repeated over an `each` array. | JSX element with `id` + `each` + a render function |\n| `Toil.RawHtml` | A block of **pre-rendered HTML** you trust (Markdown you rendered, say). | JSX element with `id` + `html` (+ optional `as`) |\n| `Toil.attr` | A dynamic value in **attribute position** (an `href`, a `class`). | a **function** you call inside the attribute |\n| `Toil.Island` | Content that must render **only in the browser** (the escape hatch). | JSX element with children |\n\n`Toil.attr` is a function rather than an element because an attribute is not a child node, so it cannot be a JSX element. You call it right where the value goes.\n\nHere is a full SSR route: a blog post whose title, body HTML, tag list, and author link all come from the route's `loader`.\n\n```tsx\n// client/routes/blog/[id].tsx\nexport const ssr = true;\n\n// Runs on the server for the first paint, then again on the client to reproduce\n// the same data so hydration matches (see \"Keeping a route SSR-safe\" below).\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n // Illustrative shape: { title, bodyHtml, tags, authorUrl }.\n return Server.REST.blog.get({ params: { id: params.id } });\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return (\n <article>\n <h1>\n <Toil.Hole id=\"title\">{post.title}</Toil.Hole>\n </h1>\n\n {/* A dynamic attribute: call attr() in attribute position. */}\n <a href={Toil.attr('authorUrl', post.authorUrl)}>By the author</a>\n\n {/* A block of trusted, pre-rendered HTML. */}\n <Toil.RawHtml id=\"body\" html={post.bodyHtml} />\n\n {/* A list: one row template, stamped once per item on the server. */}\n <ul>\n <Toil.Repeat id=\"tags\" each={post.tags}>\n {(tag) => <li key={tag}>{tag}</li>}\n </Toil.Repeat>\n </ul>\n </article>\n );\n}\n```\n\nA few rules that keep the template valid:\n\n- **Every marker needs a stable `id`**: a short name unique within the page. The build maps each id to a numbered slot, so keep the ids constant across builds.\n- **`Toil.Repeat` needs at least one row at build time.** It captures that first row as the sub-template for every row, so the build render must see sample data with one or more items (an empty `each` gives it nothing to capture).\n- **`Toil.RawHtml` renders inside a wrapper element** (a `<div>` by default; pass `as=\"section\"` to change the tag), and you own sanitising that HTML, exactly like React's `dangerouslySetInnerHTML`.\n\nAnything that genuinely cannot run on the server (it reads `window`, calls `Date.now()`, or depends on the live URL) goes inside a `Toil.Island`, which renders nothing on the server and reveals its children only after hydration:\n\n```tsx\n<Toil.Island>\n <LiveClock />\n</Toil.Island>\n```\n\nThis is the flow from your marked-up JSX to the first-paint HTML:\n\n```mermaid\nflowchart TD\n A[\"Your SSR route JSX<br/>dynamic bits wrapped in Toil.Hole / Repeat / RawHtml / attr\"] --> B[\"Build render (once, in sentinel mode)\"]\n B --> C[\".tmpl: static HTML with numbered holes\"]\n B --> D[\".slots: which hole is text / raw / attr / repeat\"]\n B --> E[\"Slot enum + coherence HASH\"]\n C --> F[\"Deployed to the Dacely edge\"]\n E --> G[\"Compiled backend render(req)\"]\n G -->|\"per request: fill only the hole values\"| F\n D --> G\n F -->|\"splice values into the template\"| H[\"First-paint HTML to the browser\"]\n```\n\n#### Advanced: hand-writing the server `render`\n\nYou almost never do this. The compiler generates the server `render(req)` for an SSR route from the JSX above, so the hole ids line up automatically. But the server side is backed by a plain, hand-writable API for the rare case you need full control (an unusual template, or a value the compiler cannot derive). Your `render` returns a `SlotValues` object, filled with:\n\n- `setText(slot, value)`: a text hole (React-escaped for you).\n- `setRaw(slot, html)`: a raw-HTML hole (you own sanitising).\n- `setAttr(slot, value)`: an attribute hole.\n- `setRepeat(slot, rows)`: a repeat region, with the rows assembled through an `HtmlBuilder` (chain `.raw(...)`, `.text(...)`, `.attr(...)`).\n- `setHeader(name, value)`, `setTitle(title)`, and `setStatus(code)`: response headers, a per-request `<title>`, and the status code.\n\n`SlotValues` and `HtmlBuilder` live in `server/runtime/ssr/slots.ts`. The `setTitle` helper is the supported way to give an SSR page a data-driven `<title>` (a blog post's real title from its loader), overriding the one baked into the template. This is server (backend) code, so it belongs with the [backend](../backend/README.md).\n\n### Keeping a route SSR-safe\n\nServer rendering happens where there is no browser: no `window`, no `document`, no mouse. So an SSR route (and every layout above it) must render without touching browser-only APIs during that first render. Anything that must run only in the browser (reading `window`, using `Date.now()`, or router hooks that need the live URL) goes inside an **island**, a marker that renders nothing on the server and appears only after hydration.\n\nIf a route or one of its layouts throws while rendering on the server, toiljs does not ship a broken page. It **skips SSR for that route at build with a warning** and falls back to plain client rendering. So adding `ssr = true` is always safe: worst case you get client rendering plus a build warning telling you what to move into an island.\n\n### Suspense markers and self-healing hydration\n\nUnder the hood the client wraps each route and layout in React `Suspense` boundaries that line up with what the server emitted, so hydration matches. If hydration ever does mismatch (the server HTML and the client's idea of the page disagree), React does the safe thing: it discards the server markup for that part and re-renders it on the client. You get a correct page either way. The cost of a mismatch is a small flash and some wasted work, not a broken page, which is why the guidance above (keep it SSR-safe, put browser-only bits in islands) is about smoothness, not correctness.\n\n## Known SSR limitations\n\nBe aware of these honest gaps as of today:\n\n- **`template.tsx` is not server-rendered.** A `template.tsx` wrapper (the re-mounting cousin of a layout) is not part of the SSR output. A route under one still works: hydration self-heals to client rendering for that part.\n- **Parallel slots (`@slot`) are not server-rendered.** Slot content (including intercepted modals) renders on the client after hydration, not in the first paint. Since slots are typically modals and overlays that appear on interaction, this is rarely a problem.\n- **Islands have no first paint or SEO.** That is by design: an island is your \"client only\" escape hatch, so anything inside it is intentionally absent from the server HTML and from what crawlers see.\n- **The client loader must reproduce the server's data.** For a hole whose value comes from the request (a query param), the route's client `loader` has to derive the same value the server used, or hydration will re-render that part. If the client cannot reproduce a value, put that content in an island.\n\n## Which mode should I use?\n\n- **Interactive, per-user page** (dashboard, settings): client rendering. Do nothing.\n- **Public page that needs correct link previews and titles**: you already have build-time prerender. Do nothing extra.\n- **Public page that should also show its content instantly on first load** (landing page, blog post, docs): add `export const ssr = true` and keep it SSR-safe.\n\n## Related\n\n- [Backend overview](../backend/README.md): where the server-side `render` for an SSR route lives.\n- [Metadata and SEO](./metadata.md): what gets baked into the `<head>`.\n- [Routing](./routing.md): layouts, templates, and slots.\n- [Fetching data](./data-fetching.md): loaders and how their data seeds hydration.\n",
|
|
43
44
|
"frontend/routing.md": "# Routing\n\ntoiljs uses file-based routing: every file under `client/routes/` becomes a page, and its path on disk becomes its URL. There is no route config to maintain, you create a file and the route exists.\n\nThis page is about **frontend** (browser) routing: the pages a user visits and how they navigate between them. Your **backend** HTTP endpoints use a different, decorator-based system (see [HTTP routes](../backend/rest.md)).\n\n## The basic idea\n\nDrop a React component at `client/routes/<something>.tsx`, `export default` it, and it is a page. The default export is the component that renders at that URL.\n\n```tsx\n// client/routes/about.tsx -> /about\nexport default function About() {\n return (\n <main>\n <h1>About us</h1>\n </main>\n );\n}\n```\n\nRun `toiljs dev`, open `/about`, and it is there. Nothing else to register.\n\n## How a file path becomes a URL\n\nThe compiler scans `client/routes/` and turns each file into a URL pattern with a small set of rules. Here is the whole mapping in one table:\n\n| File under `client/routes/` | URL | What it is |\n| --- | --- | --- |\n| `index.tsx` | `/` | The home page. |\n| `about.tsx` | `/about` | A static page. |\n| `blog/index.tsx` | `/blog` | An index inside a folder. |\n| `blog/[id].tsx` | `/blog/:id` | A **dynamic** page (one param). |\n| `docs/[...slug].tsx` | `/docs/*` | A **catch-all** (one or more segments). |\n| `files/[[...slug]].tsx` | `/files` and `/files/*` | An **optional** catch-all (zero or more). |\n| `(marketing)/pricing.tsx` | `/pricing` | A **route group**: parens add no URL segment. |\n| `@modal/photo/[id].tsx` | (a parallel slot, see below) | A named **slot**. |\n\nThe rules, spelled out:\n\n- **`index`** means \"the folder itself\", so `blog/index.tsx` is `/blog`, and the top `index.tsx` is `/`.\n- **Square brackets** mark a dynamic segment. `[id]` captures one path segment into a param named `id`. You read it with `Toil.useParams()`.\n- **`[...name]`** is a catch-all: it captures the rest of the path (one segment or more) as a single `/`-joined string.\n- **`[[...name]]`** is an *optional* catch-all: like `[...name]`, but it also matches the bare parent URL with nothing after it (the param is then absent).\n- **Parentheses** like `(marketing)` create a **route group**: a folder that organizes files without adding anything to the URL. Handy for grouping pages that share a layout.\n\n```mermaid\nflowchart LR\n A[\"client/routes/blog/[id].tsx\"] -->|scan| B[\"/blog/:id\"]\n B -->|visit /blog/42| C[\"params = { id: '42' }\"]\n C -->|Toil.useParams| D[\"render Blog post 42\"]\n```\n\n### Dynamic pages: reading the param\n\nA file named with brackets gets its captured values from `Toil.useParams()`:\n\n```tsx\n// client/routes/blog/[id].tsx -> /blog/:id\nexport default function BlogPost() {\n const { id } = Toil.useParams();\n return (\n <main>\n <h1>Blog post {id}</h1>\n </main>\n );\n}\n```\n\nVisiting `/blog/42` renders \"Blog post 42\". Param values are URL-decoded for you.\n\nFor a catch-all, the param is the whole tail joined with slashes:\n\n```tsx\n// client/routes/docs/[...slug].tsx -> /docs/*slug\nexport default function Docs() {\n const { slug } = Toil.useParams();\n // /docs/getting-started/install -> slug === \"getting-started/install\"\n return <main>{slug}</main>;\n}\n```\n\nAn optional catch-all (`[[...slug]]`) matches the bare parent too, so `slug` can be empty:\n\n```tsx\n// client/routes/files/[[...slug]].tsx -> matches /files AND /files/a/b\nexport default function Files() {\n const { slug } = Toil.useParams();\n // \"/files\" -> slug is undefined; \"/files/a/b\" -> slug === \"a/b\"\n return <main>{slug ?? '(the base /files page)'}</main>;\n}\n```\n\n### When two routes could match\n\nIf a URL could match more than one pattern, toiljs picks the most specific one. Static segments win over dynamic (`:id`) segments, which win over catch-alls (`*slug`), and deeper routes win over shallower ones. So `/blog/new` prefers a literal `blog/new.tsx` over `blog/[id].tsx` if both exist. You do not configure this, it just does the intuitive thing.\n\n## Special files\n\nSome filenames are not pages, they are helpers that wrap or replace pages. They live alongside your routes and are never matched as a URL:\n\n| Filename | Role |\n| --- | --- |\n| `layout.tsx` | Wraps the pages in its folder (and below). **Persists** across navigation. |\n| `template.tsx` | Like a layout, but **re-mounts** on every navigation within it. |\n| `loading.tsx` | Shown while the page (and its data) is loading. |\n| `error.tsx` | Shown when the page throws (an error boundary). |\n| `global-error.tsx` | The last-resort error boundary, wraps even the root layout. |\n| `404.tsx` (or `not-found.tsx`) | Shown when no route matches. |\n\n### Layouts\n\nA `layout.tsx` receives the page (or nested layout) as `children` and renders around it. The root `client/layout.tsx` wraps every page. It is the natural home for your header, footer, and site-wide `<head>` defaults:\n\n```tsx\n// client/layout.tsx\nimport type { ReactNode } from 'react';\nimport Header from './components/Header';\nimport Footer from './components/Footer';\n\nexport default function Layout({ children }: { children?: ReactNode }) {\n return (\n <div className=\"app\">\n <Header />\n <main className=\"content\">{children}</main>\n <Footer />\n </div>\n );\n}\n```\n\nLayouts nest by folder. A `client/routes/dashboard/layout.tsx` wraps every page under `/dashboard`, inside the root layout. Because a layout **persists** across navigations, state inside it (a sidebar's open/closed flag, a scroll position) survives when you move between its child pages.\n\n### Templates vs layouts\n\nA `template.tsx` looks like a layout but does the opposite on navigation: it **re-mounts** every time you move to a new page within it. Use a layout when state should persist (a nav sidebar), and a template when it should reset (an enter animation that should replay, or a counter that should start fresh per page).\n\n### Loading and error states\n\nPut a `loading.tsx` next to a page (or in a folder) and it shows automatically while that page's chunk and its `loader` data are still resolving:\n\n```tsx\n// client/routes/dashboard/loading.tsx\nexport default function Loading() {\n return <p>Loading dashboard...</p>;\n}\n```\n\nPut an `error.tsx` there and it catches any error the page throws, showing a fallback instead of a blank screen. `global-error.tsx` sits outside the root layout, so it catches errors thrown by the layout itself.\n\n## Route groups: shared layout, no URL change\n\nWrap a folder name in parentheses to group files without changing their URLs. This is the trick for \"these three pages share one layout, but I do not want a `/legal` prefix\":\n\n```\nclient/routes/\n (legal)/\n layout.tsx -> wraps both pages below\n privacy.tsx -> /privacy (NOT /legal/privacy)\n terms.tsx -> /terms\n```\n\n## Parallel slots (`@slot`) and intercepting routes\n\nThis is an advanced feature; skip it until you need a modal that keeps the page behind it alive.\n\nA folder starting with `@`, like `@modal`, is a **named slot**. It is a whole second route tree that matches the current URL *independently* of the main page, and renders wherever you place a `<Toil.Slot>`:\n\n```tsx\n// client/routes/gallery/layout.tsx\nimport type { ReactNode } from 'react';\n\nexport default function GalleryLayout({ children }: { children?: ReactNode }) {\n return (\n <div>\n {children}\n <Toil.Slot name=\"modal\" /> {/* renders the @modal slot for this URL */}\n </div>\n );\n}\n```\n\nThe `@` folder adds nothing to the URL; it just says \"these routes fill the slot named `modal`\". A slot with no match renders nothing (or a `fallback` you pass).\n\nAn **intercepting route** is a slot route whose folder name starts with `(.)`, `(..)`, or `(...)`. It hijacks a *soft* (in-app) navigation to another URL and renders that URL's content inside the slot instead, while the main page stays mounted behind it. That is exactly how a \"click a photo, see it in a modal over the gallery\" pattern works, where reloading the page (a hard load) shows the full photo page instead:\n\n```\nclient/routes/gallery/\n index.tsx -> /gallery\n photo/[id].tsx -> /gallery/photo/:id (full page, on hard load)\n @modal/(.)photo/[id].tsx -> fills @modal on a soft click to that URL\n```\n\nThe `(.)` markers mean: `(.)` same level, `(..)` up one level, `(...)` from the routes root. This tells the interceptor which real URL it is standing in for.\n\n## Links and navigation\n\nThis is a quick tour. For the full treatment, programmatic navigation, typed hrefs, prefetching, scroll restoration, and animated transitions, see [Navigation](./navigation.md).\n\n### `Toil.Link`\n\nUse `Toil.Link` instead of a plain `<a>` for in-app links. It navigates client-side (no full page reload), and prefetches the target route's code on hover or focus so the click feels instant:\n\n```tsx\n<Toil.Link href=\"/about\">About</Toil.Link>\n```\n\n`Link` accepts every normal anchor attribute (`className`, `target`, `rel`, `download`, and so on), plus a few toiljs controls:\n\n| Prop | Default | What it does |\n| --- | --- | --- |\n| `replace` | `false` | Replace the current history entry instead of pushing a new one. |\n| `scroll` | `true` | Scroll to the top after navigating. |\n| `prefetch` | `true` | Prefetch the route on hover/focus. Set `false` to opt out. |\n\n`Link` is smart about when *not* to intercept: external URLs, `target=\"_blank\"`, `download`, `#hash` links, and modified clicks (Ctrl/Cmd/middle-click) all fall through to normal browser behavior. The `href` is typed to your project's real routes, so a typo is a compile error.\n\n### `Toil.NavLink` and active state\n\n`NavLink` is a `Link` that knows whether it points at the current page. When active it adds the class `active` (configurable) and `aria-current=\"page\"`. This is what you want for a navigation bar that highlights the current section:\n\n```tsx\n<Toil.NavLink href=\"/blog\" activeClassName=\"is-current\">\n Blog\n</Toil.NavLink>\n```\n\nYou can also drive `className`, `style`, or `children` from the active state with a function:\n\n```tsx\n<Toil.NavLink href=\"/blog\" className={({ isActive }) => (isActive ? 'on' : 'off')}>\n Blog\n</Toil.NavLink>\n```\n\nBy default a parent link is active for its sub-paths too (`/blog` is active on `/blog/42`). Pass `end` to require an exact match:\n\n```tsx\n<Toil.NavLink href=\"/\" end>Home</Toil.NavLink>\n```\n\n### Navigating in code\n\nFor navigation that is not a link (after a form submit, a redirect), use the router hook or the free `navigate` function:\n\n```tsx\nexport default function Login() {\n const router = Toil.useRouter();\n const onDone = () => router.push('/dashboard');\n // router.replace(href), router.back(), router.forward(), router.refresh()\n return <button onClick={onDone}>Continue</button>;\n}\n```\n\n`useRouter()` returns a handle with `push`, `replace`, `back`, `forward`, `refresh` (re-run the current page's data loader), `revalidate` (refetch data), and `prefetch`.\n\nIf you just need to jump to a URL and nothing else, `Toil.useNavigate()` returns the bare `navigate(href, options)` function:\n\n```tsx\nexport default function Login() {\n const navigate = Toil.useNavigate();\n return (\n <button onClick={() => navigate('/dashboard', { replace: true })}>\n Continue\n </button>\n );\n}\n```\n\nThe options are `{ replace?: boolean; scroll?: boolean }`, the same two controls `Toil.Link` exposes above. The same function is also available free of any hook as `Toil.navigate(href, options)`, which is handy in code that is not a React component (a plain event handler, a utility module).\n\n### Typed hrefs and the `href()` escape hatch\n\nEvery href you pass to `Toil.Link`, `navigate`, or `router.push` is type-checked against your project's real routes. The compiler scans `client/routes/` and generates a file called `toil-routes.d.ts` that registers the union of your route paths (it fills in the `Href` type). So a typo is a compile error before you ever run the app:\n\n```tsx\n<Toil.Link href=\"/abuot\">About</Toil.Link>\n// ^ Type error: \"/abuot\" is not assignable to type 'Href'. (No such route.)\n```\n\nDynamic routes show up in that union as template-literal types. A file at `client/routes/blog/[id].tsx` contributes the type `` `/blog/${string}` ``, so a link built with one interpolation usually still checks:\n\n```tsx\n<Toil.Link href={`/blog/${post.id}`}>Read</Toil.Link> // fine: matches `/blog/${string}`\n```\n\nThe catch is a URL assembled from several data pieces (or one TypeScript cannot prove the shape of): it is typed as a plain `string`, and `string` is not assignable to `Href`, so you get a type error:\n\n```tsx\nconst cat = product.category; // string\nconst slug = product.slug; // string\nnavigate(`/${cat}/${slug}`);\n// ^ Argument of type 'string' is not assignable to parameter of type 'Href'.\n```\n\nThe escape hatch is `Toil.href()`. It takes a `string` and asserts it is a valid href, returning the `Href` type the navigation APIs expect. Use it right at the call site once you are sure the path is a real route:\n\n```tsx\nnavigate(Toil.href(`/${cat}/${slug}`)); // asserted valid\n<Toil.Link href={Toil.href(`/${cat}/${slug}`)}>Open</Toil.Link>;\n```\n\n`href()` is a pure type assertion (it returns the string unchanged), so reach for it only when the type-check is in your way, not to paper over a genuine typo. Before the routes are generated (a fresh project, the first build), `Href` is just `string`, so nothing complains until `toil-routes.d.ts` exists.\n\n### Reading the current location\n\nA handful of hooks let a component read where it is:\n\n| Hook | Returns |\n| --- | --- |\n| `Toil.useParams()` | The dynamic route params, e.g. `{ id }`. |\n| `Toil.usePathname()` | The current path, e.g. `\"/blog/42\"`. |\n| `Toil.useLocation()` | The current path, e.g. `\"/blog/42\"` (an alias of `usePathname()`). |\n| `Toil.useSearchParams()` | The query string as a `URLSearchParams`. |\n| `Toil.useNavigationPending()` | `true` while a navigation is in flight (for a loading bar). |\n\n## Loading data for a route\n\nA route file can `export const loader` alongside its component. The loader runs on navigation, in parallel with loading the page's code, and the page reads the result with `Toil.useLoaderData`. This keeps data fetching out of `useEffect` and lets `loading.tsx` show while it runs. Loaders are covered in depth in [Fetching data](./data-fetching.md):\n\n```tsx\nexport const loader = async ({ params }: Toil.LoaderArgs) => {\n const post = await Server.REST.blog.get({ params: { id: params.id } });\n return post;\n};\n\nexport default function BlogPost() {\n const post = Toil.useLoaderData<typeof loader>();\n return <article><h1>{post.title}</h1></article>;\n}\n```\n\n## Gotchas\n\n- **`export default` is required.** A route file without a default-exported component is not a page. The special files (`layout`, `loading`, and so on) also use the default export.\n- **Reserved filenames are not pages.** `layout.tsx`, `template.tsx`, `loading.tsx`, `error.tsx`, `global-error.tsx`, `404.tsx`, and `not-found.tsx` are helpers, not routes. Do not name a real page one of these.\n- **Use `Toil.Link`, not `<a>`, for in-app links.** A plain `<a href=\"/about\">` triggers a full page reload, throwing away the SPA speed. Reserve `<a>` for external links.\n- **Params are always strings.** `useParams()` gives you `{ id: \"42\" }`, not a number. Convert if you need a number.\n- **Route groups and slots are invisible in the URL.** `(group)` and `@slot` folders never appear in the address bar. If a URL looks wrong, check for a stray bracket or paren in the file path.\n\n## Related\n\n- [Rendering and SSR](./rendering.md): what renders on the server versus the browser.\n- [Fetching data](./data-fetching.md): loaders, the typed backend clients, and forms.\n- [Metadata and SEO](./metadata.md): set the title and tags per route.\n- [Backend HTTP routes](../backend/rest.md): the separate, decorator-based server routing.\n",
|
|
44
45
|
"frontend/scripts.md": "# Scripts\n\n`Toil.Script` loads an external or inline `<script>` for you, with control over *when* it runs and a guarantee it runs only *once* across your whole app. Use it instead of a hand-written `<script>` tag for third-party snippets (analytics, chat widgets, embeds). It is the toiljs analog of Next.js `next/script`.\n\n## Why not just write a `<script>` tag?\n\nDropping a raw `<script>` into your JSX is unreliable in a single-page app (an app where the browser loads one HTML shell and JavaScript swaps pages in place, with no full reload). Two problems:\n\n1. **It may not execute.** When React inserts a `<script>` element into the page, the browser does not always run it the way it runs scripts present in the original HTML.\n2. **It runs too often.** As the user navigates between routes that both render that script, React can mount it more than once, so an analytics library or a widget initialises twice.\n\n`Toil.Script` fixes both: it injects a real `<script>` into `<head>` so the browser runs it, and it **deduplicates** by a key so a given script executes at most once for the whole life of the app, even across client-side navigations. It renders nothing into your layout.\n\n## The simplest usage: an external script\n\nGive it a `src`. By default it loads once the app is interactive, which is right for most third-party scripts:\n\n```tsx\n// client/layout.tsx\nimport { type ReactNode } from 'react';\n\nexport default function Layout({ children }: { children?: ReactNode }) {\n return (\n <div className=\"app\">\n <Toil.Script src=\"https://cdn.example-analytics.com/analytics.js\" />\n {children}\n </div>\n );\n}\n```\n\nPutting it in the root layout means it loads once for the whole app and stays loaded as the user navigates. For an external script, the dedup key defaults to its `src`, so you never need an `id`.\n\n## Load strategies\n\nThe `strategy` prop decides *when* the script is injected, relative to your app becoming interactive:\n\n| Strategy | When it runs | Use it for |\n| --- | --- | --- |\n| `afterInteractive` (default) | On mount, once the app is running. | Analytics, chat widgets, most third-party scripts. |\n| `lazyOnload` | Deferred until the browser is idle, after the page's `load` event. | Low-priority extras (a feedback button, a social embed) that should not compete with the initial render. |\n| `beforeInteractive` | As early as possible: injected immediately on first mount. | A script other code depends on immediately. |\n\nOne honest caveat about `beforeInteractive`: a toiljs frontend is a client-only single-page app, so there is no server-rendered `<script>` to run before hydration. `beforeInteractive` therefore still runs *after* hydration, just as early and eagerly as possible on the first mount. It is a priority hint, not a true \"before the page is interactive\" guarantee.\n\n```tsx\n<Toil.Script\n src=\"https://widget.example.com/embed.js\"\n strategy=\"lazyOnload\"\n/>\n```\n\n## Inline scripts\n\nTo run a snippet of code instead of loading a URL, put the code in `children` (as a **string**) and give the script an `id`. An inline script has no `src`, so the `id` is what identifies it for dedup, and it is required:\n\n```tsx\n<Toil.Script id=\"init-theme\" strategy=\"beforeInteractive\">\n {`document.documentElement.dataset.theme =\n localStorage.getItem('theme') ?? 'light';`}\n</Toil.Script>\n```\n\nWithout an `id` (and no `src`), an inline script has nothing to dedup on, so `Toil.Script` does nothing at all. This is a deliberate no-op, not an error, so remember the `id`.\n\n## Reacting to load\n\nThree optional callbacks let you run code around the script's lifecycle:\n\n```tsx\n<Toil.Script\n src=\"https://widget.example.com/embed.js\"\n strategy=\"lazyOnload\"\n onReady={() => {\n // The global the script defines is now available.\n window.MyWidget?.init();\n }}\n onError={(err) => {\n console.warn('widget failed to load', err);\n }}\n/>\n```\n\n- `onLoad` fires **once**, when an external script finishes loading (or an inline script is inserted).\n- `onReady` fires after load **and on every later mount** once the script is already loaded. So if a route that renders the `Toil.Script` is left and revisited, `onReady` runs again, which is the right place to re-initialise a widget.\n- `onError` fires if an **external** script fails to load. After an error the script's key is cleared, so a later remount retries the load.\n\n## All props\n\nRead them straight from the source (`src/client/components/Script.tsx`):\n\n| Prop | Type | Default | What it does |\n| --- | --- | --- | --- |\n| `src` | `string` | (none) | URL of an external script. Omit it when you provide an inline body via `children`. |\n| `children` | `string` | (none) | Inline script body. Mutually exclusive with `src`. |\n| `strategy` | `'beforeInteractive' \\| 'afterInteractive' \\| 'lazyOnload'` | `'afterInteractive'` | When to inject the script (see above). |\n| `id` | `string` | `src` for external scripts | Stable identity for dedup. **Required** for inline scripts. |\n| `type` | `string` | (none) | The `type` attribute, e.g. `'module'` or `'application/json'`. |\n| `onLoad` | `() => void` | (none) | Fired once the script has loaded (external) or been inserted (inline). |\n| `onReady` | `() => void` | (none) | Fired after load, and on every later mount once the script is already loaded. |\n| `onError` | `(error: unknown) => void` | (none) | Fired if an external script fails to load. |\n\nExternal scripts are injected with `async` set, so they never block other work while downloading.\n\n## Gotchas\n\n- **Inline scripts need an `id`.** No `id` and no `src` means nothing to dedup on, so the component quietly does nothing.\n- **Dedup is app-wide and survives navigation.** The same `Toil.Script` rendered on two different routes runs a total of once, keyed by `id` (or `src`). This is the point, but it means you cannot use two copies of the same key to run something twice.\n- **Props are read at injection time, and the script re-injects only if its key or strategy changes.** Changing a handler or an inline body *without* changing the `id` will not re-run the script. If you truly need to re-inject a changed inline script, change its `id`.\n- **`onReady` versus `onLoad`.** Use `onLoad` for one-time setup that must happen exactly once; use `onReady` for setup that should also run each time a component remounts against an already-loaded script.\n- **`beforeInteractive` is not truly before hydration** in a client-only app (see the caveat above). Do not rely on it running before your React tree mounts.\n- **`Toil.Script` renders nothing.** It returns `null`, so you can place it anywhere in your component tree; a layout is the usual home for app-wide scripts.\n\n## Related\n\n- [Rendering and SSR](./rendering.md): how the client-only app and its first paint fit together.\n- [Metadata and SEO](./metadata.md): for `<head>` tags (title, meta, Open Graph), which is a different job from loading scripts.\n- [Frontend overview](./README.md): the `Toil` global and the rest of the client API.\n",
|
|
45
46
|
"frontend/search.md": "# Page search\n\ntoiljs ships a built-in search over your own pages: at build time it indexes every route's metadata (title, description, keywords, Open Graph), and at runtime you query that index to build a search box or a command palette that jumps straight to a page. It is entirely static (no server, no search service), and there is no equivalent in Next.js, so this page explains it from scratch.\n\n## What it is\n\nEvery route in your app can export `metadata` (its title, description, and so on, see [Metadata and SEO](./metadata.md)). The toiljs compiler reads that metadata from all your routes at build time and bakes a small **page index** into your bundle: a list of `{ path, title, description, keywords, ... }` for every page. When your app starts, that index is registered in the browser. You can then search it instantly, offline, with zero network calls, and turn a match into a navigation.\n\nThis is what powers a \"jump to any page\" box or a `Cmd+K` command palette without you maintaining a list of pages or standing up a search backend.\n\n```mermaid\nflowchart TD\n A[\"Your routes<br/>each with export const metadata\"] -->|build: scan + extract| B[\"Static page index<br/>{ path, dynamic, metadata } per page\"]\n B -->|baked into the bundle| C[\"App startup: Toil.registerPages(index)\"]\n C --> D[\"Browser: usePageSearch / searchPages query the index\"]\n D --> E[\"Ranked results -> goTo() navigates\"]\n```\n\nOnly **statically-known** metadata is indexed. A route whose `<head>` comes from a dynamic `generateMetadata` (a per-request title) has nothing to index by default; to include it anyway, export `searchHints` (see below).\n\n## The `usePageSearch` hook\n\n`usePageSearch` is the React way in. Give it the current query string, and it hands back ranked results plus a helper to navigate:\n\n```tsx\nconst { results, pages, goTo } = Toil.usePageSearch(query);\n```\n\nIt returns an object with three fields:\n\n| Field | Type | What it is |\n| --- | --- | --- |\n| `results` | `readonly PageSearchResult[]` | The ranked matches for `query`, best first. Empty when the query is blank. |\n| `pages` | `readonly PageMeta[]` | The full page index, handy for showing an \"all pages\" listing. |\n| `goTo` | `(target, options?) => void` | Navigates to a result, a page, or a raw path string. A stable reference (safe to pass to a child or destructure). |\n\nEach item in `results` is a `PageSearchResult`:\n\n- `page`: the matched `PageMeta`, which is `{ path, dynamic, metadata }`. `path` is the route URL (`'/about'`), `dynamic` says whether it has `:param` segments, and `metadata` is the indexed title/description/etc.\n- `score`: a relevance number (higher is better; always above zero for a returned result).\n- `matches`: which fields matched, for example `['title', 'keywords']`.\n\nThe results are memoised, so they recompute only when the query (or options) change, not on every render.\n\n### A full search box\n\nHere is a complete search page: an input, a ranked result list, and a click that navigates to the chosen page.\n\n```tsx\n// client/routes/search.tsx\nimport { useState } from 'react';\n\nexport const metadata: Toil.Metadata = {\n title: 'Search',\n description: 'Find any page and jump straight to it.',\n keywords: ['search', 'find', 'pages'],\n};\n\nexport default function SearchPage() {\n const [query, setQuery] = useState('');\n const { results, pages, goTo } = Toil.usePageSearch(query);\n\n return (\n <main>\n <h1>Search</h1>\n <input\n type=\"search\"\n value={query}\n onChange={(e) => {\n setQuery(e.target.value);\n }}\n placeholder={`Search ${pages.length} pages...`}\n aria-label=\"Search pages\"\n autoFocus\n />\n\n {query.trim() !== '' && (\n <ul>\n {results.length === 0 && <li>No pages match \"{query}\".</li>}\n {results.map((r) => (\n <li key={r.page.path}>\n <button\n type=\"button\"\n onClick={() => {\n goTo(r);\n }}\n >\n <strong>{r.page.metadata.title ?? r.page.path}</strong>{' '}\n <code>{r.page.path}</code>\n {r.page.metadata.description !== undefined && (\n <p>{r.page.metadata.description}</p>\n )}\n </button>\n </li>\n ))}\n </ul>\n )}\n </main>\n );\n}\n```\n\n`goTo` accepts a result (as above), a `PageMeta`, or a plain path string, and takes the same options as `navigate` (see [Routing](./routing.md)). Passing a result is the common case.\n\n### Options\n\n`usePageSearch` takes a second argument to tune the search:\n\n```tsx\nconst { results } = Toil.usePageSearch(query, {\n limit: 8, // cap the number of results (after ranking)\n includeDynamic: true, // include :param routes (see below); default false\n fields: ['title', 'keywords'], // only match these fields; default all fields\n});\n```\n\n| Option | Type | Default | Effect |\n| --- | --- | --- | --- |\n| `limit` | `number` | no cap | Keep only the top N results after ranking. |\n| `includeDynamic` | `boolean` | `false` | Include dynamic (`:param` / `*catch-all`) routes. Off by default because you cannot navigate to them without filling in the params. |\n| `fields` | array of field names | all fields | Restrict matching to a subset of `'title'`, `'description'`, `'keywords'`, `'path'`, `'openGraph'`. |\n\n## Making dynamic routes searchable with `searchHints`\n\nA dynamic route like `client/routes/blog/[id].tsx` usually produces its `<head>` with a per-post `generateMetadata`, so there is nothing static to index, and the route is missing from search. To surface it anyway, export `searchHints`: a small static object the compiler merges into the index for that route (winning ties against any static `metadata`).\n\n```tsx\n// client/routes/blog/[id].tsx\n\n// The per-post <title> is dynamic, so it cannot be indexed. These static hints\n// put the blog into the search index so it shows up when someone searches \"blog\".\nexport const searchHints: Toil.SearchHints = {\n title: 'Blog',\n description: 'Articles and updates.',\n keywords: ['blog', 'posts', 'articles'],\n};\n```\n\n`SearchHints` has three optional fields: `title`, `description`, and `keywords` (a string or an array of strings).\n\nTwo things to keep in mind. First, `searchHints` only affects the index; the route is still dynamic, so it appears in results only when you pass `includeDynamic: true`. Second, you cannot `goTo` a dynamic page directly (it needs concrete params), so `goTo` is a no-op for one unless you hand it a real, filled-in path string. In practice you use `searchHints` to make a *section* discoverable (typing \"blog\" surfaces the blog), and point the user at a concrete landing URL.\n\n## The lower-level API\n\nThe hook is a thin wrapper over a small, framework-agnostic core you can use directly (outside React, in tests, or to build your own UI). All of these are on the `Toil` global:\n\n- `Toil.searchPages(query, options?)`: the pure ranking function behind the hook. Returns `PageSearchResult[]`. Same options as the hook.\n- `Toil.getPages()`: the full registered index (`readonly PageMeta[]`), including dynamic routes. Empty before registration.\n- `Toil.pagePath(target)`: normalises a result, a page, or a raw string down to its path string.\n- `Toil.registerPages(pages)`: replaces the live index. toiljs calls this for you at startup with the compiler-built index, so you rarely call it yourself. It exists for tests and advanced setups that build a custom index.\n\n```tsx\nimport { searchPages } from 'toiljs/client';\n\n// Outside a component: the top 5 title/keyword matches for \"billing\".\nconst hits = searchPages('billing', { limit: 5, fields: ['title', 'keywords'] });\n```\n\n## How ranking works\n\nThe matcher is simple and predictable:\n\n- The query is lower-cased and split on whitespace into terms. **Every** term must match somewhere (AND semantics), or the page is dropped. A blank query returns nothing.\n- Matching is case-insensitive and substring-based, with bonuses: an exact field match ranks highest, then a match at the start of the field, then a match at the start of a word inside the field, then a plain mid-word substring.\n- Each field carries a weight, so a hit in the title counts for much more than a hit in the description:\n\n| Field | Weight |\n| --- | --- |\n| `title` | 10 |\n| `path` | 6 |\n| `keywords` | 5 |\n| `description` | 3 |\n| `openGraph` | 2 |\n\n- The `path` field is made word-searchable: `/get-started` matches \"get\" or \"started\".\n- Results are sorted by score (highest first), ties broken alphabetically by path for a stable order, then cut to `limit` if you set one.\n\n## Gotchas\n\n- **Only static metadata is indexed.** A route whose title comes from `generateMetadata` is invisible to search until you add `searchHints`.\n- **Dynamic routes are excluded by default.** They need `includeDynamic: true` to appear, and even then `goTo` cannot navigate to one without concrete params.\n- **The index is built, not live.** It reflects the metadata at build time. Add a route or change a title and you must rebuild for search to see it.\n- **A blank query returns no results**, not every page. If you want an \"all pages\" listing for an empty box, render `pages` yourself.\n- **The type names** `PageSearchResult`, `PageSearchOptions`, and `SearchField` are importable from `'toiljs/client'` if you need to annotate them; `PageMeta` and `SearchHints` are also available as `Toil.PageMeta` / `Toil.SearchHints`.\n\n## Related\n\n- [Metadata and SEO](./metadata.md): the `metadata` the search index is built from.\n- [Routing](./routing.md): route paths, dynamic segments, and `navigate` (what `goTo` calls).\n- [Rendering and SSR](./rendering.md): where `searchHints` fits alongside `generateStaticParams` for dynamic routes.\n- [Frontend overview](./README.md): the `Toil` global and the rest of the client API.\n",
|
|
@@ -65,7 +66,7 @@ export const TOIL_DOCS: Record<string, string> = {
|
|
|
65
66
|
"realtime/streams.md": "# Streams (`@stream`)\n\nA `@stream` is a server class that handles one live connection from start to finish. You mark a class with `@stream`, add lifecycle hooks, and the Dacely edge keeps one instance of that class alive for as long as the browser stays connected.\n\n## What a stream is (and why it is different)\n\nWhen you write an [HTTP route](../backend/rest.md), the server builds a **fresh** handler for every request and throws it away afterward. Anything you stored on the handler's fields is gone the moment the response is sent. That is perfect for one-shot requests, but it means the handler cannot \"remember\" anything between requests on its own.\n\nA `@stream` is the opposite. It is a **resident box**: one live instance, created when a connection opens and kept alive until it closes. Because it is the *same* instance for every message on that connection, values you store on its fields **persist across messages**. That is what makes it the right tool for a conversation, a game session, or anything stateful that lasts for the life of a connection.\n\nThe word \"resident\" just means \"stays in memory and keeps running.\" The word \"box\" is toiljs's name for one sandboxed instance of your compiled server code.\n\n```ts\n@stream('echo')\nclass Echo {\n private count: i32 = 0;\n\n @connect\n onConnect(): void {\n this.count = 0; // a fresh connection starts a fresh box, so count begins at 0\n }\n\n @message\n onMessage(packet: StreamPacket): StreamOutbound {\n this.count = this.count + 1; // survives across messages: same box every time\n const text = 'pong #' + this.count.toString();\n return StreamOutbound.reply(Uint8Array.wrap(String.UTF8.encode(text)));\n }\n\n @close\n onClose(): void {\n // the box is destroyed after this hook runs\n }\n}\n```\n\nConnect, send three messages, and you get back `pong #1`, `pong #2`, `pong #3`. The advancing number is proof that the same box handled all three.\n\n## Declaring a stream\n\nMark a class with `@stream` and give it a **name**. The name becomes the route the browser connects to.\n\n```ts\n@stream('echo') // mounted at /echo\nclass Echo { /* ... */ }\n\n@stream // bare form: the route is the class name (/Echo)\nclass Echo { /* ... */ }\n```\n\nThere are three forms:\n\n- `@stream('name')`: an explicit mount name (connect at `/name`).\n- `@stream` (bare): the mount name is the class name.\n- `@stream({ ... })`: a config object (see [Configuration](#configuration) below).\n\n## The four lifecycle hooks\n\nA stream method becomes a lifecycle hook when you tag it with one of these decorators. Every hook is **optional**: declare only the ones you need, and a missing hook is simply a no-op (it does nothing, it never crashes).\n\n| Decorator | Fires when... |\n| ------------- | ----------------------------------------------------------------------- |\n| `@connect` | the connection opens (the box has just been created). |\n| `@message` | an inbound frame arrives from the browser. |\n| `@close` | the connection closes cleanly (the box is destroyed after this hook). |\n| `@disconnect` | the connection is lost abruptly (network dropped, browser killed). |\n\nA **frame** is one message: one call to `send()` on the client becomes one `@message` on the server.\n\n```mermaid\nsequenceDiagram\n participant B as Browser\n participant S as @stream box\n B->>S: open\n activate S\n Note over S: @connect runs, box is created\n B->>S: frame \"a\"\n S-->>B: @message -> reply\n B->>S: frame \"b\"\n S-->>B: @message -> reply\n B->>S: close\n Note over S: @close runs, box is destroyed\n deactivate S\n```\n\n### `@connect`\n\nRuns once, right after the box is created. Use it to set up per-connection state (reset a counter, read the requested path, decide whether to accept the connection). It can return a `StreamOutbound` to accept or reject (see below). The Echo example uses it to zero its counter.\n\n**What it receives.** `@connect` is handed a `StreamInbound`, a small read-only object the host fills in with the details of the connection that just opened. It lets you look at *where* the connection came from before you decide to keep it.\n\n| Member | Type | What it gives you |\n| ------------- | -------- | ----------------------------------------------------------------------- |\n| `streamId` | `u64` | A unique id for this connection. |\n| `transport` | `i32` | A numeric tag for the transport that carried the connection. |\n| `authority()` | `string` | The host the browser connected to (for example `example.com`). |\n| `path()` | `string` | The path the browser opened (for example `/echo`). |\n\nWatch the shape: `authority()` and `path()` are **methods** (call them with `()`), while `streamId` and `transport` are plain properties (no parentheses).\n\n```ts\n@connect\nonConnect(info: StreamInbound): StreamOutbound {\n // Inspect where the connection is headed, then decide whether to keep it.\n if (info.path() != '/echo') {\n return StreamOutbound.reject(1); // refuse: any u16 reason code\n }\n this.count = 0; // fresh connection, fresh state\n return StreamOutbound.accept(); // keep the connection open\n}\n```\n\nYou do not have to take the argument. If the hook needs none of these details, declare it with no parameter (`onConnect(): void`), exactly like the Echo example above.\n\n### `@message`\n\nRuns for every inbound frame. This is where most of your logic lives. It receives the frame and may reply. Details in [Reading and replying](#reading-and-replying-to-messages).\n\n### `@close` and `@disconnect`\n\nBoth mean \"the connection is over,\" and both are your chance to clean up. The difference is *how* it ended:\n\n- `@close` is a **graceful** close: the browser (or your server) ended it on purpose.\n- `@disconnect` is an **abrupt** loss: the network dropped or the tab was killed with no goodbye.\n\nAfter either one, the box is destroyed.\n\n**What they receive.** Both hooks are handed a `StreamConnectionEvent`, a read-only summary of the connection that just ended.\n\n| Member | Type | What it gives you |\n| -------------- | ----- | ---------------------------------------------------------- |\n| `connectionId` | `u64` | The id of the connection that ended. |\n| `reason` | `u16` | A numeric close code explaining why it ended. |\n| `durationMs` | `u64` | How long the connection stayed open, in milliseconds. |\n\nAll three are plain properties (getters), so read them without `()`.\n\n```ts\n@close\nonClose(ev: StreamConnectionEvent): void {\n // Last chance to clean up. `ev` tells you how the connection ended.\n const heldSeconds = ev.durationMs / 1000;\n // e.g. record the session length or flush a buffer here.\n}\n\n@disconnect\nonDisconnect(ev: StreamConnectionEvent): void {\n // Same shape, but this fired because the connection dropped abruptly.\n // ev.reason carries the close code the edge assigned to the drop.\n}\n```\n\nAs with `@connect`, the argument is optional: declare `onClose(): void` if you do not need it.\n\n## Per-connection state (and its limits)\n\nState on the box's fields lasts for **one connection**. It does **not** survive:\n\n- a **reconnect**: if the browser drops and reopens, it gets a brand-new box that starts clean.\n- a **different user**: every connection gets its own box, so one connection's state can never leak into another. This is a safety property, not just a convenience.\n\nSo treat box fields as **per-connection scratch space** only. For anything that must outlive the connection (a saved message, a score, who a user is across reconnects), write it to [the database](../database/README.md), not to a class field.\n\n## Reading and replying to messages\n\nBy default, a `@message` receives a `StreamPacket`, which is a thin view over the raw bytes that arrived, and returns a `StreamOutbound`, which stages the reply.\n\n```ts\n@message\nonMessage(packet: StreamPacket): StreamOutbound {\n const raw = packet.bytes(); // the inbound frame as bytes\n return StreamOutbound.reply(raw); // echo the same bytes back\n}\n```\n\n`StreamPacket` (the inbound frame):\n\n| Member | What it gives you |\n| ------------ | ---------------------------------------------------------- |\n| `bytes()` | the whole frame as a `Uint8Array` (copy it if you keep it). |\n| `length` | the number of bytes in the frame. |\n| `at(i)` | the byte at index `i`. |\n\n`StreamOutbound` (what you return):\n\n| Call | Meaning |\n| ----------------------------- | ------------------------------------------------------------------ |\n| `StreamOutbound.reply(bytes)` | send one frame back to the browser. |\n| `StreamOutbound.empty()` | accept the frame and send nothing back. |\n| `StreamOutbound.reject(code)` | refuse (used from `@connect` to turn a connection away). |\n| `StreamOutbound.accept()` | accept a connection with no reply frame. |\n\nA `@message` may also return `void` when it never replies.\n\n> **Bytes, not strings.** A frame is raw bytes on the wire. To send text, encode it with `String.UTF8.encode(...)` (server) or `new TextEncoder().encode(...)` (browser), and decode it with `new TextDecoder().decode(...)` on the other side.\n\n### Typed messages\n\nRaw bytes are flexible but fiddly. If your messages are structured, declare a [`@data`](../backend/data.md) class and pass it as the stream's `message` type. Your `@message` hook then receives the **decoded object** instead of raw bytes.\n\n```ts\n@data\nclass ChatMsg {\n text: string = '';\n constructor(text: string = '') { this.text = text; }\n}\n\n@stream({ message: ChatMsg })\nclass Chat {\n @message\n onMessage(msg: ChatMsg): StreamOutbound { // decoded @data, not raw bytes\n const echoed = 'you said: ' + msg.text;\n return StreamOutbound.reply(Uint8Array.wrap(String.UTF8.encode(echoed)));\n }\n}\n```\n\nThe reply is still raw (`StreamOutbound` deals in bytes). Only the **inbound** side is decoded for you. On the client, a typed stream lets you `send(new ChatMsg('hi'))` and toiljs encodes it for you.\n\n## Games and interactive apps\n\nA stream box remembers state for the life of a connection, which is exactly what a game session needs. Each connected player gets their **own** box, and that box holds the player's live state (position, health, score) in memory, right next to the core handling their packets. The lifecycle hooks map cleanly onto a session: `@connect` spawns the player, `@message` handles each input, `@close` and `@disconnect` remove them.\n\n```ts\n// server/streams/Match.ts\n@data\nclass Move { // the input a player sends each tick\n dx: i32 = 0;\n dy: i32 = 0;\n constructor(dx: i32 = 0, dy: i32 = 0) { this.dx = dx; this.dy = dy; }\n}\n\n@stream({ message: Move, scope: StreamScope.Regional })\nclass Match {\n // Per-connection state: this player's position. It lives in memory for the\n // whole session, on the one core that owns this connection.\n private x: i32 = 0;\n private y: i32 = 0;\n private moves: u32 = 0;\n\n @connect\n onConnect(info: StreamInbound): StreamOutbound {\n this.x = 50; // spawn point\n this.y = 50;\n this.moves = 0;\n return StreamOutbound.accept();\n }\n\n @message\n onMove(move: Move): StreamOutbound {\n // Apply the input to this player's live position, then confirm it.\n this.x = this.x + move.dx;\n this.y = this.y + move.dy;\n this.moves = this.moves + 1;\n const state = '{\"x\":' + this.x.toString() + ',\"y\":' + this.y.toString() + '}';\n return StreamOutbound.reply(Uint8Array.wrap(String.UTF8.encode(state)));\n }\n\n @disconnect\n onDrop(ev: StreamConnectionEvent): void {\n // The player left (tab closed, network died). Their box is destroyed\n // next. Persist a score here if it must outlive the session.\n }\n}\n```\n\nEvery input a player sends is one `@message`, applied to **their** box's position and confirmed straight back to them with low latency. Because the box is resident, the player's position is always there in memory, on the same core, with no database round trip per move. That is what makes fast input loops (a game tick, a cursor drag, a live editor) feel instant.\n\n**What this version does, and does not, do.** Each player has their own box, so this handles per-player input, per-player state, and instant confirmation back to that player. To make players see **each other** (the other half of multiplayer), one player's move must fan out to everyone else in the match. That cross-connection broadcast is the `@channel` feature, which is **planned, not live yet** (see [Channels](./channels.md)). Until it ships, the common patterns are to write shared match state to [the database](../database/README.md) and have clients read it, or to keep a match to a single authoritative box. The per-connection pieces above (input, state, and presence via `@connect` / `@disconnect`) work today.\n\nFor **presence** (\"who is online\"), `@connect` and `@disconnect` are your join and leave signals: bump a counter or write a row when a player connects, and undo it when they drop.\n\n## The `main.stream.ts` file (a separate tier)\n\nStreams live in their **own entry file**, `server/main.stream.ts`, separate from the request entry `server/main.ts`. Importing your `@stream` classes there pulls them into a **separate compiled artifact**, `build/server/release-stream.wasm`.\n\n```ts\n// server/main.stream.ts\nimport { revertOnError } from 'toiljs/server/runtime/abort/abort';\n\nimport './streams/Echo'; // add each @stream module here as you grow\nimport './streams/Chat';\n\n// Re-export the WASM entry points the host binds, exactly like main.ts.\nexport * from 'toiljs/server/runtime/exports';\nexport function abort(message: string, fileName: string, line: u32, column: u32): void {\n revertOnError(message, fileName, line, column);\n}\n```\n\nWhy a separate file? A stream box and a request handler are **different kinds of program** that run on different parts of the edge (see [Compute tiers](../concepts/tiers.md)). toiljs compiles each into its own `.wasm`:\n\n```sh\n$ toiljs build\n$ ls build/server/*.wasm\nbuild/server/release.wasm # L1 request (@rest / @service)\nbuild/server/release-stream.wasm # L2/L3 stream (@stream)\nbuild/server/release-cold.wasm # L4 daemon (@daemon)\n```\n\nYou do not run this by hand. `toiljs build` produces `release-stream.wasm` automatically whenever your project has a `@stream` surface, and shared helper code and `@data` types are compiled into every artifact.\n\n> **One file cannot be both a stream and an RPC surface.** A single source file may not declare both a `@stream` and a `@service` / `@remote` ([RPC](../backend/rpc.md)), because one compiled artifact cannot be two tiers at once. Keep them in separate files: `@stream` in `main.stream.ts`, `@rest` / `@service` in `main.ts`. They coexist happily side by side in the same project, just not in the same file.\n\n## Configuration\n\nThe config-object form lets you tune a stream:\n\n```ts\n@stream({\n scope: StreamScope.Regional, // where the box runs (see below)\n message: ChatMsg, // decode inbound frames into this @data type\n maxFrameBytes: 65536, // reject frames larger than this\n ingressRingBytes: 262144 // size of the inbound buffer\n})\nclass Chat { /* ... */ }\n```\n\n- **`scope`** picks how close to the user the box runs. `StreamScope.Regional` (the default) runs it at a regional node; `StreamScope.Continental` runs it at a wider continental node. See [Compute tiers](../concepts/tiers.md) for what L2 and L3 mean.\n- **`message`** is the typed-message shortcut described above.\n- **`maxFrameBytes`** and **`ingressRingBytes`** cap frame size and buffer size to protect the box from oversized or flooding input.\n\n## Reaching a stream from the browser\n\nEvery `@stream` class gets a generated, typed client at `Server.Stream.<ClassName>`, wired up for you in `shared/server.ts` (the same place the [RPC](../backend/rpc.md) client lands). Call `connect()` to open the connection:\n\n```ts\nimport '../shared/server'; // attaches globalThis.Server (browser-only)\n\nconst chat = await Server.Stream.Echo.connect();\nchat.onMessage((bytes) => { /* a reply frame, always raw bytes */ });\nchat.send(new TextEncoder().encode('hello'));\nchat.onClose((code) => { /* the connection ended */ });\nchat.close();\n```\n\nThe client is keyed by the **class name** (`Server.Stream.Echo`) and connects to the class's **mount route** (`/echo`). Inbound replies are always raw bytes. The full client walkthrough, plus the lower-level `useChannel` hook, is in [Channels](./channels.md).\n\n## How placement works (you do not manage it)\n\nOn the production edge, your box lives in **one place** for the connection's whole life. Every message from that connection is routed to the exact instance holding your box, so its in-memory state is always there. You never configure this; the edge does it automatically. In `toiljs dev` there is only one process, so this is a non-issue.\n\n## Why this scales\n\nA `@stream` box is deliberately cheap to run at scale, and the edge is built to spread a very large number of them across many machines and cities. Three design choices do the heavy lifting:\n\n- **Connections spread across the fleet in parallel.** Each connection is handled on its own, with no shared lock in the middle for connections to fight over. There is no central bottleneck every connection has to pass through, so adding machines adds capacity in a straight line.\n- **The connection survives network changes.** If a client's network changes (Wi-Fi to cellular, or a NAT rebind), the edge keeps the connection attached to the same box. The user keeps their session and their in-memory state; the box is never orphaned.\n- **Each box is isolated and bounded.** Every connection gets its own sandboxed box with its own linear memory. One tenant's boxes are capped to a slice of the node's RAM (a noisy-neighbor guard), and a single box is hard-capped (64 MiB) so a runaway connection can only fill itself, then it traps. Each lifecycle hook also runs under a hard compute cap, so a hook that loops forever is stopped instead of hogging the machine.\n\nPut together: connections spread across the fleet, run in parallel, the session sticks to its box even when the network moves, and each box is walled off from the others. That is what lets one deployment hold a very large number of live connections at once. How large depends on your hardware and message sizes; these docs do not publish a benchmarked number.\n\n> **The wider picture.** For how the edge spreads connections across the fleet and the mesh adds up to world-wide fan-out, see [Built for massive fan-out and world-wide sync](./README.md#built-for-massive-fan-out-and-world-wide-sync) in the overview.\n\n## Gotchas\n\n- **Box fields are per-connection only.** They reset on reconnect and are never shared between users. Persist anything durable to [the database](../database/README.md).\n- **Frames are bytes.** Encode and decode text yourself, or use a typed `message` so toiljs does it.\n- **Copy `packet.bytes()` if you keep it.** The inbound buffer is reused after the hook returns, so store a copy if you need the bytes later.\n- **A file cannot mix `@stream` with `@service` / `@remote`.** Keep streams in `main.stream.ts`.\n- **`@channel` is not live yet.** A stream that declares a `@channel` hook is rejected by the edge today. Broadcasting to many subscribers is a planned feature; see [Channels](./channels.md).\n\n## Related\n\n- [Realtime overview](./README.md): the big picture and when to reach for realtime.\n- [Channels](./channels.md): the client `useChannel` hook and a chat-style example.\n- [Compute tiers (L1 to L4)](../concepts/tiers.md): where the stream artifact runs.\n- [Data types (`@data`)](../backend/data.md): typed messages.\n- [The database (ToilDB)](../database/README.md): where to keep state that outlives a connection.\n",
|
|
66
67
|
"services/analytics.md": "# Analytics\n\nRead your site's own usage numbers (requests, bytes, cache hits, database ops, errors, and more) straight from a route, with no extra code and no third-party service.\n\nReach for analytics to build a status or usage dashboard for a site: how much traffic it serves, how effective its cache is, how many database operations it runs. It is **infrastructure metering per domain**, not per-user product analytics. There are no custom events; you read a fixed catalog of counters the edge already keeps for you.\n\n`Analytics` is an **ambient global**: use it with no import, like [`crypto`](./crypto.md) or `Time`.\n\n## What \"metering\" means\n\nEvery time the edge (the Dacely server running your code) serves a request, runs a database operation, opens a stream, or sends an email, it **counts** that into a set of per-domain counters. Those counters live in the same worldwide, distributed database that backs [ToilDB](../database/README.md), so the total for your site is correct across every edge location, not just the one machine that happened to serve a request.\n\n`Analytics.self()` reads a **snapshot** of your site's current counter values. You do not record anything yourself; the numbers are already there.\n\n```mermaid\nflowchart LR\n R[\"Every request, DB op,<br/>stream, email\"] --> C[\"Edge counts it into<br/>per-domain counters\"]\n C --> D[(\"Distributed counters,<br/>worldwide\")]\n D --> S[\"Analytics.self()<br/>reads a snapshot\"]\n S --> Y[\"Your route returns<br/>the numbers\"]\n```\n\n## Read your site's stats\n\n`Analytics.self()` returns a `TenantStats` object. Every counter is a **named getter** that returns a `u64` (an unsigned 64-bit integer, always zero or positive), so mapping the numbers into a response needs no casts at all.\n\n```ts\nimport { RouteContext } from 'toiljs/server/runtime';\n\n@rest('metrics')\nclass Metrics {\n @get('/')\n public overview(ctx: RouteContext): string {\n const s = Analytics.self();\n // Every getter is a u64. cacheRatio is the one f64 (a fraction 0..1).\n return `requests=${s.requests} cacheHits=${s.cacheHits} ratio=${s.cacheRatio}`;\n }\n}\n```\n\nYou can also read any counter by its numeric id with `s.metric(MetricId.Requests)`, which returns `0` for an unknown id. The named getters are preferred: they are self-documenting and there are no magic strings to mistype.\n\n### The counter catalog\n\nEvery getter below is a lifetime running total: it only ever goes up, and reading it never resets it. They are grouped by area. (`MetricId` is the matching numeric id, an ambient enum you can use with no import.)\n\n**Requests and the L1 edge**\n\n| Getter | What it counts |\n| --- | --- |\n| `requests` | HTTP requests served |\n| `bytesOutL1` | Response bytes sent |\n| `bytesInL1` | Request bytes received |\n| `status2xx` | 2xx responses (success) |\n| `status3xx` | 3xx responses (redirects) |\n| `status4xx` | 4xx responses (client errors) |\n| `status5xx` | 5xx responses (server errors) |\n| `staticHits` | Static assets served without running your code |\n| `wasmDispatches` | Times your compiled handler ran |\n| `executorFullRejects` | Requests rejected because the worker pool was full |\n| `unknownHostRejects` | Requests rejected for an unknown host |\n| `rateLimitedRejects` | Requests rejected by rate limiting |\n| `gasUsed` | Total compute \"gas\" your handlers consumed |\n\n**Database (ToilDB)**\n\n| Getter | What it counts |\n| --- | --- |\n| `dbOps` | Database operations issued |\n| `dbReads` | Read operations |\n| `dbWrites` | Write operations |\n| `dbErrors` | Operations that errored |\n| `dbLatencyNsSum` | Summed operation latency, in nanoseconds |\n| `meanDbLatencyNs` | Derived: `dbLatencyNsSum / dbOps` (0 with no ops) |\n\n**Streams (WebTransport realtime)**\n\n| Getter | What it counts |\n| --- | --- |\n| `streamAccepts` | Stream connections accepted |\n| `streamRejectWrongNode` | Streams that reached the wrong node |\n| `streamRejectCapacity` | Streams rejected for capacity |\n| `streamRejectArtifact` | Streams rejected for a missing or invalid build |\n| `streamRejectGuest` | Streams your code rejected |\n| `streamTraps` | Stream handler crashes |\n| `streamIdleTimeouts` | Streams closed for being idle |\n| `streamBytesIn` / `streamBytesOut` | Bytes received / sent on streams |\n| `streamBackpressureEvents` | Times a stream had to slow down |\n| `streamCloses` | Clean closes |\n| `streamDisconnects` | Abrupt disconnects |\n\n**Daemons (L4 background jobs)**\n\n| Getter | What it counts |\n| --- | --- |\n| `daemonStarts` / `daemonStartFailures` | Daemon starts / failed starts |\n| `daemonTicksFired` | Scheduled ticks that ran |\n| `daemonTicksSkippedNotLeader` | Ticks skipped because this node was not the leader |\n| `daemonTicksFailed` | Ticks that failed |\n| `daemonLeaderAcquires` / `daemonLeaderFenced` | Leadership gained / lost |\n| `daemonHttpCallAttempts` / `daemonHttpCallFailures` | Outbound HTTP calls attempted / failed |\n\n**Memory, email, and cache**\n\n| Getter | What it counts |\n| --- | --- |\n| `memGrownBytes` | Wasm memory grown, in bytes |\n| `emails` | Emails sent |\n| `cacheHits` | Responses served from the edge cache |\n| `cacheMisses` | Cacheable responses that missed the cache |\n| `cacheRatio` | Derived **`f64`**: `cacheHits / (cacheHits + cacheMisses)`, a fraction from 0 to 1 (0 when there were no cacheable responses) |\n\n### Live gauges and request windows\n\nA few fields are not lifetime totals. They still read as `u64`.\n\n**Live gauges** are the current level right now, not a running total:\n\n- `connectedStreams`: streams connected at this moment.\n- `committedMemory`: wasm memory committed right now, in bytes.\n\n**Request windows** show your current rate-limit usage against your plan cap (a cap of `0` means unlimited):\n\n- `reqMinuteUsed` / `reqMinuteCap`: requests used and the cap for the current 1-minute window.\n- `reqDayUsed` / `reqDayCap`: requests used and the cap for the current 24-hour window.\n\nThere is also `nowMs`, the edge wall-clock time (in milliseconds) when the snapshot was taken, so you can show \"as of\" and compute how fresh the numbers are.\n\n## Time series (for graphs)\n\nThe totals above are single numbers. To draw a graph over time, use `Analytics.series(metric, range)`, which returns a `Series`: a list of per-bucket totals from oldest to newest.\n\n```ts\n@get('/requests-24h')\npublic requests24h(ctx: RouteContext): string {\n const s = Analytics.series(MetricId.Requests, AnalyticsRange.H24);\n // s.points -> per-bucket totals, oldest to newest\n // s.bucketSecs -> seconds each bucket covers\n // s.headMs -> end time (ms) of the newest bucket\n // s.ratePerSec(i) -> requests/second for bucket i (points[i] / bucketSecs)\n return `buckets=${s.points.length} bucketSecs=${s.bucketSecs}`;\n}\n```\n\n`AnalyticsRange` picks the window and the resolution. Short ranges use per-minute buckets for a smooth line; the rest use per-hour buckets (kept for 30 days):\n\n| Range | Span | Bucket width |\n| --- | --- | --- |\n| `H1` | 1 hour | 1 minute |\n| `H6` | 6 hours | 1 minute |\n| `H12` | 12 hours | 1 hour |\n| `H24` | 24 hours | 1 hour |\n| `D3` | 3 days | 1 hour |\n| `D7` | 7 days | 1 hour |\n| `D14` | 14 days | 1 hour |\n| `D30` | 30 days | 1 hour |\n\nThe series stores **totals per bucket**, never rates. A rate is derived: `ratePerSec(i)` divides bucket `i` by its width for you, or you can compute `points[i] / bucketSecs` yourself. Most metrics you will chart are counters (`MetricId.Requests`, `MetricId.CacheHits`, and so on).\n\n## Permissions: who can read what\n\n- **Your own site.** `Analytics.self()` and `Analytics.series(...)` always read the site the request landed on. The calling domain is decided by the edge; your code cannot forge it.\n- **Cross-site reads are for `dacely.com` only.** The privileged `dacely.com` domain (the platform dashboard) may read any site with `Analytics.site(domain)`, `Analytics.siteSeries(domain, ...)`, and `Analytics.listSites(...)`. For any other caller, or an unknown domain, `site` and `siteSeries` return `null` and `listSites` returns an empty page. There is no way for a normal site to read another site's numbers.\n\n`Analytics.listSites(cursor, limit)` enumerates site names for the dashboard, one page at a time. It returns a `SiteList` with `sites: string[]` and `hasMore: bool`. When `hasMore` is true, pass the last name back as the next `cursor` to get the following page.\n\n```ts\n// Only meaningful when the calling domain is dacely.com.\nconst page = Analytics.listSites('', 100); // first 100 site names\nfor (let i = 0; i < page.sites.length; i++) {\n const other = Analytics.site(page.sites[i]); // TenantStats | null\n if (other != null) {\n // read other.requests, other.cacheRatio, ...\n }\n}\n```\n\n## Worked example: return everything\n\nThis route reads the full snapshot and maps it into a typed response. Mapping is mechanical: every getter is a `u64` and every response field is a `u64`, so there are no casts. The example groups a representative field from each area; add the rest of the getters from the catalog above the same way.\n\n```ts\nimport { RouteContext } from 'toiljs/server/runtime';\n\n/** The shape returned to the client. All fields are u64 except the one ratio. */\n@data\nexport class UsageReport {\n // requests / L1\n requests: u64 = 0;\n bytesOut: u64 = 0;\n status2xx: u64 = 0;\n status5xx: u64 = 0;\n gasUsed: u64 = 0;\n // database\n dbOps: u64 = 0;\n dbErrors: u64 = 0;\n meanDbLatencyNs: u64 = 0;\n // cache\n cacheHits: u64 = 0;\n cacheMisses: u64 = 0;\n cacheRatio: f64 = 0;\n // live gauges (current level, not a total)\n connectedStreams: u64 = 0;\n committedMemory: u64 = 0;\n // request windows (cap 0 = unlimited)\n requestsThisMinute: u64 = 0;\n requestsThisMinuteCap: u64 = 0;\n requestsToday: u64 = 0;\n requestsTodayCap: u64 = 0;\n // when the snapshot was read (edge ms)\n nowMs: u64 = 0;\n}\n\n@rest('usage')\nclass Usage {\n @get('/')\n public report(ctx: RouteContext): UsageReport {\n const s = Analytics.self();\n const out = new UsageReport();\n\n out.requests = s.requests;\n out.bytesOut = s.bytesOutL1;\n out.status2xx = s.status2xx;\n out.status5xx = s.status5xx;\n out.gasUsed = s.gasUsed;\n\n out.dbOps = s.dbOps;\n out.dbErrors = s.dbErrors;\n out.meanDbLatencyNs = s.meanDbLatencyNs;\n\n out.cacheHits = s.cacheHits;\n out.cacheMisses = s.cacheMisses;\n out.cacheRatio = s.cacheRatio;\n\n out.connectedStreams = s.connectedStreams;\n out.committedMemory = s.committedMemory;\n\n out.requestsThisMinute = s.reqMinuteUsed;\n out.requestsThisMinuteCap = s.reqMinuteCap;\n out.requestsToday = s.reqDayUsed;\n out.requestsTodayCap = s.reqDayCap;\n\n out.nowMs = s.nowMs;\n return out;\n }\n}\n```\n\nBecause this is a `@data` return type, the client gets a fully typed result from `Server.REST.usage.report()`. See [structured data types](../backend/data.md) for how `@data` works.\n\n## Local dev behavior\n\nUnder `toiljs dev`, the real per-domain metering does not exist (there is only one machine and no fleet), so the dev server returns a **fixed sample** `TenantStats` and a gentle synthetic ramp for `series(...)`. That lets you build and style a dashboard locally. Dev is also permissive: it returns sample data for any domain, because the real `dacely.com`-only check is enforced on the edge. Your code is unchanged; only the numbers differ.\n\n## Gotchas and limits\n\n- **Totals never reset.** The counter getters are lifetime running totals. To see \"requests in the last hour\", use `series(...)` and sum the buckets, not the totals.\n- **Values are `u64`.** Map them into `u64` fields with no casts. The only non-integer field is `cacheRatio` (and `Series.ratePerSec`), which are `f64`.\n- **Not per-user analytics.** These are infrastructure counters per domain. There is no way to record a custom event or attribute a number to a specific user.\n- **Cross-site reads need `dacely.com`.** `Analytics.site(...)` returns `null` for everyone else. Do not build a feature that reads another site's stats.\n- **Snapshots, not live streams.** Each call reads the current values once. `nowMs` tells you when.\n\n## Related\n\n- [Caching](./caching.md), the source of `cacheHits`, `cacheMisses`, and `cacheRatio`.\n- [ToilDB database](../database/README.md), the source of the `db*` counters and where these counters are stored.\n- [Structured data types](../backend/data.md), for the `@data` return type in the worked example.\n- [Compute tiers](../concepts/tiers.md), for what \"L1\", \"streams\", and \"daemons\" mean in the catalog.\n",
|
|
67
68
|
"services/caching.md": "# Caching\n\n**Caching lets the edge keep a copy of a response and hand it back instantly, without re-running your code.** You opt in per route, either with the `@cache` decorator or by calling `Response.cache(...)`.\n\n## What \"cache\" means here\n\nA **cache** is a fast, temporary store of a previous answer. When a request comes in, the edge can check: \"have I already computed this exact response recently?\" If yes, that is a **cache hit**: it returns the saved copy in microseconds and never wakes up your WebAssembly program. If no, that is a **cache miss**: it runs your handler, sends the result, and (if you asked it to) saves a copy for next time.\n\nThere are two places a response can be cached:\n\n- **The edge cache** is shared: one saved copy on an edge server answers many users. This is where the big speedups come from.\n- **The browser cache** is private to one user's browser. You control it with a standard `Cache-Control` header.\n\nYou can use one or both.\n\n## When to cache (and when not)\n\nCache a response only when it is the **same for everyone** and safe to reuse for a little while:\n\n- Good: a public leaderboard, a product catalog, a blog index, an exchange-rate snapshot, a \"server status\" endpoint.\n- Bad: anything personalized (a user's dashboard, their cart, \"hello, Alice\"). If you cached that, one user could be served another user's data. The edge has strong safety rails against this (below), but the simplest rule is: **do not cache per-user data on the shared edge.**\n\nCaching is **always opt-in**. A response you do not mark is never stored. There is no \"cache every GET\" mode, because the edge cannot tell a personalized response from a public one on its own.\n\n## How: the `@cache` decorator\n\nA **decorator** is an annotation you put right above a route method. `@cache` tells the edge how long it may reuse the response. The numbers are positional:\n\n```ts\nimport { Response } from 'toiljs/server/runtime';\n\n@rest('leaderboard')\nclass Leaderboard {\n @cache(60) // 60 MINUTES at the edge\n @get('/')\n public top(): Response {\n // ...expensive query...\n return Response.json(standings);\n }\n}\n```\n\nThe full form is `@cache(edgeTtlMinutes, browserTtlSeconds?, privateScope?, allowAuth?)`:\n\n```ts\n@cache(60) // 60 minutes at the edge, browser does not cache\n@cache(60, 300) // + tell the browser to cache for 5 minutes (300s)\n@cache(60, 300, true) // + mark it \"private\": browser only, never the shared edge\n@cache(60, 300, true, true) // + allow caching even for logged-in requests (see rails)\n```\n\n**TTL** means \"time to live\": how long a saved copy stays fresh before the edge throws it away and recomputes. Notice the units differ on purpose: the edge TTL is in **minutes**, the browser TTL is in **seconds**.\n\nEvery argument must be a plain number or `true`/`false` literal. If you pass something computed (a variable, a function call), the decorator safely degrades to \"not cached\" instead of misbehaving.\n\n## How: `Response.cache(...)`\n\n`@cache` is just a shorthand. You can do the same thing by hand on any `Response`, which is handy when you decide at runtime:\n\n```ts\npublic cache(\n edgeTtlMinutes: u16,\n browserTtlSeconds: u32 = 0,\n privateScope: bool = false,\n allowAuth: bool = false,\n): Response\n```\n\n```ts\nreturn Response.json(body).cache(60, 300);\n```\n\n`cache(...)` returns the same `Response`, so it chains. There is also a shorthand for \"edge only, no browser caching\":\n\n```ts\nreturn Response.bytes(blob).cacheFor(5); // edge-cache 5 minutes\n```\n\nUnder the hood, both set one header, `Dacely-Cache-Control`, that the edge reads and then strips (it never reaches the client).\n\n## The parameters, in detail\n\n| Parameter | What it does |\n| --- | --- |\n| `edgeTtlMinutes` | How long the shared edge may serve the saved copy. Clamped to a 24-hour maximum, no matter what you pass. `0` means no edge caching. |\n| `browserTtlSeconds` | The `max-age` sent to the browser. `0` (default) means the browser is told not to cache. |\n| `privateScope` | Marks the response `private`: it may only live in the browser's own cache, never the shared edge or a CDN. |\n| `allowAuth` | Permits caching a response to a logged-in request. Off by default (see safety rails). |\n\n## A hit vs a miss, visually\n\n```mermaid\nflowchart TD\n R[\"Request arrives<br/>at the edge\"] --> Q{\"Saved copy<br/>still fresh?\"}\n Q -->|Yes: cache HIT| H[\"Return the saved response<br/>(microseconds, no code runs)\"]\n Q -->|No: cache MISS| M[\"Run your WebAssembly handler\"]\n M --> D{\"Marked cacheable<br/>and eligible?\"}\n D -->|Yes| S[\"Save a copy for the TTL,<br/>then return it\"]\n D -->|No| X[\"Return it, save nothing\"]\n```\n\nEvery response the edge sends carries a `Dacely-Cache` header so you can see what happened while debugging:\n\n| `Dacely-Cache` value | Meaning |\n| --- | --- |\n| `HIT` | Served from the cache; your code did not run. |\n| `MISS` | Your code ran, and the response was saved (the next identical request should `HIT`). |\n| `DYNAMIC` | Your code ran, and the response was not cached (no directive, or not eligible). |\n| `BYPASS` | A fallback path (for example the compute layer was momentarily unavailable); not a cached answer. |\n\n## Safety rails (why cross-user leaks cannot happen)\n\nThe edge refuses to store certain responses even if you asked it to. These rules exist because a shared cache is exactly where one user's data could accidentally leak to another:\n\n- **5xx responses are never cached.** A server error is temporary; caching a `500` would keep serving the failure for the whole TTL. (2xx, 3xx, and 4xx are cacheable: a redirect or a `404`/`410` is a predictable function of the request.)\n- **A response with a `Set-Cookie` is never cached.** A cookie is per-user state.\n- **A response to a logged-in request is not cached** unless you pass `allowAuth = true`. This stops one user's personalized response from being handed to someone else. Even with `allowAuth = true`, such an entry is only ever served back to other **authenticated** requests, never to an anonymous one (which would skip the auth check entirely).\n- **The edge TTL is clamped to 24 hours.**\n\nThe cache is also keyed precisely: an entry is identified by the **host, method, path, and the exact request body**. So a `POST` with body `A` never returns the copy saved for body `B`, and a `GET` never returns a `POST`'s copy.\n\nBecause auth checks and body decoding run **before** the cache directive is applied, an unauthorized request is rejected with `401` before anything is stored, and a cached copy only ever comes from a handler that actually ran successfully.\n\n## ETag, Last-Modified, and 304 (static files)\n\nThe `@cache` decorator above is for **dynamic** responses (ones your code computes). Your project's **static files** (the built HTML, JS, CSS, and images) get a second, automatic layer of caching that you do not configure. It is worth understanding because it is what makes repeat page loads cheap.\n\nEvery static file the edge serves carries three headers:\n\n- **`ETag`**: a short fingerprint of the file's contents (a \"version tag\"). toiljs uses a *weak* ETag (it starts with `W/`), which survives compression by a CDN in front of the edge.\n- **`Last-Modified`**: when the file last changed.\n- **`Cache-Control`**: content-hashed assets (files whose name contains their content hash, like `app-CfvHW67Y.js`) get `immutable` with a one-year max-age, because the URL itself changes whenever the content does. Everything else (HTML, non-hashed assets) gets `must-revalidate`, so the browser always checks freshness.\n\n**Revalidation** is the trick. Instead of re-downloading a file it already has, a browser asks \"has this changed?\" by echoing the tag back in an `If-None-Match` header (or the date in `If-Modified-Since`). If the file is unchanged, the edge replies **`304 Not Modified`** with an empty body, and the browser reuses its copy. A `304` is tiny and fast.\n\n```mermaid\nsequenceDiagram\n participant B as Browser\n participant E as Dacely edge\n B->>E: GET /app.js<br/>If-None-Match: W/\"1a2b-c3d4\"\n alt file unchanged\n E-->>B: 304 Not Modified<br/>(no body)\n else file changed\n E-->>B: 200 OK<br/>(new bytes + new ETag)\n end\n```\n\nThe precedence follows the HTTP spec (RFC 7232): if the request has `If-None-Match`, that decides it; only when it is absent does `If-Modified-Since` apply. You get all of this for free; there is nothing to turn on.\n\n## Where cached bodies live (memory bounds)\n\nYou do not need this to use caching, but it explains the limits you may hit.\n\nThe edge response cache is bounded and hard-capped so it can never exhaust a node's memory:\n\n- **RAM tier**: small, short-lived responses live in memory, within a bounded budget (on the order of ~128 MB) plus a cap on the number of entries. A single response over ~256 KB does not go in RAM.\n- **Disk tier (optional)**: when the operator enables an on-disk spill directory, a **big** response (over ~256 KB) or a **long-lived** one (TTL of 10 minutes or more) is written to disk and served back with near-zero RAM, the same way static files are. If disk spill is not enabled, a big response is simply not cached (you will see `Dacely-Cache: DYNAMIC`).\n\nExpired entries are dropped on read (a stale entry is treated as a miss) and reclaimed when the cache needs room. Nothing survives a process restart.\n\n## Gotchas\n\n- **Units differ.** Edge TTL is in minutes; browser TTL is in seconds. `@cache(60)` is one hour at the edge, not one minute.\n- **Do not cache personalized data on the shared edge.** If a response depends on who is asking, either do not cache it, or use `privateScope` so it is browser-only.\n- **A `Set-Cookie` disables edge caching for that response.** If you set a cookie, that response will not be edge-cached, by design.\n- **Very large or slow-to-change bodies may not be cached** unless disk spill is enabled on the node. Check the `Dacely-Cache` header if a response you expected to cache shows `DYNAMIC`.\n- **Nothing persists across restarts.** The cache is a speed optimization, never a source of truth.\n\n## Related\n\n- [Rate limiting](./ratelimit.md): protect a route before it runs.\n- [Analytics](./analytics.md): see your cache hit ratio per site.\n- [Cookies](./cookies.md): why a `Set-Cookie` opts a response out of edge caching.\n- [Auth, sessions, and `@user`](../auth/usage.md): why logged-in responses are not cached by default.\n- [Every decorator](../concepts/decorators.md).\n",
|
|
68
|
-
"services/cookies.md": "# Cookies\n\n**Cookies let you store a small piece of state in the user's browser and read it back on the next request.** toiljs ships a complete cookie layer: read incoming cookies, set them on a response, and (optionally) sign or encrypt their values so they cannot be tampered with.\n\n## What a cookie is\n\nA **cookie** is a tiny named value the browser holds for your site. The browser sends it back on every request to your site (in a `Cookie` request header), and you can change or add cookies by putting a `Set-Cookie` header on your response. Cookies are how a site remembers things between requests: a theme preference, a feature flag, a session id.\n\nTwo directions to keep straight:\n\n- **Reading** happens on the `Request` (the browser sent its cookies to you).\n- **Writing** happens on the `Response` (you tell the browser to store or clear a cookie).\n\n```mermaid\nsequenceDiagram\n participant B as Browser\n participant S as Your handler\n B->>S: Request<br/>Cookie: theme=dark\n Note over S: req.cookie('theme') -> \"dark\"\n S-->>B: Response<br/>Set-Cookie: theme=light; ...\n Note over B: browser stores the new cookie<br/>and sends it next time\n```\n\nThe cookie types (`Cookie`, `Cookies`, `CookieMap`, `SecureCookies`, and the `SameSite` / `CookieEncoding` / `CookiePrefix` enums) are **ambient globals**: use them with no import, like `crypto`. They are also exported from `toiljs/server/runtime` if you prefer an explicit import.\n\n## Reading incoming cookies\n\nOn the `Request`, two methods cover almost everything:\n\n```ts\nconst theme = req.cookie('theme'); // one value: string | null\nconst jar = req.cookies(); // all of them, parsed once and cached\njar.get('theme'); // \"dark\", or null\njar.has('theme'); // true / false\n```\n\n`req.cookies()` returns a `CookieMap`, an ordered name-to-value view. It is parsed once per request and cached, so calling it repeatedly is cheap.\n\n## Setting a cookie on a response\n\nBuild a cookie with the fluent `Cookie` builder, then attach it with `setCookie`. Every setter returns the cookie, so calls chain:\n\n```ts\nimport { Response } from 'toiljs/server/runtime';\n\nreturn Response.json('{\"ok\":true}').setCookie(\n Cookie.create('theme', 'dark')\n .path('/')\n .maxAge(60 * 60 * 24 * 365) // one year, in seconds\n .sameSite(SameSite.Lax)\n .secure(),\n);\n```\n\nShorthands for the common cases:\n\n```ts\nresp.setCookieKV('theme', 'dark'); // name=value, no attributes\nresp.clearCookie('theme'); // delete it (empty value, expired)\n```\n\nEach `setCookie` adds its own `Set-Cookie` header; cookies are never folded into one header.\n\n### The attributes you will actually use\n\n| Method | Sets | What it means |\n| --- | --- | --- |\n| `path('/')` | `Path` | Which URL paths the cookie applies to. `/` means the whole site. |\n| `maxAge(seconds)` | `Max-Age` | How long the browser keeps it. `0` or negative deletes it now. |\n| `secure()` | `Secure` | Only send over HTTPS. Use it for anything that matters. |\n| `httpOnly()` | `HttpOnly` | Hide it from JavaScript in the browser (blocks theft via XSS). Use it for session cookies. |\n| `sameSite(SameSite.Lax)` | `SameSite` | Controls whether the cookie is sent on cross-site requests. `Lax` is a good default; `Strict` is tighter; `None` allows cross-site (and forces `Secure`). |\n\nYou rarely need the rest, but they exist: `domain`, `expires` (an absolute time instead of a duration), `partitioned` (CHIPS), `priority`, and `extension` for anything custom.\n\n## The `__Host-` and `__Secure-` prefixes\n\nTwo special name prefixes give the browser extra guarantees. They are not magic strings you invent; browsers recognize them and **refuse** to accept a cookie that carries the prefix without meeting its rules.\n\n- **`__Secure-`**: the cookie must be `Secure` (HTTPS only).\n- **`__Host-`**: the cookie must be `Secure`, have `Path=/`, and have **no** `Domain`. This locks the cookie to exactly your host, so a sibling subdomain cannot set or read it. It is the strongest option and the right choice for session cookies.\n\nYou do not spell the prefix yourself. Two helpers apply it and enforce the rules for you:\n\n```ts\nCookie.create('sid', 'abc123')\n .httpOnly()\n .sameSite(SameSite.Lax)\n .maxAge(3600)\n .asHostPrefixed(); // prepends __Host-, forces Secure + Path=/ + no Domain\n```\n\n`asSecurePrefixed()` is the lighter version (prepends `__Secure-` and forces `Secure`).\n\n> Under `toiljs dev`, browsers treat `http://localhost` as a secure context, so `Secure` and `__Host-` cookies work over plain HTTP locally. You do not need HTTPS to test them.\n\n## Worked example: a theme preference cookie\n\nA tiny handler that reads a `theme` cookie and lets the user flip it. This is the canonical \"remember a preference\" pattern: a plain, non-secret value.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('prefs')\nclass Prefs {\n // GET /prefs/theme -> current theme (defaults to \"light\")\n @get('/theme')\n read(ctx: RouteContext): Response {\n const theme = ctx.request.cookie('theme');\n return Response.text((theme != null ? theme : 'light') + '\\n');\n }\n\n // POST /prefs/theme/dark -> remember \"dark\" for a year\n @post('/theme/dark')\n setDark(ctx: RouteContext): Response {\n return Response.text('saved\\n').setCookie(\n Cookie.create('theme', 'dark')\n .path('/')\n .maxAge(60 * 60 * 24 * 365)\n .sameSite(SameSite.Lax)\n .secure(),\n );\n }\n}\n```\n\nA preference like this does not need to be secret or tamper-proof (the worst a user can do is change their own theme), so a plain cookie is fine. When the value **does** matter, reach for `SecureCookies`.\n\n## Secure cookies: signing and encryption\n\nSometimes a cookie value must not be forged or read by the user. `SecureCookies` covers both, built on the `crypto` global (no extra setup):\n\n- **Signed** (`SecureCookies.signed(key)`): the value stays readable but is stamped with a signature (HMAC-SHA256) bound to the cookie's name. The user can see it but cannot change it or move it to another cookie without the signature failing. Use this for a value the client may read but must not tamper with.\n- **Encrypted** (`SecureCookies.encrypted(key)`): the value is scrambled (AES-GCM) so the user cannot read **or** change it. Use this for something confidential.\n\nThe `key` is raw secret bytes you supply (load it from a secret via [`Environment.getSecure`](./environment.md), never hard-code it):\n\n```ts\n// In a real app, load this from a secret, do not inline it.\nconst key = Uint8Array.wrap(String.UTF8.encode('0123456789abcdef0123456789abcdef'));\n\n// Signed: readable but tamper-proof.\nconst signer = SecureCookies.signed(key);\nconst sealed = signer.sign('session', 'user-42');\nconst user = signer.unsign('session', sealed); // \"user-42\", or null if tampered\n\n// Encrypted: unreadable and authenticated.\nconst box = SecureCookies.encrypted(key);\nresp.setCookie(box.seal(Cookie.create('secret', 'top-secret').httpOnly()));\nconst secret = box.open(req.cookies(), 'secret'); // \"top-secret\", or null\n```\n\nTwo important safety properties:\n\n- **Verification never crashes.** `unsign` and `decrypt` return `null` on a tampered, truncated, or wrong-key value; they never throw. That makes them safe to call directly on attacker-controlled input.\n- **Values are bound to the cookie name.** A sealed value made for cookie `a` will not verify or decrypt under cookie `b`.\n\nFor HMAC keys, use 32 or more bytes. For AES, the key must be exactly 16 or 32 bytes (a wrong length is rejected immediately). You can rotate keys by sealing with a new key while still accepting an old one:\n\n```ts\nconst signer = SecureCookies.signed(newKey).addKey(oldKey); // seal with new, open with either\n```\n\n## Encoding is not encryption\n\nOne common mix-up. **Encoding** (the `CookieEncoding` on a `Cookie`, default percent-encoding) only makes a value safe to put on the wire; anyone can reverse it. **Signing / encryption** (`SecureCookies`) is the cryptographic protection and needs a secret key. If you want a value the user cannot forge, you need `SecureCookies`, not a fancier encoding.\n\n## Relationship to auth session cookies\n\nYou do not manage login cookies by hand. The [auth system](../auth/usage.md) sets and reads its own hardened, `__Host-` prefixed, signed session cookie for you, and enforces access with the `@auth` decorator. This page is for **your own** cookies (preferences, flags, small bits of state). If you find yourself building a login cookie from scratch, use auth instead; it already does the hard, security-sensitive parts correctly.\n\n## Advanced reference: `Cookie` and `Cookies` helpers\n\nMost handlers only need the builders above. These extra members are here for lower-level work: validating a cookie before you send it, parsing a `Set-Cookie` header back into a `Cookie` (for a proxy or a test), or controlling the exact wire encoding.\n\n### More `Cookie` methods\n\n| Method | Returns | What it does |\n| --- | --- | --- |\n| `validate()` | `CookieValidation` | Check the cookie against RFC 6265bis (name is a token, name+value within the 4096-byte cap, `Path` form, prefix rules, `SameSite=None` / `Partitioned` imply `Secure`, the 400-day lifetime cap) **without** sending it. Never throws. |\n| `serialize(strict)` | `string` | Build the `Set-Cookie` value. Lenient by default (always emits a best-effort cookie); pass `serialize(true)` to **throw** on a hard violation instead. `setCookie(...)` calls this for you. |\n| `withEncoding(enc)` | `Cookie` | Choose how the value goes on the wire: `CookieEncoding.Percent` (default), `.Raw`, or `.Base64Url`. Chains like the other setters. |\n| `detectedPrefix()` | `CookiePrefix` | Report which special prefix the name carries (`CookiePrefix.Host`, `.Secure`, or `.None`), detected case-insensitively. |\n| `encodedValue()` | `string` | The value after the chosen encoding is applied, exactly as it will appear on the wire. |\n| `expiresRaw(date)` | `Cookie` | Set `Expires` to a verbatim date string (an escape hatch; it is **not** validated, unlike `expires(epochSeconds)`). |\n\n`validate()` returns a `CookieValidation`, a small result object:\n\n- `valid: bool` is `true` when there were no problems.\n- `errors: Array<string>` holds one human-readable message per problem (empty when valid).\n\n```ts\n// A __Host- cookie must be Secure with Path=/; here we forgot both.\nconst c = Cookie.create('__Host-sid', 'abc');\nconst check = c.validate();\nif (!check.valid) {\n // check.errors includes \"__Host- prefix requires the Secure attribute\"\n // (asHostPrefixed() would have set those attributes for you)\n}\n```\n\n### `Cookies` static helpers\n\n`Cookies` is the read side, plus a couple of codec shortcuts. Every method is static, so call them as `Cookies.xxx(...)`.\n\n| Call | Returns | What it does |\n| --- | --- | --- |\n| `Cookies.parse(header)` | `CookieMap` | Parse a request `Cookie` header (`a=1; b=2`) into a name-to-value map. Values are percent-decoded; malformed pairs are skipped, never thrown. |\n| `Cookies.get(header, name)` | `string \\| null` | Shorthand: parse `header` and return one value (or `null`). |\n| `Cookies.serialize(name, value)` | `string` | One-shot `Set-Cookie` value for `name=value` with no attributes. For attributes, build a `Cookie` and call `serialize()`. |\n| `Cookies.parseSetCookie(header)` | `Cookie` | Parse a `Set-Cookie` field value back into a `Cookie` (handy for proxies or tests). The value is kept verbatim (`CookieEncoding.Raw`) so re-serializing reproduces the original. |\n| `Cookies.encodeValue(raw)` | `string` | Percent-encode a value the way the default `Cookie` encoding would. |\n| `Cookies.decodeValue(enc)` | `string` | Percent-decode a value (the inverse of `encodeValue`). |\n\n```ts\n// Round-trip a Set-Cookie header (for example, inspecting an upstream response in a proxy):\nconst cookie = Cookies.parseSetCookie('sid=abc123; Path=/; HttpOnly; SameSite=Lax');\ncookie.name; // \"sid\"\ncookie.serialize(); // \"sid=abc123; Path=/; SameSite=Lax; HttpOnly\"\n```\n\n## Gotchas\n\n- **Read on the `Request`, write on the `Response`.** Setting a cookie does not change what `req.cookie(...)` returns for the current request; it takes effect on the browser's next request.\n- **`maxAge` is in seconds.** `maxAge(3600)` is one hour, not one millisecond.\n- **To delete a cookie, the `path` (and `domain`) must match** the ones you set it with. Use `clearCookie(name, path, domain)`.\n- **A `Set-Cookie` opts a response out of edge caching.** By design, a response that sets a cookie is never edge-cached, because a cookie is per-user state. See [Caching](./caching.md).\n- **Do not put secrets in a plain cookie value.** Use `SecureCookies.encrypted(...)`, and load the key from a [secret](./environment.md).\n\n## Related\n\n- [Auth, sessions, and `@user`](../auth/usage.md): the built-in, hardened session cookie.\n- [Crypto](./crypto.md): the primitives under `SecureCookies`.\n- [Environment and secrets](./environment.md): where to store your signing / encryption key.\n- [Caching](./caching.md): why setting a cookie disables edge caching.\n",
|
|
69
|
+
"services/cookies.md": "# Cookies\n\n**Cookies let you store a small piece of state in the user's browser and read it back on the next request.** toiljs ships a complete cookie layer: read incoming cookies, set them on a response, and (optionally) sign or encrypt their values so they cannot be tampered with.\n\n## What a cookie is\n\nA **cookie** is a tiny named value the browser holds for your site. The browser sends it back on every request to your site (in a `Cookie` request header), and you can change or add cookies by putting a `Set-Cookie` header on your response. Cookies are how a site remembers things between requests: a theme preference, a feature flag, a session id.\n\nTwo directions to keep straight:\n\n- **Reading** happens on the `Request` (the browser sent its cookies to you).\n- **Writing** happens on the `Response` (you tell the browser to store or clear a cookie).\n\n```mermaid\nsequenceDiagram\n participant B as Browser\n participant S as Your handler\n B->>S: Request<br/>Cookie: theme=dark\n Note over S: req.cookie('theme') returns \"dark\"\n S-->>B: Response<br/>Set-Cookie: theme=light\n Note over B: browser stores the new cookie<br/>and sends it next time\n```\n\nThe cookie types (`Cookie`, `Cookies`, `CookieMap`, `SecureCookies`, and the `SameSite` / `CookieEncoding` / `CookiePrefix` enums) are **ambient globals**: use them with no import, like `crypto`. They are also exported from `toiljs/server/runtime` if you prefer an explicit import.\n\n## Reading incoming cookies\n\nOn the `Request`, two methods cover almost everything:\n\n```ts\nconst theme = req.cookie('theme'); // one value: string | null\nconst jar = req.cookies(); // all of them, parsed once and cached\njar.get('theme'); // \"dark\", or null\njar.has('theme'); // true / false\n```\n\n`req.cookies()` returns a `CookieMap`, an ordered name-to-value view. It is parsed once per request and cached, so calling it repeatedly is cheap.\n\n## Setting a cookie on a response\n\nBuild a cookie with the fluent `Cookie` builder, then attach it with `setCookie`. Every setter returns the cookie, so calls chain:\n\n```ts\nimport { Response } from 'toiljs/server/runtime';\n\nreturn Response.json('{\"ok\":true}').setCookie(\n Cookie.create('theme', 'dark')\n .path('/')\n .maxAge(60 * 60 * 24 * 365) // one year, in seconds\n .sameSite(SameSite.Lax)\n .secure(),\n);\n```\n\nShorthands for the common cases:\n\n```ts\nresp.setCookieKV('theme', 'dark'); // name=value, no attributes\nresp.clearCookie('theme'); // delete it (empty value, expired)\n```\n\nEach `setCookie` adds its own `Set-Cookie` header; cookies are never folded into one header.\n\n### The attributes you will actually use\n\n| Method | Sets | What it means |\n| --- | --- | --- |\n| `path('/')` | `Path` | Which URL paths the cookie applies to. `/` means the whole site. |\n| `maxAge(seconds)` | `Max-Age` | How long the browser keeps it. `0` or negative deletes it now. |\n| `secure()` | `Secure` | Only send over HTTPS. Use it for anything that matters. |\n| `httpOnly()` | `HttpOnly` | Hide it from JavaScript in the browser (blocks theft via XSS). Use it for session cookies. |\n| `sameSite(SameSite.Lax)` | `SameSite` | Controls whether the cookie is sent on cross-site requests. `Lax` is a good default; `Strict` is tighter; `None` allows cross-site (and forces `Secure`). |\n\nYou rarely need the rest, but they exist: `domain`, `expires` (an absolute time instead of a duration), `partitioned` (CHIPS), `priority`, and `extension` for anything custom.\n\n## The `__Host-` and `__Secure-` prefixes\n\nTwo special name prefixes give the browser extra guarantees. They are not magic strings you invent; browsers recognize them and **refuse** to accept a cookie that carries the prefix without meeting its rules.\n\n- **`__Secure-`**: the cookie must be `Secure` (HTTPS only).\n- **`__Host-`**: the cookie must be `Secure`, have `Path=/`, and have **no** `Domain`. This locks the cookie to exactly your host, so a sibling subdomain cannot set or read it. It is the strongest option and the right choice for session cookies.\n\nYou do not spell the prefix yourself. Two helpers apply it and enforce the rules for you:\n\n```ts\nCookie.create('sid', 'abc123')\n .httpOnly()\n .sameSite(SameSite.Lax)\n .maxAge(3600)\n .asHostPrefixed(); // prepends __Host-, forces Secure + Path=/ + no Domain\n```\n\n`asSecurePrefixed()` is the lighter version (prepends `__Secure-` and forces `Secure`).\n\n> Under `toiljs dev`, browsers treat `http://localhost` as a secure context, so `Secure` and `__Host-` cookies work over plain HTTP locally. You do not need HTTPS to test them.\n\n## Worked example: a theme preference cookie\n\nA tiny handler that reads a `theme` cookie and lets the user flip it. This is the canonical \"remember a preference\" pattern: a plain, non-secret value.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('prefs')\nclass Prefs {\n // GET /prefs/theme -> current theme (defaults to \"light\")\n @get('/theme')\n read(ctx: RouteContext): Response {\n const theme = ctx.request.cookie('theme');\n return Response.text((theme != null ? theme : 'light') + '\\n');\n }\n\n // POST /prefs/theme/dark -> remember \"dark\" for a year\n @post('/theme/dark')\n setDark(ctx: RouteContext): Response {\n return Response.text('saved\\n').setCookie(\n Cookie.create('theme', 'dark')\n .path('/')\n .maxAge(60 * 60 * 24 * 365)\n .sameSite(SameSite.Lax)\n .secure(),\n );\n }\n}\n```\n\nA preference like this does not need to be secret or tamper-proof (the worst a user can do is change their own theme), so a plain cookie is fine. When the value **does** matter, reach for `SecureCookies`.\n\n## Secure cookies: signing and encryption\n\nSometimes a cookie value must not be forged or read by the user. `SecureCookies` covers both, built on the `crypto` global (no extra setup):\n\n- **Signed** (`SecureCookies.signed(key)`): the value stays readable but is stamped with a signature (HMAC-SHA256) bound to the cookie's name. The user can see it but cannot change it or move it to another cookie without the signature failing. Use this for a value the client may read but must not tamper with.\n- **Encrypted** (`SecureCookies.encrypted(key)`): the value is scrambled (AES-GCM) so the user cannot read **or** change it. Use this for something confidential.\n\nThe `key` is raw secret bytes you supply (load it from a secret via [`Environment.getSecure`](./environment.md), never hard-code it):\n\n```ts\n// In a real app, load this from a secret, do not inline it.\nconst key = Uint8Array.wrap(String.UTF8.encode('0123456789abcdef0123456789abcdef'));\n\n// Signed: readable but tamper-proof.\nconst signer = SecureCookies.signed(key);\nconst sealed = signer.sign('session', 'user-42');\nconst user = signer.unsign('session', sealed); // \"user-42\", or null if tampered\n\n// Encrypted: unreadable and authenticated.\nconst box = SecureCookies.encrypted(key);\nresp.setCookie(box.seal(Cookie.create('secret', 'top-secret').httpOnly()));\nconst secret = box.open(req.cookies(), 'secret'); // \"top-secret\", or null\n```\n\nTwo important safety properties:\n\n- **Verification never crashes.** `unsign` and `decrypt` return `null` on a tampered, truncated, or wrong-key value; they never throw. That makes them safe to call directly on attacker-controlled input.\n- **Values are bound to the cookie name.** A sealed value made for cookie `a` will not verify or decrypt under cookie `b`.\n\nFor HMAC keys, use 32 or more bytes. For AES, the key must be exactly 16 or 32 bytes (a wrong length is rejected immediately). You can rotate keys by sealing with a new key while still accepting an old one:\n\n```ts\nconst signer = SecureCookies.signed(newKey).addKey(oldKey); // seal with new, open with either\n```\n\n## Encoding is not encryption\n\nOne common mix-up. **Encoding** (the `CookieEncoding` on a `Cookie`, default percent-encoding) only makes a value safe to put on the wire; anyone can reverse it. **Signing / encryption** (`SecureCookies`) is the cryptographic protection and needs a secret key. If you want a value the user cannot forge, you need `SecureCookies`, not a fancier encoding.\n\n## Relationship to auth session cookies\n\nYou do not manage login cookies by hand. The [auth system](../auth/usage.md) sets and reads its own hardened, `__Host-` prefixed, signed session cookie for you, and enforces access with the `@auth` decorator. This page is for **your own** cookies (preferences, flags, small bits of state). If you find yourself building a login cookie from scratch, use auth instead; it already does the hard, security-sensitive parts correctly.\n\n## Advanced reference: `Cookie` and `Cookies` helpers\n\nMost handlers only need the builders above. These extra members are here for lower-level work: validating a cookie before you send it, parsing a `Set-Cookie` header back into a `Cookie` (for a proxy or a test), or controlling the exact wire encoding.\n\n### More `Cookie` methods\n\n| Method | Returns | What it does |\n| --- | --- | --- |\n| `validate()` | `CookieValidation` | Check the cookie against RFC 6265bis (name is a token, name+value within the 4096-byte cap, `Path` form, prefix rules, `SameSite=None` / `Partitioned` imply `Secure`, the 400-day lifetime cap) **without** sending it. Never throws. |\n| `serialize(strict)` | `string` | Build the `Set-Cookie` value. Lenient by default (always emits a best-effort cookie); pass `serialize(true)` to **throw** on a hard violation instead. `setCookie(...)` calls this for you. |\n| `withEncoding(enc)` | `Cookie` | Choose how the value goes on the wire: `CookieEncoding.Percent` (default), `.Raw`, or `.Base64Url`. Chains like the other setters. |\n| `detectedPrefix()` | `CookiePrefix` | Report which special prefix the name carries (`CookiePrefix.Host`, `.Secure`, or `.None`), detected case-insensitively. |\n| `encodedValue()` | `string` | The value after the chosen encoding is applied, exactly as it will appear on the wire. |\n| `expiresRaw(date)` | `Cookie` | Set `Expires` to a verbatim date string (an escape hatch; it is **not** validated, unlike `expires(epochSeconds)`). |\n\n`validate()` returns a `CookieValidation`, a small result object:\n\n- `valid: bool` is `true` when there were no problems.\n- `errors: Array<string>` holds one human-readable message per problem (empty when valid).\n\n```ts\n// A __Host- cookie must be Secure with Path=/; here we forgot both.\nconst c = Cookie.create('__Host-sid', 'abc');\nconst check = c.validate();\nif (!check.valid) {\n // check.errors includes \"__Host- prefix requires the Secure attribute\"\n // (asHostPrefixed() would have set those attributes for you)\n}\n```\n\n### `Cookies` static helpers\n\n`Cookies` is the read side, plus a couple of codec shortcuts. Every method is static, so call them as `Cookies.xxx(...)`.\n\n| Call | Returns | What it does |\n| --- | --- | --- |\n| `Cookies.parse(header)` | `CookieMap` | Parse a request `Cookie` header (`a=1; b=2`) into a name-to-value map. Values are percent-decoded; malformed pairs are skipped, never thrown. |\n| `Cookies.get(header, name)` | `string \\| null` | Shorthand: parse `header` and return one value (or `null`). |\n| `Cookies.serialize(name, value)` | `string` | One-shot `Set-Cookie` value for `name=value` with no attributes. For attributes, build a `Cookie` and call `serialize()`. |\n| `Cookies.parseSetCookie(header)` | `Cookie` | Parse a `Set-Cookie` field value back into a `Cookie` (handy for proxies or tests). The value is kept verbatim (`CookieEncoding.Raw`) so re-serializing reproduces the original. |\n| `Cookies.encodeValue(raw)` | `string` | Percent-encode a value the way the default `Cookie` encoding would. |\n| `Cookies.decodeValue(enc)` | `string` | Percent-decode a value (the inverse of `encodeValue`). |\n\n```ts\n// Round-trip a Set-Cookie header (for example, inspecting an upstream response in a proxy):\nconst cookie = Cookies.parseSetCookie('sid=abc123; Path=/; HttpOnly; SameSite=Lax');\ncookie.name; // \"sid\"\ncookie.serialize(); // \"sid=abc123; Path=/; SameSite=Lax; HttpOnly\"\n```\n\n## Gotchas\n\n- **Read on the `Request`, write on the `Response`.** Setting a cookie does not change what `req.cookie(...)` returns for the current request; it takes effect on the browser's next request.\n- **`maxAge` is in seconds.** `maxAge(3600)` is one hour, not one millisecond.\n- **To delete a cookie, the `path` (and `domain`) must match** the ones you set it with. Use `clearCookie(name, path, domain)`.\n- **A `Set-Cookie` opts a response out of edge caching.** By design, a response that sets a cookie is never edge-cached, because a cookie is per-user state. See [Caching](./caching.md).\n- **Do not put secrets in a plain cookie value.** Use `SecureCookies.encrypted(...)`, and load the key from a [secret](./environment.md).\n\n## Related\n\n- [Auth, sessions, and `@user`](../auth/usage.md): the built-in, hardened session cookie.\n- [Crypto](./crypto.md): the primitives under `SecureCookies`.\n- [Environment and secrets](./environment.md): where to store your signing / encryption key.\n- [Caching](./caching.md): why setting a cookie disables edge caching.\n",
|
|
69
70
|
"services/crypto.md": "# Crypto\n\nA small, safe cryptography toolkit (hashing, random bytes, HMAC, symmetric encryption, key derivation, and signatures) available with no import through the ambient `crypto` global.\n\nReach for `crypto` to fingerprint a value (hashing), generate an unguessable token or ID (random bytes), prove a value has not been tampered with (HMAC or a signature), or encrypt data you store yourself (AES). Skip it for **login passwords**: use [auth](../auth/README.md), which is purpose-built and does the hard parts for you. `crypto` is also the engine under [signed cookies](./cookies.md) and auth, so most apps use it indirectly without ever calling it.\n\n## The shape of the API\n\n`crypto` mirrors the browser Web Crypto API, with one big difference: **there are no Promises**. ToilScript (the language your backend is written in) has no `async`, so every call returns its result **directly**, right away.\n\n```ts\nconst digest = crypto.sha256Text('hello'); // Uint8Array, no await\nconst id = crypto.randomUUID(); // string, no await\n```\n\nThere are two layers:\n\n- **`crypto.*`**: ergonomic one-call helpers (hash this string, give me a UUID, HMAC these bytes). Start here.\n- **`crypto.subtle`**: the full primitive surface (import a key, encrypt, sign, derive bits) for when you need more control.\n\nBoth are ambient globals (no import). The small helper classes and the `ALG_*` / `FMT_*` / `USAGE_*` / `CURVE_*` constants you pass to `subtle` are imported from the `'crypto'` module.\n\n## Quick helpers (`crypto.*`)\n\nEvery helper is synchronous and returns its value directly.\n\n| Helper | Signature | What it does |\n| --- | --- | --- |\n| `getRandomValues` | `(array: Uint8Array): void` | Fill the array with cryptographically strong random bytes. |\n| `randomUUID` | `(): string` | A random RFC 4122 v4 UUID string. |\n| `sha1` / `sha256` / `sha384` / `sha512` | `(data: Uint8Array): Uint8Array` | Hash raw bytes. |\n| `sha1Text` … `sha512Text` | `(s: string): Uint8Array` | UTF-8 encode the string, then hash. |\n| `hmacSha256` | `(key: Uint8Array, msg: Uint8Array): Uint8Array` | Keyed fingerprint (HMAC-SHA-256) of bytes. |\n| `hmacSha256Text` | `(key: Uint8Array, msg: string): Uint8Array` | HMAC-SHA-256 over a string. |\n| `toHex` | `(bytes: Uint8Array): string` | Lowercase hex string (for display or storage). |\n| `subtle` | `SubtleCrypto` | The full primitive surface (below). |\n\n**Hashing** turns any input into a fixed-size fingerprint. The same input always gives the same output, and you cannot work backward from the fingerprint to the input. Use it to compare or index a value without storing the original (for example, keying a cache by the hash of a URL).\n\n**Random bytes** come from a CSPRNG (a cryptographically secure random generator), which means the output is unpredictable and safe for tokens, session IDs, and IVs. Never use `Math.random()` for anything security-related.\n\n**HMAC** is a fingerprint computed *with a secret key*. Anyone can hash a value, but only someone who holds the key can produce the correct HMAC, so it proves a value came from you and was not altered. This is exactly what [`TwoFactor`](./email.md) tokens and [signed cookies](./cookies.md) use.\n\n## Worked example: hash a value and generate a token\n\n```ts\nimport { RouteContext } from 'toiljs/server/runtime';\n\n@rest('demo')\nclass Demo {\n @get('/')\n public example(ctx: RouteContext): string {\n // Hash a value to a hex fingerprint (safe to log or use as a key).\n const digest = crypto.sha256Text('alice@example.com');\n const fingerprint = crypto.toHex(digest); // 64 hex chars\n\n // Generate a 128-bit unguessable token as hex.\n const bytes = new Uint8Array(16);\n crypto.getRandomValues(bytes);\n const token = crypto.toHex(bytes); // 32 hex chars\n\n // A ready-made unique id.\n const id = crypto.randomUUID();\n\n return `fp=${fingerprint} token=${token} id=${id}`;\n }\n}\n```\n\nTo fingerprint a value *with a secret* (so only your server can produce or check it), use HMAC:\n\n```ts\nconst key = Uint8Array.wrap(String.UTF8.encode(Environment.getSecure('SIGNING_KEY')!));\nconst mac = crypto.hmacSha256Text(key, 'order:42');\nconst macHex = crypto.toHex(mac);\n```\n\n## The primitive surface (`crypto.subtle`)\n\nUse `subtle` when the helpers are not enough: symmetric encryption, signatures, or key derivation. Every method is synchronous.\n\n| Method | Signature |\n| --- | --- |\n| `digest` | `digest(algorithm: i32, data: Uint8Array): Uint8Array` |\n| `importKey` | `importKey(format: i32, keyData: Uint8Array, algorithm: AlgorithmParams, extractable: bool, usages: i32): CryptoKey` |\n| `exportKey` | `exportKey(format: i32, key: CryptoKey): Uint8Array` |\n| `encrypt` | `encrypt(algorithm: AlgorithmParams, key: CryptoKey, data: Uint8Array): Uint8Array` |\n| `decrypt` | `decrypt(algorithm: AlgorithmParams, key: CryptoKey, data: Uint8Array): Uint8Array` |\n| `sign` | `sign(algorithm: AlgorithmParams, key: CryptoKey, data: Uint8Array): Uint8Array` |\n| `verify` | `verify(algorithm: AlgorithmParams, key: CryptoKey, signature: Uint8Array, data: Uint8Array): bool` |\n| `deriveBits` | `deriveBits(algorithm: AlgorithmParams, baseKey: CryptoKey, length: i32): Uint8Array` |\n| `deriveKey` | `deriveKey(algorithm, baseKey, lengthBits, derivedKeyAlgorithm, extractable, usages): CryptoKey` |\n\nTwo things differ from the browser API, both because of the missing-Promise design:\n\n- **`algorithm` and `format` are integer constants, not strings.** You pass `ALG_SHA_256` (not `\"SHA-256\"`) and `FMT_RAW` (not `\"raw\"`). The constants are listed below.\n- **`verify` returns a `bool`.** A signature mismatch returns `false` (it does not throw). A real error, like an invalid key, does throw.\n\n### Keys are per-request handles\n\nYou never hold raw key bytes in your code for long. `importKey` gives you back a **`CryptoKey`**, which is an opaque handle into a host-side keystore. That keystore is **wiped when the request ends**, so a `CryptoKey` is valid only within the request that created it. Never cache one across requests; import it again each time.\n\n```mermaid\nsequenceDiagram\n participant G as Your handler\n participant H as Host keystore (per request)\n G->>H: importKey(...) -> a CryptoKey handle\n G->>H: sign(params, key, data)\n H-->>G: signature bytes\n Note over H: the keystore is wiped when the request ends\n```\n\n### Algorithm parameter classes\n\nEach algorithm takes a small parameters object you build and pass in. Import the class you need from `'crypto'`:\n\n```ts\nimport { AesGcmParams, HmacImportParams, ALG_SHA_256, USAGE_ENCRYPT, USAGE_DECRYPT } from 'crypto';\n```\n\n| Class | Constructor | Used for |\n| --- | --- | --- |\n| `AesGcmParams` | `(iv, additionalData?, tagLength = 128)` | AES-GCM encrypt/decrypt |\n| `AesCbcParams` | `(iv)` | AES-CBC |\n| `AesCtrParams` | `(counter, length = 128)` | AES-CTR |\n| `HmacImportParams` | `(hash)` | importing an HMAC key |\n| `HmacParams` | `()` | HMAC sign/verify (hash comes from the key) |\n| `Pbkdf2Params` | `(hash, salt, iterations)` | deriving a key from a password |\n| `HkdfParams` | `(hash, salt, info?)` | deriving a key from key material |\n| `EcdsaParams` | `(hash)` | ECDSA sign/verify |\n| `EcKeyImportParams` | `(alg, namedCurve)` | importing an EC key |\n| `Ed25519Params` | `()` | Ed25519 sign/verify |\n| `X25519ImportParams` | `()` | importing an X25519 key |\n| `EcdhParams` | `(alg, publicKeyHandle)` | ECDH / X25519 key agreement |\n\n### Constants\n\n- **Hashes and algorithms:** `ALG_SHA_1`, `ALG_SHA_256`, `ALG_SHA_384`, `ALG_SHA_512`, `ALG_SHA3_256`, `ALG_SHA3_384`, `ALG_SHA3_512`, `ALG_AES_GCM`, `ALG_AES_CBC`, `ALG_AES_CTR`, `ALG_AES_KW`, `ALG_HMAC`, `ALG_ECDSA`, `ALG_ED25519`, `ALG_ECDH`, `ALG_X25519`, `ALG_HKDF`, `ALG_PBKDF2`.\n- **Key formats:** `FMT_RAW`, `FMT_PKCS8`, `FMT_SPKI`. (`FMT_JWK` is rejected.)\n- **Key usages (a bitmask, OR them together):** `USAGE_ENCRYPT`, `USAGE_DECRYPT`, `USAGE_SIGN`, `USAGE_VERIFY`, `USAGE_DERIVE_KEY`, `USAGE_DERIVE_BITS`, `USAGE_WRAP_KEY`, `USAGE_UNWRAP_KEY`.\n- **Named curves:** `CURVE_P256`, `CURVE_P384`. (`CURVE_P521` is not supported.)\n\nThe `digest` selector also accepts the SHA3 constants, so `crypto.subtle.digest(ALG_SHA3_256, data)` works even though there is no `sha3` quick helper.\n\n### `CryptoKey`\n\nA `CryptoKey` is a handle plus metadata: `handle: i32`, `type: string` (`\"secret\"`, `\"public\"`, or `\"private\"`), `extractable: bool`, `algorithm: i32`, and `usages: i32`, with `algorithmName()` and `hasUsage(u)` helpers. Remember it is only valid within the current request.\n\n## Example: AES-256-GCM encrypt and decrypt\n\nAES-GCM is authenticated encryption: it both hides the data and detects tampering. It needs a 32-byte key and a fresh 12-byte IV (a \"number used once\"). **Never reuse an IV with the same key**; generate a new one every time.\n\n```ts\nimport { AesGcmParams, ALG_AES_GCM, FMT_RAW, USAGE_ENCRYPT, USAGE_DECRYPT } from 'crypto';\n\n// 32-byte key and a fresh 12-byte IV.\nconst rawKey = new Uint8Array(32); crypto.getRandomValues(rawKey);\nconst iv = new Uint8Array(12); crypto.getRandomValues(iv);\n\n// Import the key once; grant it both usages so it can round-trip.\nconst key = crypto.subtle.importKey(\n FMT_RAW, rawKey, new AesGcmParams(iv), false, USAGE_ENCRYPT | USAGE_DECRYPT,\n);\n\nconst plaintext = Uint8Array.wrap(String.UTF8.encode('secret note'));\nconst ciphertext = crypto.subtle.encrypt(new AesGcmParams(iv), key, plaintext);\nconst recovered = crypto.subtle.decrypt(new AesGcmParams(iv), key, ciphertext);\n// recovered == plaintext\n```\n\nStore the `iv` next to the ciphertext (it is not secret); you need the same IV to decrypt.\n\n## Example: HMAC with `subtle`\n\nThe quick `crypto.hmacSha256` helper does this in one line, but here is the explicit form, which mirrors how [`TwoFactor`](./email.md) signs its tokens:\n\n```ts\nimport { HmacImportParams, HmacParams, ALG_SHA_256, FMT_RAW, USAGE_SIGN, USAGE_VERIFY } from 'crypto';\n\nconst key = crypto.subtle.importKey(\n FMT_RAW, rawKeyBytes, new HmacImportParams(ALG_SHA_256), false, USAGE_SIGN | USAGE_VERIFY,\n);\n\nconst mac = crypto.subtle.sign(new HmacParams(), key, message);\nconst ok: bool = crypto.subtle.verify(new HmacParams(), key, mac, message); // true\n\n// verify compares in constant time and returns false on a mismatch (it does not throw).\n```\n\n## Limitations\n\n- **No Promises.** Every call is synchronous.\n- **No RSA.** It was dropped because the only pure-Rust implementation had an unfixable timing weakness. Use ECDSA or Ed25519 for signatures.\n- **No JWK key format.** Use `FMT_RAW`, `FMT_PKCS8`, or `FMT_SPKI`.\n- **No on-host key generation.** Keys are always **imported**, never generated on the server. Generate them elsewhere and import the bytes.\n- **No P-521.** `CURVE_P256` and `CURVE_P384` are supported.\n- **Keys do not survive the request.** A `CryptoKey` is a per-request handle; re-import each request.\n- **Every call is metered.** Crypto operations cost compute (charged up front from the sizes involved), so an over-budget call fails cleanly instead of burning CPU.\n\n### Post-quantum signature verify\n\nAuth's login uses a post-quantum signature scheme (ML-DSA) that the host can **verify** (it never holds a private key). This underpins the [auth login stack](../auth/how-it-works.md); you reach it through `AuthService`, not through `crypto` directly, so it is documented there rather than here.\n\n## Related\n\n- [Auth: how it works](../auth/how-it-works.md), the post-quantum login stack built on host-side verify.\n- [Cookies](./cookies.md), signed and encrypted cookies built on `crypto`.\n- [Email](./email.md), whose `TwoFactor` codes are HMAC tokens.\n- [Environment and secrets](./environment.md), where you keep signing and encryption keys.\n",
|
|
70
71
|
"services/email.md": "# Email\n\nSend transactional email (verification codes, password resets, receipts) straight from a route handler.\n\nReach for email when your app needs to message one user because of something they did: confirm a signup, send a 2FA code, mail a receipt. It is built for **transactional** email (one message, triggered by an action), not bulk marketing or newsletters. There is no attachment support and no per-recipient dynamic content beyond simple placeholders (see [gotchas](#gotchas-and-limits)).\n\nEverything here is an **ambient global**: an object you can use without importing it, exactly like [`crypto`](./crypto.md) or `Environment`. A backend that never sends email pulls none of this into its compiled program.\n\n- **`EmailService`** sends one email.\n- **`EmailTemplate`** is a reusable message with `{{placeholder}}` holes (plain text and/or HTML).\n- **`emails/*.tsx`** lets you author emails as React components; the build turns each into a typed `Emails.<Name>.send(...)`.\n- **`TwoFactor`** issues and checks email verification codes with no database.\n\n## How a send works\n\nWhen your handler calls `EmailService.send(...)`, it does not talk to the email provider itself. The **edge** (the Dacely server running your code) validates the recipient, checks your sending limits, then hands the message to a single background \"mailer\" thread that talks to the provider. Your handler **suspends** (pauses) until the mailer reports a result, so a slow provider never freezes the worker that is serving other requests.\n\n```mermaid\nsequenceDiagram\n participant H as Your handler (wasm)\n participant E as Edge host\n participant M as Off-core mailer\n participant P as Email provider\n H->>E: EmailService.send(to, subject, body, purpose, html)\n E->>E: validate recipient, check caps + dedup\n E->>M: hand off the message\n Note over H: your handler pauses here\n M->>P: deliver over the network\n P-->>M: accepted / rejected\n M-->>E: status\n E-->>H: EmailStatus (Sent, Budget, ...)\n```\n\n## Send one email\n\n`EmailService.send` takes the recipient, a subject, a plain-text body, a short `purpose` tag, and an optional HTML body. It returns an `EmailStatus` telling you what happened.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('notify')\nclass Notify {\n @post('/welcome')\n welcome(ctx: RouteContext): Response {\n const status = EmailService.send(\n 'alice@example.com', // to: exactly one address\n 'Welcome!', // subject\n 'Thanks for signing up.', // plain-text body\n 'welcome', // purpose tag (used for dedup + abuse limits)\n '<h1>Thanks for signing up.</h1>', // optional HTML body\n );\n\n return status == EmailStatus.Sent\n ? Response.text('sent\\n')\n : Response.text('could not send\\n', 503);\n }\n}\n```\n\nThe full signature is `send(to, subject, body, purpose = 'tx', html = '')`. Pass a non-empty `html` to send an HTML message; then `body` is the plain-text fallback (set both for the best deliverability, or leave `body` empty for HTML-only).\n\n`EmailStatus` is a global enum, so you can compare against it (`status == EmailStatus.Sent`) with no import. `Sent` and `Deduped` mean success; the rest tell you why the message was not delivered and whether retrying could help.\n\n| Status | Meaning | Retry? |\n| --- | --- | --- |\n| `Sent` | Accepted by the provider. | done |\n| `Deduped` | An identical recent `(recipient, purpose)` send was collapsed into one. | treat as sent |\n| `Budget` | Your per-minute (or per-day) send budget is exhausted. | yes, later |\n| `TryLater` | The mailer was momentarily saturated. | yes, back off |\n| `RecipientCapped` | This recipient hit their hourly cap. | no (this window) |\n| `BadRecipient` | The address failed validation (multiple addresses, control characters). | no |\n| `Disabled` | This site has no email configured. | no |\n| `ProviderError` | The provider rejected it, or delivery failed after retries. | no |\n\nThe `purpose` is a short, non-sensitive tag like `\"welcome\"` or `\"reset\"`. The edge folds it into two things: a **dedup key** (an identical `(site, recipient, purpose)` within about 30 seconds becomes one send, which returns `Deduped`) and the **abuse counters**. It is never logged in the clear, so keep real user data out of it.\n\nThe recipient is checked on the host: exactly one address, no carriage returns, line feeds, or angle brackets. That means a bad or hostile input can never smuggle a second recipient or inject an email header.\n\n## Configure email\n\nEmail is **off by default**. A site that has not configured it gets `Disabled` from every send. You configure it in two places: non-secret settings in `toil.config.ts`, and the provider credential as a **secret** in the [environment store](./environment.md) (never in your code or your compiled program).\n\n```ts\n// toil.config.ts\nimport { defineConfig } from 'toiljs/compiler';\n\nexport default defineConfig({\n server: {\n email: {\n provider: 'resend', // 'resend' | 'gmail' | 'smtp'\n from: 'you@example.com', // the From address (validated; single address)\n maxPerMin: 60, // per-site cap: at most 60 sends per rolling minute\n },\n },\n});\n```\n\nThe credential lives in your secrets file as a reserved `TOIL_EMAIL_*` key. These keys are **host-only**: the edge reads them in Rust, and they are stripped out of the buckets your code can read, so a tenant can never fish the API key out with `Environment.getSecure`.\n\n```bash\n# .env.secrets (gitignored; never committed, never in the .wasm)\nTOIL_EMAIL_API_KEY=re_xxxxxxxxxxxx\n```\n\nOn the production edge the same keys live in the site's secrets file (mode `0600`, kept out of the config directory the file watcher sees), so a credential is never written to a log, to `/_admin`, or into your deployed module. See [Environment and secrets](./environment.md) for how the store works.\n\n### Providers\n\nYou pick one provider. Each maps `TOIL_EMAIL_API_KEY` to the right credential.\n\n| Provider | `provider` | What `TOIL_EMAIL_API_KEY` holds | Extra keys |\n| --- | --- | --- | --- |\n| **Resend** | `resend` | A Resend API key (a JSON API). | none |\n| **Gmail** | `gmail` | A Gmail **App Password** (SMTP over `smtp.gmail.com:587`). | none |\n| **Generic SMTP** | `smtp` | The SMTP password. | `TOIL_EMAIL_SMTP_HOST` (required), optional `TOIL_EMAIL_SMTP_PORT` (defaults `587`), `TOIL_EMAIL_SMTP_USER` (defaults to `from`) |\n\nFor Gmail you need 2-Step Verification on the account, then create an App Password at `https://myaccount.google.com/apppasswords`. The username is your `from` address. For generic SMTP, port `587` uses STARTTLS and port `465` uses implicit TLS.\n\nEvery setting can also be given as a `TOIL_EMAIL_*` environment variable, and those override the `toil.config.ts` values. That means the exact same secrets file works in local dev and on the edge.\n\n### Local dev behavior\n\n`toiljs dev` runs the **full email pipeline** in Node: recipient validation, dedup, and every cap behave exactly like the edge. Once you set a provider plus `TOIL_EMAIL_API_KEY`, dev **really sends** (Resend over its API, Gmail/SMTP over nodemailer). If you have **not** configured a provider, `EmailService.send` becomes a log-only mock that returns `Sent`, so a signup flow that emails a code still works end to end without any setup.\n\n> One dev-only nuance: the dev server runs your code synchronously, so the actual network send is fire-and-forget. Validation and the caps return their exact status immediately, but a `Sent` is optimistic; the real delivery outcome (or a `ProviderError`) is logged, not returned. The programming interface is identical to the edge, so code that runs in dev runs unchanged in production.\n\n## Templates with `{{placeholders}}`\n\nWhen you send the same shape of email with different values, define an `EmailTemplate` once. `{{name}}` holes are filled from a `Map` at send time.\n\n```ts\nconst welcome = new EmailTemplate(\n 'Welcome, {{name}}!', // subject\n 'Hi {{name}}, your code is {{code}}.', // plain-text body\n '<h1>Welcome, {{name}}</h1><p>Code: <b>{{code}}</b></p>', // html (optional)\n);\n\nconst vars = new Map<string, string>();\nvars.set('name', 'Alice');\nvars.set('code', '123456');\n\nconst status = welcome.send('alice@example.com', vars, 'welcome');\n```\n\n- `{{ name }}` with surrounding spaces is accepted; an unknown placeholder renders to an empty string.\n- Omit the third argument for a plain-text-only template.\n- `template.render(vars)` returns the rendered `{ subject, body, html }` **without** sending, which is handy for previews and tests.\n\nFor anything richer than token substitution (real layout, brand styling), author the email as a React component instead.\n\n## React email templates\n\nYou can write emails as React components in an **`emails/`** folder. At `toiljs build` each one is rendered **once, at build time**, into a static string of HTML with inline styles, and its props become `{{token}}` holes. The build then generates a typed `Emails.<Name>.send(...)` for your server to call.\n\nWhy build-time? Email clients (Gmail, Outlook, Apple Mail) run **no JavaScript** and strip `<style>` blocks and external CSS. So an HTML email has to be a finished, inline-styled string. Rendering it once at build gives you React ergonomics while shipping the inbox exactly what it can display.\n\n```tsx\n// emails/Welcome.tsx\nexport const subject = 'Welcome, {{name}}!';\n\nexport default function Welcome({ name, code }: { name: string; code: string }) {\n return (\n <table\n width=\"100%\"\n style={{ fontFamily: 'Arial, sans-serif' }}>\n <tbody>\n <tr>\n <td style={{ padding: '24px' }}>\n <h1 style={{ color: '#111' }}>Welcome, {name}!</h1>\n <p>\n Your code is <b>{code}</b>.\n </p>\n </td>\n </tr>\n </tbody>\n </table>\n );\n}\n```\n\nThe generated `Emails.Welcome.send(...)` takes the recipient, then one argument per `{{token}}` **in alphabetical order**, then an optional `purpose`:\n\n```ts\n// Welcome.tsx uses {{code}} and {{name}} -> params are (code, name), alphabetical\nconst status = Emails.Welcome.send('alice@example.com', '123456', 'Alice');\n```\n\nAuthoring notes:\n\n- **Styles must end up inline.** Write inline `style={{ ... }}`, or import a stylesheet and its rules are inlined into each element for you at build. A bare CSS import on its own has no effect on the rendered email.\n- **Optional exports:** `subject` (a token template; defaults to the file name), `text` (a plain-text alternative; otherwise derived from the HTML), and `purpose`.\n- **Field substitution only.** Because the component renders once at build time, a runtime `{items.map(...)}` bakes in at build; it does not re-run per recipient. That is perfect for verification, confirmation, and receipt emails; a truly dynamic list needs a different approach.\n\nWhile `toiljs dev` runs, open **`/__toil/emails`** (the dev banner prints the link) to preview every `emails/*.tsx`, fill each `{{token}}`, and toggle the HTML and plain-text parts. It refreshes as you edit.\n\n## Verification codes with `TwoFactor`\n\n`TwoFactor` issues and checks short numeric codes (for 2FA, email confirmation, or magic-code login) with **no database**. That is possible because it is **stateless**: instead of storing the code server-side, it emails the code and returns a signed **token** that commits to the code using [HMAC](./crypto.md) (a keyed fingerprint). The code itself is only ever in the email; the token holds a fingerprint of it, not the code.\n\nTo verify, the server recomputes the fingerprint from the token plus the code the user typed and compares them in constant time. A valid `(token, code)` pair can only come from someone who **both** received the email and holds the token, and the fingerprint binds the recipient, the purpose, and an expiry so a token cannot be replayed for another address, another flow, or after it expires.\n\n```ts\n// 1. Issue and email a code. Hand the returned token to the client\n// (a cookie or a hidden form field).\nconst challenge = TwoFactor.send('alice@example.com', 'login');\n// challenge.token -> give this to the client\n// challenge.status -> the EmailStatus of the send (check it was Sent/Deduped)\n\n// 2. Later, when the user submits the code they received:\nconst ok: bool = TwoFactor.verify(challenge.token, 'alice@example.com', userEntered);\nif (ok) {\n // the code is valid and unexpired\n}\n```\n\nThe three entry points:\n\n- **`send(recipient, purpose, ttlSecs = 600, digits = 6)`** issues a code, emails it with a built-in template, and returns `{ token, status }`.\n- **`issue(recipient, purpose, ttlSecs, digits)`** returns `{ code, token }` **without** sending, so you can email `code` yourself with a branded `EmailTemplate` or `Emails.*` message.\n- **`verify(token, recipient, code)`** returns `true` only for a code issued for that recipient that has not expired.\n\nOne-time setup: call **`TwoFactor.setSecret(secret)`** once at startup in `main.ts`. This is the HMAC key that signs the tokens. It must be identical on every edge instance and must never end up in a client bundle. (It is separate from your email provider key.)\n\n> **Important limitation: not single-use.** `TwoFactor` gives you integrity and expiry, but because it stores no state, a valid code verifies **repeatedly** until its TTL runs out. Keep the TTL short. If you need true single-use, record a per-recipient \"last verified at\" (in your database or the edge store) and reject any code at or before it.\n\n## Worked example: welcome email plus a 2FA code\n\nA signup route that emails a welcome message and starts a 2FA challenge in one handler:\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('signup')\nclass Signup {\n @post('/start')\n start(ctx: RouteContext): Response {\n const email = ctx.query('email'); // in real code, validate this first\n\n // 1. Friendly welcome (fire it, but note the status for logging).\n Emails.Welcome.send(email, '123456', 'Alice');\n\n // 2. Start a 2FA challenge; the code is emailed, the token comes back.\n const challenge = TwoFactor.send(email, 'signup');\n if (challenge.status != EmailStatus.Sent && challenge.status != EmailStatus.Deduped) {\n return Response.text('could not send code\\n', 503);\n }\n\n // Hand the token to the client (here, a JSON field; a cookie also works).\n return Response.json(`{\"token\":${JSON.stringify(challenge.token)}}`);\n }\n\n @post('/confirm')\n confirm(ctx: RouteContext): Response {\n const email = ctx.query('email');\n const token = ctx.query('token');\n const code = ctx.query('code');\n\n const ok: bool = TwoFactor.verify(token, email, code);\n return ok ? Response.text('confirmed\\n') : Response.text('bad code\\n', 400);\n }\n}\n```\n\nTo make `TwoFactor` verify across restarts and across the fleet, set the secret once at startup:\n\n```ts\n// main.ts\nTwoFactor.setSecret(crypto.sha256Text(Environment.getSecure('TWOFA_SECRET')!));\n```\n\nFor where email and codes fit into a full login flow, see [extending auth](../auth/extending.md).\n\n## Gotchas and limits\n\n- **Email is opt-in.** No config means every send returns `Disabled`. Check the status.\n- **The provider key is host-only.** It never appears in your compiled module, in logs, or in `/_admin`. You cannot read it with `Environment.getSecure`; that is by design.\n- **No attachments, no dynamic per-recipient lists.** The API sends a subject plus a text and/or HTML body. React templates substitute fields; they do not re-render per recipient.\n- **`TwoFactor` is not single-use.** See the limitation above. Keep TTLs short.\n- **Caps are enforced on the host, exactly.** A per-site per-minute cap (`maxPerMin`) and an optional per-day cap, plus a per-recipient hourly cap and about 30 seconds of dedup. Over a cap you get `Budget`, `RecipientCapped`, or `Deduped`, never a silent drop. Editing the caps takes effect on the next send with no restart.\n- **Put nothing sensitive in `purpose`.** It keys dedup and abuse counters; use a short tag like `\"reset\"`.\n- **Observability:** `GET /_admin/email` returns counts by reason (submitted, sent, deduped, budget, and so on). It exposes counts only, never a recipient, subject, body, or code.\n\n## Related\n\n- [Environment and secrets](./environment.md), where `TOIL_EMAIL_API_KEY` lives.\n- [Crypto](./crypto.md), the HMAC and random primitives `TwoFactor` is built on.\n- [Rate limiting](./ratelimit.md), to protect any route that triggers an email (like \"send me a code\").\n- [Extending auth](../auth/extending.md), for email and codes inside a login flow.\n- [HTTP routes](../backend/rest.md), for `@rest` / `@post` and `RouteContext`.\n",
|
|
71
72
|
"services/environment.md": "# Environment and secrets\n\n**`Environment` gives your app configuration values and secrets that are set outside your code**, so your compiled backend (`.wasm`) never carries a password or an API key. You read them at runtime; you never write them from code.\n\n## Why this exists\n\nYour backend often needs two kinds of outside values:\n\n- **Config**: non-sensitive settings that can change per deployment, like a public API base URL, a region name, or a feature flag.\n- **Secrets**: sensitive values you must never leak, like a payment provider's API key.\n\nThe wrong way to handle these is to type them into your source code. Then your secret is in your git history, in your build output, and in the `.wasm` shipped to every edge node. The right way, and the way toiljs uses, is the **GitHub Actions model**: you set these values out of band (today a dashboard, backed by the edge database), and your code reads them by name at runtime. Your compiled program carries **no credentials**.\n\n`Environment` is an ambient global: you use it with no import, like `Time` or `crypto`.\n\n## The two buckets: `get` vs `getSecure`\n\nThere are two **disjoint** buckets, exactly like GitHub Actions' `vars` versus `secrets`:\n\n- **`Environment.get(key)`** reads a **plain variable** (config). Returns the string, or `null` if it is not set.\n- **`Environment.getSecure(key)`** reads a **secret**. Returns the string, or `null` if it is not set.\n\n\"Disjoint\" means the buckets never overlap: a secret is **never** returned by `get()`, and a plain var is never returned by `getSecure()`. This is a deliberate safety property. It means a secret cannot leak through a code path that logs the result of a `get()`, because `get()` can never see a secret in the first place. Keys are case-sensitive and matched exactly.\n\n```mermaid\nflowchart LR\n subgraph Store[\"Per-app environment (set out of band)\"]\n V[\"vars<br/>(config)\"]\n S[\"secrets\"]\n end\n V -->|Environment.get key| G[\"your handler\"]\n S -->|Environment.getSecure key| G\n V -. never .-x GS[\"getSecure\"]\n S -. never .-x GT[\"get\"]\n```\n\n## How: a worked example reading a secret\n\nHere a route reads a public config value and a secret, then uses the secret to authorize a call to a third party. Note what it does **not** do: it never logs the secret and never returns it to the client.\n\n```ts\nimport { Response, RouteContext } from 'toiljs/server/runtime';\n\n@rest('billing')\nclass Billing {\n @get('/status')\n show(ctx: RouteContext): Response {\n const base = Environment.get('PUBLIC_API_BASE'); // plain var, or null\n const key = Environment.getSecure('STRIPE_KEY'); // secret, or null\n\n if (key == null) {\n return Response.text('billing not configured\\n', 503);\n }\n\n // Use `key` to authorize an outbound call. NEVER log it,\n // NEVER put it in the response, NEVER copy it into the client bundle.\n // (Outbound calls are made from a daemon or service; see the links below.)\n\n return Response.text(base != null ? base : 'unset');\n }\n}\n```\n\nBoth getters return `string | null`. Always handle the `null` case: a value that is not configured comes back as `null`, not an empty string or a crash.\n\n> **Secrets are plaintext in your module at runtime.** That is the point: you need the actual bytes to call out to a third party. So the rule is simple. Do not log a secret, do not put it in a response, and do not copy it into anything the browser receives.\n\n## Where the values live\n\nYou set vars and secrets in **two separate places**, so the split is structural (the secrets store can be locked down on its own).\n\n### On the edge\n\nEach host has its own environment, backed by the edge database and, as a fallback, by two dotenv files kept **outside** your deployed project (so the config watcher never even sees a credential):\n\n```bash\n# $TOIL_ENV_DIR/<host>.env (default dir: /run/toil/env) -> Environment.get(...)\nPUBLIC_API_BASE=https://api.example.com\nREGION=eu\n\n# $TOIL_ENV_DIR/<host>.env.secrets (mode 0600) -> Environment.getSecure(...)\nSTRIPE_KEY=sk_live_xxx\n```\n\nA **dotenv file** is just `KEY=value`, one per line, with `#` comments, optional `export`, and optional quotes.\n\nFramework-reserved keys use the `TOIL_` prefix (today, the `TOIL_EMAIL_*` email provider config). These are **host-only**: resolved and used in Rust where the framework needs them, and **stripped from both guest buckets**, so your code can never read them with `get` or `getSecure`. You do not read email config yourself; see [Email](./email.md).\n\n### In local development\n\n`toiljs dev` reads two files at your project root:\n\n```bash\n# .env (plain vars)\nPUBLIC_API_BASE=http://localhost:4000\n\n# .env.secrets (secrets; mode 0600; gitignored by the scaffold)\nSTRIPE_KEY=sk_test_xxx\n```\n\nIt also overlays `process.env` as plain vars, for convenience. The behavior is identical to the edge, so code that reads env in dev reads it the same way in production. Both files are gitignored by the project scaffold so you do not commit them by accident.\n\n## Secrets never enter the `.wasm`\n\nThis is the whole point, so it is worth stating plainly. Your source code contains only the **names** you look up (`'STRIPE_KEY'`), never the values. The compiler turns your names into calls to a host function; the actual value is resolved on the trusted server side, from that host's environment, at the moment you ask. The value is never baked into the compiled module and never shipped to the browser.\n\nOn the edge, a secret is **zeroized** (wiped from memory) when the host goes cold, so it does not linger.\n\n## How caching works (and why it is safe)\n\nReading env goes through a host lookup, so the edge caches values so a busy app is not paying for a fresh lookup on every request. The cache is designed to be abuse-proof:\n\n- **Lazy**: env is loaded the first time your code reads it, not eagerly. A host that never reads env costs nothing.\n- **Shared and read-only**: the data lives in one place and is never copied per request.\n- **Bounded with idle eviction (a TTL cache)**: entries are dropped after they sit unused, and the total size is capped. So a flood of requests to many different hosts can never grow memory without bound, and secrets are wiped when a host goes cold.\n\nYou do not manage any of this. You just call `get` / `getSecure`.\n\n## Build-time config vs runtime environment\n\nDo not confuse two different things that both feel like \"configuration\":\n\n- **Build-time config** lives in your project's config file (for example `toil.config.ts`). It shapes how your app is *built and wired*: routes, tiers, which features are on. It is baked into the build. See [Configuration](../concepts/config.md).\n- **Runtime environment** (this page) is resolved *while your code runs*, per host, and is never baked in. It is where secrets and per-deployment values belong.\n\nRule of thumb: if a value is a secret, or it differs between your dev machine and production, it belongs in the runtime environment, not in build-time config.\n\n## Gotchas\n\n- **`Environment` is read-only.** There is no `set`. You configure values out of band (dashboard / dotenv files), never from the module.\n- **Handle `null`.** An unset key returns `null`. Do not assume a value is present.\n- **Never log or return a secret.** `getSecure` hands you real credentials; treat them like one.\n- **Do not put a secret in build-time config.** That would bake it into the `.wasm`, defeating the whole design.\n- **`TOIL_`-prefixed keys are invisible to your code.** They are host-only framework config; you cannot read them with `get`/`getSecure`.\n\n## Related\n\n- [Email](./email.md): configured through the reserved, host-only `TOIL_EMAIL_*` keys.\n- [Configuration](../concepts/config.md): build-time config, and how it differs from runtime env.\n- [Crypto](./crypto.md): use a secret from `getSecure` as a signing or encryption key.\n- [Cookies](./cookies.md): `SecureCookies` takes a key you can source from a secret.\n",
|