toiljs 0.0.85 → 0.0.87

package/docs/auth/how-it-works.md

@@ -0,0 +1,138 @@
+# How Toil auth works
+
+Toil auth is a **post-quantum, password-based, mutually-authenticated** login. The design goal: the user
+types a password, but the server never sees it, never stores it, and can't be phished into leaking a
+verifier, and every primitive is quantum-resistant.
+
+This page explains the protocol. You do not need to understand it to use auth (see [Usage](./usage.md)),
+but you should understand the guarantees before you ship.
+
+## The building blocks
+
+| Primitive | Role |
+| --- | --- |
+| **OPRF** (RFC 9497, ristretto255-SHA512) | A *server-keyed* salt. The client blinds its password, the server evaluates it under a per-user key, the client unblinds. The result can't be computed without the server, so a stolen database can't be brute-forced offline. |
+| **Argon2id** | Stretches the OPRF output into key material (memory-hard, GPU/ASIC-resistant). Runs in the browser. |
+| **ML-DSA-44** (FIPS 204) | The user's identity key pair is *derived* from the Argon2id output. The server stores only the **public** key. Login is proved by an ML-DSA signature. |
+| **ML-KEM-768** (FIPS 203) | Key encapsulation on login: the server proves it holds its KEM secret by returning a confirmation tag only derivable from the decapsulated shared secret → **mutual** auth (anti-phishing). |
+| **HMAC-SHA256** | Signs the session cookie (`AUTH_SESSION_SECRET`). |
+
+> The whole thing is sometimes called an *aPAKE* (asymmetric password-authenticated key exchange). Do not
+> call it OPAQUE, this is Toil's own OPRF + KEM + signature construction.
+
+## What the server stores
+
+Per account, in ToilDB (`@database AuthDb`):
+
+- `username`, a deterministic `salt`, the **ML-DSA public key**, and the Argon2id params.
+
+That's it. **No password, no password hash, no verifier that can be brute-forced without the OPRF key.**
+Login challenges are a second collection, consumed exactly once (atomic `getDelete`).
+
+## Registration
+
+```
+ Browser (client)                                   Toil edge (server)
+ ─────────────────                                  ──────────────────
+ blind(password)                                    OPRF key = f(seed, username)
+        │  username, blinded                                │
+        │ ───────── POST /auth/register/start ───────────► │
+        │                                                   │  evaluated = OPRF(blinded)
+        │  ◄──── {mem,iters,par, salt, evaluated} ───────── │  (salt is deterministic per user)
+        │                                                   │
+ unblind → oprfOut                                          │
+ seed  = Argon2id(oprfOut, salt, params)                    │
+ (pk, sk) = ML-DSA-44.keygen(seed)                          │
+ proof = ML-DSA.sign(sk, "register|username|pk")            │
+        │  username, pk, proof                              │
+        │ ───────── POST /auth/register/finish ──────────► │
+        │                                                   │  verifyRegister(pk, msg, proof)  ✓ proof-of-possession
+        │  ◄──────────── {status: 0 ok | 1 taken} ──────── │  store AuthAccount{username, salt, pk, params}
+```
+
+The server never learns the password or the secret key `sk`, only the public key `pk` and a proof the
+client holds the matching `sk`. A duplicate username returns a **distinguishable** `status = 1` (so the UI
+can say "taken, log in instead"); everything else fails generically.
+
+## Login (with mutual auth)
+
+```
+ Browser (client)                                   Toil edge (server)
+ ─────────────────                                  ──────────────────
+ blind(password)                                            │
+        │  username, blinded                                │
+        │ ───────── POST /auth/login/start ──────────────► │  evaluated = OPRF(blinded)   (ALWAYS, even unknown user)
+        │                                                   │  if known: store Challenge{cid, nonce, iat, exp}
+        │  ◄─ {cid, aud, params, salt, nonce, iat, exp, ── │  (identical response whether the user exists or not)
+        │       evaluated}                                  │
+ unblind → Argon2id → (pk, sk) = ML-DSA.keygen              │
+ (ct, ssС) = ML-KEM-768.encapsulate(serverKemPublicKey)     │
+ msg = "login|username|aud|cid|nonce|iat|exp|ct|params|kid" │
+ sig = ML-DSA.sign(sk, msg)                                 │
+        │  cid, ct, sig                                     │
+        │ ───────── POST /auth/login/finish ─────────────► │  ch = challenges.getDelete(cid)   (consume once; check exp)
+        │                                                   │  rebuild msg from OUR stored values + ct
+        │                                                   │  verifyLogin(acct.pk, msg, sig)   ✓ it's really this user
+        │                                                   │  ssS = ML-KEM.decapsulate(ct)     ✓ we hold the KEM key
+        │                                                   │  K = deriveSessionKey(ssS, H(msg))
+        │  ◄──── {0, sessionToken, serverConfirm} ──────── │  serverConfirm = tag(K, H(msg))
+        │      + Set-Cookie: __Host-toil_sess=…             │  mint session cookie
+ verify serverConfirm using ssC   ✓ the server is genuine   │
+```
+
+Two verifications, both required:
+1. **The server verifies the client**: the ML-DSA signature over a message bound to the challenge, the
+   Argon2id params, and the server's KEM key id. Replays fail (the challenge is consumed).
+2. **The client verifies the server**: the `serverConfirm` tag is derivable only from the ML-KEM shared
+   secret, which only the holder of the KEM secret key can decapsulate. A phishing site can't forge it.
+
+## Anti-enumeration
+
+`/login/start` behaves identically for a known and an unknown user: it always runs the OPRF (a decoy key
+for unknown users), returns a **deterministic** per-user salt and constant params, and a fresh challenge.
+The challenge is persisted only for a real account, and `/login/finish` fails generically at consume for an
+unknown user. So an attacker can't probe which usernames exist. Every failure path returns the same
+`401 auth: request failed`.
+
+## Sessions & cookies
+
+On successful login the server mints **two** cookies:
+
+- **`__Host-toil_sess`**: the authoritative session. HMAC-SHA256 signed with `AUTH_SESSION_SECRET`,
+  `HttpOnly`, `Secure`, `SameSite=Lax`. It carries the `@user` codec payload. `@auth` and
+  `AuthService.getUser()` open + verify this cookie server-side; a forged or tampered cookie fails.
+- **`__Secure-toil_user`**: a **readable** companion carrying the same payload, so the browser can show
+  "logged in as …" via the client's `getUser()`. The server **never trusts it**, it is display-only.
+
+Each request runs in a fresh wasm instance, but the signed cookie is self-contained, so no server-side
+session store is needed. `AUTH_SESSION_SECRET` must be identical across every edge instance (it is, via the
+env store) so a cookie minted anywhere verifies everywhere.
+
+## The stable user identity: `ToilUserId`
+
+At login the server derives a stable, tenant-scoped id and stores it in the session (the first field of the
+built-in `@user`):
+
+```
+toilUserId = SHA-256( mldsaPublicKey ‖ username ‖ domain )      // 256 bits
+```
+
+- **Stable:** same login key + username on the same tenant `domain` → same id, forever, across sessions
+  and devices. Key your own data on it.
+- **Opaque + one-way:** it's a hash: safe to store, log, or expose without leaking the key or the address.
+- Read it anywhere with `AuthService.userId()`. See [Extending](./extending.md#toiluserid) for the
+  `ToilUserId` API (O(1) `==` / `!=`, `toHex()`, …).
+
+## Threat-model summary
+
+- **Server database stolen** → attacker gets public keys + salts, not passwords. Brute-forcing needs the
+  OPRF key (server-side) *and* Argon2id work per guess.
+- **Server compromised / malicious** → still can't recover passwords (only public keys) and can't forge a
+  past session without `AUTH_SESSION_SECRET`.
+- **Phishing site** → can't produce the `serverConfirm` tag (no KEM secret), so a correct client aborts.
+- **Quantum adversary** → ML-DSA + ML-KEM are post-quantum; the OPRF/Argon2id/HMAC pieces are classical but
+  not the long-term identity or key-exchange.
+- **Replay** → challenges are single-use (`getDelete`) with a short TTL.
+
+Residual responsibilities are yours: set the [secrets](./configuration.md), pin your deployment's KEM
+public key in the client, and raise the Argon2id params for production.

package/docs/auth/usage.md

@@ -0,0 +1,188 @@
+# Using auth
+
+## 1. Enable it
+
+Either the config flag (canonical) or a one-line import, both do the same thing (the build appends the
+framework's auth controller + user shape to your server as compiled entries, so `/auth/*` self-mounts):
+
+```ts
+// toil.config.ts, canonical
+export default defineConfig({ server: { auth: true } });
+```
+
+```ts
+// server/main.ts, escape hatch (equivalent)
+import 'toiljs/server/auth';
+```
+
+There is also an `AuthService.enable()` no-op you can call for discoverability, but enabling is done by the
+flag/import above (it happens at build time). Turn it off by removing the flag/import, with neither, no
+`/auth` routes and no accounts collection are generated. It is strictly opt-in.
+
+## 2. The client
+
+`toiljs/client` ships the browser half, it runs the OPRF blinding, Argon2id, ML-DSA keygen, and ML-KEM
+encapsulation, and talks to `/auth/*`. You never touch the crypto.
+
+```ts
+import { Auth } from 'toiljs/client';
+
+// Create an account. Throws on a taken username or a transport error.
+await Auth.register(username, password);
+
+// Log in. On success the browser holds the signed session cookie; subsequent
+// requests to @auth routes are authorized automatically.
+await Auth.login(username, password);
+```
+
+Options (second argument, `AuthOptions`):
+
+```ts
+await Auth.login(username, password, {
+    baseUrl: '/auth',                 // default; change if you mount elsewhere
+    serverKemPublicKey: MY_KEM_PUB,   // REQUIRED in production, pin your deployment's KEM key
+});
+```
+
+> **Production:** the client ships with the DEV KEM public key pinned. A real deployment MUST pass its own
+> `serverKemPublicKey` (derived from `AUTH_KEM_SK`). See [Configuration](./configuration.md).
+
+## 2b. Client-side: who's logged in, and protecting pages
+
+`register`/`login` set the session cookies; to *render* login state on the client, use the generated
+`getUser()` (emitted from the built-in `@user`). It reads the **readable** `__Secure-toil_user` companion
+cookie and returns the typed user or `null`, instant, no network call. It is **display-only**; the server
+still enforces access via `@auth`, so never trust it for authorization.
+
+```tsx
+import { getUser } from 'shared/server';   // generated; typed to the built-in @user
+
+function Nav() {
+    const user = getUser();                // { toilUserId, username } | null
+    return user
+        ? <span>Hi {user.username} <button onClick={logout}>Log out</button></span>
+        : <a href="/login">Log in</a>;
+}
+
+async function logout() {
+    await fetch('/auth/logout', { method: 'POST' });
+    location.href = '/login';              // cookies are cleared; bounce to login
+}
+```
+
+Gate a whole client route by redirecting when there's no session:
+
+```tsx
+export default function Dashboard() {
+    const user = getUser();
+    if (user == null) { location.href = '/login'; return null; }   // not logged in -> to /login
+    return <main>Welcome, {user.username}</main>;
+}
+```
+
+The authoritative check is still the server: any data this page fetches should come from an `@auth` route,
+so a user who forged/deleted the readable cookie sees the redirect OR a `401`, never real data.
+
+## 2c. Handling errors
+
+`Auth.register` / `Auth.login` reject with a message you can show. The important distinguishable cases:
+
+```ts
+try {
+    await Auth.register(username, password);
+} catch (e) {
+    const m = String(e);
+    if (m.includes('already registered')) setError('That username is taken, log in instead.');
+    else setError('Could not register, try again.');
+}
+
+try {
+    await Auth.login(username, password);
+} catch {
+    // Wrong password OR unknown user both fail generically (anti-enumeration), one message.
+    setError('Incorrect username or password.');
+}
+```
+
+- **Username taken** → `register` throws `auth: username already registered (log in instead)` (a
+  distinguishable case so you can guide the user).
+- **Wrong password / unknown user** → `login` throws generically (`auth: request failed`): by design,
+  the two are indistinguishable, so use ONE "incorrect username or password" message.
+- **Rate limited** → after 5 attempts / 60s a `429` surfaces as the same generic throw; back off and tell
+  the user to wait.
+
+## 3. Guard your routes: `@auth`
+
+Put `@auth` on a route method or a whole `@rest` class. The generated dispatcher checks for a valid signed
+session **before** your handler runs and returns `401` otherwise. `@auth` is unchanged by built-in auth,
+it's the same decorator you'd use with hand-written auth.
+
+```ts
+@rest('account')
+class AccountApi {
+    @auth                       // this route needs a session
+    @get('/settings')
+    public settings(): Response { /* … */ }
+
+    @get('/public')             // this one is open
+    public open(): Response { /* … */ }
+}
+
+@auth                           // …or guard the ENTIRE class
+@rest('admin')
+class AdminApi { /* every route requires a session */ }
+```
+
+## 4. Read the current user
+
+Inside any handler:
+
+```ts
+// The typed logged-in user (the built-in `@user`: toilUserId + username), or null.
+const user = AuthService.getUser();
+if (user != null) {
+    user.username;    // string
+    user.toilUserId;  // Uint8Array(32), the stable id bytes
+}
+
+// The stable identity as a ToilUserId (gate on hasSession() first, see note).
+if (AuthService.hasSession()) {
+    const id = AuthService.userId()!;   // ToilUserId
+    // key your own data on id (see Extending)
+}
+```
+
+> `ToilUserId` overloads `==`, so `AuthService.userId() == null` does not type-check. Gate with
+> `AuthService.hasSession()` and then use `userId()!`, or use `getUser()` and null-check that.
+
+## 5. The endpoint wire contract
+
+You normally use the `Auth` client, but the raw endpoints are binary (`DataWriter`/`DataReader`, never
+JSON):
+
+| Route | Request body | Response |
+| --- | --- | --- |
+| `POST /auth/register/start` | `str(username) bytes(blinded)` | `u8(0) u32(mem) u32(iters) u32(par) bytes(salt) bytes(evaluated)` |
+| `POST /auth/register/finish` | `str(username) bytes(pubkey) bytes(proof)` | `u8(status)`, `0` ok, `1` username taken |
+| `POST /auth/login/start` | `str(username) bytes(blinded)` | `bytes(cid) str(aud) u32(mem) u32(iters) u32(par) bytes(salt) bytes(nonce) u64(iat) u64(exp) bytes(evaluated)` |
+| `POST /auth/login/finish` | `bytes(cid) bytes(ct) bytes(sig)` | `u8(0) bytes(sessionToken) bytes(serverConfirm)` + `Set-Cookie`, or `u8(≠0)` on failure |
+| `GET /auth/me` *(`@auth`)* | (none) | `bytes(toilUserId) str(username)` |
+| `POST /auth/logout` *(`@auth`)* | (none) | `200` + cookie-clearing `Set-Cookie` |
+
+Rate limiting: every register/login POST carries `@ratelimit(SlidingWindow, 5, 60)` (5 requests / 60s per
+client) out of the box, so brute-force is throttled before it reaches the crypto.
+
+## 6. Under `toiljs dev`
+
+Everything runs locally with **zero setup**: the dev server emulates the ToilDB account/challenge storage
+and the ML-DSA/ML-KEM/OPRF host functions in process, and the auth secrets fall back to insecure DEV
+values. Register and login span requests (the accounts persist for the dev session). You'll see a warning
+that `AUTH_SESSION_SECRET` is unset, that's expected in dev; set it before you deploy (see
+[Configuration](./configuration.md)).
+
+## 7. `Server.REST.auth.*`
+
+Because the controller is a normal `@rest('auth')` class, a typed `Server.REST.auth.*` fetch client is
+generated for free (`me`, `logout`, and the register/login methods). Use it for `/me` and `/logout`; but
+**drive register/login through `toiljs/client` `Auth`**, not the generated client, only the `Auth` helper
+runs the required browser-side OPRF/Argon2id/ML-DSA/ML-KEM crypto.

package/docs/backend/README.md

@@ -0,0 +1,143 @@
+# Backend overview
+
+Your backend is TypeScript that toilscript compiles into a small, sandboxed WebAssembly program, which runs on the Dacely edge and answers every request.
+
+## What the backend is
+
+In a toiljs project, everything under `server/` is your backend. You write it in TypeScript, the same language as your frontend. But it does not run in a browser and it does not run in Node. Instead, a compiler called **toilscript** turns it into **WebAssembly** (often shortened to **WASM**): a compact, fast, portable binary format that many kinds of servers can run safely.
+
+That compiled program is then deployed to the **Dacely edge**. Two pieces of jargon to unpack there:
+
+- **The edge** means a fleet of servers spread across many cities around the world. When a user makes a request, it is served by the edge node physically closest to them. Close means fast: less distance for the data to travel, so lower latency. You do not pick a region or manage servers; your one compiled backend runs everywhere at once.
+- **Sandboxed** means your WASM program runs inside a locked box. It cannot open files, reach the operating system, or make raw network connections on its own. The only way it can touch the outside world is through a small, fixed set of **host functions** that toiljs provides (read the request, build a response, query the database, send an email, and so on). Every one of those calls is metered and bounded, so a buggy or hostile backend cannot crash the node or read another app's data. This is what makes it safe to run thousands of different apps on the same shared edge.
+
+You never call the WASM boundary by hand. You write normal TypeScript classes and functions, tag them with decorators (like `@rest` or `@service`), and the compiler wires everything up.
+
+> New to decorators? A **decorator** is the `@name` you write just above a class or method. It attaches meaning to that code without changing what the code does line by line. toiljs uses decorators to say "this class is an HTTP controller" or "this method is callable from the browser." See [Decorators](../concepts/decorators.md).
+
+## The request lifecycle
+
+Here is what happens, end to end, when a browser talks to your backend.
+
+```mermaid
+flowchart TD
+    U["User's browser"] -->|"HTTP request"| E["Nearest Dacely edge node"]
+    E --> S{"Static file<br/>for this path?"}
+    S -->|"Yes (GET/HEAD)"| F["Serve the file<br/>(HTML, JS, images)"]
+    S -->|"No"| W["Your compiled backend<br/>(server.wasm handle)"]
+    W --> H["Your handler runs<br/>and returns a Response"]
+    H --> E
+    F --> E
+    E -->|"HTTP response"| U
+```
+
+Step by step:
+
+1. The request lands on the closest edge node.
+2. The edge first checks whether the path is a **static file** it can serve directly (your built frontend: HTML pages, JavaScript bundles, images). If so, it serves the file and never wakes your code. This is fast and free.
+3. Otherwise the edge hands the request to your compiled backend by calling its single WASM export, `handle`.
+4. Inside, toiljs decodes the raw bytes into a friendly [`Request`](./rest.md#the-request-object) object and calls your handler's `handle(req)` method.
+5. Your handler returns a [`Response`](./rest.md#building-a-response). toiljs encodes it back into bytes.
+6. The edge sends that response to the browser.
+
+The key mental model: your backend is a pure function of the request. Bytes in, bytes out, one request at a time.
+
+## Stateless by default
+
+A **fresh copy** of your handler serves each request. Any fields you set on a controller do not survive to the next request, and the request might even be served by a different edge node on the other side of the world. This is called being **stateless**.
+
+That is a feature, not a limitation: it is what lets your backend scale to the whole planet with no coordination. When you need data to persist between requests (a user account, a counter, a list of posts), you store it in the built-in global database, **ToilDB**. See [the database section](../database/README.md).
+
+```mermaid
+flowchart LR
+    R1["Request 1"] --> B1["Handler copy A<br/>(fields reset)"]
+    R2["Request 2"] --> B2["Handler copy B<br/>(fields reset)"]
+    B1 --> DB[("ToilDB<br/>(shared, persistent)")]
+    B2 --> DB
+```
+
+## The three surfaces
+
+Your backend can expose three different kinds of endpoint. Each is opted into with a decorator, and each has its own page:
+
+| Surface | Decorator | What it is | When to use it |
+| --- | --- | --- | --- |
+| **HTTP REST** | `@rest` + `@get`/`@post`/... | Plain HTTP routes with paths, methods, and status codes. | A public API, webhooks, anything a browser, `curl`, or a third party calls directly. See [REST](./rest.md). |
+| **Typed RPC** | `@service` / `@remote` | Server functions your own frontend calls like local async functions, fully type-checked end to end. | Talking from your own React app to your own backend. See [RPC](./rpc.md). |
+| **Realtime** | `@stream` | A long-lived connection where the server keeps state per connected client. | Chat, live cursors, notifications, anything push-based. See [Realtime](../realtime/README.md). |
+
+REST and RPC are the everyday tools. Most apps use both: REST for anything the outside world calls, RPC for your own frontend. They are not exclusive; you can use all three in one project.
+
+All of these share the same building block for their data: **`@data` classes**, which are typed structs that travel safely between the browser and your WASM backend. See [Data types](./data.md).
+
+## Where your handler lives
+
+REST, RPC, and streams self-register: you tag a class and the compiler adds it to the right dispatcher. A tiny amount of glue lives in `server/main.ts`, which imports your route files and names your top-level handler class. In a typical project you rarely touch `main.ts`; you add route and service files and they are discovered automatically.
+
+Your handler class extends `ToilHandler` and overrides `handle`. The common pattern is to try the REST dispatcher first, then fall back to your own logic:
+
+```ts
+import { Method, Request, Response, Rest, ToilHandler } from 'toiljs/server/runtime';
+
+export class AppHandler extends ToilHandler {
+    public handle(req: Request): Response {
+        // Try every @rest controller. Returns the first match, or null.
+        const hit = Rest.dispatch(req);
+        if (hit != null) return hit;
+
+        // Your own hand-written endpoints can go here.
+        if (req.path == '/health') return Response.text('ok\n');
+
+        // "I have no answer for this path": let the edge serve it (a static
+        // file, the client app) instead of returning a hard 404.
+        return Response.unhandled();
+    }
+}
+```
+
+RPC calls (to the reserved path `/__toil_rpc`) are handled by the framework before your `handle` runs, so you do not dispatch them yourself.
+
+### Per-request hooks: `onRequestStarted` and `onRequestCompleted`
+
+`ToilHandler` gives you two optional hooks that run around every request, so you can add cross-cutting logic (logging, timing, metrics) once instead of repeating it in every route, and without re-implementing `handle`:
+
+- **`onRequestStarted(req: Request): void`** runs just **before** `handle(req)`. Override it for per-request setup: start a timer, log the incoming method and path, read a header you need everywhere.
+- **`onRequestCompleted(req: Request, resp: Response): void`** runs just **after** `handle` returns, and it is handed the `Response` that is about to go out. It **also runs when your handler throws**, after the runtime has turned that throw into a `500`, so it is the right place for teardown you always want to happen (record the outcome, stop the timer, emit a metric).
+
+Both are empty by default, so override only the one you need. The framework calls them around every request; you never call them yourself.
+
+```ts
+import { Request, Response, Rest, ToilHandler } from 'toiljs/server/runtime';
+
+export class AppHandler extends ToilHandler {
+    public handle(req: Request): Response {
+        const hit = Rest.dispatch(req);
+        return hit != null ? hit : Response.unhandled();
+    }
+
+    // Runs before every handle().
+    public onRequestStarted(req: Request): void {
+        // per-request setup, e.g. note the path you are about to serve
+    }
+
+    // Runs after every handle(), including after a throw became a 500.
+    public onRequestCompleted(req: Request, resp: Response): void {
+        // always-run teardown: record the outcome, stop a timer, emit a metric
+    }
+}
+```
+
+If your project is REST-only, you do not even need a custom handler; toiljs ships a ready-made one. See [REST](./rest.md#dispatch-and-the-404-fallback).
+
+## Compute tiers
+
+The request/response backend described here is the default and most common tier, called **L1**. toiljs also has higher tiers for long-lived connections (streams) and scheduled background work (daemons), each compiled into its own WASM artifact from the same project. You opt into a tier just by adding its entry file and surface decorator. For the full picture, see [Compute tiers](../concepts/tiers.md).
+
+## Related
+
+- [HTTP routes (`@rest`)](./rest.md): paths, methods, params, and responses.
+- [Typed RPC (`@service`/`@remote`)](./rpc.md): calling the server from your frontend with end-to-end types.
+- [Data types (`@data`)](./data.md): the serializable structs everything uses.
+- [The database (ToilDB)](../database/README.md): where persistent state lives.
+- [Compute tiers](../concepts/tiers.md): L1 request, L2/L3 stream, L4 daemon.
+- [Realtime streams](../realtime/README.md): the `@stream` surface.