esoul-sdk 0.7.0 → 0.16.0

package/README.md

@@ -1,183 +1,605 @@
 # esoul-sdk
 
-Build a **full product** on ExternalSoul — not a widget. An app you write with this SDK gets its
-own database tables, its own server, its own background jobs, its own realtime, its own words for
-the people who use it, and the same agent tools that chat, voice and MCP already call. Once it
-ships it is indistinguishable from the platform's own apps.
+esoul-sdk is a toolkit for building applications on [ExternalSoul](https://externalsoul.com). An
+application written with it is compiled into the platform and runs as a native app: it has its own
+database tables, its own server endpoints, its own background tasks, its own realtime channel, its
+own vocabulary of roles, and a set of tools that agents call from chat, voice, the agent builder
+and MCP. Access control is declared in the application's manifest and enforced by the platform at
+every entry point; application code does not check permissions itself.
+
+The package provides the types, the manifest schema and validator, the rule compiler, the request
+helpers and an in-memory test harness. Inside ExternalSoul the same import names resolve to the
+platform's implementations.
+
+## Useful links
+
+- [Getting started](docs/01-getting-started.md)
+- [Manifest reference](docs/02-manifest.md)
+- [Documentation index](#documentation)
+- `llms.txt` and `llms-full.txt` — the documentation in one file, for coding models
+
+## Features
+
+Data model
+
+- Tables are declared in the manifest (`db`) and created on install; migrations are additive-only
+- Every query is scoped to the application instance and filtered for the caller by compiled rules
+- Rows can be owned by their creator, by the workspace, or by a user across instances
+- Fields can be sealed (encrypted at rest, readable only by the row's owner)
+- Index kinds for equality, list containment and case-insensitive text search, with cursor paging
+- Application state on the workspace timeline as a fold of typed events (replayable, scrubbable)
+
+Access control
+
+- A `viewer` on every entry point: owner, member, visitor, anonymous, agent, or the app itself
+- Roles as the application's own words (`customer`, `staff`, …), mapped from the platform's kinds
+- Per-surface access levels: `write`, `read`, `public`, `token`, or a list of roles
+- Rules scoped by a person's typed attributes — a customer linked to a number sees only rows carrying it
+- Roles composed by the workspace owner at runtime, inside an envelope the manifest declares
+- Refusals distinguish `login-required` from `forbidden`, so a UI can show a sign-in wall
+
+Server
+
+- Ops: server-side functions with a declared input schema, shared with the derived tools
+- Routes: HTTP endpoints mounted per instance, including server-sent event streams
+- Token routes: endpoints a machine reaches with a bearer token the application minted
+- Background tasks on a durable executor, with steps, retries and concurrency limits
+- Webhooks, OAuth connections held by the platform, workspace files and file providers
+- Access to other applications through bindings (typed contracts) and workspace tools
+
+Realtime
+
+- Per-instance channels with topics; each topic declares its audience (everyone, one person, one role)
+- Tokens are issued per channel, so a subscriber never receives another audience's messages
+
+Tools
+
+- One declaration per op is used by the server, the derived tool and the UI
+- Tools run on every surface: typed chat, voice, agent builder, MCP, and the browser
+
+Development
+
+- The Forge: a cloud workbench with a live preview, a persona switcher (*view as*), the application's
+  tools callable before installation, tests and the full checks
+- An in-memory database built from the manifest, with the same compiled rules production uses
+- A validator for the package folder (`esoul-app validate`)
+
+Deployment
+
+- Installed from the application's own GitHub repository at a commit
+- Additive migrations applied on install; the install card lists every public surface
+
+## Installation
 
 ```bash
 npm install --save-dev esoul-sdk
 ```
 
-Inside ExternalSoul the package name resolves to the platform's real implementations. Outside it
-(your editor, your tests) it gives you the types, the manifest validator, the rule compiler and
-the test helpers; the server functions throw "host only" if called, because they run in the
-platform.
+Outside ExternalSoul the package gives you types, the manifest validator, the rule compiler and the
+test helpers. Functions that need the platform (`pluginDb`, `viewerProfile`, `mintRouteToken`, …)
+throw `host only` when called outside it; they are exercised through the test harness or in a
+Forge workbench.
 
----
+Requirements: Node.js 20 or newer, TypeScript 5, React 19 for the UI entry point.
 
-## What you can build with it
+## Concepts
 
-Take a help desk. Strangers may read the published articles. Someone signed in may raise a ticket
-and see their own. The people on duty see every ticket. The owner decides who is on duty. A
-confirmation must go out exactly once, even if the mail service is down for a minute. That is
-every hard thing at once, and it is the shape most real products have.
+**Application.** A folder containing a manifest (`plugin.json`), a schema (`app.tsx`), a UI, and
+optionally a server module (`server.ts`) and tests. The folder name is the application id. The
+application type is `plugin_` followed by the id with underscores.
 
-An app like that, written with this SDK, contains **no access checks at all**. They are
-declarations, and the platform enforces them at the seam.
+**Manifest.** `plugin.json` declares everything the platform enforces: tables, roles, access levels
+of ops, routes and tasks, realtime topics, bindings, connections and dependencies. It is validated
+against `schemas/plugin.schema.json`.
 
-| You want | You declare | You never write |
-|---|---|---|
-| Your own tables | `db` in the manifest | migrations, a Prisma schema, an ORM |
-| Per-person data | `owner: "creator"` + `rules` | a `WHERE userId = …` anywhere |
-| A public page | `"access": "public"` on an op | an auth check in the handler |
-| A sign-in wall | `"requires": "account"` | the difference between "sign in" and "never" |
-| Words for your people | `roles` | a permissions table |
-| One person's live updates | `"audience": "viewer"` on a topic | a filter on arrival |
-| Durable follow-up work | a `task` | a queue, retries, idempotency plumbing |
-| Another app's help | `uses` + a contract | an integration |
-
----
-
-## The seven capabilities
-
-**1. Viewer — every seam knows who is calling.** Ops, routes, tasks, tools and the UI all receive
-a `viewer`: the owner, a member, a signed-in visitor, an anonymous one, an agent acting for
-someone, or your app's own server code. The platform resolves it; an app cannot supply, widen or
-forge one. `useViewer()` in the UI, `ctx.viewer` on the server. When you need the person's name
-or email — to address them, or to fill a form in — `viewerProfile(ctx.viewer)` reads the esoul
-account of the CALLER and nobody else, so people use your app with the account they already have.
-
-**2. Roles — your app's own vocabulary.** Declare `roles: { vocabulary: ["requester", "agent",
-"supervisor"] }` and a mapping from the platform's kinds. The workspace owner can then assign a role
-per app, per person, so an organisation sees the pages their job needs. A role is a WORD, never a
-permission: it decides what your rules mean, not what the platform allows.
-
-**3. Access levels — nothing opens by default.** Every op, route and task is `write` unless you
-say otherwise. `read` admits a read-only collaborator; `public` admits someone on a share link
-who has no workspace access at all; `public` + `requires: "account"` answers `login-required`
-instead of `forbidden`, which is the difference between showing a sign-in wall and showing an
-error. Every public door is listed on the install card before anyone installs your app.
-
-**4. The database — tables you declare, scoped by the platform.** A `db` block becomes real
-tables, generated typings, and rules compiled once and enforced everywhere. `pluginDb(ctx)` hands
-you a typed client whose every query is already scoped to this instance and filtered for this
-caller: one person's `findMany` returns their own rows, someone on duty gets the whole queue, and
-neither had to ask.
-Fields can be `sealed` (encrypted at rest, readable only by their owner). Migrations are
-additive-only and applied on install; a change that would drop or retype a column is refused with
-the column named.
-
-**5. Realtime with an audience.** A topic declares who hears it: everyone on the app, only the
-person it concerns, or only a role. The platform mints a token per channel, so one person is never
-handed another's messages — not filtered out on arrival, never issued. Aiming a message is a
-separate permission from hearing one.
-
-**6. Background tasks.** Durable work on the platform's own scheduler: retried, replay-safe,
-steps on the timeline. This is where the slow and failure-prone things go — sending the
-confirmation, calling somebody else's API — so the request that changed the record stays fast and
-honest.
-
-**7. Bindings.** Declare a SLOT and a CONTRACT (`uses: { rooms: { contract: "rooms/v1" } }`) and
-the owner picks which app fills it. The platform checks at bind time that the provider really has
-every tool, event and table the contract names. Your app reaches it through `ctx.apps.rooms` —
-the contract's tools only, with the caller's identity travelling along.
-
----
-
-## Where to start
-
-- **Build it in the Forge, not on your laptop.** Open a Forge board in your workspace and ask it
-  to open a workbench for your app: a cloud machine with the platform on it, a live preview in
-  your frame, your app's tools callable before it is installed, and **VIEW AS** — the switcher
-  that shows you your app as each kind of person who will use it, including the stranger. Read
-  [docs/01-getting-started.md](docs/01-getting-started.md).
-- **The contract, one page per part:**
-  1. [Getting started](docs/01-getting-started.md) — the loop, the package layout
-  2. [The manifest](docs/02-manifest.md) — `plugin.json`, every field
-  3. [Events and state](docs/03-events-and-state.md) — the heart: dataCreator, processor, replay
-  4. [Tools](docs/04-tools.md) — what agents call, on every surface
-  5. [The UI](docs/05-ui.md) — React, hooks, theme, responsive rules
-  6. [The server half](docs/06-server.md) — ops, routes, webhooks, calling other apps
-  7. [Background tasks](docs/07-background-tasks.md) — durable work, polling, the replay model
-  8. [Connections and OAuth](docs/08-connections.md) — tokens the platform holds for you
-  9. [Files](docs/09-files.md) — workspace files, Drive, your own provider
-  10. [Testing](docs/10-testing.md) — the fold contract, and your rules, as tests
-  11. [Shipping](docs/11-shipping.md) — submit, review, release, install; the import wall
-  12. [Rules and failures](docs/12-rules.md) — every rule with the failure that earned it
-  13. [People and access](docs/13-people-and-access.md) — viewer, roles, levels, the sign-in wall
-  14. [Your own tables](docs/14-database.md) — `db`, rules, scopes, sealed fields, migrations
-  15. [Realtime](docs/15-realtime.md) — topics, audiences, who may address whom
-  16. [Bindings](docs/16-bindings.md) — slots, contracts, reaching another app
-- **For a coding model:** `llms.txt` (short) and `llms-full.txt` (the whole contract in one file).
-
-## The one rule that explains the others
-
-**Events are the truth.** Your app's fold is its events re-run on every replay, scrub and sync. So
-a reducer is pure and idempotent, ids and timestamps are minted in the `dataCreator` (never in a
-reducer), whole-replace events carry a collapse key, and a state description never claims
-something it could not read.
-
-The second rule, for everything that is not in the fold: **the platform decides who sees what.**
-Your rules are declarations the platform enforces at the seam. An app that checks access in its
-own handler has two answers to one question, and one of them will be wrong.
-
-## What is in the package
-
-| Entry | What it gives you |
+**Events and state.** Application state is the result of folding typed events from the workspace
+timeline. Each event has a `dataCreator` (mints ids and timestamps) and a `processor` (a pure
+reducer). The same events are dispatched by the UI, by tools and by tasks.
+
+**Viewer.** The identity of the caller, resolved by the platform for every op, route, task, tool and
+UI render. A viewer has a kind, an account id, a set of ids it has acted under, the application's
+role word for it, and whether it may write.
+
+**Op.** A server-side function of the application, called from the UI or from a tool. Its input
+schema is declared once with `defineOps`; the server parses with `handleOp` and the tool is derived
+with `opTool`.
+
+**Route.** An HTTP endpoint the application mounts at `/api/plugins/<id>/route/<name>`.
+
+**Task.** A background job on the platform's durable executor. A task is kicked by an event, runs
+in steps that are memoised across retries, and may dispatch events and publish realtime messages.
+
+**Topic.** A named message stream on the application's realtime channel. Its audience is declared in
+the manifest.
+
+**Binding.** A slot the application declares (`uses`) and the workspace owner fills with another
+application that provides the named contract.
+
+## Package entry points
+
+| Import | Contents |
 |---|---|
-| `esoul-sdk` | `ApplicationSchema`, `EventDefinition`, `EventTypes`, `ApplicationIdentifier`, `definePluginChannel`, `defineBindingEvent`, `checkBinding`, `subscriptionsFor`, `resolveAppRole`, `incompleteStateNotice`, `deterministicReducerId`, `timingSafeEqual`, `nanoid`, `callPluginOp`, `kickPluginTask`, `pluginRouteUrl`, the manifest schema |
-| `esoul-sdk/react` | `useViewer`, `useSignInWall`, `useAppCanEdit`, `usePluginEventDispatch`, `usePluginRealtime`, `useWorkspaceTools`, file hooks |
-| `esoul-sdk/server` | `pluginDb`, `viewerProfile`, `PluginServerModule` (ops, routes, webhooks), `sseStream`, `readAppState`, `callWorkspaceTool`, `emitPluginAppEvent`, `getPluginConnectionCredentials`, file provider types |
-| `esoul-sdk/testing` | `memoryDb`, `fakeViewer`, `runOp`, `fakeApps`, `capture`, `startMockOAuth` — your rules run against the client the platform compiles from your own manifest |
-| `esoul-app validate <dir>` | validates a package folder against the manifest schema |
+| `esoul-sdk` | Schema and event types, `defineOps`, `handleOp`, `opTool`, `definePluginChannel`, `defineBindingEvent`, `checkBinding`, `resolveAppRole`, `chartSvg`, `incompleteStateNotice`, `deterministicReducerId`, `callPluginOp`, `kickPluginTask`, `pluginRouteUrl`, `nanoid`, the manifest schema, `compileEnvelope`, `compileCustomRole` |
+| `esoul-sdk/server` | `pluginDb`, `viewerProfile`, `sseStream`, `mintRouteToken`, `readAppState`, `callWorkspaceTool`, `computer`, `emitPluginAppEvent`, `generateAppImage`, `renderChartImage`, `setAppRole`, `listAppRoles`, `defineAppRole`, `removeAppRole`, `getPluginConnectionCredentials`, `pluginFiles`, and the context types (`PluginOpContext`, `PluginRouteContext`, `PluginViewer`, `PluginServerModule`) |
+| `esoul-sdk/react` | `useViewer`, `useSignInWall`, `useAppCanEdit`, `usePluginEventDispatch`, `usePluginRealtime`, `useWorkspaceTools`, `usePluginWorkspaceFiles`, `usePluginFileUpload`, `useFileSources`, `useFileSourceEntries`; the types `PluginViewerPublic`, `SignInWall` |
+| `esoul-sdk/testing` | `memoryDb`, `fakeViewer`, `runOp`, `fakeApps`, `capture`, `startMockOAuth` |
+| `esoul-sdk/schemas/plugin.schema.json` | The JSON Schema of the manifest |
+| `esoul-app validate <dir>` | Command-line validator for a package folder |
+
+## Getting started
+
+The recommended way to build an application is the Forge, a workbench inside ExternalSoul. It does
+not require a checkout of the platform.
+
+1. Add a **Forge** board to a workspace.
+2. Ask the assistant to open a workbench for your application. The platform starts a cloud machine
+   with the platform's source, scaffolds the application folder, and shows a live preview on the
+   board.
+3. Edit files with the board's tools (`write_app_file`, `edit_app_file`). Each change reloads the
+   preview and reports its health.
+4. Inspect the result: `look_at_app` renders the application on desktop and phone, light and dark,
+   as any persona (`owner`, `member`, `visitor-a`, `anonymous`, or a composed role such as
+   `role:packer`). `call_app_tool` runs the application's tools before installation. `test_app`
+   runs the tests; `check_app` runs the full gate.
+5. Push the application to its own GitHub repository from the board, and install it from
+   Settings → Apps in any workspace.
+
+From your own editor, `npm install --save-dev esoul-sdk` gives you the same types and the
+validator, and lets you run tests locally with `esoul-sdk/testing`.
+
+### Package layout
+
+```
+<id>/
+  plugin.json            the manifest
+  ops.ts                 op input schemas (defineOps)
+  app.tsx                the schema: events, state, tools, state description
+  ui/<id>-ui.tsx         the React UI ("use client")
+  server.ts              ops, routes, webhooks
+  <id>.test.ts           tests
+  .esoul/                generated: db.d.ts, rules.json, migration.sql
+```
+
+## Example: a shop
+
+The reference application is a shop. Anyone with the link may browse the catalogue. A signed-in
+customer places orders and sees only their own. Staff see every order of the shop. The owner sets
+prices and decides who is staff, and may compose narrower roles at runtime — for example a
+*packer* who sees only orders being prepared, never the customer's note, and may only mark them
+shipped.
+
+### Manifest
+
+```json
+{
+  "manifestVersion": 1,
+  "id": "shop",
+  "name": "Shop",
+  "version": "1.0.0",
+  "applicationType": "plugin_shop",
+  "entry": "app",
+  "roles": {
+    "vocabulary": ["customer", "staff", "owner"],
+    "default": {
+      "owner": "owner",
+      "member-edit": "staff",
+      "member-readonly": "staff",
+      "visitor": "customer",
+      "anonymous": "customer",
+      "agent": "inherit"
+    },
+    "custom": {
+      "models": {
+        "Order": { "where": ["status"], "hide": ["note"], "update": { "transitions": "status" } }
+      },
+      "ops": ["set-order-status"]
+    }
+  },
+  "ops": {
+    "browse": { "access": "public" },
+    "place-order": { "access": "public", "requires": "account" },
+    "list-orders": { "access": "public", "requires": "account" },
+    "set-order-status": { "access": ["staff", "owner"] },
+    "add-product": {}
+  },
+  "routes": {
+    "order-updates": { "access": "read" }
+  },
+  "channel": {
+    "topics": {
+      "catalogue": {},
+      "order-status": { "audience": "viewer", "mayAddress": ["staff", "owner"] },
+      "new-order": { "audience": "role:staff", "mayAddress": ["customer", "staff", "owner"] }
+    }
+  },
+  "db": {
+    "Product": {
+      "scope": "instance",
+      "fields": { "name": "string", "priceCents": "int", "active": "boolean=true", "tags": "string[]" },
+      "indexes": [["active", "createdAt"], { "fields": ["tags"], "kind": "contains" }, { "fields": ["name"], "kind": "text" }],
+      "rules": { "read": "anyone", "write": ["owner"] }
+    },
+    "Order": {
+      "scope": "instance",
+      "owner": "creator",
+      "fields": { "status": "string=new", "totalCents": "int", "lines": "json", "shipTo": "json", "note": "text?" },
+      "indexes": [["status"], ["createdAt"]],
+      "rules": {
+        "read": ["creator", "staff", "owner"],
+        "create": { "roles": ["customer", "staff", "owner"], "requires": "account" },
+        "update": { "roles": ["staff", "owner"], "creatorMay": ["note"] },
+        "delete": ["owner"]
+      }
+    },
+    "Address": {
+      "scope": "user",
+      "fields": { "label": "string", "lines": "json" },
+      "sealed": ["lines"]
+    }
+  }
+}
+```
+
+An op with no `access` is `write`: only the owner and edit members may call it. `"access":
+"public"` admits anyone who can reach the application, including through a share link;
+`"requires": "account"` makes an anonymous call fail with `login-required` instead of `forbidden`.
+
+### Op inputs
+
+```ts
+// ops.ts
+import { z } from "zod";
+import { defineOps } from "esoul-sdk";
+
+export const ORDER_STATUSES = ["new", "preparing", "shipped", "fulfilled", "refunded"] as const;
+
+export const ops = defineOps({
+  browse: z.object({}),
+  "add-product": z.object({ name: z.string().min(1).max(80), priceCents: z.number().int().min(0) }),
+  "place-order": z.object({
+    lines: z.array(z.object({ productId: z.string().min(1), qty: z.number().int().min(1).max(99) })).min(1),
+    shipTo: z.object({ name: z.string().min(1), street: z.string().min(1), city: z.string().min(1) }),
+    note: z.string().max(280).optional(),
+  }),
+  "list-orders": z.object({}),
+  "set-order-status": z.object({
+    orderId: z.string().min(1).describe("The order id, as list_orders shows it"),
+    status: z.enum(ORDER_STATUSES),
+  }),
+});
+```
+
+### Server
+
+```ts
+// server.ts
+import "server-only";
+import { handleOp } from "esoul-sdk";
+import { pluginDb, type PluginOpContext, type PluginServerModule } from "esoul-sdk/server";
+import type { ShopDb } from "./.esoul/db";
+import { ops } from "./ops";
+
+const db = (ctx: PluginOpContext) => pluginDb<ShopDb>(ctx);
+
+export const pluginServer: PluginServerModule = {
+  ops: {
+    browse: handleOp(ops, "browse", async (ctx) => {
+      const d = await db(ctx);
+      return d.product.findMany({ where: { active: true }, orderBy: { createdAt: "asc" }, take: 200 });
+    }),
+
+    "add-product": handleOp(ops, "add-product", async (ctx, input) => {
+      const d = await db(ctx);
+      return d.product.create({ data: { name: input.name, priceCents: input.priceCents, tags: [] } });
+    }),
+
+    "place-order": handleOp(ops, "place-order", async (ctx, input) => {
+      const d = await db(ctx);
+      // Price on the server, from the catalogue.
+      const products = await d.product.findMany({ where: { id: { in: input.lines.map((l) => l.productId) } } });
+      const totalCents = input.lines.reduce((sum, l) => sum + l.qty * (products.find((p) => p.id === l.productId)?.priceCents ?? 0), 0);
+      const order = await d.order.create({ data: { status: "new", totalCents, lines: input.lines, shipTo: input.shipTo, note: input.note } });
+      await ctx.notify("new-order", { orderId: order.id });
+      return { orderId: order.id, totalCents };
+    }),
+
+    "list-orders": handleOp(ops, "list-orders", async (ctx) => {
+      const d = await db(ctx);
+      return d.order.findMany({ orderBy: { createdAt: "desc" } });
+    }),
+
+    "set-order-status": handleOp(ops, "set-order-status", async (ctx, input) => {
+      const d = await db(ctx);
+      const order = await d.order.update({ where: { id: input.orderId }, data: { status: input.status } });
+      await ctx.notify("order-status", { orderId: order.id, status: order.status }, { to: { viewerIds: [order.ownerId] } });
+      return order;
+    }),
+  },
+};
+```
 
-## Testing your app
+There is no access check in this file. `pluginDb(ctx)` returns a client scoped to the instance and
+filtered for `ctx.viewer`. A customer's `order.findMany` returns their own rows; a staff member's
+returns every row of the shop. A customer calling `set-order-status` is refused before the handler
+runs, because the manifest opens it to `staff` and `owner` only. A packer — a composed role scoped
+to `status: ["preparing"]` — receives only those rows, with `note` blanked, and an update from
+`preparing` to anything other than `shipped` is refused with the allowed move named.
 
-The helpers run your REAL server code against an in-memory database built from your own
-`plugin.json`, with the same compiled rules the production client uses. What passes here is what
-the real database will do.
+### Tools
 
 ```ts
+// app.tsx (excerpt)
+import { opTool } from "esoul-sdk";
+import { ops } from "./ops";
+
+toolkitCreator: (identifier) => {
+  const base = identifier.instanceName.replace(/[^a-zA-Z0-9]/g, "_");
+  const cfg = { pluginId: "shop", nodeId: identifier.nodeId };
+  return {
+    [`browse_${base}`]: opTool(ops, "browse", {
+      ...cfg,
+      description: `List the products of "${identifier.instanceName}".`,
+      readOnly: true,
+      publicSafe: true,
+      say: (rows) => ({ text: rows.map((p) => `${p.name} — ${(p.priceCents / 100).toFixed(2)} (id ${p.id})`).join("\n") || "No products." }),
+    }),
+    [`set_order_status_${base}`]: opTool(ops, "set-order-status", {
+      ...cfg,
+      description: `Move an order of "${identifier.instanceName}" to a new status.`,
+      say: (o) => ({ text: `Order ${o.id} is now ${o.status}.` }),
+    }),
+  };
+},
+```
+
+The tool's parameters are the op's input schema; a field the op does not declare is refused with
+the field named. The tool runs as the person who invoked it, on every surface.
+
+### UI
+
+```tsx
+// ui/shop-ui.tsx (excerpt)
+"use client";
+import { callPluginOp } from "esoul-sdk";
+import { useSignInWall, useViewer } from "esoul-sdk/react";
+
+export function ShopUi({ nodeId }: { nodeId: string }) {
+  const viewer = useViewer();           // { kind, userId, role, canEdit, signedIn, customRole, can }
+  const wall = useSignInWall();
+  const order = async (lines, shipTo) => {
+    try {
+      return await callPluginOp("shop", "place-order", nodeId, { lines, shipTo });
+    } catch (err) {
+      wall.raise(err);                   // shows the sign-in wall on `login-required`, rethrows otherwise
+    }
+  };
+  // viewer.can.models.Order.moves tells a packer's screen which buttons to show;
+  // the server enforces the same rule regardless.
+  …
+}
+```
+
+### Test
+
+```ts
+// shop.test.ts
 import { fakeViewer, memoryDb, runOp } from "esoul-sdk/testing";
 import manifest from "./plugin.json";
 import { pluginServer } from "./server";
 
-const ada = fakeViewer("visitor", { userId: "u_ada", role: "requester" });
-const lin = fakeViewer("visitor", { userId: "u_lin", role: "requester" });
+const owner = fakeViewer("owner");
+const ada = fakeViewer("visitor", { userId: "u_ada", role: "customer" });
+const lin = fakeViewer("visitor", { userId: "u_lin", role: "customer" });
 
-it("does not let the OTHER one see it", async () => {
+it("a customer sees only their own orders", async () => {
   const db = memoryDb(manifest);
-  await runOp(pluginServer, "raise-ticket", { viewer: ada, args: TICKET, db: db.as(ada) });
-  const { result } = await runOp(pluginServer, "my-tickets", { viewer: lin, args: {}, db: db.as(lin) });
+  const { result: p } = await runOp(pluginServer, "add-product", { viewer: owner, args: { name: "Tea", priceCents: 350 }, db: db.as(owner) });
+  const shipTo = { name: "Ada", street: "Main 1", city: "Brno" };
+  await runOp(pluginServer, "place-order", { viewer: ada, args: { lines: [{ productId: p.id, qty: 1 }], shipTo }, db: db.as(ada) });
+  const { result } = await runOp(pluginServer, "list-orders", { viewer: lin, args: {}, db: db.as(lin) });
   expect(result).toEqual([]);
 });
+
+it("an anonymous caller is asked to sign in", async () => {
+  const db = memoryDb(manifest);
+  const anon = fakeViewer("anonymous");
+  await expect(runOp(pluginServer, "list-orders", { viewer: anon, args: {}, db: db.as(anon) })).rejects.toMatchObject({ code: "login-required" });
+});
+```
+
+`memoryDb(manifest)` builds an in-memory database from the manifest with the same compiled rules the
+platform applies to the real database. `runOp` calls the op with a platform-shaped context and
+records what it notified and emitted.
+
+## API reference
+
+### Manifest (`plugin.json`)
+
+| Field | Type | Description |
+|---|---|---|
+| `manifestVersion` | `1` | Schema version. |
+| `id` | string | Application id: lower-case letters, digits, hyphens. Equals the folder name. |
+| `name`, `description`, `version`, `icon` | string | Shown in the store and the install card. `version` is semver. |
+| `applicationType` | string | `plugin_` + id with underscores. Never changes after release. |
+| `entry` | string | The schema module (`app`). |
+| `roles.vocabulary` | string[] | The application's role words. |
+| `roles.default` | object | Mapping from platform kinds (`owner`, `member-edit`, `member-readonly`, `visitor`, `anonymous`, `agent`) to role words. `"inherit"` for `agent` uses the role of the person it acts for. |
+| `roles.describe` | object | One line per role, for the install card. |
+| `roles.attributes` | object | Name → `string`, `int` or `boolean`: what a grant may say about a person. Typed when granted; carried on `viewer.attrs`; read by scoped rules as `viewer.<name>`. |
+| `roles.custom` | object | The envelope for roles the owner composes: per model, which indexed fields may scope (`where`), which fields may be hidden (`hide`), which fields and transition field may be updated (`update`); `ops` a composed role may be handed; `attributes` a grant may carry. |
+| `ops` | object | Op name → `{ access, requires }`. `access`: `write` (default), `read`, `public`, or a list of role words. `requires: "account"` turns an anonymous refusal into `login-required`. |
+| `routes` | object | Route name → `{ access }`, additionally `token` for routes reached with a minted bearer token. |
+| `kickableTasks` | object | Tasks the browser may kick, with their access level. |
+| `pollTasks` | array | `[{ task, everyMinutes }]`: tasks the platform kicks on a schedule (5 to 1440 minutes). |
+| `channel.topics` | object | Topic name → `{ audience, mayAddress, description }`. `audience`: `all` (default), `viewer`, or `role:<word>`. `mayAddress`: roles that may publish on the topic. |
+| `db` | object | Model name → table declaration (see below). |
+| `uses` | object | Slot name → `{ contract, label, optional }`. |
+| `provides` | object | Contracts this application provides, with the tools, events and tables each names. |
+| `connections` | array | OAuth or API-key connections the platform holds for the application. |
+| `webhooks` | string[] | Webhook names handled in `server.ts`. |
+| `workspaceTools` | array | Tools of other applications the server side may call. |
+| `platformApi` | object | `{ min, max? }`: the platform contract versions the application accepts. |
+
+npm packages the application needs are declared in a `package.json` in the application folder
+with a `dependencies` field only.
+
+Table declaration:
+
+| Field | Description |
+|---|---|
+| `scope` | `instance` (rows belong to this instance), `workspace`, or `user` (rows follow the account across instances). |
+| `owner` | `creator` marks each row with the creator's ids so rules can name `creator`. |
+| `fields` | Name → type: `string`, `text`, `int`, `float`, `boolean`, `json`, `datetime`, `string[]`, `ref:<Model>`; `?` for optional, `=value` for a default. |
+| `unique`, `indexes` | Field groups. An index entry may be `{ fields, kind }` with `kind` `btree` (default), `contains` (list membership) or `text` (case-insensitive substring). |
+| `sealed` | Fields encrypted at rest and readable only by the row's owner. |
+| `rules` | `read`, `create`, `update`, `delete`: `"anyone"`, a list of principals (`creator`, role words, or a scoped role `{ "role": "customer", "where": { "customer": "viewer.customer" } }`), or `{ roles, requires, creatorMay }`. A scoped role reads only the rows its `where` names, creates inside them, and grants nothing to a holder without the attribute. A model without rules is writable by the owner role only. |
+
+Platform columns on every table: `id`, `workspaceId`, `nodeId`, `ownerId`, `createdBy`,
+`createdAt`, `updatedAt`, `deletedAt`.
+
+### `esoul-sdk`
+
+| Export | Description |
+|---|---|
+| `ApplicationSchema`, `EventDefinition`, `EventTypes`, `ApplicationIdentifier`, `ApplicationPort` | The schema contract: events, `stateCreator`, `toolkitCreator`, `getStateDescription`, `tasks`, `channel`, `reconstructStateFromEventLog`. |
+| `defineOps(inputs)` | Declares the input schema of every op once. |
+| `handleOp(ops, name, fn)` | Wraps an op handler so it receives parsed input; an undeclared or malformed field is refused as `invalid`, naming the field. |
+| `opTool(ops, name, { pluginId, nodeId, description, say, then?, readOnly?, publicSafe? })` | Derives an agent tool from an op. `say` turns the op's result into the tool's text; `readOnly` admits a read-scoped token; `publicSafe` allows use on a public storefront. |
+| `definePluginChannel({ applicationType, topics })` | Declares the realtime channel placed on the schema as `channel`. |
+| `defineBindingEvent`, `checkBinding`, `bindingsOf` | Bindings: the event that records a slot being filled, the check that a provider satisfies a contract, and the current bindings from state. |
+| `resolveAppRole`, `roleKeyFor` | The mapping from platform kinds to role words that the platform itself uses. |
+| `compileEnvelope`, `compileCustomRole`, `CustomRoleError` | The compiler for composed roles, for tests that assert on them. |
+| `coerceAttr`, `coerceAttrs`, `attrProblem`, `ATTR_TYPES` | Attribute typing: what a typed value becomes, and the sentence for one that does not fit. |
+| `chartSvg(spec)` | Renders a time-series chart with panels, markers and gaps to SVG; text is drawn as glyph outlines so it renders on a server without fonts. |
+| `incompleteStateNotice`, `missingStateKeys` | For `getStateDescription`: reports a state that did not load instead of describing it as empty. |
+| `deterministicReducerId(prefix, seed)` | A stable id for use inside a reducer. |
+| `callPluginOp(pluginId, op, nodeId, args?)` | Calls an op from the browser or from a tool. Throws `PluginCallError` carrying `code` (`invalid`, `forbidden`, `login-required`), `detail` and `signInPath`. |
+| `kickPluginTask(args)`, `pluginRouteUrl(...)` | Kicks a task; builds a route's URL. |
+| `timingSafeEqual`, `nanoid` | Utilities. |
+
+### `esoul-sdk/server`
+
+| Export | Description |
+|---|---|
+| `PluginServerModule` | `{ ops, routes, webhooks }` exported as `pluginServer` from `server.ts`. |
+| `PluginOpContext` | `pluginId`, `opName`, `workspaceId`, `nodeId`, `instanceName`, `viewer`, `args`, `origin`, `cloudConnectionId`, `apps` (bound applications), `notify(topic, data, { to? })`, `emit(eventName, eventData)`. |
+| `PluginRouteContext` | The op context plus `request`, `method`, `searchParams`, `canWrite`. Handlers return a `Response`. |
+| `PluginViewer` | `kind`, `userId`, `viewerIds`, `role`, `customRole`, `attrs`, `canWrite`, `shareId`, `agent`. |
+| `pluginDb<T>(ctx)` | A typed client over the application's tables, scoped and filtered for `ctx.viewer`. Per model: `findMany`, `findUnique`, `count`, `aggregate`, `groupBy`, `create`, `createMany`, `update`, `updateMany`, `upsert`, `delete`, `deleteMany`; `$transaction`. Refusals throw with `code` `forbidden`, `login-required` or `invalid`. |
+| `viewerProfile(viewer)` | The caller's own account (`name`, `email`, `picture`), or `null`. |
+| `sseStream(fn)` | A server-sent event response for a route. |
+| `mintRouteToken(ctx, { route, ttlSeconds?, label? })` | Mints a bearer token for a route declared `"access": "token"`. Returns `{ token, url, expiresAt }`; `url` is built on `ctx.origin`. Expiry is the only revocation; 30 days at most. |
+| `readAppState(nodeId)` | The folded state of another application in the workspace. |
+| `callWorkspaceTool({ ... })` | Calls a tool of another application, gated by the manifest's `workspaceTools`. |
+| `computer(ctx, machineNodeId)` | A paired machine: `status`, `run`, `runToEnd`, `claude`, `claudeToEnd`, `readFile`, `fetchJson`, `python`, `runJson`. Waits are capped and the approval gate is reported. |
+| `emitPluginAppEvent(args)` | Appends an event to the application's own timeline from server code. |
+| `generateAppImage(ctx, args)` | Generates an image, scales it to web weight, stores it and returns its URL. Refused in a Forge preview. |
+| `renderChartImage(ctx, { name, svg })` | Renders an SVG to PNG and returns `{ url }` (installed) or `{ base64 }` (preview). |
+| `setAppRole(ctx, { email, role, attrs? })` | The owner gives an account one of the application's role words, or a composed role, with attribute values (`{ customer: "204" }`) validated against `roles.attributes`. Recorded as an event. |
+| `listAppRoles(ctx)` | `{ people, roles, custom, envelope }`. |
+| `defineAppRole(ctx, definition)`, `removeAppRole(ctx, name)` | The owner composes or removes a role inside `roles.custom`. |
+| `getPluginConnectionCredentials(ctx)` | The credentials of the instance's bound connection. |
+| `pluginFiles(ctx)`, `filesForOp(ctx)` | Workspace files and file sources from server code. |
+
+### `esoul-sdk/react`
+
+| Hook | Description |
+|---|---|
+| `useViewer()` | The public viewer: `kind`, `userId`, `role`, `canEdit`, `signedIn`, `attrs`, `customRole`, `can`. `can` lists the ops a composed role may call and, per model, the hidden fields, updatable fields and allowed moves. |
+| `useSignInWall()` | `{ needed, reason, signIn, raise, ask, serverSays }`. `raise(err)` shows the wall when `err.code === "login-required"` and rethrows anything else; `ask(call)` attempts a call and returns `null` (wall raised) on `login-required`; `serverSays` is the server's answer so far (`null`, `"account"`, `"no-account"`). |
+| `useAppCanEdit()` | Whether the current viewer may write. |
+| `usePluginEventDispatch()` | Dispatches one of the application's events from the UI. |
+| `usePluginRealtime({ channel, workspaceId, nodeId, topics, enabled? })` | Subscribes to the instance's channel. Returns `{ data, latestData, error, state }`. |
+| `useWorkspaceTools(identity)` | Lists and calls tools of other applications from the UI, subject to `workspaceTools`. |
+| `usePluginWorkspaceFiles()`, `usePluginFileUpload()`, `useFileSources()`, `useFileSourceEntries()` | Workspace files, upload, and file sources (workspace, Google Drive, providers). |
+
+### `esoul-sdk/testing`
+
+| Export | Description |
+|---|---|
+| `memoryDb(manifest, options?)` | An in-memory database compiled from the manifest. Starts as the application's own code; `.as(viewer)` returns the same rows as another caller. `$rules` exposes the compiled rules. |
+| `fakeViewer(kind, { userId?, viewerIds?, role?, name?, email?, attrs? })` | A viewer of a kind. `name`/`email` are what `viewerProfile` returns for it; `attrs` are what a scoped rule reads. |
+| `runOp(server, opName, { viewer, args?, db?, apps?, notifyFails? })` | Calls `pluginServer.ops[opName]` with a platform-shaped context. Returns `{ result, notified, emitted }`; throws the op's refusals. |
+| `fakeApps(fixtures)` | Bound applications for `ctx.apps`. |
+| `capture()` | A recorder for callbacks. |
+| `startMockOAuth(opts?)` | A local OAuth server for connection tests. |
+
+### Command line
+
+```bash
+npx esoul-app validate <dir>
 ```
 
-`runOp` also records what your op NOTIFIED and who it addressed, and `notifyFails` makes
-notifying throw — the question worth asking of any op that tells somebody after it has written
-something: does the write survive?
+Exits 0 when the folder is a well-formed application package; otherwise prints each problem.
+
+## Rules
+
+> **Note.** State is a fold of events. A processor must be a pure, idempotent function of
+> `(state, event)`. Ids and timestamps are minted in `dataCreator`, never in a processor. An event
+> that replaces a whole value carries a collapse key so a burst folds to one event.
+
+> **Note.** Access is declared, not checked. An op, route or task that tests the viewer itself has
+> a second answer to a question the platform already answered; the platform's answer is the one
+> enforced. Use `viewer.role` and `viewer.can` to decide what a screen shows, not what a request
+> may do.
+
+> **Warning.** A task handler is re-run from the top after every step boundary. Every side effect
+> — a dispatch, a fetch, an emit, a random id — belongs inside `ctx.step.run`. A side effect outside
+> a step runs once per replay.
+
+> **Warning.** Application code imports the platform only through `esoul-sdk`. A relative import
+> into the platform's source is refused by the checks and by the install.
+
+> **Note.** `viewer.signedIn` in the UI is the page's estimate. Whether a request needs an account
+> is decided by the op; attempt the call and let `useSignInWall().raise` act on `login-required`.
+
+## Documentation
+
+| | |
+|---|---|
+| [1. Getting started](docs/01-getting-started.md) | The Forge loop, package layout, the smallest complete application |
+| [2. The manifest](docs/02-manifest.md) | Every field of `plugin.json` |
+| [3. Events and state](docs/03-events-and-state.md) | `dataCreator`, `processor`, collapse keys, replay |
+| [4. Tools](docs/04-tools.md) | Tools on every surface; deriving a tool from its op |
+| [5. The UI](docs/05-ui.md) | React, hooks, theme, responsive rules |
+| [6. The server](docs/06-server.md) | Ops, routes, webhooks, `computer`, charts, calling other applications |
+| [7. Background tasks](docs/07-background-tasks.md) | The durable executor, the replay model, concurrency |
+| [8. Connections](docs/08-connections.md) | OAuth and API-key connections held by the platform |
+| [9. Files](docs/09-files.md) | Workspace files, Drive, file providers |
+| [10. Testing](docs/10-testing.md) | The fold contract, ops, tasks and rules as tests |
+| [11. Shipping](docs/11-shipping.md) | The repository, review, release, install, dependencies |
+| [12. Rules and failures](docs/12-rules.md) | Each rule with the failure it prevents |
+| [13. People and access](docs/13-people-and-access.md) | Viewer, roles, access levels, composed roles, the sign-in wall |
+| [14. Your own tables](docs/14-database.md) | `db`, rules, scopes, sealed fields, indexes, migrations |
+| [15. Realtime](docs/15-realtime.md) | Topics, audiences, addressing |
+| [16. Bindings](docs/16-bindings.md) | Slots, contracts, reaching another application |
 
 ## Versions
 
-- **0.7.0** — what a catalogue-sized app needs. **Index kinds**: a plain group is a btree,
-  `{ fields: ["tags"], kind: "contains" }` answers `{ tags: { has: … } }` on a list, and
-  `{ fields: ["title"], kind: "text" }` answers `{ title: { contains: … } }` — so a department
-  and a search term are questions for the database, with `cursor` paging. `contains` ignores
-  case in both clients (a search box that misses "earl grey" is broken quietly). Asking a list a
-  scalar's question (or the reverse) is refused with the right one named. **`ctx.emit`** — an op
-  records on its OWN timeline, so a fact reaches the fold whether a person or an agent caused it.
-  A list field defaults to `[]` (it used to refuse every create), a rule-less model defaults to
-  the role your manifest calls the owner, and `viewerProfile` is a real export rather than a
-  declaration. Documented the client surface that was already there: `aggregate`, `groupBy`,
-  `$transaction`, the `*Many` writes.
-- **0.6.0** — the full-stack release. Your own tables (`db`, `pluginDb`, rules, scopes, sealed
-  fields, additive migrations applied on install). `viewer` on every seam, app `roles`, per-surface
-  access levels with the sign-in wall. Realtime audiences: a topic says who hears it and the mint
-  issues a token per channel. Bindings: `uses`/`provides`, contracts, `ctx.apps.<slot>`. A tool now
-  acts for the person who invoked it rather than for the platform. `viewerProfile` — the caller's
-  own esoul account, server-side only. Testing: `memoryDb`, `fakeViewer`, `runOp`.
-- 0.5.0 — server routes (`pluginServer.routes`), `sseStream`, plugin realtime.
-- 0.3.0 — renamed from `@externalsoul/plugin-sdk`. `readAppState`, `callWorkspaceTool`, `nanoid`
-  on the index. The import wall: an app reaches the platform only through this package.
-- 0.2.0 — file sources and providers.
+| Version | Changes |
+|---|---|
+| 0.15.0 | **Files work in a Forge box**, and do more: `FilesApi.list(…, opts)` narrowed at the source, `listAll`, `resolvePath`, `readMany`, `write` (make or replace by name — a label file beside its image; manifest `fileSources.write`), `readGrant` + `fileGrantUrl` (signed, expiring URLs for an `<img>` in a preview and for a machine). UI: `useFolderAutocomplete`, `useFileUrls`, `useFileSourceEntries(…, opts)` with `loadMore`, `listFileEntries`, `resolveFilePath`. `ImageLabeler` / `PolygonCanvas` — the Explorer's polygon editor as a component. Pure: `pairImagesWithLabels`, `serializeLabelMe`, `parseLabelMe`, `labelFileNameFor`. Testing: `memoryFiles`. |
+| 0.14.0 | Attribute-scoped access: `roles.attributes` (typed; validated when granted; `viewer.attrs`); scoped rule principals `{ role, where }` with `viewer.<attribute>`, applied to reads, aggregates, updates and deletes and filling or refusing creates, in both database clients; a rule combining `creator` with a scoped role is refused. `compileManifestRules` is the one reader of a manifest's rules. VIEW AS personas carry attributes (`visitor-a?customer=204`). |
+| 0.13.0 | Composed roles: `roles.custom` envelope; `defineAppRole`, `removeAppRole`; `setAppRole` with attributes; `listAppRoles` returns composed roles and the envelope. Reads, updates and deletes are narrowed by the composition in both database clients; surfaces not handed to the role are refused; `useViewer().customRole` and `can`. `access: [<roles>]` on a surface. Composed roles are personas in the Forge. |
+| 0.12.0 | Token routes: `"access": "token"` and `mintRouteToken`. `ctx.origin` on ops. |
+| 0.11.0 | `chartSvg`, `renderChartImage`. `computer().python()` and `runJson()` with `truncated` outputs. `APPROVAL_WAIT`. Table refusals name the constraint. |
+| 0.10.0 | `computer(ctx, machineNodeId)`. Server code in a Forge preview reaches the workspace (`callWorkspaceTool`, `readAppState`, `computer`), gated by `workspaceTools`. |
+| 0.9.0 | `defineOps`, `handleOp`, `opTool`: one declaration for an op's input, its server parser and its tool. |
+| 0.8.0 | `generateAppImage`; `setAppRole` and `listAppRoles`; `useSignInWall().ask()` and `serverSays`. |
+| 0.7.0 | Index kinds (`contains`, `text`) and cursor paging; `ctx.emit`; list field defaults; `aggregate`, `groupBy`, `$transaction`, `*Many` documented. |
+| 0.6.0 | Tables (`db`, `pluginDb`, rules, scopes, sealed fields, additive migrations). `viewer` on every entry point, roles, access levels, the sign-in wall. Realtime audiences. Bindings. `viewerProfile`. Testing: `memoryDb`, `fakeViewer`, `runOp`. |
+| 0.5.0 | Routes, `sseStream`, realtime channels. |
+| 0.3.0 | Renamed from `@externalsoul/plugin-sdk`. `readAppState`, `callWorkspaceTool`. The import wall. |
+| 0.2.0 | File sources and providers. |
+
+## Releasing (maintainers)
+
+```bash
+cd packages/esoul-sdk
+npm run release:check   # fresh docs + build, dist completeness, pack, clean-room install, every entry point, the CLI
+npm publish             # prepack rebuilds docs and dist; the owner's npm login
+npm view esoul-sdk version
+```
+
+The check refuses a version that is already on npm and a `dist/` that lacks a module `src/` has.
+Commit the regenerated `llms.txt`, `llms-full.txt` and `api-reference.md` with the version bump.