esoul-sdk 0.3.0

package/llms-full.txt

@@ -0,0 +1,934 @@
+# esoul-sdk — the complete contract for building ExternalSoul apps (llms-full.txt)
+
+Generated from README.md and docs/. Read it whole. Every rule carries the failure that earned it.
+
+# esoul-sdk
+
+Build **ExternalSoul apps**: native apps for an event-sourced workspace where people and AI
+agents share one canvas. An app you write with this SDK is indistinguishable from the platform's
+own once it ships — the same events, the same timeline, the same durability, the same tools that
+chat, voice, agents and MCP call.
+
+```bash
+npm install --save-dev esoul-sdk
+```
+
+Inside ExternalSoul the package name resolves to the platform's real implementations. Outside it
+(your editor, your tests) the package gives you the types, the manifest validator, and the test
+helpers; the server functions throw "host only" if called, because they run in the platform.
+
+## Where to start
+
+- **Build it in the Forge, not on your laptop.** Open a Forge board in your workspace and ask it
+  to open a workbench for your app. You get a cloud machine with the platform on it, a live
+  preview in your frame, your app's tools callable before it is installed, checks, and a submit
+  button. Read [docs/01-getting-started.md](docs/01-getting-started.md).
+- **The contract, one page per part:**
+  1. [Getting started](docs/01-getting-started.md) — the loop, the package layout
+  2. [The manifest](docs/02-manifest.md) — `plugin.json`, every field
+  3. [Events and state](docs/03-events-and-state.md) — the heart: dataCreator, processor, replay
+  4. [Tools](docs/04-tools.md) — what agents call, on every surface
+  5. [The UI](docs/05-ui.md) — React, hooks, theme, responsive rules
+  6. [The server half](docs/06-server.md) — ops, webhooks, reading state, calling other apps
+  7. [Background tasks](docs/07-background-tasks.md) — durable work, polling, the replay model
+  8. [Connections and OAuth](docs/08-connections.md) — tokens the platform holds for you
+  9. [Files](docs/09-files.md) — workspace files, Drive, your own provider
+  10. [Testing](docs/10-testing.md) — the fold contract as tests
+  11. [Shipping](docs/11-shipping.md) — submit, review, release, install; the import wall
+  12. [Rules and failures](docs/12-rules.md) — every rule with the failure that earned it
+- **For a coding model:** `llms.txt` (short) and `llms-full.txt` (the whole contract in one file).
+
+## The one rule that explains the others
+
+**Events are the truth.** Your app's state is the fold of its events, re-run on every replay,
+scrub and sync. So a reducer must be pure and idempotent, ids and timestamps are minted in the
+`dataCreator` (never in a reducer), whole-replace events carry a collapse key, and the agent-facing
+state description never claims something it could not read. Everything in the docs follows from
+that.
+
+## What is in the package
+
+| Entry | What it gives you |
+|---|---|
+| `esoul-sdk` | `ApplicationSchema`, `EventDefinition`, `EventTypes`, `ApplicationIdentifier`, `incompleteStateNotice`, `deterministicReducerId`, `stableStringify`, `timingSafeEqual`, `nanoid`, `callPluginOp`, the manifest schema |
+| `esoul-sdk/react` | `usePluginEventDispatch`, `useAppCanEdit`, `usePluginCurrentChatId`, `useWorkspaceTools`, file hooks |
+| `esoul-sdk/server` | `PluginServerModule`, `readAppState`, `callWorkspaceTool`, `emitPluginAppEvent`, `getPluginConnectionCredentials`, file provider types |
+| `esoul-sdk/testing` | a mock OAuth server for connection tests |
+| `esoul-app validate <dir>` | validates a package folder against the manifest schema |
+
+## Versions
+
+- **0.3.0** — renamed from `@externalsoul/plugin-sdk` (still resolved as an alias inside the
+  platform). `readAppState`, `callWorkspaceTool`, `nanoid` on the index. The import wall: an app
+  reaches the platform only through this package. Docs rewritten for the Forge workbench loop.
+- 0.2.0 — file sources and providers.
+- 0.1.0 — the contract: manifest, schema, events, tools, tasks, webhooks, ops, connections.
+
+
+
+==============================================================================
+# 1. Getting started
+
+## What an app is
+
+An ExternalSoul app is a folder. Compiled into the platform, it becomes a native app: it renders
+in the same frame as the spreadsheet next to it, its state is a fold of typed events on the
+workspace timeline, and its tools are callable from chat, voice, the agent builder and MCP. There
+is no dynamic loading and no sandbox at runtime: the review of your submission is the trust
+boundary, and this SDK plus the platform's checks make your package correct on arrival.
+
+## The loop
+
+You do not need the repository, a local checkout, or a running platform.
+
+1. In your ExternalSoul workspace, add a **Forge** board.
+2. Tell the assistant (or your own Claude over MCP) to **open a workbench** for your app. The
+   platform boots a cloud machine holding the platform's source, scaffolds your app folder from
+   the template, and shows a **live preview** in the board's frame.
+3. Write the files with the board's tools (`write_app_file`, `edit_app_file`). Every change
+   hot-reloads into the preview and comes back with the preview's health, the compiler's own
+   words if you broke it.
+4. **Look** at it (`look_at_app`: desktop and phone, light and dark), **call its tools** before it
+   is installed (`call_app_tool`, the events land in the frame), **read its state**
+   (`read_app_state`), **run its tests** (`test_app`, seconds) and **the full checks**
+   (`check_app`: registry sync, the platform suites, types, the import wall).
+5. **Submit** (`ship_app`). The owner reviews the diff, approves, and releases it with one
+   script. The board then says "released — add it to a workspace".
+6. Add it to a workspace like any other app. Its tools are live everywhere.
+
+From your own editor, `npm install --save-dev esoul-sdk` gives you the same types and the
+manifest validator, and lets you write tests locally. The code still runs in the platform.
+
+## Package layout
+
+```
+src/plugins/<id>/
+  plugin.json                 the manifest (docs/02)
+  app.tsx                     the SCHEMA — exports `pluginSchema`; never "use client" (docs/03, 04)
+  ui/<id>-ui.tsx              the React UI — "use client" (docs/05)
+  server.ts                   only if you declare ops or webhooks (docs/06)
+  <id>.test.ts                the fold contract as tests (docs/10)
+  <id>-ui.test.tsx            renders the UI once, checks the empty state
+  package.json                only if you need an npm package the platform lacks (docs/11)
+```
+
+The folder name is the app id: lower-case, hyphens. The application type is `plugin_` plus the
+id with underscores (`sticky-notes` → `plugin_sticky_notes`). Neither changes after shipping.
+
+## The smallest complete app
+
+```ts
+// app.tsx
+import { nanoid, EventTypes, incompleteStateNotice, type ApplicationIdentifier,
+  type ApplicationSchema, type EventData, type EventDefinition, type ApplicationPort } from "esoul-sdk";
+import { z } from "zod";
+import { NotesUi } from "./ui/notes-ui";
+
+export interface NotesData extends ApplicationIdentifier {
+  notes: { id: string; text: string; at: number }[];
+}
+
+export const notedEvent: EventDefinition<NotesData> = {
+  eventName: "plugin_notes_noted",
+  type: EventTypes.Client,
+  dataCreator: (args) => ({
+    eventName: "plugin_notes_noted",
+    eventData: { id: args.id ?? nanoid(), text: String(args.text ?? ""), at: args.at ?? Date.now() },
+    timestamp: Date.now(),
+    workspaceId: args.workspaceId,
+    applicationId: args.applicationId || args.nodeId,
+    instanceName: args.instanceName,
+    chatIdSource: args.chatIdSource,
+  }) as EventData<any>,
+  processor: (state, event) => {
+    const { id, text, at } = event.eventData ?? {};
+    if (typeof id !== "string" || typeof text !== "string" || !text.trim()) return state;
+    if (state.notes.some((n) => n.id === id)) return state; // replay-safe
+    return { ...state, notes: [...state.notes, { id, text: text.trim(), at: typeof at === "number" ? at : 0 }] };
+  },
+};
+
+export const pluginSchema: ApplicationSchema<NotesData> = {
+  applicationType: "plugin_notes",
+  description: "Notes on the timeline.",
+  reactNode: NotesUi,
+  reconstructStateFromEventLog: true,
+  events: [notedEvent],
+  getPorts: (): ApplicationPort[] => [],
+  stateCreator: (identifier) => ({ ...identifier, notes: [] }),
+  toolkitCreator: (identifier, forChatId, eventCallback) => {
+    const base = identifier.instanceName.replace(/[^a-zA-Z0-9]/g, "_");
+    const tools = {
+      [`add_note_${base}`]: {
+        description: `Add a note to "${identifier.instanceName}". Returns its id.`,
+        parameters: z.object({ text: z.string().min(1) }),
+        execute: async ({ text }: { text: string }) => {
+          const id = nanoid();
+          await eventCallback(notedEvent.dataCreator({ ...identifier, applicationId: identifier.nodeId, chatIdSource: forChatId, id, text }));
+          return `Added note ${id}.`;
+        },
+      },
+    };
+    for (const t of Object.values(tools)) (t as any).onClient = (t as any).execute; // every surface
+    return tools;
+  },
+  getStateDescription: (state) => {
+    const notLoaded = incompleteStateNotice({ title: "Notes", instanceName: state?.instanceName, shape: { lists: { notes: state?.notes } } });
+    if (notLoaded) return notLoaded;
+    return [`## Notes — "${state.instanceName}"`, ...(state.notes.length ? state.notes.slice(-10).map((n) => `- ${n.text} \`id:${n.id}\``) : ["- No notes yet."])].join("\n");
+  },
+};
+```
+
+Every line of that is explained in the next pages. The workbench scaffolds you an equivalent.
+
+
+
+==============================================================================
+# 2. The manifest — `plugin.json`
+
+Validated by `esoul-app validate <dir>`, by the workbench's checks and at release. The JSON
+schema is shipped at `schemas/plugin.schema.json`.
+
+```json
+{
+  "manifestVersion": 1,
+  "id": "sticky-notes",
+  "name": "Sticky notes",
+  "version": "0.1.0",
+  "description": "A wall of paper notes: pin, colour, drag, and let agents add to it.",
+  "applicationType": "plugin_sticky_notes",
+  "entry": "app",
+  "icon": "StickyNote",
+  "author": { "name": "Sticky notes", "url": "esoul:user:<your id>" },
+
+  "ops": ["read-notes"],
+  "webhooks": ["inbound"],
+  "kickableTasks": ["sync"],
+  "pollTasks": [{ "task": "refresh", "everyMinutes": 30 }],
+  "workspaceTools": ["spreadsheet:add_row", "my_computer:claude_task"],
+  "connections": [{ "key": "github", "kind": "oauth2", "label": "GitHub",
+    "authorizeUrl": "https://github.com/login/oauth/authorize", "tokenUrl": "https://github.com/login/oauth/access_token",
+    "oauthScopes": ["repo"], "clientIdEnv": "GITHUB_CLIENT_ID", "clientSecretEnv": "GITHUB_CLIENT_SECRET" }],
+  "fileSources": { "workspace": "read", "providers": ["google-drive"] },
+  "platformApi": { "min": "1.1.0" }
+}
+```
+
+| Field | Rule |
+|---|---|
+| `id` | lower-case, hyphens; equals the folder name; **never changes** after shipping |
+| `name` | what people see in the picker (≤ 80) |
+| `version` | semver; bump on every submission |
+| `description` | one honest paragraph (≤ 500) |
+| `applicationType` | `plugin_` + id with underscores; the platform tells plugins from built-ins by the prefix; **never changes** |
+| `entry` | the schema module without extension (`app`) |
+| `icon` | a lucide-react name from the curated set (the workbench lists them when yours is wrong); missing → the generic glyph, never a broken tile |
+| `author` | stamped by the workbench at scaffold: `url` is `esoul:user:<id>`, and it is what the release script and the id-collision check read. No email — a manifest may become public |
+| `ops` | server operations exported by `server.ts` (docs/06) |
+| `webhooks` | inbound routes `POST /api/plugins/<id>/webhook/<hook>` served by `server.ts` (docs/06) |
+| `kickableTasks` | task names the **browser** may kick (docs/07) |
+| `pollTasks` | tasks the platform kicks on a cadence, 5-minute granularity, minimum 5 (docs/07) |
+| `workspaceTools` | the grant wall for calling OTHER apps' tools — `<applicationType>:<tool base name>`; absent = no cross-app access (docs/04, 06) |
+| `connections` | OAuth2 / API-key connections the user grants once and the platform holds sealed; **names of env vars, never values** (docs/08) |
+| `fileSources` / `fileProviders` | consent to read workspace files and providers; providers you contribute (docs/09) |
+| `platformApi` | refuse install outside `[min, max]` of the platform contract version |
+| `scopes` | reserved |
+
+Secrets never live in a manifest, and a `package.json` in the folder declares npm dependencies
+the platform does not have yet (docs/11).
+
+
+
+==============================================================================
+# 3. Events and state — the heart of the contract
+
+## The model
+
+An app's state is **the fold of its events**. The platform stores the events (a per-workspace,
+per-app log with a sequence number) and derives the state by running your processors over them:
+on the client as a person taps, on the server as the canonical fold, again on every replay,
+scrub, snapshot, cross-device sync and share redaction. There is no other state. That is what
+makes the timeline scrubbable and an agent's action indistinguishable from a person's.
+
+So three things must hold, and the checks and the review look for them:
+
+1. **A processor is pure and idempotent.** Same input, same output, on every replay; the same
+   event applied twice leaves the state as after once.
+2. **Ids and timestamps are minted in the `dataCreator`, never in a processor.** A processor that
+   calls `nanoid()` or `Date.now()` folds differently every replay and breaks time-scrubbing.
+3. **A processor refuses what it cannot trust** — returns `state` unchanged rather than throw.
+   Malformed payloads exist (a retried webhook, an old client), and a throw in a fold takes the
+   whole workspace view down.
+
+## Anatomy of an event
+
+```ts
+export const noteTextSetEvent: EventDefinition<StickyNotesData> = {
+  eventName: "plugin_sticky_notes_text_set",        // globally unique; prefix with your type
+  type: EventTypes.Client,                          // Client = dispatched by UI/tools; Server = from tasks/webhooks
+  dataCreator: (args) => ({                         // MINTS the envelope. Called by the UI and by tools.
+    eventName: "plugin_sticky_notes_text_set",
+    eventData: { noteId: args.noteId, text: String(args.text ?? "").slice(0, 2000), at: args.at ?? Date.now() },
+    timestamp: Date.now(),
+    workspaceId: args.workspaceId,
+    applicationId: args.applicationId || args.nodeId,
+    instanceName: args.instanceName,
+    chatIdSource: args.chatIdSource,
+  }) as EventData<any>,
+  processor: (state, event) => {                    // PURE. The fold.
+    const d = event.eventData ?? {};
+    if (typeof d.noteId !== "string" || typeof d.text !== "string") return state;
+    const i = state.notes.findIndex((n) => n.id === d.noteId);
+    if (i === -1) return state;                     // an unknown id is ignored, never invented
+    const next = [...state.notes]; next[i] = { ...next[i], text: d.text, updatedAt: d.at };
+    return { ...state, notes: next };
+  },
+  // One row on the timeline per typing burst, not one per keystroke (see below).
+  collapseConfig: {
+    collapseKeyFn: (eventData, ctx) => `${ctx.applicationId}:${(eventData as { noteId?: string }).noteId}:text`,
+    collapseWindowMs: 2500,
+  },
+};
+```
+
+`stateCreator(identifier)` returns the empty state — always including the identifier fields
+(`{ ...identifier, notes: [] }`), because tools and describers read `instanceName` from it.
+
+## Collapse keys: bursts are one event
+
+A person typing, dragging or resizing produces a burst. Without a collapse key that is a hundred
+timeline rows and a scrub that steps through every keystroke. `collapseConfig` merges consecutive
+events with the same key inside the window into one. Key by the **entity and the field**
+(`<app>:<noteId>:text`), never by the app alone: two notes edited in the same window must not
+merge. Whole-replace events (an editor's content, a canvas snapshot) **must** carry a collapse key.
+
+## What is local and what is an event
+
+What the workspace remembers goes through events. What is honestly local to one device — a drag
+in progress, text still being typed, which tab is open — stays in component state and lands as
+ONE event when done. Never mirror app state into a second store; never write derived data the
+fold could recompute.
+
+## Bounds
+
+Cap collections in the processor (`if (state.notes.length >= 300) return state;`) and cap
+strings in the `dataCreator`. An app with an unbounded array is an app that one day cannot be
+folded.
+
+## `reconstructStateFromEventLog: true`
+
+Set it. It tells the platform your state IS the fold, which is what enables replay-on-load,
+snapshot baselines and the durability layers. Every shipped plugin and the scaffold set it.
+
+## Server-typed events
+
+`EventTypes.Server` marks events that only server code emits (a task, a webhook). The UI must
+not dispatch them, and the processor still obeys the three rules.
+
+## Durability you get for free
+
+Because your events flow through the platform's dispatch chokepoint, an installed app inherits
+the write-ahead log with retry, session stamping, the stale-base write fence, the pagehide flush
+that walks every open app, server-side deduplication of re-sent events, one canonical server fold
+and the monotonic merge guard. You do nothing for this — except keep the three rules, because
+every one of those layers re-runs your processors.
+
+
+
+==============================================================================
+# 4. Tools — what agents call
+
+A tool is how chat, voice, the agent builder, MCP, a WebMCP page, and other apps' server code act
+on your app. **Every UI action has a tool twin**: if a person can do it by hand, an agent can do
+it by tool, through the same event. The platform mints your tools per instance as
+`<verb>_<instance name>` (spaces and punctuation → underscores).
+
+```ts
+toolkitCreator: (identifier, forChatId, eventCallback) => {
+  const base = identifier.instanceName.replace(/[^a-zA-Z0-9]/g, "_");
+  const idArgs = { ...identifier, applicationId: identifier.nodeId, chatIdSource: forChatId };
+  const tools = {
+    [`add_note_${base}`]: {
+      description: `Pin a new sticky note on "${identifier.instanceName}". Returns the note's id.`,
+      parameters: z.object({ text: z.string().min(1).max(2000), color: z.enum(["butter", "rose", "mint", "sky", "lilac"]).optional() }),
+      execute: async (args) => {
+        const noteId = nanoid();
+        await eventCallback(noteAddedEvent.dataCreator({ ...idArgs, noteId, ...args }));
+        return `Pinned note ${noteId}.`;
+      },
+    },
+    [`read_notes_${base}`]: {
+      description: `Read every note on "${identifier.instanceName}" with the ids the other tools take.`,
+      parameters: z.object({}),
+      readOnly: true,
+      publicSafe: true,
+      execute: async () => {
+        const { callPluginOp } = await import("esoul-sdk");
+        const s = await callPluginOp<Pick<StickyNotesData, "notes">>("sticky-notes", "read-notes", identifier.nodeId);
+        return describeNotes(s);
+      },
+    },
+  };
+  // Browser surfaces (voice, WebMCP) run `onClient`. Same work — never an empty stub:
+  // the voice runtime reports a tool that returns nothing as a SUCCESS.
+  for (const t of Object.values(tools)) (t as any).onClient = (t as any).execute;
+  return tools;
+},
+```
+
+## The two surfaces
+
+- **`execute`** runs on the server: chat, agents, MCP, the workbench's `call_app_tool`.
+- **`onClient`** runs in the browser: WebRTC voice, WebMCP. Events dispatch there too, and
+  `callPluginOp` rides the session. Setting `onClient = execute` for every tool is the simplest
+  correct form; the checks refuse an empty `onClient: () => {}` stub because it makes an agent
+  claim work it never did.
+
+## Flags
+
+- `readOnly: true` — a read-scoped token may call it.
+- `publicSafe: true` — safe to expose on a public storefront (no writes, no private data).
+
+## Honest results
+
+A tool result is what the model believes happened. Return what you **did**, and refuse what you
+could not do: an update to an id that is not on the wall says "No note X — nothing changed", not
+"updated". When a check needs server truth (an op) and it is unavailable, say so
+("could not verify the id here"). Name what the caller should do next; say where ids come from.
+
+## Server truth from a tool
+
+A tool that must read the real state (not the caller's cache) calls a **plugin op** through
+`callPluginOp(pluginId, op, nodeId, args)` — never `fetch("/api/v1/…")` itself (token-gated, it
+refuses the tool). Ops are yours to write (docs/06). In the workbench there is no database behind
+the preview, so such a tool is refused with a message naming `read_app_state`; that is expected.
+
+## Calling other apps
+
+From the UI: `useWorkspaceTools()` (docs/05). From server code: `callWorkspaceTool` (docs/06).
+Both are gated by the manifest's `workspaceTools` grants, `"<applicationType>:<tool base>"`, and
+both stay inside the app's own workspace.
+
+## Describing the app to agents
+
+`getStateDescription(state)` is what an agent reads about your app every turn. Compact, truthful,
+with ids. And **never claim a count or an emptiness you could not read**:
+
+```ts
+getStateDescription: (state) => {
+  const notLoaded = incompleteStateNotice({ title: "Sticky notes", instanceName: state?.instanceName, shape: { lists: { notes: state?.notes } } });
+  if (notLoaded) return notLoaded;   // a MISSING collection means the state did not load; [] is real
+  return describeNotes(state);
+},
+```
+
+
+
+==============================================================================
+# 5. The UI
+
+```tsx
+"use client";
+import React from "react";
+import { useAppCanEdit, usePluginEventDispatch, useWorkspaceTools } from "esoul-sdk/react";
+import { noteAddedEvent, type StickyNotesData } from "../app";
+
+export function StickyNotesUi({ state }: { state: StickyNotesData }) {
+  const dispatch = usePluginEventDispatch();   // the ONLY way to change state
+  const canEdit = useAppCanEdit();             // readers of a share see the app and cannot mutate
+  const tools = useWorkspaceTools(state);      // other apps' tools, per the manifest's grants
+  // …
+}
+```
+
+- Props are `{ state }`: the folded state, live. You never fetch it.
+- `dispatch(noteAddedEvent.dataCreator({ …identifier fields from state, …args }))` is a write.
+  Disable writes when `!canEdit`.
+- The schema module (`app.tsx`) is **never** `"use client"`; the UI module is. The schema imports
+  the UI, not the other way around (it closes a module cycle).
+
+## Theme
+
+Read the `.dark` class on `<html>`, and **observe it** (the person toggles at runtime):
+
+```ts
+function useIsDark() {
+  const [dark, setDark] = React.useState(false);
+  React.useEffect(() => {
+    const read = () => setDark(document.documentElement.classList.contains("dark"));
+    read();
+    const mo = new MutationObserver(read);
+    mo.observe(document.documentElement, { attributes: true, attributeFilter: ["class"] });
+    return () => mo.disconnect();
+  }, []);
+  return dark;
+}
+```
+
+Keep one palette object per app with `LIGHT` and `DARK` of the same shape. Light is warm sepia
+(`#faf6f0` / `#efe7d8` / `#e8dfd0`, brown text). Dark is translucent data surfaces over the frame's
+gradient, with editors, dialogs, popovers and sticky headers **opaque**.
+
+## Responsive
+
+The app renders inside the platform frame at any size: a desktop window, a maximised canvas, a
+390 px phone.
+
+- The root fills its frame: `width:100%; height:100%; overflow:hidden`, a flex column.
+- Chrome rows `flexShrink:0`; the content area `flex:1; minHeight:0` with its own scroll.
+- Measure the VISIBLE scroller for layout decisions (column counts), never a large inner canvas;
+  re-measure on resize.
+- Toolbars wrap; labels collapse on narrow widths; nothing needs a horizontal page scroll.
+- Touch is first-class: tap to act (never double-tap), `touch-action: manipulation`, larger hit
+  targets on `(pointer: coarse)`.
+- Custom clickables are real buttons: `role="button"`, `tabIndex={0}`, Enter and Space,
+  `aria-label` on icon-only controls. Distinct icons for distinct actions. Status is never
+  colour-only.
+- Popovers and menus portal to `document.body` with a solid backing.
+- An empty state that says what to do first, in the app's own voice.
+
+`look_at_app` in the workbench screenshots desktop-light, desktop-dark and phone-light. Open the
+images; a phone shot with a horizontal scrollbar is a bug.
+
+## Cross-app from the UI
+
+```ts
+const { call, listApps, available } = useWorkspaceTools(state);
+await call({ appType: "spreadsheet", tool: "add_row", args: { cells: { When: new Date().toISOString() } } });
+```
+
+In the workbench this works through the board's own tab, so you can test it before install;
+installed, only the grants in `plugin.json` `workspaceTools` are allowed. Declare each as
+`"<applicationType>:<tool base name>"`.
+
+## Files
+
+`usePluginWorkspaceFiles()`, `usePluginFileUpload()`, `useFileSources()`, `useFileSourceEntries()`
+— docs/09.
+
+## Character
+
+The platform is built as craft: a sticky-notes wall is linen and paper with a gummed strip, not a
+grid of yellow rectangles. Derive any randomness from stable ids, never `Math.random()` in
+render, so nothing twitches between renders. Motion is brief and respects reduced-motion.
+
+
+
+==============================================================================
+# 6. The server half — `server.ts`
+
+Only if you declare `ops`, `webhooks` or `fileProviders`. Marked `"server-only"`, it never
+reaches the client graph.
+
+```ts
+import "server-only";
+import { readAppState, callWorkspaceTool, emitPluginAppEvent, type PluginOpContext, type PluginServerModule, type PluginWebhookContext } from "esoul-sdk/server";
+
+async function readNotes(ctx: PluginOpContext) {
+  const app = await readAppState(ctx.nodeId);                    // folded to head; null = no such app
+  if (!app) throw new Error(`no app ${ctx.nodeId}`);
+  const s = app.state as Partial<StickyNotesData>;
+  if (!Array.isArray(s.notes)) throw new Error("the wall did not fold (notes missing)");   // MISSING ≠ empty
+  return { notes: s.notes };
+}
+
+export const pluginServer: PluginServerModule = {
+  ops: { "read-notes": readNotes },
+  webhooks: { inbound: async (ctx: PluginWebhookContext) => { /* verify, validate, kick a task, ACK */ } },
+};
+```
+
+## Ops — server truth for a tool
+
+`POST /api/plugins/<id>/op/<name>` with `{ nodeId, args }`. The route checks the caller's
+workspace write access and that `nodeId` is an instance of THIS plugin's type, then calls your
+handler with `ctx = { nodeId, workspaceId, args, cloudConnectionId, … }`. A tool reaches it through
+`callPluginOp` (docs/04). Ops are for reads that must be true now and for actions that need a
+credential; they must be idempotent (a tool call can be retried).
+
+## Webhooks — push in
+
+`POST /api/plugins/<id>/webhook/<hook>`. **Verify first** (a shared secret with
+`timingSafeEqual`), validate the body, then **ACK and hand the work to a task** through
+`ctx.sendInngestEvent("<applicationType>/<task>", payload)`. Never do the work in the webhook: a
+sender retries, a request has a time limit, and a task is durable. Give every push an
+idempotency key and derive the event's id from it (`deterministicReducerId`), so a redelivery
+folds to one item. Fail CLOSED when no secret is configured.
+
+## `readAppState(nodeId)`
+
+One app, folded to head: `{ nodeId, workspaceId, applicationType, foldedSeq, state }`, or `null`
+when absent. Use it for your own instance and, with care, for other apps in the same workspace.
+Never a raw database read — the import wall refuses `prisma` and the state column lags the log.
+
+## `callWorkspaceTool` — orchestrate other apps
+
+```ts
+const r = await callWorkspaceTool({
+  pluginId: "research-graph", nodeId: ctx.nodeId,
+  appType: "my_computer", tool: "claude_task",
+  args: { prompt: "Summarise the three papers in ~/reading", cwd: "~/reading" },
+});
+// r = { ok, text }
+```
+
+Gated exactly like the UI: the manifest must declare `"my_computer:claude_task"` in
+`workspaceTools`, and the target must live in the calling app's own workspace. Name a specific
+instance with `targetNodeId` instead of `appType`. This is how an app runs a `my_computer`
+(commands, Claude Code sessions, results), fills a spreadsheet from a job, or books a calendar.
+
+## `emitPluginAppEvent`
+
+Emit ANOTHER app's own events (its `dataCreator` shape) into the same workspace, actor-stamped as
+your plugin — for mirroring into a native app whose processors you cannot call directly.
+Same-workspace only.
+
+## Connections in server code
+
+`ctx.cloudConnectionId` is the instance's bound connection; resolve it with
+`getPluginConnectionCredentials(connectionId, pluginId)` (docs/08).
+
+## What server code may import
+
+Only `esoul-sdk` / `esoul-sdk/server`, relative files, `server-only`, and npm packages the
+platform already depends on. No `prisma`, no `node:*`, no `next`, no internals — the import wall
+(docs/11) refuses them, and a reviewer relies on that.
+
+
+
+==============================================================================
+# 7. Background tasks — durable work
+
+Tasks are declared on the schema and run on the platform's durable executor (Inngest). Each task
+becomes an event `<applicationType>/<taskName>`.
+
+```ts
+tasks: [
+  {
+    taskName: "ingest",
+    description: "Fold one pushed item — idempotent by pushId.",
+    concurrency: { limit: 1, scope: "per-app" },
+    handler: async (ctx) => {
+      await ctx.step.run("fold-pushed-item", async () => {
+        const { text, pushId } = ctx.eventData ?? {};
+        if (typeof text !== "string" || typeof pushId !== "string") return;
+        await ctx.dispatchEvent("plugin_todo_add_item", { itemId: `push-${pushId}`, text: text.trim(), createdAt: Date.now() });
+      });
+    },
+  },
+],
+```
+
+## The context
+
+- `ctx.identifier` — workspaceId, nodeId, applicationType, instanceName.
+- `ctx.eventData` — the kick's payload.
+- `ctx.step` — the durable step API; `await ctx.step.run("name", fn)` memoises `fn`'s result.
+- `ctx.getState()` — the app's state, fresh.
+- `ctx.dispatchEvent(eventName, eventData)` — through the full pipeline (append, fold, watermark),
+  using YOUR processors, so a task's mutation is identical to a tap's.
+- `ctx.notify(topic, data)` — a realtime nudge to the app's channel; the UI refetches.
+- `ctx.logger`.
+
+## The replay model — read this twice
+
+The executor **re-runs your handler from the top** after every step boundary, replacing each
+completed `step.run` with its cached result. Anything outside a `step.run` runs again on every
+replay. So: **every side effect lives inside a `step.run`** — a dispatch, a fetch, an emit — or
+it fires N+1 times. `nanoid()` and `Date.now()` outside a step produce different values per
+replay; mint them inside. Step names are unique per logical operation; loops include the index.
+
+## How a task gets kicked
+
+- **From a webhook**: `ctx.sendInngestEvent("<applicationType>/<task>", payload)` (docs/06).
+- **From the browser**: list the task in `kickableTasks`; the UI posts the event through the
+  platform's send-event route, which allows only listed tasks.
+- **On a cadence**: `pollTasks: [{ task, everyMinutes }]` — one shared platform sweep kicks each
+  live instance at 5-minute granularity (minimum 5). This is your cron. There are deliberately
+  no per-app Inngest functions: function ids are fixed at module load, and the plan caps
+  concurrency; a shared sweep costs nothing per app. Handlers must be idempotent anyway, because
+  sweeps and pushes overlap by design.
+
+## Concurrency
+
+`concurrency: { limit, scope: "per-app" | "global" }`. Keep the limit small; a parked wait holds
+no slot.
+
+## Testing tasks
+
+Unit-test the handler with a fake `ctx` (`step.run` that just calls the function, a recording
+`dispatchEvent`), and assert idempotency by running it twice with the same `eventData` (docs/10).
+
+
+
+==============================================================================
+# 8. Connections and OAuth — tokens the platform holds for you
+
+An app that talks to an outside service declares the connection; the person grants it once in
+Account settings; the platform stores the tokens sealed and refreshes them; your code asks for a
+credential when it needs one. **Secrets never live in your package**: the manifest names
+environment variables, not values.
+
+```json
+"connections": [{
+  "key": "github", "kind": "oauth2", "label": "GitHub",
+  "authorizeUrl": "https://github.com/login/oauth/authorize",
+  "tokenUrl": "https://github.com/login/oauth/access_token",
+  "oauthScopes": ["repo"],
+  "clientIdEnv": "GITHUB_CLIENT_ID", "clientSecretEnv": "GITHUB_CLIENT_SECRET"
+}]
+```
+
+`kind: "apiKey"` instead declares `headerNames` — the header names the app will send; the person
+pastes the key once.
+
+## Using a credential
+
+```ts
+import { getPluginConnectionCredentials } from "esoul-sdk/server";
+
+async function listRepos(ctx: PluginOpContext) {
+  if (!ctx.cloudConnectionId) throw new Error("connect GitHub in Account settings first");
+  const cred = await getPluginConnectionCredentials(ctx.cloudConnectionId, "my-app");
+  if (cred.kind !== "oauth2") throw new Error("expected an OAuth connection");
+  const r = await fetch("https://api.github.com/user/repos", { headers: { Authorization: `Bearer ${cred.accessToken}` } });
+  // …
+}
+```
+
+The platform unseals, checks expiry and refreshes with the stored refresh token before handing
+you `accessToken`. Never log it, never put it in an event, never return it from a tool.
+
+An instance is **bound** to one connection (`ctx.cloudConnectionId`); a person with two GitHub
+accounts adds two instances. File providers you contribute receive `ctx.connectionId` per active
+connection (docs/09).
+
+## The platform's own connections
+
+The platform already holds the person's Google connection for its mail and calendar apps. Reusing
+it from an app ("the mail app needs the Google tokens") is designed and not yet built: the
+manifest would declare `platformConnections` with scopes, the person would consent per app, and
+the SDK would hand out a token scoped to those scopes. Until then an app declares its own Google
+OAuth connection with the platform's client id and secret named as env vars, and the person
+grants it once more.
+
+
+
+==============================================================================
+# 9. Files — workspace files, Drive, your own provider
+
+Files are a consented surface. The manifest declares what the app may read:
+
+```json
+"fileSources": { "workspace": "read", "providers": ["google-drive"] }
+```
+
+- `workspace: "read" | "readwrite"` — the workspace's own files.
+- `providers` — provider keys the app may browse (the person's connected sources).
+- Absent = no file access at all. The person sees the declaration at install.
+
+## In the UI
+
+`usePluginWorkspaceFiles()` (the workspace's files, live), `usePluginFileUpload()` (upload as the
+person), `useFileSources(workspaceId)` and `useFileSourceEntries(source, path)` for browsing
+sources with entries.
+
+## In server code
+
+`pluginFiles(ctx)` / `filesForOp(ctx)` give a `FilesApi`: list a source, read a file (capped at
+`FILE_READ_CAP_BYTES`), resolve a `FileRef`. Source ids are `"workspace"`, `"google-drive"`,
+`"local:<mountId>"`, or your own provider's key; `parseSourceId` splits them.
+
+## Contributing a provider
+
+```json
+"fileProviders": [{ "key": "dropbox", "connectionKey": "dropbox", "label": "Dropbox" }]
+```
+
+`server.ts` exports `fileProviders: { dropbox: impl }` where `impl` implements
+`PluginFileProviderImpl` (`list`, `read`, …). The host derives one mount per ACTIVE connection
+of `connectionKey` and calls you with `ctx.connectionId` set; resolve the credential with
+`getPluginConnectionCredentials`. Reads are capped; errors are typed (`FileSourceError`,
+`FileSourceErrorKind`) so the UI can say "not found" or "too large" instead of failing blank.
+
+
+
+==============================================================================
+# 10. Testing — the fold contract as tests
+
+The workbench runs your tests with jest (`test_app`, seconds; `check_app` adds the platform's
+suites and a type check). Write the contract the reviewer wants to trust:
+
+```ts
+import { noteAddedEvent, noteTextSetEvent, pluginSchema, type StickyNotesData } from "./app";
+
+const IDENT = { workspaceId: "ws1", nodeId: "node1", applicationType: "plugin_sticky_notes", instanceName: "My wall" };
+const fresh = (): StickyNotesData => pluginSchema.stateCreator(IDENT as any, {} as any);
+const apply = (def: any, s: StickyNotesData, args: Record<string, any> = {}) => def.processor(s, def.dataCreator({ ...IDENT, ...args }));
+
+it("starts empty, adds exactly what it says, and a replay is a no-op", () => {
+  expect(fresh().notes).toEqual([]);
+  const once = apply(noteAddedEvent, fresh(), { noteId: "n1", text: "milk" });
+  const ev = noteAddedEvent.dataCreator({ ...IDENT, noteId: "n1", text: "milk" });
+  const twice = noteAddedEvent.processor(noteAddedEvent.processor(fresh(), ev), ev);
+  expect(once.notes.map((n) => n.id)).toEqual(["n1"]);
+  expect(twice).toEqual(noteAddedEvent.processor(fresh(), ev));
+});
+
+it("refuses a payload it cannot trust, and ignores an unknown id", () => {
+  expect(noteAddedEvent.processor(fresh(), { eventData: { noteId: 42 } } as any)).toEqual(fresh());
+  expect(apply(noteTextSetEvent, fresh(), { noteId: "ghost", text: "x" })).toEqual(fresh());
+});
+
+it("no processor mints an id or a time: two folds of one log are identical", () => {
+  const log = [noteAddedEvent.dataCreator({ ...IDENT, noteId: "n1", text: "a" }), noteTextSetEvent.dataCreator({ ...IDENT, noteId: "n1", text: "b" })];
+  const fold = () => log.reduce((s, e) => (e.eventName === noteAddedEvent.eventName ? noteAddedEvent : noteTextSetEvent).processor(s, e as any), fresh());
+  expect(fold()).toEqual(fold());
+});
+
+it("the description never claims an emptiness it could not read", () => {
+  expect(pluginSchema.getStateDescription({ ...IDENT } as any)).toMatch(/not loaded|incomplete/i);
+  expect(pluginSchema.getStateDescription(fresh())).toContain("No notes yet");
+});
+```
+
+A UI test renders once with a fixed state and checks the visible chrome and the empty state.
+Keep suites fast; they run beside the live preview on the same machine.
+
+## Tools
+
+Call `execute` with a recording `eventCallback` and assert the events it emitted and the text it
+returned — including the refusal text for a bad id.
+
+## Tasks
+
+Fake the context: `step.run` that calls its function, a recording `dispatchEvent`. Run the
+handler twice with the same `eventData`; the second run must not add a second item.
+
+## Connections
+
+`esoul-sdk/testing` ships `startMockOAuth()` — a local OAuth server that issues, refreshes and
+revokes tokens, so a connection-backed op can be tested end to end without a real provider.
+
+## The fold corpus
+
+Once an app has real history, record it (`scripts/plugins/record-fold-corpus.mjs` in the
+platform) and commit `fold-corpus.json` + hashes. From then on the checks refuse a change that
+alters what history MEANS — the strongest protection an app with data can have. Re-recording is
+declaring a migration; say so in the changelog.
+
+
+
+==============================================================================
+# 11. Shipping — submit, review, release, install
+
+## The path
+
+1. **Build and test in the Forge workbench** (docs/01). No repository access is needed: the
+   platform clones with its own installation for the app arm.
+2. **Submit** with `ship_app`. It refuses unless every check is green, commits, pushes the app's
+   branch and opens (or updates) a pull request labelled `user-app` and `submitted-by:<you>`.
+   The board shows "submitted — awaiting review". An app id already on the base branch under
+   another author is refused before a pull request exists — pick another id.
+3. **The owner reviews** the diff on GitHub and approves.
+4. **One script releases it**: checks the pull request out cleanly, runs the import wall, the
+   sync, your tests and the type check, merges any dependencies you declared onto your branch so
+   the reviewer sees them, squash-merges, waits for the production deploy, grants you the
+   entitlement, and tells your board "released — add it to a workspace".
+5. **Add it** in any of your workspaces from the app picker, by chat (`add_app`), or over MCP.
+   Its tools are live in chat, voice, agents and MCP; its events ride the timeline.
+
+## The import wall
+
+Review is the security boundary — your server code will run with the platform's credentials.
+Review is only tractable if nothing can reach around the SDK, so the wall refuses at check time,
+at sync and at release:
+
+- allowed: `esoul-sdk` (and `/server`, `/react`, `/testing`), relative files in your folder,
+  `server-only`, npm packages the platform already depends on;
+- refused: `@/…` platform internals, `node:*` and the node built-ins, `prisma`, `next`,
+  `inngest`, the Vercel SDKs, and any package not in the platform's `package.json`.
+
+Need something the SDK lacks? Ask for it in the SDK — that is the review's pressure valve. Need
+an npm package? Declare it (below); it becomes a visible line in the pull request.
+
+## Declaring npm dependencies
+
+Put a `package.json` in your app folder with only `dependencies`:
+
+```json
+{ "dependencies": { "date-fns": "^4.1.0" } }
+```
+
+The release script merges them into the platform's `package.json`, updates the lockfile on your
+branch, and pushes that commit onto your pull request before merging. In the workbench the
+package is not yet installed until that happens; a later platform version merges declared
+dependencies at `open`.
+
+## Entitlement — your app on your instance
+
+A released app is available to **its submitter** and to no one else until the owner grants
+everyone (`--public`). A plugin that predates entitlements stays public. The picker only offers
+what you may add; the add path enforces it.
+
+## Versions and upgrades
+
+Bump `version` on every submission. Events are forever: a change that alters what an existing
+event MEANS rewrites every workspace's history on the next fold — record a fold corpus (docs/10)
+so the checks catch it, and treat such a change as a migration with its own events.
+
+## Dev and deployed are symmetric
+
+In the workbench your app can already call other apps (through the board's tab) and its own
+tools can be called (`call_app_tool`) before install. Installed, the same calls go through the
+manifest's grants, and the platform mints your tools natively. Exposing an in-development app's
+tools to the whole workspace — so chat could call them before install — is the next step on
+the platform's side (the board minting proxy tools per app under construction).
+
+## Cost and lifetime
+
+A workbench dies ten minutes after the last touch (a tool call, or a board tab watching it).
+`close_workbench` stops it at once. Files and commits persist; reopening resumes and updates the
+platform it runs on.
+
+
+
+==============================================================================
+# 12. Rules and failures — every rule with the failure that earned it
+
+| Rule | The failure it prevents |
+|---|---|
+| Processors are pure and idempotent | a replay duplicated items; a scrub showed a state that never existed |
+| Ids and timestamps are minted in `dataCreator`, never in a processor | two folds of one log disagreed; snapshots diverged from replay |
+| A processor refuses a bad payload instead of throwing | one malformed webhook took a whole workspace view down |
+| Whole-replace and burst events carry a collapse key per entity and field | fifty timeline rows per sentence; two notes edited together merged into one |
+| `reconstructStateFromEventLog: true` | an app hydrated from a stale column and lost its last edits |
+| `getStateDescription` uses `incompleteStateNotice`; missing ≠ empty | an agent "saw" an empty list, invited a duplicate, and the notebook crashed on a throw |
+| Every tool has a real `onClient` (`= execute`) | the voice agent said "added the note" while nothing ran — an empty stub reads as success |
+| Tool results say what was DONE, refuse what was not | "updated" for an id that did not exist |
+| Server truth through a plugin op, never `fetch("/api/v1/…")` | the token-gated route refused the tool with "not readable" |
+| Cross-app calls only through `useWorkspaceTools` / `callWorkspaceTool` with manifest grants | an app could act on any app in the workspace unseen |
+| Server code reaches the platform only through `esoul-sdk/server` (the import wall) | a demo plugin read the database directly; a reviewer would have had to read every line |
+| Webhooks verify, validate, kick a task, ACK; idempotent by the sender's key | a retried push duplicated an item; a slow webhook timed out mid-work |
+| Every task side effect lives in a `step.run` | a dispatch fired four times, once per replay |
+| No per-app crons; `pollTasks` at 5-minute granularity | Inngest function ids and the plan's concurrency cap |
+| Secrets are env-variable NAMES in the manifest, never values | a client secret in a package that becomes public |
+| The schema module is never `"use client"` | the schema became a server proxy and every processor vanished at build |
+| The UI never imports the store or the registry, only `esoul-sdk/react` | a module cycle that crashed the client on load |
+| Root fills the frame, `overflow:hidden`, content `flex:1 minHeight:0` | the whole app scrolled sideways on a phone |
+| Measure the visible scroller, not the canvas | a 2400 px canvas laid new notes off the right edge |
+| Observe `.dark` at runtime | wrong colours until the person clicked something |
+| Bound every collection and string | an app that could no longer be folded |
+| Bump the manifest version; record a fold corpus once there is data | a "small" event change rewrote every workspace's history |
+
+## Symptom → rule
+
+- "It works in the preview and nothing changes after install" → the tool only had `onClient`, or
+  the event was `Server`-typed and dispatched from the UI.
+- "The timeline has hundreds of rows for one edit" → no collapse key.
+- "State reverts after reload" → `reconstructStateFromEventLog` unset, or a processor minted ids.
+- "check_app is red on registry" → the manifest failed validation or the import wall refused a
+  file; the detail names it.
+- "My tool needs the database" → write an op; call it with `callPluginOp`; expect a refusal in
+  the workbench (no database behind a preview) and use `read_app_state` there.